初试PL/SQL并行编程
2024-09-25 04:54:26
并行处理能大大提高数据的处理速度,它依赖于硬件资源、网络资源等环境.
并行处理的硬件资源环境分为5大类:
1.传统的单台计算机、单处理器、单内核的机器.(无法进行并行处理,但是并行程序还是可以运行的,此时就和普通程序一样了)
2.单台计算机、单处理器、多内核的机器.(目前本人的测试环境,没钱买多处理器的
)
3.单台计算机、多处理器、多内核的机器.
4.集群系统
5.分布式系统
PS:如何查看几个处理器、几核,常用的软件就是CPU-Z.
本人的测试机器CPU:
单CPU,双核心处理器
下面开始,编写一个普通的管道函数(这里演示的就是管道函数的并行处理):
CREATE OR REPLACE FUNCTION pipe_test
(
c_empno SYS_REFCURSOR
,p VARCHAR2 DEFAULT ','
) RETURN EMP_element
PIPELINED IS
v_element VARCHAR2(1000);
BEGIN
FETCH c_empno
INTO v_element;
LOOP
EXIT WHEN c_empno%NOTFOUND;
PIPE ROW(substr(v_element, 0, instr(v_element, p) - 1));
LOOP
v_element := substr(v_element
,instr(v_element, p) + 1
,length(v_element) - instr(v_element, p) + 1);
EXIT WHEN instr(v_element, p) = 0;
PIPE ROW(substr(v_element, 0, instr(v_element, p) - 1));
END LOOP;
PIPE ROW(v_element);
FETCH c_empno
INTO v_element;
END LOOP;
RETURN;
END pipe_test;
注意需要先建立类型EMP_element,如下:
CREATE OR REPLACE TYPE EMP_element as table of varchar2(100);
然后我们建立一个并行处理的管道函数:
CREATE OR REPLACE FUNCTION parallel_test
(
p_empno SYS_REFCURSOR
,p VARCHAR2 DEFAULT ','
) RETURN EMP_element
PIPELINED
PARALLEL_ENABLE(PARTITION p_empno BY ANY) IS
v_element VARCHAR2(1000);
BEGIN
FETCH p_empno
INTO v_element;
LOOP
EXIT WHEN p_empno%NOTFOUND;
PIPE ROW(substr(v_element, 0, instr(v_element, p) - 1));
LOOP
v_element := substr(v_element
,instr(v_element, p) + 1
,length(v_element) - instr(v_element, p) + 1);
EXIT WHEN instr(v_element, p) = 0;
PIPE ROW(substr(v_element, 0, instr(v_element, p) - 1));
END LOOP;
PIPE ROW(v_element);
FETCH p_empno
INTO v_element;
END LOOP;
RETURN;
END parallel_test;
留意下启用并行的关键字PARALLEL_ENABLE.
接下来我们构造大表进行测试,根据v$pq_sesstat视图的结果,我们来判断数据库系统是否真的进行了并行处理.
12:57:52 SYS@orcl> create table big_data as select 'Cc' a,'Dd' b from dual connect by level<1000000; 表已创建。 已用时间: 00: 00: 01.92
12:59:00 SYS@orcl> select count(*) from big_data; COUNT(*)
----------
999999 已选择 1 行。 已用时间: 00: 00: 00.31
13:47:08 SYS@orcl> set autot trace exp stat
--普通管道函数耗时40.48秒
13:48:42 SYS@orcl> select SCOTT.pipe_test(CURSOR(SELECT a||','||b from big_data)) FROM DUAL; 已选择 1 行。 已用时间: 00: 00: 40.48 执行计划
----------------------------------------------------------
Plan hash value: 2452824241 ------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 2 (0)| 00:00:01 |
| 1 | VIEW | | 8168 | 414K| 29 (0)| 00:00:01 |
| 2 | COLLECTION ITERATOR PICKLER FETCH| PIPE_TEST | 8168 | | 29 (0)| 00:00:01 |
| 3 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 |
------------------------------------------------------------------------------------------------ Note
-----
- dynamic sampling used for this statement (level=2) 统计信息
----------------------------------------------------------
1000515 recursive calls
0 db block gets
1000325 consistent gets
1548 physical reads
0 redo size
6027769 bytes sent via SQL*Net to client
1214 bytes received via SQL*Net from client
10 SQL*Net roundtrips to/from client
7 sorts (memory)
0 sorts (disk)
1 rows processed 13:53:24 SYS@orcl> alter system flush shared_pool; 系统已更改。 已用时间: 00: 00: 01.43
--使用启用并行的管道函数,耗时40.32,与普通的管道函数耗时差不多
--奇怪的是执行计划了多了一个BIG_DATA的全表扫描操作了
13:53:26 SYS@orcl> select SCOTT.parallel_test(CURSOR(SELECT a||','||b from big_data)) FROM DUAL; 已选择 1 行。 已用时间: 00: 00: 40.32 执行计划
----------------------------------------------------------
Plan hash value: 1682567826 ---------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ---------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | | 2 (0)| 00:00:01 | | 1 | VIEW | | 8168 | 414K| 29 (0)| 00:00:01 | | 2 | COLLECTION ITERATOR PICKLER FETCH| PARALLEL_TEST | 8168 | | 29 (0)| 00:00:01 | | 3 | TABLE ACCESS FULL | BIG_DATA | 968K| 7565K| 422 (3)| 00:00:06 | | 4 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | ---------------------------------------------------------------------------------------------------- Note
-----
- dynamic sampling used for this statement (level=2) 统计信息
----------------------------------------------------------
1000867 recursive calls
0 db block gets
1000353 consistent gets
1560 physical reads
0 redo size
6026847 bytes sent via SQL*Net to client
1006 bytes received via SQL*Net from client
8 SQL*Net roundtrips to/from client
9 sorts (memory)
0 sorts (disk)
1 rows processed
--通过v$pq_sesstat视图我们发现此时系统并没有进行并行处理,(其实通过执行计划也可看出没有使用并行)
13:54:21 SYS@orcl> select * from v$pq_sesstat; STATISTIC LAST_QUERY SESSION_TOTAL
------------------------------ ---------- -------------
Queries Parallelized 0 0
DML Parallelized 0 0
DDL Parallelized 0 0
DFO Trees 0 0
Server Threads 0 0
Allocation Height 0 0
Allocation Width 0 0
Local Msgs Sent 0 0
Distr Msgs Sent 0 0
Local Msgs Recv'd 0 0
Distr Msgs Recv'd 0 0 已选择11行。 已用时间: 00: 00: 00.00 13:54:40 SYS@orcl> alter system flush buffer_cache; 系统已更改。 已用时间: 00: 00: 00.00
13:55:27 SYS@orcl> alter system flush shared_pool; 系统已更改。 已用时间: 00: 00: 01.40
--接下来我们加HINT,强制并行处理,此时耗时25.03秒,降低了一半
13:55:30 SYS@orcl> select /*+ parallel */scott.PARALLEL_TEST(CURSOR(SELECT a||','||b from big_data))
FROM DUAL; 已选择 1 行。 已用时间: 00: 00: 25.03 执行计划
----------------------------------------------------------
Plan hash value: 3127237831 ----------------------------------------------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib | ----------------------------------------------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | | 2 (0)| 00:00:01 | | | | | 1 | PX COORDINATOR | | | | | | | | | | 2 | PX SEND QC (RANDOM) | :TQ10000 | 8168 | 414K| 16 (0)| 00:00:01 | Q1,00 | P->S | QC (RAND) | | 3 | VIEW | | 8168 | 414K| 16 (0)| 00:00:01 | Q1,00 | PCWP | | | 4 | COLLECTION ITERATOR PICKLER FETCH| PARALLEL_TEST | 8168 | | 16 (0)| 00:00:01 | Q1,00 | PCWP | | | 5 | PX BLOCK ITERATOR | | 968K| 7565K| 234 (3)| 00:00:03 | Q1,00 | PCWC | | | 6 | TABLE ACCESS FULL | BIG_DATA | 968K| 7565K| 234 (3)| 00:00:03 | Q1,00 | PCWP | | | 7 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 | | | | ----------------------------------------------------------------------------------------------------------------------------------- Note
-----
- dynamic sampling used for this statement (level=2)
- automatic DOP: Computed Degree of Parallelism is 2 统计信息
----------------------------------------------------------
1000941 recursive calls
4 db block gets
1000516 consistent gets
1842 physical reads
0 redo size
6026847 bytes sent via SQL*Net to client
1006 bytes received via SQL*Net from client
8 SQL*Net roundtrips to/from client
9 sorts (memory)
0 sorts (disk)
1 rows processed
--上面的执行计划及v$pq_sesstat都显示系统启用了并行处理,并行度为2
13:57:44 SYS@orcl> select * from v$pq_sesstat; STATISTIC LAST_QUERY SESSION_TOTAL
------------------------------ ---------- -------------
Queries Parallelized 0 1
DML Parallelized 0 0
DDL Parallelized 0 0
DFO Trees 0 1
Server Threads 0 0
Allocation Height 0 0
Allocation Width 0 0
Local Msgs Sent 0 553
Distr Msgs Sent 0 0
Local Msgs Recv'd 0 553
Distr Msgs Recv'd 0 0 已选择11行。 已用时间: 00: 00: 00.01
为了使示例更加有说服力,我们再来试一下普通管道函数+HINT强制并行看看:
16:02:36 SYS@orcl> select /*+ parallel */SCOTT.pipe_test(CURSOR(SELECT a||','||b from big_data)) FROM DUAL; 已选择 1 行。 已用时间: 00: 00: 43.34 执行计划
----------------------------------------------------------
Plan hash value: 2452824241 ------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 2 (0)| 00:00:01 |
| 1 | VIEW | | 8168 | 414K| 16 (0)| 00:00:01 |
| 2 | COLLECTION ITERATOR PICKLER FETCH| PIPE_TEST | 8168 | | 16 (0)| 00:00:01 |
| 3 | FAST DUAL | | 1 | | 2 (0)| 00:00:01 |
------------------------------------------------------------------------------------------------ Note
-----
- dynamic sampling used for this statement (level=2)
- automatic DOP: Computed Degree of Parallelism is 1 统计信息
----------------------------------------------------------
1003093 recursive calls
18 db block gets
1000973 consistent gets
1628 physical reads
0 redo size
6026843 bytes sent via SQL*Net to client
1006 bytes received via SQL*Net from client
8 SQL*Net roundtrips to/from client
100 sorts (memory)
0 sorts (disk)
1 rows processed
由于普通函数不支持并行,所以即使+HINT强制并行,系统还是没有使用并行处理.耗时依然为40多秒.
如果不使用函数,使用普通的SQL,又是可以使用HINT强制并行的,如下:
15:59:07 SYS@orcl> select /*+ parallel */ a||','||b from big_data; 已选择999999行。 已用时间: 00: 00: 04.09 执行计划
----------------------------------------------------------
Plan hash value: 2638980575 --------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | TQ |IN-OUT| PQ Distrib |
--------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 968K| 7565K| 234 (3)| 00:00:03 | | | |
| 1 | PX COORDINATOR | | | | | | | | |
| 2 | PX SEND QC (RANDOM)| :TQ10000 | 968K| 7565K| 234 (3)| 00:00:03 | Q1,00 | P->S | QC (RAND) |
| 3 | PX BLOCK ITERATOR | | 968K| 7565K| 234 (3)| 00:00:03 | Q1,00 | PCWC | |
| 4 | TABLE ACCESS FULL| BIG_DATA | 968K| 7565K| 234 (3)| 00:00:03 | Q1,00 | PCWP | |
-------------------------------------------------------------------------------------------------------------- Note
-----
- dynamic sampling used for this statement (level=2)
- automatic DOP: Computed Degree of Parallelism is 2 统计信息
----------------------------------------------------------
6 recursive calls
0 db block gets
1577 consistent gets
1520 physical reads
0 redo size
13467035 bytes sent via SQL*Net to client
733742 bytes received via SQL*Net from client
66668 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
999999 rows processed
通过上面的示例可以看到,并行处理可以大大加速数据的处理,几乎成倍的提升性能.
但并不是并行度为几,就会提示几倍的性能,它受硬件环境,优化器内部算法,PL/SQL引擎解析等等的约束.
总之,并行处理给我们带来的性能提升是很大的,所以在某些场景使用并行编程也是很必要的.
最新文章
- selenium 页面截图并保存
- ie 7/8不支持trim的属性的解决方案
- asp中 grideview 更新 无法获取值 解决办法
- storm kafka整合
- Json学习篇
- PHP实现http与https转化
- POJ_2385_Apple_Catching_(动态规划)
- Unity3d IOS中的IGUI控件
- 纯js实现分页
- 批量删除Kindle Personal Documents
- linux命令——scp
- C 真正理解二级指针
- 调用本地摄像头拍照（H5和画布）
- chrome浏览器下载内容存放位置
- postgresql-分页数据重复问题探索
- ionic3应用的Android打包签名发布步骤
- [JS] Topic - Object.create vs new
- gitbash上使用tree
- css3背景颜色渐变属性 兼容性测试基础环境为：windows系统；IE6.0+, Firefox4.0+, Chrome4.0+, Safari4.0+, Opera15.0+
- Adobe Reader 2019 Offline Installer, Free Download - Best PDF Reader