KingbaseES例程之快速删除表数据
概述
快速删除表中的数据
delete语句删除数据
表中的数据被删除了，但是这个数据在硬盘上的真实存储空间不会被释放。
这种删除缺点是：删除效率比较低。
这种删除优点是：支持删除部分数据，支持回滚。
truncate语句删除数据
这种删除效率比较高，表被一次截断，物理删除。
这种删除缺点：不支持删除部分数据。
这种删除优点：快速，支持回滚。
案例：删除大表数据，但保留少量数据
一张表有100万条数据，分为1000组信息，仅保留每组的最后一条数据，如何快速删除其它99万余条数据？
方法一：删除每组非最大值的数据
explain (analyse,buffers )
delete
from test10
where (c1,id) not in (select c1,max(id) from test10 group by c1)
returning *;
Delete on test10 (cost=36508.94..56943.94 rows=500000 width=6) (actual time=221.183..1732.834 rows=998999 loops=1)
Buffers: shared hit=2012980
-> Seq Scan on test10 (cost=36508.94..56943.94 rows=500000 width=6) (actual time=221.128..583.449 rows=998999 loops=1)
Filter: (NOT (hashed SubPlan 1))
Rows Removed by Filter: 1001
Buffers: shared hit=9547
SubPlan 1
-> GroupAggregate (cost=0.42..36506.44 rows=1001 width=8) (actual time=0.067..219.780 rows=1001 loops=1)
Group Key: test10_1.c1
Buffers: shared hit=4112
-> Index Only Scan using idx01 on test10 test10_1 (cost=0.42..31496.42 rows=1000000 width=8) (actual time=0.010..126.628 rows=1000000 loops=1)
Heap Fetches: 0
Buffers: shared hit=4112
Planning Time: 0.120 ms
Execution Time: 1799.063 ms
方法二：CTE获取每组最新行，删除每组非CTE的数据
explain (analyse,buffers )
with recursive cte as (
(select c1, ctid from test10 order by c1, id desc limit 1)
union all
(select test10.c1, test10.CTID
from cte,
lateral ( select CTID, c1
from test10
where cte.c1 < test10.c1
order by test10.c1, test10.id desc
limit 1) test10
))
delete from test10
where not exists (select 1 from cte where cte.ctid = test10.ctid )
returning *
;
Delete on test10 (cost=62.30..28121.41 rows=999899 width=36) (actual time=10.799..1627.548 rows=998999 loops=1)
Buffers: shared hit=2013025
CTE cte
-> Recursive Union (cost=0.42..59.02 rows=101 width=10) (actual time=0.012..9.888 rows=1001 loops=1)
Buffers: shared hit=4157
" -> Subquery Scan on ""*SELECT* 1"" (cost=0.42..0.49 rows=1 width=10) (actual time=0.010..0.013 rows=1 loops=1)"
Buffers: shared hit=4
-> Limit (cost=0.42..0.48 rows=1 width=14) (actual time=0.010..0.011 rows=1 loops=1)
Buffers: shared hit=4
-> Index Scan using idx02 on test10 test10_1 (cost=0.42..54240.28 rows=1000000 width=14) (actual time=0.010..0.010 rows=1 loops=1)
Buffers: shared hit=4
-> Nested Loop (cost=0.42..5.65 rows=10 width=10) (actual time=0.009..0.009 rows=1 loops=1001)
Buffers: shared hit=4153
-> WorkTable Scan on cte cte_1 (cost=0.00..0.20 rows=10 width=4) (actual time=0.000..0.000 rows=1 loops=1001)
-> Limit (cost=0.42..0.53 rows=1 width=14) (actual time=0.009..0.009 rows=1 loops=1001)
Buffers: shared hit=4153
-> Index Scan using idx02 on test10 test10_2 (cost=0.42..33409.58 rows=333333 width=14) (actual time=0.009..0.009 rows=1 loops=1001)
Index Cond: (c1 > cte_1.c1)
Buffers: shared hit=4153
-> Hash Anti Join (cost=3.28..28062.39 rows=999899 width=36) (actual time=10.727..422.146 rows=998999 loops=1)
Hash Cond: (test10.ctid = cte.ctid)
Buffers: shared hit=9592
-> Seq Scan on test10 (cost=0.00..15435.00 rows=1000000 width=6) (actual time=0.005..141.828 rows=1000000 loops=1)
Buffers: shared hit=5435
-> Hash (cost=2.02..2.02 rows=101 width=36) (actual time=10.713..10.714 rows=1001 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 77kB
Buffers: shared hit=4157
-> CTE Scan on cte (cost=0.00..2.02 rows=101 width=36) (actual time=0.049..10.400 rows=1001 loops=1)
Buffers: shared hit=4157
Planning Time: 0.201 ms
Execution Time: 1691.687 ms
方法三：数组变量与truncate组合，支持事务回滚
do
$$
declare
v_rec test10[];
begin
v_rec := array(
with recursive cte as (
(select id, c1, c2 from test10 order by c1, id desc limit 1)
union all
(select test10.id, test10.c1, test10.c2
from cte,
lateral ( select test10.id, test10.c1, test10.c2
from test10
where cte.c1 < test10.c1
order by test10.c1, test10.id desc
limit 1) test10
))
select (id, c1, c2)
from cte);
truncate test10;
insert into test10
select (t).*
from (select unnest(v_rec) t) t;
commit;
exception
when others then
rollback;
end;
$$
;
ANONYMOUS BLOCK
Time: 99.299 ms
TRUNCATE与DML操作的组合，实现通过少量数据的DML操作，实现DELETE大部分数据操作，可以减少执行时长。由于truncate支持事务回滚，可以在发生异常时回滚事务，或主动回滚事务，保证数据的完整性。