PostgreSQL 11 1Kw TPCC , 1亿 TPCB 7*24 强压耐久测试

本文详细记录了TPCC和TPCB两种数据库性能测试的案例,包括1000万TPCC和1亿TPCB的7*24小时耐久测试。展示了不同阶段的TPS变化、数据库资源消耗及最终性能指标。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

背景
TPCC, TPCB是工业标准的OLTP类型业务的数据库测试,包含大量的读、写、更新、删除操作。

7*24小时强压耐久测试,主要看数据库在长时间最大压力下的 性能、稳定性、可靠性。

测试CASE :

1、1000万 tpcc

2、1亿 tpcb

测试时长7天。

测试环境
与如下测试同一台ECS虚拟机环境。

《PostgreSQL 11 tpcc 测试(103万tpmC on ECS) - use sysbench-tpcc by Percona-Lab》

《(TPC-H测试 SF=10,SF=200) PostgreSQL 11 vs 10 vs Deepgreen》

《PostgreSQL 11 100亿 tpcb 性能测试 on ECS》

https://github.com/digoal/sysbench-tpcc

shared_buffer=300G

use huge page
PS:
实际上最初测试的是TPCC 2亿耐久(2.5TB左右的活跃数据),发现压力基本上都在IO上(因为活跃数据2.5TB,已远远超出机器的内存),平均TPS 3千(18万 tpmC)。

1、1000万 tpcc 耐久测试
1、清空

./tpcc.lua --pgsql-host=/tmp --pgsql-port=1921 --pgsql-user=postgres --pgsql-db=newdb --threads=64 --tables=10 --scale=100 --trx_level=RC --db-ps-mode=auto --db-driver=pgsql cleanup
2、初始化数据

export pgsql_table_options="tablespace tbs1"
export pgsql_index_options="tablespace tbs2"
./tpcc.lua --pgsql-host=/tmp --pgsql-port=1921 --pgsql-user=postgres --pgsql-db=newdb --threads=64 --tables=10 --scale=100 --trx_level=RC --db-ps-mode=auto --db-driver=pgsql --use_fk=0 prepare
3、持续7*24小时,强压耐久测试

./tpcc.lua --pgsql-host=/tmp --pgsql-port=1921 --pgsql-user=postgres --pgsql-db=newdb --threads=64 --tables=10 --scale=100 --trx_level=RC --db-ps-mode=auto --db-driver=pgsql --time=604800 --report-interval=1 --enable_purge=no run
初始化结束时,100多GB。

4、shell

!/bin/bash

./tpcc.lua --pgsql-host=/tmp --pgsql-port=1921 --pgsql-user=postgres --pgsql-db=newdb --threads=64 --tables=10 --scale=100 --trx_level=RC --db-ps-mode=auto --db-driver=pgsql cleanup

export pgsql_table_options="tablespace tbs1"
export pgsql_index_options="tablespace tbs2"
./tpcc.lua --pgsql-host=/tmp --pgsql-port=1921 --pgsql-user=postgres --pgsql-db=newdb --threads=64 --tables=10 --scale=100 --trx_level=RC --db-ps-mode=auto --db-driver=pgsql --use_fk=0 prepare

./tpcc.lua --pgsql-host=/tmp --pgsql-port=1921 --pgsql-user=postgres --pgsql-db=newdb --threads=64 --tables=10 --scale=100 --trx_level=RC --db-ps-mode=auto --db-driver=pgsql --time=604800 --report-interval=1 --enable_purge=no run
结果
1、初始化结束时,100多GB。

截取运行至8000秒(约2.2小时)时的TPS: 2万左右。

[ 8000s ] thds: 64 tps: 20485.43 qps: 581769.53 (r/w/o: 265359.48/275433.19/40976.85) lat (ms,95%): 7.98 err/s 89.01 reconn/s: 0.00
[ 8001s ] thds: 64 tps: 20501.67 qps: 583473.15 (r/w/o: 266219.73/276250.08/41003.34) lat (ms,95%): 7.98 err/s 82.99 reconn/s: 0.00
[ 8002s ] thds: 64 tps: 20494.29 qps: 579007.65 (r/w/o: 264151.49/273867.58/40988.58) lat (ms,95%): 7.84 err/s 84.01 reconn/s: 0.00
[ 8003s ] thds: 64 tps: 20233.62 qps: 570370.25 (r/w/o: 260329.09/269574.92/40466.24) lat (ms,95%): 7.98 err/s 93.00 reconn/s: 0.00
[ 8004s ] thds: 64 tps: 20367.20 qps: 580154.75 (r/w/o: 264423.62/274995.72/40735.40) lat (ms,95%): 7.98 err/s 96.00 reconn/s: 0.00
[ 8005s ] thds: 64 tps: 20411.02 qps: 585439.65 (r/w/o: 267331.30/277285.31/40823.05) lat (ms,95%): 7.98 err/s 100.00 reconn/s: 0.00
[ 8006s ] thds: 64 tps: 20651.82 qps: 583243.91 (r/w/o: 266004.68/275936.59/41302.64) lat (ms,95%): 7.98 err/s 73.00 reconn/s: 0.00
[ 8007s ] thds: 64 tps: 20324.62 qps: 579310.20 (r/w/o: 264444.07/274218.89/40647.24) lat (ms,95%): 7.98 err/s 111.00 reconn/s: 0.00
[ 8008s ] thds: 64 tps: 20419.62 qps: 585216.87 (r/w/o: 267119.16/277256.47/40841.25) lat (ms,95%): 7.84 err/s 93.00 reconn/s: 0.00
[ 8009s ] thds: 64 tps: 20563.73 qps: 583747.26 (r/w/o: 266217.47/276403.34/41126.45) lat (ms,95%): 7.84 err/s 78.00 reconn/s: 0.00
[ 8010s ] thds: 64 tps: 20441.18 qps: 582812.00 (r/w/o: 265923.28/276005.37/40883.35) lat (ms,95%): 7.84 err/s 85.00 reconn/s: 0.00
[ 8011s ] thds: 64 tps: 20501.97 qps: 585269.16 (r/w/o: 267044.62/277220.60/41003.94) lat (ms,95%): 7.98 err/s 104.00 reconn/s: 0.00
[ 8012s ] thds: 64 tps: 20511.63 qps: 579944.71 (r/w/o: 264265.36/274656.10/41023.25) lat (ms,95%): 7.84 err/s 94.95 reconn/s: 0.00
[ 8013s ] thds: 64 tps: 20411.52 qps: 580261.93 (r/w/o: 264764.40/274674.50/40823.03) lat (ms,95%): 7.84 err/s 106.05 reconn/s: 0.00
[ 8014s ] thds: 64 tps: 20070.84 qps: 573689.42 (r/w/o: 261630.91/271918.83/40139.68) lat (ms,95%): 8.13 err/s 114.00 reconn/s: 0.00
[ 8015s ] thds: 64 tps: 20600.03 qps: 584514.83 (r/w/o: 266629.38/276684.39/41201.06) lat (ms,95%): 7.84 err/s 79.00 reconn/s: 0.00
[ 8016s ] thds: 64 tps: 20317.84 qps: 578408.02 (r/w/o: 263900.96/273870.39/40636.68) lat (ms,95%): 7.98 err/s 80.00 reconn/s: 0.00
[ 8017s ] thds: 64 tps: 20468.49 qps: 581658.79 (r/w/o: 265438.29/275285.53/40934.97) lat (ms,95%): 7.98 err/s 88.00 reconn/s: 0.00
[ 8018s ] thds: 64 tps: 20420.97 qps: 581415.10 (r/w/o: 264919.59/275653.57/40841.94) lat (ms,95%): 7.98 err/s 91.00 reconn/s: 0.00
[ 8019s ] thds: 64 tps: 20635.64 qps: 580629.97 (r/w/o: 264969.20/274388.49/41272.28) lat (ms,95%): 7.84 err/s 100.00 reconn/s: 0.00
[ 8020s ] thds: 64 tps: 20333.19 qps: 581171.36 (r/w/o: 264986.46/275517.52/40667.39) lat (ms,95%): 7.98 err/s 77.99 reconn/s: 0.00
[ 8021s ] thds: 64 tps: 20413.23 qps: 584491.09 (r/w/o: 266667.01/276999.63/40824.45) lat (ms,95%): 7.84 err/s 93.01 reconn/s: 0.00
[ 8022s ] thds: 64 tps: 20356.76 qps: 582531.18 (r/w/o: 265727.89/276090.77/40712.52) lat (ms,95%): 7.98 err/s 110.00 reconn/s: 0.00
[ 8023s ] thds: 64 tps: 20382.28 qps: 580142.43 (r/w/o: 264541.61/274833.26/40767.56) lat (ms,95%): 7.98 err/s 88.01 reconn/s: 0.00
[ 8024s ] thds: 64 tps: 20416.72 qps: 578534.32 (r/w/o: 263951.49/273750.40/40832.44) lat (ms,95%): 7.98 err/s 99.99 reconn/s: 0.00
[ 8025s ] thds: 64 tps: 20476.80 qps: 580704.08 (r/w/o: 264754.29/274994.19/40955.60) lat (ms,95%): 7.98 err/s 90.01 reconn/s: 0.00
[ 8026s ] thds: 64 tps: 20440.61 qps: 583615.78 (r/w/o: 266265.88/276469.69/40880.21) lat (ms,95%): 7.84 err/s 83.00 reconn/s: 0.00
[ 8027s ] thds: 64 tps: 20426.51 qps: 580503.41 (r/w/o: 264836.57/274813.82/40853.01) lat (ms,95%): 7.98 err/s 90.00 reconn/s: 0.00
[ 8028s ] thds: 64 tps: 20533.18 qps: 582233.85 (r/w/o: 265680.44/275488.05/41065.37) lat (ms,95%): 7.98 err/s 88.00 reconn/s: 0.00
[ 8029s ] thds: 64 tps: 20316.48 qps: 579150.80 (r/w/o: 264125.29/274392.54/40632.97) lat (ms,95%): 7.98 err/s 94.00 reconn/s: 0.00
[ 8030s ] thds: 64 tps: 20477.27 qps: 581592.57 (r/w/o: 265383.46/275252.58/40956.53) lat (ms,95%): 7.98 err/s 94.00 reconn/s: 0.00
[ 8031s ] thds: 64 tps: 20575.51 qps: 586335.17 (r/w/o: 267565.60/277619.54/41150.03) lat (ms,95%): 7.84 err/s 88.93 reconn/s: 0.00
[ 8032s ] thds: 64 tps: 20330.22 qps: 579527.50 (r/w/o: 264197.85/274671.20/40658.45) lat (ms,95%): 7.98 err/s 87.07 reconn/s: 0.00
[ 8033s ] thds: 64 tps: 20513.19 qps: 580624.41 (r/w/o: 264746.47/274850.56/41027.38) lat (ms,95%): 7.98 err/s 93.00 reconn/s: 0.00
[ 8034s ] thds: 64 tps: 20378.91 qps: 580302.54 (r/w/o: 264830.88/274713.83/40757.83) lat (ms,95%): 7.98 err/s 93.00 reconn/s: 0.00
[ 8035s ] thds: 64 tps: 20448.74 qps: 581127.58 (r/w/o: 265028.61/275200.48/40898.48) lat (ms,95%): 7.98 err/s 99.00 reconn/s: 0.00
[ 8036s ] thds: 64 tps: 20344.99 qps: 582162.67 (r/w/o: 265618.85/275853.84/40689.98) lat (ms,95%): 7.98 err/s 89.00 reconn/s: 0.00
[ 8037s ] thds: 64 tps: 20255.29 qps: 581262.31 (r/w/o: 265043.79/275707.94/40510.58) lat (ms,95%): 7.98 err/s 70.00 reconn/s: 0.00
[ 8038s ] thds: 64 tps: 20697.54 qps: 581712.56 (r/w/o: 265270.89/275048.58/41393.09) lat (ms,95%): 7.84 err/s 95.93 reconn/s: 0.00
[ 8039s ] thds: 64 tps: 20382.69 qps: 579676.79 (r/w/o: 264292.48/274616.92/40767.38) lat (ms,95%): 7.98 err/s 96.07 reconn/s: 0.00
[ 8040s ] thds: 64 tps: 20460.71 qps: 580293.70 (r/w/o: 264518.22/274855.07/40920.41) lat (ms,95%): 7.84 err/s 78.00 reconn/s: 0.00
[ 8041s ] thds: 64 tps: 20321.56 qps: 578692.52 (r/w/o: 264122.31/273927.09/40643.12) lat (ms,95%): 7.98 err/s 80.00 reconn/s: 0.00
[ 8042s ] thds: 64 tps: 20031.57 qps: 570578.33 (r/w/o: 260355.45/270157.73/40065.15) lat (ms,95%): 8.13 err/s 85.00 reconn/s: 0.00
[ 8043s ] thds: 64 tps: 20412.66 qps: 578667.83 (r/w/o: 263881.59/273964.92/40821.33) lat (ms,95%): 7.98 err/s 92.00 reconn/s: 0.00
[ 8044s ] thds: 64 tps: 19949.81 qps: 570346.51 (r/w/o: 260248.50/270198.40/39899.62) lat (ms,95%): 8.13 err/s 78.00 reconn/s: 0.00
[ 8045s ] thds: 64 tps: 20404.18 qps: 574960.10 (r/w/o: 262190.33/271960.41/40809.36) lat (ms,95%): 7.98 err/s 73.00 reconn/s: 0.00
[ 8046s ] thds: 64 tps: 20059.60 qps: 571523.19 (r/w/o: 260615.84/270788.14/40119.21) lat (ms,95%): 8.13 err/s 80.00 reconn/s: 0.00
[ 8047s ] thds: 64 tps: 20106.65 qps: 574613.54 (r/w/o: 262414.44/271983.80/40215.31) lat (ms,95%): 8.13 err/s 101.99 reconn/s: 0.00
[ 8048s ] thds: 64 tps: 20085.47 qps: 570847.83 (r/w/o: 260594.07/270082.82/40170.93) lat (ms,95%): 8.13 err/s 86.00 reconn/s: 0.00
[ 8049s ] thds: 64 tps: 20229.09 qps: 573145.08 (r/w/o: 261418.18/271270.73/40456.17) lat (ms,95%): 7.98 err/s 68.00 reconn/s: 0.00
[ 8050s ] thds: 64 tps: 20205.93 qps: 571454.49 (r/w/o: 260576.85/270466.79/40410.85) lat (ms,95%): 7.98 err/s 74.01 reconn/s: 0.00
[ 8051s ] thds: 64 tps: 20501.31 qps: 581237.93 (r/w/o: 265126.07/275106.23/41005.63) lat (ms,95%): 7.98 err/s 82.00 reconn/s: 0.00
[ 8052s ] thds: 64 tps: 20178.41 qps: 576336.27 (r/w/o: 262996.37/272983.08/40356.83) lat (ms,95%): 8.13 err/s 93.00 reconn/s: 0.00
截取运行至60314秒(约16小时)时的TPS: 1.8万左右。

[ 60314s ] thds: 64 tps: 19341.74 qps: 543204.76 (r/w/o: 247957.48/256563.81/38683.48) lat (ms,95%): 8.43 err/s 93.00 reconn/s: 0.00
[ 60315s ] thds: 64 tps: 19402.02 qps: 553901.95 (r/w/o: 252584.21/262512.71/38805.03) lat (ms,95%): 8.28 err/s 93.00 reconn/s: 0.00
[ 60316s ] thds: 64 tps: 18301.57 qps: 519270.68 (r/w/o: 236803.38/245865.17/36602.13) lat (ms,95%): 8.43 err/s 61.00 reconn/s: 0.00
[ 60317s ] thds: 64 tps: 16248.96 qps: 460563.21 (r/w/o: 210044.41/218019.88/32498.92) lat (ms,95%): 8.74 err/s 81.00 reconn/s: 0.00
[ 60318s ] thds: 64 tps: 19251.90 qps: 550591.20 (r/w/o: 251161.72/260926.67/38502.80) lat (ms,95%): 8.43 err/s 96.00 reconn/s: 0.00
[ 60319s ] thds: 64 tps: 19509.95 qps: 548363.50 (r/w/o: 249973.55/259370.05/39019.90) lat (ms,95%): 8.28 err/s 73.00 reconn/s: 0.00
[ 60320s ] thds: 64 tps: 19543.97 qps: 549761.22 (r/w/o: 250722.65/259949.63/39088.94) lat (ms,95%): 8.13 err/s 76.00 reconn/s: 0.00
[ 60321s ] thds: 64 tps: 19363.48 qps: 549986.10 (r/w/o: 250868.20/260390.95/38726.95) lat (ms,95%): 8.28 err/s 82.00 reconn/s: 0.00
[ 60322s ] thds: 64 tps: 19371.95 qps: 551287.99 (r/w/o: 251553.31/260992.78/38741.90) lat (ms,95%): 8.28 err/s 80.00 reconn/s: 0.00
[ 60323s ] thds: 64 tps: 15691.03 qps: 452727.03 (r/w/o: 206630.23/214713.73/31383.06) lat (ms,95%): 8.90 err/s 64.00 reconn/s: 0.00
[ 60324s ] thds: 64 tps: 16274.47 qps: 458086.44 (r/w/o: 208798.89/216740.61/32546.94) lat (ms,95%): 8.90 err/s 61.01 reconn/s: 0.00
[ 60325s ] thds: 64 tps: 19378.81 qps: 550473.61 (r/w/o: 251081.54/260633.45/38758.62) lat (ms,95%): 8.28 err/s 77.00 reconn/s: 0.00
[ 60326s ] thds: 64 tps: 19208.78 qps: 546456.65 (r/w/o: 249474.10/258563.99/38418.55) lat (ms,95%): 8.43 err/s 68.00 reconn/s: 0.00
[ 60327s ] thds: 64 tps: 16284.20 qps: 461247.29 (r/w/o: 210580.59/218099.29/32567.40) lat (ms,95%): 8.74 err/s 73.87 reconn/s: 0.00
[ 60328s ] thds: 64 tps: 18873.26 qps: 533919.22 (r/w/o: 243502.03/252668.67/37748.52) lat (ms,95%): 8.28 err/s 84.15 reconn/s: 0.00
[ 60329s ] thds: 64 tps: 19181.30 qps: 548828.91 (r/w/o: 250310.84/260155.48/38362.60) lat (ms,95%): 8.28 err/s 73.00 reconn/s: 0.00
[ 60330s ] thds: 64 tps: 19163.09 qps: 544791.56 (r/w/o: 248654.17/257811.21/38326.18) lat (ms,95%): 8.43 err/s 84.00 reconn/s: 0.00
[ 60331s ] thds: 64 tps: 15370.60 qps: 436963.74 (r/w/o: 199441.86/206782.67/30739.21) lat (ms,95%): 8.90 err/s 58.00 reconn/s: 0.00
[ 60332s ] thds: 64 tps: 18666.11 qps: 535700.06 (r/w/o: 244396.39/253969.45/37334.21) lat (ms,95%): 8.58 err/s 81.00 reconn/s: 0.00
[ 60333s ] thds: 64 tps: 19166.30 qps: 542122.53 (r/w/o: 247496.90/256293.03/38332.60) lat (ms,95%): 8.28 err/s 75.00 reconn/s: 0.00
[ 60334s ] thds: 64 tps: 17808.04 qps: 511114.51 (r/w/o: 233230.46/242268.97/35615.08) lat (ms,95%): 8.74 err/s 89.00 reconn/s: 0.00
[ 60335s ] thds: 64 tps: 15754.88 qps: 447359.62 (r/w/o: 204130.46/211719.40/31509.76) lat (ms,95%): 8.90 err/s 64.00 reconn/s: 0.00
[ 60336s ] thds: 64 tps: 18478.48 qps: 529327.12 (r/w/o: 241701.20/250668.95/36956.96) lat (ms,95%): 8.74 err/s 73.00 reconn/s: 0.00
[ 60337s ] thds: 64 tps: 19124.11 qps: 544896.11 (r/w/o: 248318.39/258330.50/38247.22) lat (ms,95%): 8.43 err/s 73.01 reconn/s: 0.00
[ 60338s ] thds: 64 tps: 16187.05 qps: 462672.85 (r/w/o: 211196.76/219100.00/32376.09) lat (ms,95%): 9.06 err/s 70.92 reconn/s: 0.00
[ 60339s ] thds: 64 tps: 18098.52 qps: 517695.48 (r/w/o: 236153.76/245345.67/36196.05) lat (ms,95%): 8.74 err/s 81.09 reconn/s: 0.00
[ 60340s ] thds: 64 tps: 19052.64 qps: 545579.33 (r/w/o: 248832.36/258642.69/38104.28) lat (ms,95%): 8.58 err/s 91.00 reconn/s: 0.00
[ 60341s ] thds: 64 tps: 16571.76 qps: 474637.18 (r/w/o: 216660.89/224831.77/33144.52) lat (ms,95%): 8.74 err/s 68.00 reconn/s: 0.00
[ 60342s ] thds: 64 tps: 15923.79 qps: 450100.13 (r/w/o: 205297.32/212955.22/31847.58) lat (ms,95%): 8.74 err/s 69.00 reconn/s: 0.00
[ 60343s ] thds: 64 tps: 18764.65 qps: 537077.94 (r/w/o: 245110.41/254437.23/37530.30) lat (ms,95%): 8.43 err/s 81.00 reconn/s: 0.00
截取运行至105235秒(约29小时)时的TPS: 1.3万左右

[ 105235s ] thds: 64 tps: 10971.14 qps: 310702.84 (r/w/o: 141931.50/146827.05/21944.30) lat (ms,95%): 11.04 err/s 54.42 reconn/s: 0.00
[ 105236s ] thds: 64 tps: 16716.96 qps: 473482.17 (r/w/o: 216038.40/224008.86/33434.92) lat (ms,95%): 9.39 err/s 79.00 reconn/s: 0.00
[ 105237s ] thds: 64 tps: 12184.17 qps: 347403.15 (r/w/o: 158458.12/164578.69/24366.33) lat (ms,95%): 10.09 err/s 51.98 reconn/s: 0.00
[ 105238s ] thds: 64 tps: 13984.71 qps: 393431.48 (r/w/o: 179531.45/185928.61/27971.42) lat (ms,95%): 9.73 err/s 54.02 reconn/s: 0.00
[ 105239s ] thds: 64 tps: 15386.03 qps: 433355.56 (r/w/o: 197604.49/204981.02/30770.05) lat (ms,95%): 9.56 err/s 65.00 reconn/s: 0.00
[ 105240s ] thds: 64 tps: 8817.13 qps: 254406.00 (r/w/o: 116071.20/120699.54/17635.25) lat (ms,95%): 12.08 err/s 40.96 reconn/s: 0.00
[ 105241s ] thds: 64 tps: 7327.41 qps: 206762.43 (r/w/o: 94191.16/97917.44/14653.83) lat (ms,95%): 11.87 err/s 37.04 reconn/s: 0.00
[ 105242s ] thds: 64 tps: 16702.17 qps: 474897.45 (r/w/o: 216633.26/224857.86/33406.33) lat (ms,95%): 9.56 err/s 76.99 reconn/s: 0.00
[ 105243s ] thds: 64 tps: 13411.71 qps: 380988.62 (r/w/o: 173814.18/180351.02/26823.42) lat (ms,95%): 10.09 err/s 53.01 reconn/s: 0.00
[ 105244s ] thds: 64 tps: 9963.37 qps: 284381.14 (r/w/o: 129775.85/134680.54/19924.75) lat (ms,95%): 11.04 err/s 35.00 reconn/s: 0.00
[ 105245s ] thds: 64 tps: 14759.45 qps: 419263.48 (r/w/o: 191417.92/198324.66/29520.91) lat (ms,95%): 9.73 err/s 69.00 reconn/s: 0.00
[ 105246s ] thds: 64 tps: 12785.37 qps: 365241.33 (r/w/o: 166674.43/172997.16/25569.73) lat (ms,95%): 10.27 err/s 52.96 reconn/s: 0.00
[ 105247s ] thds: 64 tps: 10651.06 qps: 300672.14 (r/w/o: 137243.92/142125.11/21303.11) lat (ms,95%): 10.46 err/s 35.91 reconn/s: 0.00
[ 105248s ] thds: 64 tps: 14604.60 qps: 412991.62 (r/w/o: 188383.80/195400.63/29207.19) lat (ms,95%): 9.39 err/s 54.18 reconn/s: 0.00
[ 105249s ] thds: 64 tps: 16771.01 qps: 472864.24 (r/w/o: 215600.82/223720.40/33543.02) lat (ms,95%): 9.39 err/s 79.98 reconn/s: 0.00
[ 105250s ] thds: 64 tps: 11177.31 qps: 318115.91 (r/w/o: 145144.34/150616.96/22354.62) lat (ms,95%): 10.84 err/s 50.58 reconn/s: 0.00
[ 105251s ] thds: 64 tps: 8944.08 qps: 252604.23 (r/w/o: 115208.77/119506.29/17889.17) lat (ms,95%): 11.45 err/s 40.35 reconn/s: 0.00
[ 105252s ] thds: 64 tps: 12086.05 qps: 338581.29 (r/w/o: 154610.80/159798.39/24172.09) lat (ms,95%): 9.91 err/s 53.00 reconn/s: 0.00
[ 105253s ] thds: 64 tps: 16202.94 qps: 466033.79 (r/w/o: 212768.97/220859.94/32404.88) lat (ms,95%): 9.56 err/s 76.99 reconn/s: 0.00
[ 105254s ] thds: 64 tps: 14574.21 qps: 413960.09 (r/w/o: 188988.78/195822.88/29148.43) lat (ms,95%): 9.73 err/s 63.00 reconn/s: 0.00
[ 105255s ] thds: 64 tps: 13877.36 qps: 390236.01 (r/w/o: 178052.57/184427.73/27755.71) lat (ms,95%): 10.09 err/s 50.00 reconn/s: 0.00
[ 105256s ] thds: 64 tps: 15102.61 qps: 428525.80 (r/w/o: 195790.88/202531.71/30203.21) lat (ms,95%): 9.56 err/s 64.00 reconn/s: 0.00
[ 105257s ] thds: 64 tps: 16296.44 qps: 461997.43 (r/w/o: 211100.68/218301.87/32594.88) lat (ms,95%): 9.39 err/s 73.00 reconn/s: 0.00
[ 105258s ] thds: 64 tps: 10220.71 qps: 287978.86 (r/w/o: 131516.28/136022.16/20440.42) lat (ms,95%): 10.46 err/s 53.00 reconn/s: 0.00
[ 105259s ] thds: 64 tps: 3986.03 qps: 114032.80 (r/w/o: 52087.37/53973.38/7972.06) lat (ms,95%): 13.46 err/s 14.00 reconn/s: 0.00
[ 105260s ] thds: 64 tps: 15307.79 qps: 434108.13 (r/w/o: 197882.33/205609.22/30616.59) lat (ms,95%): 9.73 err/s 77.00 reconn/s: 0.00
[ 105261s ] thds: 64 tps: 10360.59 qps: 297243.50 (r/w/o: 135577.75/140944.58/20721.17) lat (ms,95%): 10.84 err/s 41.01 reconn/s: 0.00
[ 105262s ] thds: 64 tps: 9348.18 qps: 263222.94 (r/w/o: 120104.48/124424.10/18694.36) lat (ms,95%): 11.45 err/s 48.00 reconn/s: 0.00
[ 105263s ] thds: 64 tps: 10481.89 qps: 296577.98 (r/w/o: 135384.62/140227.57/20965.79) lat (ms,95%): 11.04 err/s 43.00 reconn/s: 0.00
[ 105264s ] thds: 64 tps: 14258.34 qps: 405619.15 (r/w/o: 185077.40/192025.08/28516.67) lat (ms,95%): 9.91 err/s 64.00 reconn/s: 0.00
[ 105265s ] thds: 64 tps: 15180.17 qps: 431972.85 (r/w/o: 197029.21/204583.29/30360.35) lat (ms,95%): 9.91 err/s 58.01 reconn/s: 0.00
[ 105266s ] thds: 64 tps: 16335.46 qps: 462913.83 (r/w/o: 211153.08/219089.82/32670.93) lat (ms,95%): 9.39 err/s 79.00 reconn/s: 0.00
[ 105267s ] thds: 64 tps: 14183.78 qps: 404733.81 (r/w/o: 184728.18/191639.07/28366.57) lat (ms,95%): 9.56 err/s 65.00 reconn/s: 0.00
[ 105268s ] thds: 64 tps: 8382.61 qps: 235797.93 (r/w/o: 107527.18/111504.52/16766.22) lat (ms,95%): 12.30 err/s 26.00 reconn/s: 0.00
[ 105269s ] thds: 64 tps: 15655.17 qps: 443939.57 (r/w/o: 202572.09/210057.13/31310.34) lat (ms,95%): 9.56 err/s 71.01 reconn/s: 0.00
[ 105270s ] thds: 64 tps: 11207.84 qps: 320791.50 (r/w/o: 146306.95/152070.87/22413.69) lat (ms,95%): 11.04 err/s 44.00 reconn/s: 0.00
[ 105271s ] thds: 64 tps: 13845.13 qps: 399042.68 (r/w/o: 181915.68/189434.75/27692.26) lat (ms,95%): 9.91 err/s 65.00 reconn/s: 0.00
[ 105272s ] thds: 64 tps: 8264.45 qps: 236754.99 (r/w/o: 108044.68/112181.41/16528.90) lat (ms,95%): 11.45 err/s 40.99 reconn/s: 0.00
[ 105273s ] thds: 64 tps: 10702.08 qps: 302372.04 (r/w/o: 138055.74/142912.14/21404.16) lat (ms,95%): 10.27 err/s 53.02 reconn/s: 0.00
[ 105274s ] thds: 64 tps: 16254.38 qps: 463970.25 (r/w/o: 211848.78/219611.72/32509.75) lat (ms,95%): 9.56 err/s 76.84 reconn/s: 0.00
[ 105275s ] thds: 64 tps: 13855.29 qps: 399107.99 (r/w/o: 182274.21/189125.20/27708.58) lat (ms,95%): 10.09 err/s 72.15 reconn/s: 0.00
[ 105276s ] thds: 64 tps: 11715.07 qps: 337701.73 (r/w/o: 154173.03/160100.57/23428.13) lat (ms,95%): 10.46 err/s 48.00 reconn/s: 0.00
[ 105277s ] thds: 64 tps: 13778.07 qps: 392465.82 (r/w/o: 179067.23/185841.44/27557.14) lat (ms,95%): 9.91 err/s 56.67 reconn/s: 0.00
[ 105278s ] thds: 64 tps: 11129.26 qps: 311401.06 (r/w/o: 141946.61/147193.91/22260.53) lat (ms,95%): 10.65 err/s 45.26 reconn/s: 0.00
[ 105279s ] thds: 64 tps: 11934.79 qps: 341804.12 (r/w/o: 155904.32/162031.23/23868.57) lat (ms,95%): 10.27 err/s 44.01 reconn/s: 0.00
[ 105280s ] thds: 64 tps: 14279.68 qps: 400515.10 (r/w/o: 182797.16/189158.57/28559.37) lat (ms,95%): 9.91 err/s 57.99 reconn/s: 0.00
[ 105281s ] thds: 64 tps: 10798.39 qps: 306898.05 (r/w/o: 139849.04/145452.24/21596.78) lat (ms,95%): 10.65 err/s 45.00 reconn/s: 0.00
[ 105282s ] thds: 64 tps: 15073.77 qps: 431435.46 (r/w/o: 196872.01/204415.90/30147.54) lat (ms,95%): 9.73 err/s 60.00 reconn/s: 0.00
[ 105283s ] thds: 64 tps: 10182.07 qps: 288077.44 (r/w/o: 131508.09/136204.20/20365.14) lat (ms,95%): 10.84 err/s 32.99 reconn/s: 0.00
[ 105284s ] thds: 64 tps: 12926.33 qps: 370825.74 (r/w/o: 169125.44/175848.65/25851.65) lat (ms,95%): 10.09 err/s 58.01 reconn/s: 0.00
[ 105285s ] thds: 64 tps: 15116.37 qps: 427026.63 (r/w/o: 194883.63/201909.27/30233.74) lat (ms,95%): 9.73 err/s 54.00 reconn/s: 0.00
[ 105286s ] thds: 64 tps: 16076.53 qps: 455528.33 (r/w/o: 207808.99/215567.28/32152.06) lat (ms,95%): 9.39 err/s 80.99 reconn/s: 0.00
[ 105287s ] thds: 64 tps: 16653.26 qps: 476688.40 (r/w/o: 217349.38/226031.51/33307.52) lat (ms,95%): 9.22 err/s 69.00 reconn/s: 0.00
[ 105288s ] thds: 64 tps: 13093.48 qps: 371458.36 (r/w/o: 169550.32/175721.08/26186.97) lat (ms,95%): 10.09 err/s 54.00 reconn/s: 0.00
[ 105289s ] thds: 64 tps: 9397.12 qps: 266228.12 (r/w/o: 121445.65/125989.23/18793.24) lat (ms,95%): 11.24 err/s 49.00 reconn/s: 0.00
运行至105235秒(约29小时)时,数据库写入量达到1.9TB。

postgres=# l+

                                                           List of databases    
NameOwnerEncodingCollateCtypeAccess privilegesSizeTablespaceDescription
newdbpostgresSQL_ASCIICC 1945 GBpg_default

TPS下降主要是活跃数据逐渐超过内存大小,IO waiting造成。

没有IO瓶颈时,TPS在2万左右。

统计信息

newdb=# select relname, n_tup_ins, n_tup_upd, n_tup_hot_upd,n_tup_del from pg_stat_all_tables order by (n_tup_ins+ n_tup_upd+ n_tup_del) desc;

     relname         | n_tup_ins | n_tup_upd | n_tup_hot_upd | n_tup_del     
order_line87428547737134691034701174280
order_line17428330977131186344688134230
order_line27428144537131256554692440700
order_line67427721147131017274685600150
order_line77428061277130437884689139500
order_line47427232297131086434687100150
order_line37427370237130590914690359560
order_line107426446997131284834684253780
order_line97426876997130485494710410260
order_line57426731627130191924696364250
stock8100000007127629967117732490
stock1100000007127240197117339720
stock7100000007127104547117207270
stock2100000007126997127117097200
stock6100000007126644717116750600
stock3100000007126282677116392390
stock4100000007126149597116253780
stock9100000007125785217115897350
stock5100000007125715397115818830
stock10100000007125375777115484760
order_line18120222432905928626462460578896814
order_line17120261447905608786478523078887855
order_line19120202018904665306483479478751265
order_line16120159770904276026473986978745237
order_line20120198335903956676506848878700223
order_line13120134220904125996482878278727933
order_line15120211731903789336510199278664788
order_line14120159274903236566490982878623449
order_line12120146896903455386473800278593905
order_line11120113947901881196465821678476210
customer830000001428677201422354850
customer230000001428637831422214190
customer1030000001428610011422318490
customer130000001428561181422264690
customer330000001428561001422259140
customer630000001428560661421874000
customer730000001428546151422195290
customer430000001428465131422132970
customer930000001428376541422047900
customer530000001428256411421947650
orders87443584771350501664633030
orders17443727371320879664248470
orders27443036471318489664030370
orders47442526671319415664133990
orders77442813771308212663843410
orders67442399971312104663917910
orders57442094171311488664178360
orders97441982971312456664292730
orders107441437571316975664151940
orders37441796471305524663871340
new_orders8723240350071341012
new_orders1723254530071310992
new_orders2723188240071308036
new_orders4723126230071309388
new_orders6723121880071301214
new_orders7723163460071297056
new_orders9723087090071302880
new_orders5723094790071301185
new_orders10723027060071307616
new_orders3723071850071296566
district210001429689661427145820
district710001429666561426983270
district110001429648041426462400
district310001429620341427019720
district610001429610211427207060
district1010001429508411427038530
district410001429469151426802830
district810001429468051427173570
district910001429386631427093350
district510001429284591426935410
stock171000000089481481889383660
stock151000000089476911889335460
stock121000000089462637889185170
stock111000000089439475888962850
stock141000000089438138888962210
stock181000000089437799888942430
stock131000000089432484888889800
stock161000000089398767888550390
stock201000000089386421888429730
stock191000000089385085888420930
history374424652000
history774420955000
history274420732000
history1074418428000
history674417388000
history174408236000
history474402262000
history974397395000
history874393045000
history574388763000
warehouse310071492200714503790
warehouse210071488430714440470
warehouse710071488084714421030
warehouse610071486603714411650
warehouse1010071486293714404420
warehouse110071476931714359410
warehouse410071470721714261930
warehouse910071466903714267980
warehouse810071460796714176220
warehouse510071456483714119160
orders1812108589912551088071387939496
orders1712110637912026988040067938378
orders1912105891911311087972607923660
orders2012108671910701087935727920437
orders1612099771910679087878317922956
orders1312094924910289087941517919890
orders1512103020909845087793767914356
orders1412099048909385087800127909755
orders1212093362909240087753857906052
orders1112087640907516087608717891505
history20120367420011726195
history19120348020011727724
history18120326580011724219
history16120322430011716928
history17120297370011718122
history13120263580011706908
history12120232410011705417
history14120224790011706108
history15120223640011705010
history11120144460011697237
customer18300000018036170176549430
customer17300000018033330176520390
customer13300000018031178176493650
customer16300000018025381176438310
customer12300000018023919176426130
customer19300000018021666176405860
customer20300000018015621176333190
customer14300000018014509176333090
customer15300000018010869176296050
customer11300000018002119176215020
new_orders1810007127009125290
new_orders1710009015009119249
new_orders1910004860009112640
new_orders2010007841009106570
new_orders169997658009105470
new_orders1510000794009097660
new_orders139992490009101210
new_orders149996854009092360
new_orders129990837009090990
new_orders119984771009073380
district12100018017077180160850
district17100018016760180159110
district13100018014365180133750
district18100018012427180112880
district16100018011460180104430
district14100018010505180095580
district20100018009697180087170
district15100018009694180087410
district19100018008130180072620
district11100018008011180069270
warehouse16100900834890081240
warehouse13100900767090075090
warehouse12100900701890068170
warehouse19100900622890060660
warehouse20100900607190059420
warehouse17100900511190049140
warehouse18100900464490044590
warehouse14100900235490022200
warehouse11100900059290004730
warehouse15100899984889997140
item20100000000
item13100000000
item3100000000
item9100000000
item19100000000
item7100000000
item12100000000
item17100000000
item5100000000
item2100000000
item16100000000
item8100000000
item18100000000
item4100000000
item10100000000
item15100000000
item14100000000
item1100000000
item11100000000
item6100000000

表的大小详情

newdb=# dt+

                    List of relations    
SchemaNameTypeOwnerSizeDescription
publiccustomer1tablepostgres2123 MB
publiccustomer2tablepostgres2120 MB
publiccustomer3tablepostgres2124 MB
publiccustomer4tablepostgres2124 MB
publiccustomer5tablepostgres2122 MB
publiccustomer6tablepostgres2121 MB
publiccustomer7tablepostgres2121 MB
publiccustomer8tablepostgres2123 MB
publiccustomer9tablepostgres2124 MB
publicdistrict1tablepostgres6616 kB
publicdistrict10tablepostgres6528 kB
publicdistrict2tablepostgres6944 kB
publicdistrict3tablepostgres8240 kB
publicdistrict4tablepostgres6728 kB
publicdistrict5tablepostgres6664 kB
publicdistrict6tablepostgres6272 kB
publicdistrict7tablepostgres5160 kB
publicdistrict8tablepostgres6376 kB
publicdistrict9tablepostgres6360 kB
publichistory1tablepostgres6698 MB
publichistory10tablepostgres6699 MB
publichistory2tablepostgres6700 MB
publichistory3tablepostgres6699 MB
publichistory4tablepostgres6698 MB
publichistory5tablepostgres6697 MB
publichistory6tablepostgres6699 MB
publichistory7tablepostgres6699 MB
publichistory8tablepostgres6697 MB
publichistory9tablepostgres6697 MB
publicitem1tablepostgres11 MB
publicitem10tablepostgres11 MB
publicitem2tablepostgres11 MB
publicitem3tablepostgres11 MB
publicitem4tablepostgres11 MB
publicitem5tablepostgres11 MB
publicitem6tablepostgres11 MB
publicitem7tablepostgres11 MB
publicitem8tablepostgres11 MB
publicitem9tablepostgres11 MB
publicnew_orders1tablepostgres13 MB
publicnew_orders10tablepostgres12 MB
publicnew_orders2tablepostgres13 MB
publicnew_orders3tablepostgres13 MB
publicnew_orders4tablepostgres12 MB
publicnew_orders5tablepostgres13 MB
publicnew_orders6tablepostgres13 MB
publicnew_orders7tablepostgres13 MB
publicnew_orders8tablepostgres12 MB
publicnew_orders9tablepostgres13 MB
publicorder_line1tablepostgres82 GB
publicorder_line10tablepostgres82 GB
publicorder_line2tablepostgres82 GB
publicorder_line3tablepostgres82 GB
publicorder_line4tablepostgres82 GB
publicorder_line5tablepostgres82 GB
publicorder_line6tablepostgres82 GB
publicorder_line7tablepostgres82 GB
publicorder_line8tablepostgres83 GB
publicorder_line9tablepostgres83 GB
publicorders1tablepostgres4632 MB
publicorders10tablepostgres4630 MB
publicorders2tablepostgres4631 MB
publicorders3tablepostgres4631 MB
publicorders4tablepostgres4631 MB
publicorders5tablepostgres4631 MB
publicorders6tablepostgres4631 MB
publicorders7tablepostgres4631 MB
publicorders8tablepostgres4632 MB
publicorders9tablepostgres4631 MB
publicstock1tablepostgres3806 MB
publicstock10tablepostgres3806 MB
publicstock2tablepostgres3806 MB
publicstock3tablepostgres3819 MB
publicstock4tablepostgres3809 MB
publicstock5tablepostgres3809 MB
publicstock6tablepostgres3808 MB
publicstock7tablepostgres3808 MB
publicstock8tablepostgres3809 MB
publicstock9tablepostgres3815 MB
publicwarehouse1tablepostgres1144 kB
publicwarehouse10tablepostgres1328 kB
publicwarehouse2tablepostgres1144 kB
publicwarehouse3tablepostgres1160 kB
publicwarehouse4tablepostgres1104 kB
publicwarehouse5tablepostgres936 kB
publicwarehouse6tablepostgres1048 kB
publicwarehouse7tablepostgres1080 kB
publicwarehouse8tablepostgres1056 kB
publicwarehouse9tablepostgres1120 kB

2、1亿 tpcb 耐久测试
7*24小时测试

pgbench -i -s 1000 --tablespace=tbs1 --index-tablespace=tbs2

nohup pgbench -M prepared -v -r -P 1 -c 32 -j 32 -T 604800 >./pgbench.log1 2>&1 &
压测结束,表和库的大小如下

postgres=# l+

                                                           List of databases  
NameOwnerEncodingCollateCtypeAccess privilegesSizeTablespaceDescription
newdbpostgresSQL_ASCIICC 23 MBpg_default
postgrespostgresSQL_ASCIICC 3217 GBpg_defaultdefault administrative connection database
template0postgresSQL_ASCIICC=c/postgres +7947 kBpg_defaultunmodifiable empty database
postgres=CTc/postgres
template1postgresSQL_ASCIICC=c/postgres +7947 kBpg_defaultdefault template for new databases

(4 rows)

postgres=# dt+ pgbench_*

                      List of relations  
SchemaNameTypeOwnerSizeDescription
publicpgbench_accountstablepostgres13 GB
publicpgbench_branchestablepostgres17 MB
publicpgbench_historytablepostgres2546 GB
publicpgbench_tellerstablepostgres174 MB

(4 rows)
过程记录

head -n 10 pgbench.log1
nohup: ignoring input
starting vacuum...end.
starting vacuum pgbench_accounts...end.
progress: 1.0 s, 65992.7 tps, lat 0.345 ms stddev 0.186
progress: 2.0 s, 87864.7 tps, lat 0.364 ms stddev 0.166
progress: 3.0 s, 87709.0 tps, lat 0.365 ms stddev 0.155
progress: 4.0 s, 88942.7 tps, lat 0.360 ms stddev 0.137
progress: 5.0 s, 88978.5 tps, lat 0.360 ms stddev 0.125
progress: 6.0 s, 89881.9 tps, lat 0.356 ms stddev 0.134
progress: 7.0 s, 89717.4 tps, lat 0.357 ms stddev 0.144
........
progress: 604792.0 s, 87531.8 tps, lat 0.366 ms stddev 0.112
progress: 604793.0 s, 88013.5 tps, lat 0.364 ms stddev 0.085
progress: 604794.0 s, 88438.9 tps, lat 0.362 ms stddev 0.119
progress: 604795.0 s, 87804.3 tps, lat 0.364 ms stddev 0.416
progress: 604796.0 s, 86275.8 tps, lat 0.371 ms stddev 0.103
progress: 604797.0 s, 86883.9 tps, lat 0.368 ms stddev 0.356
progress: 604798.0 s, 87882.8 tps, lat 0.364 ms stddev 0.071
progress: 604799.0 s, 87601.3 tps, lat 0.365 ms stddev 0.083
progress: 604800.0 s, 86404.5 tps, lat 0.369 ms stddev 0.091
transaction type:
scaling factor: 1000
query mode: prepared
number of clients: 32
number of threads: 32
duration: 604800 s
number of transactions actually processed: 52381795265
latency average = 0.369 ms
latency stddev = 0.800 ms
tps = 86610.109407 (including connections establishing)
tps = 86610.150411 (excluding connections establishing)
statement latencies in milliseconds:

     0.001  \set aid random(1, 100000 * :scale)  
     0.001  \set bid random(1, 1 * :scale)  
     0.001  \set tid random(1, 10 * :scale)  
     0.000  \set delta random(-5000, 5000)  
     0.026  BEGIN;  
     0.086  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;  
     0.046  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;  
     0.058  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;  
     0.057  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;  
     0.046  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);  
     0.049  END;  

结果 8.66万 tps
7*24小时压测,数据库完成了523.8亿个事务,50几次freeze,性能平稳8.66万 tps 。

小结
1、1000万 tpcc测试,在16小时压测后,约1.8W tps,108 万tpmC。
由于TPCC不断有数据写入,29小时后,活跃数据量接近2TB,IO waiting成为瓶颈,TPS下降到1.3万左右。

2、1亿 tpcb测试,7天强压,性能平稳, 完成了523.8亿个事务,平均 tps 8.66万
参考
《PostgreSQL 11 tpcc 测试(103万tpmC on ECS) - use sysbench-tpcc by Percona-Lab》

《(TPC-H测试 SF=10,SF=200) PostgreSQL 11 vs 10 vs Deepgreen》

《PostgreSQL 11 100亿 tpcb 性能测试 on ECS》

https://github.com/digoal/sysbench-tpcc
转发自https://github.com/digoal/blog/blob/master/201809/20180920_01.md

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值