timescaledb和PG写入性能测试

在小数据量下,PG和TSDB性能相近,PG略优;增加索引后,两者性能均下降。大数据量下,PG插入性能优于TSDB,资源消耗也更低。测试环境为Intel i7 CPU,32GB内存,SSD硬盘。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

目录

 

结论摘要

测试环境

数据构造

CASE 1 单TIME索引

单行写入

WAL文件增加

BATCH写入

资源占用

CASE 2 增加一个索引

单行写入

BATCH写入

资源占用

CASE 3 大量数据


结论摘要

小数据量(线程10万行),无论单多线程,有无batch,PG和TSDB性能相仿,PG略强。

增加一条索引,两者插入性能都有所下降

WAL增长约为插入数据的2倍,但是WAL有上限。

大数据量(2000万、1亿行),BATCHSIZE=1000,PG插入性能依旧强于TSDB。

CPU占用,TSDB占用略高,以8线程batch1000为例(TSDB:80%, PG: %60)

总体上单独PG 性能和资源消耗都优于TSDB。

下一步需要找到在何种场景下TSDB插入性能会优于PG。

测试环境

Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz

32G memory

SSD harddisk.

postgresql.conf


max_connections = 256
 
shared_buffers = 3GB                    # min 128kB
 
work_mem = 16MB                         # min 64kB
maintenance_work_mem = 256MB            # min 1MB
 
wal_buffers = 4MB 
checkpoint_timeout = 10min              # range 30s-1d
max_wal_size = 2GB
 
checkpoint_completion_target = 0.9  
--------------------- 
作者:Kun_Tsai 
来源:优快云 
原文:https://blog.youkuaiyun.com/jacicson1987/article/details/82988556?utm_source=copy 
版权声明:本文为博主原创文章,转载请附上博文链接!

数据构造

timestamp 每行递增1s

id  每行递增1

10个 integer 随机数 + 9个随机长度字符串

 

CASE 1 单TIME索引

timescaledb 不设置索引(默认time索引)

PG 设置time索引

ts1=# \d+ ckts1
                                            Table "public.ckts1"
 Column |           Type           | Collation | Nullable | Default | Storage  | Stats target | Description 
--------+--------------------------+-----------+----------+---------+----------+--------------+-------------
 time   | timestamp with time zone |           | not null |         | plain    |              | 
 id     | integer                  |           |          |         | plain    |              | 
 col2   | integer                  |           |          |         | plain    |              | 
 col3   | integer                  |           |          |         | plain    |              | 
 col4   | integer                  |           |          |         | plain    |              | 
 col5   | integer                  |           |          |         | plain    |              | 
 col6   | integer                  |           |          |         | plain    |              | 
 col7   | integer                  |           |          |         | plain    |              | 
 col8   | integer                  |           |          |         | plain    |              | 
 col9   | integer                  |           |          |         | plain    |              | 
 col10  | integer                  |           |          |         | plain    |              | 
 col11  | integer                  |           |          |         | plain    |              | 
 col12  | character varying(30)    |           |          |         | extended |              | 
 col13  | character varying(30)    |           |          |         | extended |              | 
 col14  | character varying(30)    |           |          |         | extended |              | 
 col15  | character varying(30)    |           |          |         | extended |              | 
 col16  | character varying(30)    |           |          |         | extended |              | 
 col17  | character varying(30)    |           |          |         | extended |              | 
 col18  | character varying(30)    |           |          |         | extended |              | 
 col19  | character varying(30)    |           |          |         | extended |              | 
 col20  | character varying(30)    |           |          |         | extended |              | 
Indexes:
    "ckts1_time_idx" btree ("time" DESC)
Triggers:
    ts_insert_blocker BEFORE INSERT ON ckts1 FOR EACH ROW EXECUTE PROCEDURE _timescaledb_internal.insert_blocker()

postgres=# \d+ cts1
                                            Table "public.cts1"
 Column |           Type           | Collation | Nullable | Default | Storage  | Stats target | Description 
--------+--------------------------+-----------+----------+---------+----------+--------------+-------------
 time   | timestamp with time zone |           | not null |         | plain    |              | 
 id     | integer                  |           |          |         | plain    |              | 
 col2   | integer                  |           |          |         | plain    |              | 
 col3   | integer                  |           |          |         | plain    |              | 
 col4   | integer                  |           |          |         | plain    |              | 
 col5   | integer                  |           |          |         | plain    |              | 
 col6   | integer                  |           |          |         | plain    |              | 
 col7   | integer                  |           |          |         | plain    |              | 
 col8   | integer                  |           |          |         | plain    |              | 
 col9   | integer                  |           |          |         | plain    |              | 
 col10  | integer                  |           |          |         | plain    |              | 
 col11  | integer                  |           |          |         | plain    |              | 
 col12  | character varying(30)    |           |          |         | extended |              | 
 col13  | character varying(30)    |           |          |         | extended |              | 
 col14  | character varying(30)    |           |          |         | extended |              | 
 col15  | character varying(30)    |           |          |         | extended |              | 
 col16  | character varying(30)    |           |          |         | extended |              | 
 col17  | character varying(30)    |           |          |         | extended |              | 
 col18  | character varying(30)    |           |          |         | extended |              | 
 col19  | character varying(30)    |           |          |         | extended |              | 
 col20  | character varying(30)    |           |          |         | extended |              | 
Indexes:
    "cts1_time_index" btree ("time")

 

单行写入

每线程10W行数据。每行约180bytes. 一个线程写入约18MB。

看起来两者性能相差不大,PG稍强5%~10%, 之前此场景下influxdb的测试数据也放在一起做个对比

线程平均速率(行/s)单线程2线程4线程8线程16线程32线程
timescaledb1075100810261045923654
PG1092102010701120982737
infuxdb422378398370341305
       
总速率(行/s)单线程2线程4线程8线程16线程32线程
timescaledb10752011400081661435820337
PG10922037403787341508722445
infuxdb4227521588294754449732


WAL文件大小两者基本一致,都是大约数据量的 2倍。WAL文件增加

线程数字节数PG_WAL_SIZETS_WAL_SIZE
1186975623742827237464824
2373792237454714475026672
474785548149803536149525488
8149548561298455240298400040
16299134911596227840596302792
3259838511511916109521194038480

BATCH写入

单线程行数设为1000万, BATCHSIZE 设为1000

PG优于TIMESCALEDB。

每线程1000W行数据batchsize=1000
线程平均速率(行/s)单线程8线程
timescaledb2673211916
PG3911314972
   
   
总速率(行/s)单线程8线程
timescaledb2673295140
PG39113119528

资源占用

TDSB 32线程, CPU平均占用 57%。 比PG高10%。

PG 32线程, CPU平均占用 46%.

CASE 2 增加一个索引

 增加id为索引

ts1=# \d+ ckts1
                                            Table "public.ckts1"
 Column |           Type           | Collation | Nullable | Default | Storage  | Stats target | Description 
--------+--------------------------+-----------+----------+---------+----------+--------------+-------------
 time   | timestamp with time zone |           | not null |         | plain    |              | 
 id     | integer                  |           |          |         | plain    |              | 
 col2   | integer                  |           |          |         | plain    |              | 
 col3   | integer                  |           |          |         | plain    |              | 
 col4   | integer                  |           |          |         | plain    |              | 
 col5   | integer                  |           |          |         | plain    |              | 
 col6   | integer                  |           |          |         | plain    |              | 
 col7   | integer                  |           |          |         | plain    |              | 
 col8   | integer                  |           |          |         | plain    |              | 
 col9   | integer                  |           |          |         | plain    |              | 
 col10  | integer                  |           |          |         | plain    |              | 
 col11  | integer                  |           |          |         | plain    |              | 
 col12  | character varying(30)    |           |          |         | extended |              | 
 col13  | character varying(30)    |           |          |         | extended |              | 
 col14  | character varying(30)    |           |          |         | extended |              | 
 col15  | character varying(30)    |           |          |         | extended |              | 
 col16  | character varying(30)    |           |          |         | extended |              | 
 col17  | character varying(30)    |           |          |         | extended |              | 
 col18  | character varying(30)    |           |          |         | extended |              | 
 col19  | character varying(30)    |           |          |         | extended |              | 
 col20  | character varying(30)    |           |          |         | extended |              | 
Indexes:
    "ckts1_id_idx" btree (id)
    "ckts1_time_idx" btree ("time" DESC)
Triggers:
    ts_insert_blocker BEFORE INSERT ON ckts1 FOR EACH ROW EXECUTE PROCEDURE _timescaledb_internal.insert_blocker()

ts1=# 

postgres=# \d+ cts1
                                            Table "public.cts1"
 Column |           Type           | Collation | Nullable | Default | Storage  | Stats target | Description 
--------+--------------------------+-----------+----------+---------+----------+--------------+-------------
 time   | timestamp with time zone |           | not null |         | plain    |              | 
 id     | integer                  |           |          |         | plain    |              | 
 col2   | integer                  |           |          |         | plain    |              | 
 col3   | integer                  |           |          |         | plain    |              | 
 col4   | integer                  |           |          |         | plain    |              | 
 col5   | integer                  |           |          |         | plain    |              | 
 col6   | integer                  |           |          |         | plain    |              | 
 col7   | integer                  |           |          |         | plain    |              | 
 col8   | integer                  |           |          |         | plain    |              | 
 col9   | integer                  |           |          |         | plain    |              | 
 col10  | integer                  |           |          |         | plain    |              | 
 col11  | integer                  |           |          |         | plain    |              | 
 col12  | character varying(30)    |           |          |         | extended |              | 
 col13  | character varying(30)    |           |          |         | extended |              | 
 col14  | character varying(30)    |           |          |         | extended |              | 
 col15  | character varying(30)    |           |          |         | extended |              | 
 col16  | character varying(30)    |           |          |         | extended |              | 
 col17  | character varying(30)    |           |          |         | extended |              | 
 col18  | character varying(30)    |           |          |         | extended |              | 
 col19  | character varying(30)    |           |          |         | extended |              | 
 col20  | character varying(30)    |           |          |         | extended |              | 
Indexes:
    "cst1_id_index" btree (id)
    "cts1_time_index" btree ("time")

单行写入

10万行数据,单条写入

和没有添加索引时性能差不多,PG看起来性能下降更多一点。

总速率(行/s)单线程
timescaledb1051
PG1019

 

BATCH写入

每线程1000万行数据,batchsize设为1000, 8线程处理,13.9G数据

增加一个索引以后,PG和TSDB比之前性能都有所降低,PG写入仍旧比TSDB快。

每线程1000W行数据batchsize=1000
线程平均速率(行/s)单线程8线程
timescaledb2614210591
PG3686113037
   
   
总速率(行/s)单线程8线程
timescaledb2614284588
PG36861103806

资源占用

8线程,batchsize=1000, TSDB CPU占用80%左右

8线程,batchsize=1000, PG CPU占用60%左右

 

CASE 3 大量数据

设置PG的表cts1索引time为DESC(倒序),测试结果与上面并无不同

postgres=# \d+ cts1
                                            Table "public.cts1"
 Column |           Type           | Collation | Nullable | Default | Storage  | Stats target | Description 
--------+--------------------------+-----------+----------+---------+----------+--------------+-------------
 time   | timestamp with time zone |           | not null |         | plain    |              | 
 id     | integer                  |           |          |         | plain    |              | 
 col2   | integer                  |           |          |         | plain    |              | 
 col3   | integer                  |           |          |         | plain    |              | 
 col4   | integer                  |           |          |         | plain    |              | 
 col5   | integer                  |           |          |         | plain    |              | 
 col6   | integer                  |           |          |         | plain    |              | 
 col7   | integer                  |           |          |         | plain    |              | 
 col8   | integer                  |           |          |         | plain    |              | 
 col9   | integer                  |           |          |         | plain    |              | 
 col10  | integer                  |           |          |         | plain    |              | 
 col11  | integer                  |           |          |         | plain    |              | 
 col12  | character varying(30)    |           |          |         | extended |              | 
 col13  | character varying(30)    |           |          |         | extended |              | 
 col14  | character varying(30)    |           |          |         | extended |              | 
 col15  | character varying(30)    |           |          |         | extended |              | 
 col16  | character varying(30)    |           |          |         | extended |              | 
 col17  | character varying(30)    |           |          |         | extended |              | 
 col18  | character varying(30)    |           |          |         | extended |              | 
 col19  | character varying(30)    |           |          |         | extended |              | 
 col20  | character varying(30)    |           |          |         | extended |              | 
Indexes:
    "cts1_id_idx" btree (id)
    "cts1_time_idx" btree ("time" DESC)

ts1=# \d+ ckts1
                                            Table "public.ckts1"
 Column |           Type           | Collation | Nullable | Default | Storage  | Stats target | Description 
--------+--------------------------+-----------+----------+---------+----------+--------------+-------------
 time   | timestamp with time zone |           | not null |         | plain    |              | 
 id     | integer                  |           |          |         | plain    |              | 
 col2   | integer                  |           |          |         | plain    |              | 
 col3   | integer                  |           |          |         | plain    |              | 
 col4   | integer                  |           |          |         | plain    |              | 
 col5   | integer                  |           |          |         | plain    |              | 
 col6   | integer                  |           |          |         | plain    |              | 
 col7   | integer                  |           |          |         | plain    |              | 
 col8   | integer                  |           |          |         | plain    |              | 
 col9   | integer                  |           |          |         | plain    |              | 
 col10  | integer                  |           |          |         | plain    |              | 
 col11  | integer                  |           |          |         | plain    |              | 
 col12  | character varying(30)    |           |          |         | extended |              | 
 col13  | character varying(30)    |           |          |         | extended |              | 
 col14  | character varying(30)    |           |          |         | extended |              | 
 col15  | character varying(30)    |           |          |         | extended |              | 
 col16  | character varying(30)    |           |          |         | extended |              | 
 col17  | character varying(30)    |           |          |         | extended |              | 
 col18  | character varying(30)    |           |          |         | extended |              | 
 col19  | character varying(30)    |           |          |         | extended |              | 
 col20  | character varying(30)    |           |          |         | extended |              | 
Indexes:
    "ckts1_id_idx" btree (id)
    "ckts1_time_idx" btree ("time" DESC)
Triggers:
    ts_insert_blocker BEFORE INSERT ON ckts1 FOR EACH ROW EXECUTE PROCEDURE _timescaledb_internal.insert_blocker()

怀疑因为数据量不够大,导致TSDB写入性能优势没能体现

增大数据量到1亿行

batchsize = 1000,单线程写入

行数字节数PG执行时间TSDB执行时间PG行速率TSDB行速率(行/s)TSDB TRUNK数量
2000万3.45GB565 s817 s35398 (行/s)24480 (行/s)30
1亿17.4GB2782 s3992 s35945 (行/s)25050 (行/s)166

结果仍然是PG写入速度更快。

To be continued...

### TimescaleDB PostgreSQL:特性、安装、配置与性能调优 #### 特性概述 TimescaleDB 是一种专为时间序列数据设计的关系型数据库,基于 PostgreSQL 构建。这使得 TimescaleDB 继承了许多 PostgreSQL 的强大功能,同时也增加了专门针对时间序列数据处理的功能[^1]。 - **自动分区**:支持按时间/或空间维度的数据分片。 - **高效查询**:提供高效的聚合函数窗口函数来加速复杂的时间序列分析。 - **SQL 支持**:完全兼容标准 SQL 查询语法,允许用户利用熟悉的工具技术栈进行开发。 - **扩展插件**:继承自 PostgreSQL 的丰富生态系统,可以轻松集成各种第三方应用服务。 #### 安装过程 对于 TimescaleDB 的安装,在大多数情况下只需要通过包管理器或者二进制文件下载最新版本即可完成基本部署工作。具体步骤取决于操作系统环境的不同而有所差异。一旦成功安装之后,则可以通过命令行客户端 psql 或者图形界面应用程序 pgAdmin 来连接到新创建好的实例上开始初步探索。 #### 配置优化 为了使 TimescaleDB 能够更好地适应特定的工作负载需求,通常还需要进一步调整其内部参数设置。这部分操作主要集中在修改 `postgresql.conf` 文件中的几个关键选项: - **共享缓冲区大小 (`shared_buffers`)** - **有效缓存大小 (`effective_cache_size`)** - **维护工作内存 (`maintenance_work_mem`)** 这些参数的选择应当依据服务器硬件资源情况以及预期的应用场景来进行合理规划。当启动配置向导时,会根据服务器特性 PostgreSQL 版本推荐相应的设定值供参考;此时按下 'y' 即可开启自动化调参流程[^2]。 如果希望深入了解如何精确调节 PostgreSQL/TimescaleDB 中涉及的各类内存相关属性,建议查阅官方文档《Tuning Your PostgreSQL Server》获取更详细的指导说明[^3]。 ```bash # 修改 postgresql.conf 文件示例 shared_buffers = 4GB # 设置共享缓冲区大小 effective_cache_size = 8GB # 设定有效的磁盘缓存容量预估值 maintenance_work_mem = 512MB # 控制作业期间可用的最大内存量 ```
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值