YSCB来测试MongoDB

本文介绍如何使用YCSB工具进行MongoDB性能测试。包括Windows环境下源码编译安装步骤,配置文件详解及测试模式说明。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

YCSB(Yahoo! Cloud Serving Benchmark是雅虎开源的一款用于测试各类云服务/NoSQL/键值对存储的性能基准测试工具。

 

介绍一下如何用YSCB来测试mongodb

 

GitHub上下载非常的缓慢和国内防火墙的限制有关,编译好的300多M包基本上是下载不下来。

Windows环境源码编译安装

1. 安装jdk,设置环境变量

2. 安装mvn

3. 下载源码编译

Github地址: https://github.com/brianfrankcooper/YCSB

下载最新的源代码ycsb-master,导入mvn工程,设置本地的mvn库,否则更新太慢。

Building from source

YCSBrequires the use of Maven 3; if you use Maven 2, you may see errors such as these.

Tobuild the full distribution, with all database bindings:

mvn clean package
mvn -pl clean package

注:可能会少依赖的包,根据报错手工下载依赖的包。


编译成功,将包E:\code\YCSB-master\distribution\target\ycsb-0.13.0-SNAPSHOT.tar.gz上传到linux环境解压

tar -xzvf ycsb-0.13.0-SNAPSHOT.tar.gz

4. 简单测试

修改配置文件

vi /data_mongo/ycsb-0.13.0-SNAPSHOT/workloads/workloada 
recordcount=1000
operationcount=1000
workload=com.yahoo.ycsb.workloads.CoreWorkload
readallfields=true
readproportion=0.5
updateproportion=0.5
scanproportion=0
insertproportion=0
requestdistribution=zipfian

mongodb.url=mongodb://10.45.66.61:27017/ycsb    
mongodb.database=ycsb
mongodb.writeConcern=replica_acknowledged
// 这些参数也可以直接写在shell执行命令的后面。
~                                     

YCSB执行分为两个阶段,测试数据加载load阶段和测试操作执行run阶段

1.     load 加载阶段

加载测试数据分为同步模式和异步模式

./bin/ycsb.sh load mongodb-async -s -P workloads/workloada
./bin/ycsb.sh load mongodb -s -P workloads/workloada

加载完成后的结果。

 

2.    run执行阶段

执行节点也分为同步和异步模式

./bin/ycsb.sh run mongodb -s -P workloads/workloada
./bin/ycsb.sh run mongodb-async -threads 100 -s -P workloads/workloada


查询mongodb里面中的一条记录


测试并发和性能参考下面配置参数说明。

5. 配置文件说明

5.1. MongoDB ConfigurationParameters

参数

说明

mongodb.url

MongoDB URI或者连接串

mongodb.batchsize

对大量inert操作的时候,分批commit提交,提高吞吐量

mongodb.upsert

Determines if the insert operation performs an update with the upsert operation or a insert.

    Upserts have the advantage that they will continue to work for a partially loaded data set.

  - Setting to `true` uses updates, `false` uses insert operations.

  - Default value is `false`.

mongodb.writeConcern

  - Allowed values are :

    - `errors_ignored`

    - `unacknowledged`

    - `acknowledged`

    - `journaled`

    - `replica_acknowledged`

    - `majority`

  - Default value is `acknowledged`.

mongodb.readPreference

  - Allowed values are :

    - `primary`

    - `primary_preferred`

    - `secondary`

    - `secondary_preferred`

    - `nearest`

  - Default value is `primary`.

mongodb.maxconnections

Use the `maxPoolSize` options on the MongoDB URI provided bythe `mongodb.url`.

  - Default value is `100`.

mongodb.threadsAllowedToBlockForConnectionMultiplier

Use the `waitQueueMultiple` options on the MongoDB URI provided bythe `mongodb.url`.

  - Default value is `5`

 

5.2. Core workload packageproperties

参数

说明

fieldcount                                             

单条记录字段个数,默认10

fieldlength

每个字段的大小,默认100B

readallfields

是否读取所有记录,默认true

readproportion

读比例(默认0.95)

updateproportion

更新比例(默认0.05)

insertproportion

插入比例(默认0)

scanproportion

扫描比例(默认0)

readmodifywriteproportion

同一记录读、修改、回写比例(默认值:0)

requestdistribution

记录选择策略:uniform,zipfian,latest(默认值:uniform)

maxscanlength

最大scan记录数(默认值:100)

scanlengthdistribution

scan记录选择策略(默认值:uniform)

insertorder

记录插入策略:ordered,hashed(默认值:hashed)

operationcount

操作执行数

maxexecutiontime

最大执行时间,单位:秒

table

表名(默认值值usertable)

recordcount

初始化记录条数(默认值:0)

5.3. ycsb提供的几个测试的模式

文件名

说明

workloada                                           

Update heavy workload

This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.

workloadb

Read mostly workload

This workload has a 95/5 reads/write mix. 

Application example: photo tagging; add a tag is an update, but most operations are to read tags.

workloadc

Read only

This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop).

workloadd

Read latest workload

In this workload, new records are inserted, and the most recently inserted records are the most popular. 

Application example: user status updates; people want to read the latest.

workloade

Short ranges

In this workload, short ranges of records are queried, instead of individual records. 

Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id).

workloadf

Read-modify-write

In this workload, the client will read a record, modify it, and write back the changes. 

Application example: user database, where user records are read and modified by the user or to record user activity.


参考文档:https://github.com/brianfrankcooper/YCSB

评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值