YSCB来测试MongoDB

YCSB（Yahoo! Cloud Serving Benchmark）是雅虎开源的一款用于测试各类云服务/NoSQL/键值对存储的性能基准测试工具。

 

介绍一下如何用YSCB来测试mongodb

 

GitHub上下载非常的缓慢和国内防火墙的限制有关，编译好的300多M包基本上是下载不下来。

Windows环境源码编译安装

1. 安装jdk，设置环境变量

2. 安装mvn

3. 下载源码编译

Github地址: https://github.com/brianfrankcooper/YCSB

下载最新的源代码ycsb-master，导入mvn工程，设置本地的mvn库，否则更新太慢。

Building from source

YCSBrequires the use of Maven 3; if you use Maven 2, you may see errors such as these.

Tobuild the full distribution, with all database bindings:

mvn clean package

mvn -pl clean package

注:可能会少依赖的包，根据报错手工下载依赖的包。

编译成功，将包E:\code\YCSB-master\distribution\target\ycsb-0.13.0-SNAPSHOT.tar.gz上传到linux环境解压

tar -xzvf ycsb-0.13.0-SNAPSHOT.tar.gz

4. 简单测试

修改配置文件

vi /data_mongo/ycsb-0.13.0-SNAPSHOT/workloads/workloada 

recordcount=1000

operationcount=1000

workload=com.yahoo.ycsb.workloads.CoreWorkload

readallfields=true

readproportion=0.5

updateproportion=0.5

scanproportion=0

insertproportion=0

requestdistribution=zipfian

mongodb.url=mongodb://10.45.66.61:27017/ycsb    

mongodb.database=ycsb

mongodb.writeConcern=replica_acknowledged

// 这些参数也可以直接写在shell执行命令的后面。

~                                     

YCSB执行分为两个阶段，测试数据加载load阶段和测试操作执行run阶段

1.     load 加载阶段

加载测试数据分为同步模式和异步模式

./bin/ycsb.sh load mongodb-async -s -P workloads/workloada

./bin/ycsb.sh load mongodb -s -P workloads/workloada

加载完成后的结果。

 

2.    run执行阶段

执行节点也分为同步和异步模式

./bin/ycsb.sh run mongodb -s -P workloads/workloada

./bin/ycsb.sh run mongodb-async -threads 100 -s -P workloads/workloada

查询mongodb里面中的一条记录

测试并发和性能参考下面配置参数说明。

5. 配置文件说明

5.1. MongoDB ConfigurationParameters

参数	说明
mongodb.url	MongoDB URI或者连接串
mongodb.batchsize	对大量inert操作的时候，分批commit提交，提高吞吐量
mongodb.upsert	Determines if the insert operation performs an update with the upsert operation or a insert.     Upserts have the advantage that they will continue to work for a partially loaded data set.   - Setting to `true` uses updates, `false` uses insert operations.   - Default value is `false`.
mongodb.writeConcern	- Allowed values are :     - `errors_ignored`     - `unacknowledged`     - `acknowledged`     - `journaled`     - `replica_acknowledged`     - `majority`   - Default value is `acknowledged`.
mongodb.readPreference	- Allowed values are :     - `primary`     - `primary_preferred`     - `secondary`     - `secondary_preferred`     - `nearest`   - Default value is `primary`.
mongodb.maxconnections	Use the `maxPoolSize` options on the MongoDB URI provided bythe `mongodb.url`.   - Default value is `100`.
mongodb.threadsAllowedToBlockForConnectionMultiplier	Use the `waitQueueMultiple` options on the MongoDB URI provided bythe `mongodb.url`.   - Default value is `5`

 

5.2. Core workload packageproperties

参数	说明
fieldcount	单条记录字段个数，默认10
fieldlength	每个字段的大小，默认100B
readallfields	是否读取所有记录，默认true
readproportion	读比例（默认0.95）
updateproportion	更新比例（默认0.05）
insertproportion	插入比例（默认0）
scanproportion	扫描比例（默认0）
readmodifywriteproportion	同一记录读、修改、回写比例（默认值：0）
requestdistribution	记录选择策略：uniform,zipfian,latest（默认值：uniform）
maxscanlength	最大scan记录数（默认值：100）
scanlengthdistribution	scan记录选择策略（默认值：uniform）
insertorder	记录插入策略：ordered,hashed（默认值：hashed）
operationcount	操作执行数
maxexecutiontime	最大执行时间，单位：秒
table	表名（默认值值usertable）
recordcount	初始化记录条数（默认值：0）

5.3. ycsb提供的几个测试的模式

文件名	说明
workloada	Update heavy workload This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions.
workloadb	Read mostly workload This workload has a 95/5 reads/write mix.  Application example: photo tagging; add a tag is an update, but most operations are to read tags.
workloadc	Read only This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop).
workloadd	Read latest workload In this workload, new records are inserted, and the most recently inserted records are the most popular.  Application example: user status updates; people want to read the latest.
workloade	Short ranges In this workload, short ranges of records are queried, instead of individual records.  Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id).
workloadf	Read-modify-write In this workload, the client will read a record, modify it, and write back the changes.  Application example: user database, where user records are read and modified by the user or to record user activity.

参考文档:https://github.com/brianfrankcooper/YCSB