YCSB(Yahoo! Cloud Serving Benchmark)是雅虎开源的一款用于测试各类云服务/NoSQL/键值对存储的性能基准测试工具。
介绍一下如何用YSCB来测试mongodb
GitHub上下载非常的缓慢和国内防火墙的限制有关,编译好的300多M包基本上是下载不下来。
Windows环境源码编译安装
1. 安装jdk,设置环境变量
2. 安装mvn
3. 下载源码编译
Github地址: https://github.com/brianfrankcooper/YCSB
下载最新的源代码ycsb-master,导入mvn工程,设置本地的mvn库,否则更新太慢。
Building from source
YCSBrequires the use of Maven 3; if you use Maven 2, you may see errors such as these.
Tobuild the full distribution, with all database bindings:
mvn clean package
mvn -pl clean package
注:可能会少依赖的包,根据报错手工下载依赖的包。
编译成功,将包E:\code\YCSB-master\distribution\target\ycsb-0.13.0-SNAPSHOT.tar.gz上传到linux环境解压
tar -xzvf ycsb-0.13.0-SNAPSHOT.tar.gz
4. 简单测试
修改配置文件
vi /data_mongo/ycsb-0.13.0-SNAPSHOT/workloads/workloada
recordcount=1000
operationcount=1000
workload=com.yahoo.ycsb.workloads.CoreWorkload
readallfields=true
readproportion=0.5
updateproportion=0.5
scanproportion=0
insertproportion=0
requestdistribution=zipfian
mongodb.url=mongodb://10.45.66.61:27017/ycsb
mongodb.database=ycsb
mongodb.writeConcern=replica_acknowledged
//
这些参数也可以直接写在
shell
执行命令的后面。
~
YCSB执行分为两个阶段,测试数据加载load阶段和测试操作执行run阶段
1. load 加载阶段
加载测试数据分为同步模式和异步模式
./bin/ycsb.sh load mongodb-async -s -P workloads/workloada
./bin/ycsb.sh load mongodb -s -P workloads/workloada
加载完成后的结果。
2. run执行阶段
执行节点也分为同步和异步模式
./bin/ycsb.sh run mongodb -s -P workloads/workloada
./bin/ycsb.sh run mongodb-async -threads 100 -s -P workloads/workloada
查询mongodb里面中的一条记录
测试并发和性能参考下面配置参数说明。
5. 配置文件说明
5.1. MongoDB ConfigurationParameters
参数 | 说明 |
mongodb.url | MongoDB URI或者连接串 |
mongodb.batchsize | 对大量inert操作的时候,分批commit提交,提高吞吐量 |
mongodb.upsert | Determines if the insert operation performs an update with the upsert operation or a insert. Upserts have the advantage that they will continue to work for a partially loaded data set. - Setting to `true` uses updates, `false` uses insert operations. - Default value is `false`. |
mongodb.writeConcern | - Allowed values are : - `errors_ignored` - `unacknowledged` - `acknowledged` - `journaled` - `replica_acknowledged` - `majority` - Default value is `acknowledged`. |
mongodb.readPreference | - Allowed values are : - `primary` - `primary_preferred` - `secondary` - `secondary_preferred` - `nearest` - Default value is `primary`. |
mongodb.maxconnections | Use the `maxPoolSize` options on the MongoDB URI provided bythe `mongodb.url`. - Default value is `100`. |
mongodb.threadsAllowedToBlockForConnectionMultiplier | Use the `waitQueueMultiple` options on the MongoDB URI provided bythe `mongodb.url`. - Default value is `5` |
5.2. Core workload packageproperties
参数 | 说明 |
fieldcount | 单条记录字段个数,默认10 |
fieldlength | 每个字段的大小,默认100B |
readallfields | 是否读取所有记录,默认true |
readproportion | 读比例(默认0.95) |
updateproportion | 更新比例(默认0.05) |
insertproportion | 插入比例(默认0) |
scanproportion | 扫描比例(默认0) |
readmodifywriteproportion | 同一记录读、修改、回写比例(默认值:0) |
requestdistribution | 记录选择策略:uniform,zipfian,latest(默认值:uniform) |
maxscanlength | 最大scan记录数(默认值:100) |
scanlengthdistribution | scan记录选择策略(默认值:uniform) |
insertorder | 记录插入策略:ordered,hashed(默认值:hashed) |
operationcount | 操作执行数 |
maxexecutiontime | 最大执行时间,单位:秒 |
table | 表名(默认值值usertable) |
recordcount | 初始化记录条数(默认值:0) |
5.3. ycsb提供的几个测试的模式
文件名 | 说明 |
workloada | Update heavy workload This workload has a mix of 50/50 reads and writes. An application example is a session store recording recent actions. |
workloadb | Read mostly workload This workload has a 95/5 reads/write mix. Application example: photo tagging; add a tag is an update, but most operations are to read tags. |
workloadc | Read only This workload is 100% read. Application example: user profile cache, where profiles are constructed elsewhere (e.g., Hadoop). |
workloadd | Read latest workload In this workload, new records are inserted, and the most recently inserted records are the most popular. Application example: user status updates; people want to read the latest. |
workloade | Short ranges In this workload, short ranges of records are queried, instead of individual records. Application example: threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id). |
workloadf | Read-modify-write In this workload, the client will read a record, modify it, and write back the changes. Application example: user database, where user records are read and modified by the user or to record user activity. |