MongoDB实战全攻略二（分片配置，高可用集群设计实战）

***mongodb的分片配置

md01 10.0.0.11
md02 10.0.0.12
md03 10.0.0.14

2、启动三台机器的mongod实例

根据Replica Set、Sharding策略部署mongod。将两个sharding组部署到三台服务器上，每个sharding组有三个replica set成员。

#Server1:

1. mkdir -p /elain/data/mongodb/shard11
2. mkdir -p /elain/data/mongodb/shard21
3. /elain/apps/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /elain/data/mongodb/shard11 --oplogSize 100 --logpath /elain/data/mongodb/shard11.log --logappend --fork --rest
4. /elain/apps/mongodb/bin/mongod --shardsvr --replSet shard2 --port 27018 --dbpath /elain/data/mongodb/shard21 --oplogSize 100 --logpath /elain/data/mongodb/shard21.log --logappend --fork --rest

#Server2:

1. mkdir -p /elain/data/mongodb/shard12/
2. mkdir -p /elain/data/mongodb/shard22/
3. /elain/apps/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /elain/data/mongodb/shard12 --oplogSize 100 --logpath /elain/data/mongodb/shard12.log --logappend --fork --rest
4. /elain/apps/mongodb/bin/mongod --shardsvr --replSet shard2 --port 27018 --dbpath /elain/data/mongodb/shard22 --oplogSize 100 --logpath /elain/data/mongodb/shard22.log --logappend --fork --rest

#Server3:

1. mkdir -p /elain/data/mongodb/shard13/
2. mkdir -p /elain/data/mongodb/shard23/
3. /elain/apps/mongodb/bin/mongod --shardsvr --replSet shard1 --port 27017 --dbpath /elain/data/mongodb/shard13 --oplogSize 100 --logpath /elain/data/mongodb/shard13.log --logappend --fork --rest
4. /elain/apps/mongodb/bin/mongod --shardsvr --replSet shard2 --port 27018 --dbpath /elain/data/mongodb/shard23 --oplogSize 100 --logpath /elain/data/mongodb/shard23.log --logappend --fork --rest

3、初始化Replica Set 通过命令行初始化两组Replica Set，通过mongo连接到一个mongod

1. /elain/apps/mongodb/bin/mongo 10.0.0.11:27017
3. config = {_id: 'shard1', members: [
4. {_id: 0, host: '10.0.0.11:27017'},
5. {_id: 1, host: '10.0.0.12:27017'},
6. {_id: 2, host: '10.0.0.14:27017'}]};

1. rs.initiate(config);
3. /elain/apps/mongodb/bin/mongo 10.0.0.11:27018
5. config = {_id: 'shard2', members: [
6. {_id: 0, host: '10.0.0.11:27018'},
7. {_id: 1, host: '10.0.0.12:27018'},
8. {_id: 2, host: '10.0.0.14:27018'}]};
10. rs.initiate(config);

4、启动并配置三台Config Server

#Server1、2、3:

1. mkdir -p /elain/data/mongodb/config/
2. /elain/apps/mongodb/bin/mongod --configsvr --dbpath /elain/data/mongodb/config/ --port 20000 --logpath /elain/data/mongodb/config1.log --logappend --fork

5、部署并配置三台Routing Server

指定所有的config sever地址参数，chunkSize是分割数据时每块(Chunk)的单位大小

#Server1、2、3:

1. /elain/apps/mongodb/bin/mongos --configdb 10.0.0.11:20000,10.0.0.12:20000,10.0.0.14:20000 --port 30000 --chunkSize 100 --logpath /elain/data/mongodb/mongos.log --logappend --fork

6、命令行添加分片

连接到mongs服务器，并切换到admin

1. /elain/apps/mongodb/bin/mongo 10.0.0.11:30000/admin

1. db.runCommand( {
2. addshard : "shard1/10.0.0.11:27017,10.0.0.12:27017,10.0.0.14:27017",
3. name:"shard1",
4. maxsize:2048,
5. allowLocal:true } );
7. db.runCommand( {
8. addshard : "shard2/10.0.0.11:27018,10.0.0.12:27018,10.0.0.14:27018",
9. name:"shard2",
10. maxsize:2048,
11. allowLocal:true } );
13. db.runCommand( { listshards : 1 } );

如果列出(sharding)了以上二个你加的shards，表示shards已经配置成功 #激活数据库分片

1. db.runCommand( { enablesharding : "elain" } );

要使单个collection也分片存储，需要给collection指定一个分片key，通过以下命令操作：

1. db.runCommand( { shardcollection : “<namespace>”,key : <shardkeypatternobject});

注：
a. 分片的collection系统会自动创建一个索引（也可用户提前创建好）
b. 分片的collection只能有一个在分片key上的唯一索引，其它唯一索引不被允许
One note: a sharded collection can have only one unique index, which must exist on the shard key. No other unique indexes can exist on the collection.

1. mongoimport -d elain -c test --type csv --headerline --file test.csv --host 10.0.0.11:30000

分片collection例子

1. db.runCommand( { shardcollection : "elain.test",key : {id: 1} } )
2. mongos> db.chujq.t1.stats();
3. {
4. "sharded" : true,
5. "flags" : 1,
6. "ns" : "elain.test",
7. "count" : 4058977,
8. "numExtents" : 34,
9. "size" : 675039816,
10. "storageSize" : 818757632,
11. "totalIndexSize" : 131854352,
12. "indexSizes" : {
13. "_id_" : 131854352
14. },
15. "avgObjSize" : 166.30786919955446,
16. "nindexes" : 1,
17. "nchunks" : 14,
18. "shards" : {
19. "shard1" : {
20. "ns" : "elain.test",
21. "count" : 1860365,
22. "size" : 309376352,
23. "avgObjSize" : 166.29873815084673,
24. "storageSize" : 408920064,
25. "numExtents" : 16,
26. "nindexes" : 1,
27. "lastExtentSize" : 77955072,
28. "paddingFactor" : 1,
29. "flags" : 1,
30. "totalIndexSize" : 60371584,
31. "indexSizes" : {
32. "_id_" : 60371584
33. },
34. "ok" : 1
35. },
36. "shard2" : {
37. "ns" : "elain.test",
38. "count" : 2198612,
39. "size" : 365663464,
40. "avgObjSize" : 166.31559547569103,
41. "storageSize" : 409837568,
42. "numExtents" : 18,
43. "nindexes" : 1,
44. "lastExtentSize" : 74846208,
45. "paddingFactor" : 1,
46. "flags" : 1,
47. "totalIndexSize" : 71482768,
48. "indexSizes" : {
49. "_id_" : 71482768
50. },
51. "ok" : 1
52. }
53. },
54. "ok" : 1
55. }

删除片操作

1. mongos> db.runCommand({"removeshard" : "10.0.0.11:27018"});
2. {
3. "msg" : "draining started successfully",
4. "state" : "started",
5. "shard" : "shard2",
6. "ok" : 1
7. }

再执行，可看到removeshard的挪动进度

1. mongos> db.runCommand({"removeshard" : "10.0.0.11:27018"});
2. {
3. "msg" : "draining ongoing",
4. "state" : "ongoing",
5. "remaining" : {
6. "chunks" : NumberLong(3),
7. "dbs" : NumberLong(0)
8. },
9. "ok" : 1
10. }

例:

1. /elain/apps/mongodb/bin/mongod --config /elain/data/mongodb/shard1/shard1.properties --rest

***MongoDB高可用集群设计实战部署（一）

环境：
CentOS 6.0 x64

md01: 10.0.0.11
md02: 10.0.0.12
md03: 10.0.0.14
md04: 10.0.0.15
md05: 10.0.0.16
md06: 10.0.0.17

设计思路：

md01、md02、md03 做一组复制集
md04、md05、md06 做一组复制集
再把这两组复制集用分片做成 shard1、shard2 用LVS 调用

下载安装mongodb

cd /root/tools wget http://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.0.0.tgz tar zxvf mongodb-linux-x86_64-2.0.0.tgz mv mongodb-linux-x86_64-2.0.0 /elain/apps/mongodb-linux-x86_64-2.0.0 ln -s /elain/apps/mongodb-linux-x86_64-2.0.0 /elain/apps/mongodb ln -s /elain/apps/mongodb/bin/* /bin/

#添加用户组

/usr/sbin/groupadd -g 690 mongodb /usr/sbin/useradd -g mongodb mongodb -u 690 -s /sbin/nologin

各节点hosts文件 添加：

true > /etc/hosts echo -ne " 10.0.0.11 md01 10.0.0.12 md02 10.0.0.14 md03 10.0.0.15 md04 10.0.0.16 md05 10.0.0.17 md06 " >>/etc/hosts 副本集配置： 启动各节点： md01

mkdir -p /elain/logs/mongodb/ mkdir -p /elain/data/mongodb/db /elain/apps/mongodb/bin/mongod --rest --replSet elain/md01:27017 --fork --port 27017 --dbpath /elain/data/mongodb/db/ --logpath /elain/logs/mongodb/mongodb.log

md02

mkdir -p /elain/logs/mongodb/ mkdir -p /elain/data/mongodb/db /elain/apps/mongodb/bin/mongod --rest --replSet elain/md02:27017 --fork --port 27017 --dbpath /elain/data/mongodb/db/ --logpath /elain/logs/mongodb/mongodb.log

md03

mkdir -p /elain/logs/mongodb/ mkdir -p /elain/data/mongodb/db /elain/apps/mongodb/bin/mongod --rest --replSet elain/md03:27017 --fork --port 27017 --dbpath /elain/data/mongodb/db/ --logpath /elain/logs/mongodb/mongodb.log

#启动仲裁节点
在md02上

mkdir /elain/data/mongodb/arb /elain/apps/mongodb/bin/mongod --fork --rest --replSet elain --dbpath /elain/data/mongodb/arb --port 27015 --logpath /elain/logs/mongodb/mongodb.log

初始化节点:
md01:(登录其中任何一个节点操作皆可)

> rs.initiate({ _id : "elain", members : [ {_id : 1, host : "md01:27017"}, {_id : 2, host : "md02:27017"}, {_id : 3, host : "md03:27017"}, {_id : 4, host : "md02:27015", "arbiterOnly": true}, ] });

#验证

PRIMARY> rs.status()

也可浏览：http://10.0.0.11:28017/_replSet 查看状态

查看副本集状态

>rs.status() >user local; >rs.isMaster() >db.system.replset.find()

查看当前主库：

>db.$cmd.findOne({ismaster:1});

另一组副本集同理操作即可;这里为省篇幅就不再写出,但切记两复制集名称不可以重复
启动并配置三台Config Server

#md01,03,05上执行

mkdir -p /elain/data/mongodb/config/ /elain/apps/mongodb/bin/mongod --configsvr --dbpath /elain/data/mongodb/config/ --port 20000 --logpath /elain/logs/mongodb/config.log --logappend --fork

5、部署并配置三台Routing Server

指定所有的config sever地址参数，chunkSize是分割数据时每块(Chunk)的单位大小

#md02,md04,md06

/elain/apps/mongodb/bin/mongos --configdb md01:20000,md03:20000,md05:20000 --port 30000 --chunkSize 100 --logpath /elain/logs/mongodb/mongos.log --logappend --fork

6、命令行添加分片

连接到mongs服务器，并切换到admin

/elain/apps/mongodb/bin/mongo 10.0.0.11:30000/admin

db.runCommand( { addshard : "elain/md01:27017", name:"shard1", maxsize:2048, allowLocal:true } ); db.runCommand( { addshard : "chujq/md04:27017", name:"shard2", maxsize:2048, allowLocal:true } );

注：添加复制集elain，其中包含一个服务器md01:27017(还饿别的服务器，如md02、md03)，如果md01挂了，mongos会知道它所连接的是一个复制集，并会使用新的主节点（md02或md03）

db.runCommand( { listshards : 1 } );

如果列出(sharding)了以上二个你加的shards，表示shards已经配置成功

导入数据：

mongoimport -d elain -c elain --type csv --headerline --file /root/bak/test.csv --host md02:30000

注：数据自备，我这里是从生产环境下MYSQL里导出的一些真实数据来做测试

#激活数据库分片

db.runCommand( { enablesharding : "elain" } ); mongos> db.runCommand( { enablesharding : "elain" } ); { "ok" : 1 } mongos> show dbs config 0.1875GB elain 0.453125GB test (empty)

#进入数据库，建立索引，为分片做准备

/elain/apps/mongodb/bin/mongo 10.0.0.12:30000/admin use elain; db.elain.ensureIndex({"client_userid":1});

查看索引：

mongos> db.elain.find({"client_userid" : 151512}).explain(); { "cursor" : "BtreeCursor client_userid_1", "nscanned" : 3, "nscannedObjects" : 3, "n" : 3, "millis" : 0, "nYields" : 0, "nChunkSkips" : 0, "isMultiKey" : false, "indexOnly" : false, "indexBounds" : { "client_userid" : [ [ 151512, 151512 ] ] } }

#添加分片：

use admin; db.runCommand( { shardcollection : "elain.elain",key : {client_userid: 1} } ) mongos> db.runCommand( { shardcollection : "elain.elain",key : {client_userid: 1} } ) { "collectionsharded" : "elain.elain", "ok" : 1 }

#查看分片状态

use elain; db.elain.stats(); mongos> db.elain.stats(); { "sharded" : true, "flags" : 1, "ns" : "elain.elain", "count" : 507372, "numExtents" : 10, "size" : 84375328, "storageSize" : 97849344, "totalIndexSize" : 29253728, "indexSizes" : { "_id_" : 16474640, "client_userid_1" : 12779088 }, "avgObjSize" : 166.29874727024747, "nindexes" : 2, "nchunks" : 1, "shards" : { "shard2" : { "ns" : "elain.elain", "count" : 507372, "size" : 84375328, "avgObjSize" : 166.29874727024747, "storageSize" : 97849344, "numExtents" : 10, "nindexes" : 2, "lastExtentSize" : 26099712, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 29253728, "indexSizes" : { "_id_" : 16474640, "client_userid_1" : 12779088 }, "ok" : 1 } }, "ok" : 1 }

在分片后新写数据第一次：

mongos> db.elain.stats(); { "sharded" : true, "flags" : 1, "ns" : "elain.elain", "count" : 676496, "numExtents" : 12, "size" : 112500436, "storageSize" : 129179648, "totalIndexSize" : 47551616, "indexSizes" : { "_id_" : 21968912, "client_userid_1" : 25582704 }, "avgObjSize" : 166.29874529930703, "nindexes" : 2, "nchunks" : 8, "shards" : { "shard1" : { "ns" : "elain.elain", "count" : 0, "size" : 0, "storageSize" : 8192, "numExtents" : 1, "nindexes" : 2, "lastExtentSize" : 8192, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 16352, "indexSizes" : { "_id_" : 8176, "client_userid_1" : 8176 }, "ok" : 1 }, "shard2" : { "ns" : "elain.elain", "count" : 676496, "size" : 112500436, "avgObjSize" : 166.29874529930703, "storageSize" : 129171456, "numExtents" : 11, "nindexes" : 2, "lastExtentSize" : 31322112, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 47535264, "indexSizes" : { "_id_" : 21960736, "client_userid_1" : 25574528 }, "ok" : 1 } }, "ok" : 1 }

分片后新写数据第二次：

mongos> db.elain.stats(); { "sharded" : true, "flags" : 1, "ns" : "elain.elain", "count" : 1189194, "numExtents" : 23, "size" : 194533928, "storageSize" : 252874752, "totalIndexSize" : 87262448, "indexSizes" : { "_id_" : 43692544, "client_userid_1" : 43569904 }, "avgObjSize" : 163.58468677103988, "nindexes" : 2, "nchunks" : 8, "shards" : { "shard1" : { "ns" : "elain.elain", "count" : 396370, "size" : 62195348, "avgObjSize" : 156.91234957236924, "storageSize" : 86114304, "numExtents" : 11, "nindexes" : 2, "lastExtentSize" : 20881408, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 35949872, "indexSizes" : { "_id_" : 17954496, "client_userid_1" : 17995376 }, "ok" : 1 }, "shard2" : { "ns" : "elain.elain", "count" : 792824, "size" : 132338580, "avgObjSize" : 166.9205019020615, "storageSize" : 166760448, "numExtents" : 12, "nindexes" : 2, "lastExtentSize" : 37588992, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 51312576, "indexSizes" : { "_id_" : 25738048, "client_userid_1" : 25574528 }, "ok" : 1 } }, "ok" : 1 }

分片后新写数据第三次：

mongos> db.elain.stats(); { "sharded" : true, "flags" : 1, "ns" : "elain.elain", "count" : 1376876, "numExtents" : 23, "size" : 225576604, "storageSize" : 252874752, "totalIndexSize" : 100826432, "indexSizes" : { "_id_" : 50249696, "client_userid_1" : 50576736 }, "avgObjSize" : 163.83218532387812, "nindexes" : 2, "nchunks" : 10, "shards" : { "shard1" : { "ns" : "elain.elain", "count" : 494202, "size" : 77551984, "avgObjSize" : 156.92365469990003, "storageSize" : 86114304, "numExtents" : 11, "nindexes" : 2, "lastExtentSize" : 20881408, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 42057344, "indexSizes" : { "_id_" : 21600992, "client_userid_1" : 20456352 }, "ok" : 1 }, "shard2" : { "ns" : "elain.elain", "count" : 882674, "size" : 148024620, "avgObjSize" : 167.70021548159343, "storageSize" : 166760448, "numExtents" : 12, "nindexes" : 2, "lastExtentSize" : 37588992, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 58769088, "indexSizes" : { "_id_" : 28648704, "client_userid_1" : 30120384 }, "ok" : 1 } }, "ok" : 1 }

分片后新写数据第四次（DOWN 掉md04的mongo服务）

mongos> db.elain.stats(); { "sharded" : true, "flags" : 1, "ns" : "elain.elain", "count" : 1686310, "numExtents" : 26, "size" : 275761376, "storageSize" : 353116160, "totalIndexSize" : 129033632, "indexSizes" : { "_id_" : 63265888, "client_userid_1" : 65767744 }, "avgObjSize" : 163.52946729842083, "nindexes" : 2, "nchunks" : 10, "shards" : { "shard1" : { "ns" : "elain.elain", "count" : 740264, "size" : 116213588, "avgObjSize" : 156.98938216636228, "storageSize" : 141246464, "numExtents" : 13, "nindexes" : 2, "lastExtentSize" : 30072832, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 61810560, "indexSizes" : { "_id_" : 32556832, "client_userid_1" : 29253728 }, "ok" : 1 }, "shard2" : { "ns" : "elain.elain", "count" : 946046, "size" : 159547788, "avgObjSize" : 168.64696642657967, "storageSize" : 211869696, "numExtents" : 13, "nindexes" : 2, "lastExtentSize" : 45109248, "paddingFactor" : 1, "flags" : 1, "totalIndexSize" : 67223072, "indexSizes" : { "_id_" : 30709056, "client_userid_1" : 36514016 }, "ok" : 1 } }, "ok" : 1 }

总结：通过以上四次的写数据测试，我们可以看到分片是成功的，每次写数据,shard1、shard2都有数据写入，且，在下面的复制集中DOWN 掉任意一台，不影响整个架构的正常服务。

删除片操作

mongos> db.runCommand({"removeshard" : "10.0.0.11:27011"}); { "msg" : "draining started successfully", "state" : "started", "shard" : "shard2", "ok" : 1 }

再执行，可看到removeshard的挪动进度

mongos> db.runCommand({"removeshard" : "10.0.0.11:27011"}); { "msg" : "draining ongoing", "state" : "ongoing", "remaining" : { "chunks" : NumberLong(3), "dbs" : NumberLong(0) }, "ok" : 1 }

转载自http://www.elain.org

转载于:https://blog.51cto.com/dadloveu/787529