zookeeper集群搭建

zookeeper

zookeeper集群搭建

1.安装jdk

略

2.设置java堆栈大小。

差不多4g的机器设置heap大小为3g。为了避免内存swap。

3.安装zookeeper

http://hadoop.apache.org/zookeeper/releases.html

4.创建一个配置文件zoo.cfg

tickTime=2000
dataDir=/var/zookeeper/
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888

集群配置格式，server.id=host:port:port

5. myid文件

id号为1到255，必须配置在datadir下面，文件名为myid。

6. 启动zookeeper

bin/zkServer.sh start

7. zookeeper 命令

ZooKeeper Commands: The Four Letter Words
ZooKeeper responds to a small set of commands. Each command is composed of four letters. You issue the commands to ZooKeeper via telnet or nc, at the client port.

Three of the more interesting commands: "stat" gives some general information about the server and connected clients, while "srvr" and "cons" give extended details on server and connections respectively.

conf
New in 3.3.0: Print details about serving configuration.

cons
New in 3.3.0: List full connection/session details for all clients connected to this server. Includes information on numbers of packets received/sent, session id, operation latencies, last operation performed, etc...

crst
New in 3.3.0: Reset connection/session statistics for all connections.

dump
Lists the outstanding sessions and ephemeral nodes. This only works on the leader.

envi
Print details about serving environment

ruok
Tests if server is running in a non-error state. The server will respond with imok if it is running. Otherwise it will not respond at all.

A response of "imok" does not necessarily indicate that the server has joined the quorum, just that the server process is active and bound to the specified client port. Use "stat" for details on state wrt quorum and client connection information.

srst
Reset server statistics.

srvr
New in 3.3.0: Lists full details for the server.

stat
Lists brief details for the server and connected clients.

wchs
New in 3.3.0: Lists brief information on watches for the server.

wchc
New in 3.3.0: Lists detailed information on watches for the server, by session. This outputs a list of sessions(connections) with associated watches (paths). Note, depending on the number of watches this operation may be expensive (ie impact server performance), use it carefully.

wchp
New in 3.3.0: Lists detailed information on watches for the server, by path. This outputs a list of paths (znodes) with associated sessions. Note, depending on the number of watches this operation may be expensive (ie impact server performance), use it carefully.

Here's an example of the ruok command:

$ echo ruok | nc 127.0.0.1 5111
imok

很简单，可以用nc来发命令取zookeeper的信息。

8. data 目录

data目录下其实只有2个文件，

• myid ，上面介绍过了，集群id，1~255。
• snapshot. ,zookeeper 持久化的数据。

9. kazoo python库

# coding=utf-8
# kazoo库
# kazoo的一些特性
# A wide range of recipe implementations, like Lock, Election or Queue
# Data and Children Watchers
# Simplified Zookeeper connection state tracking
# Unified asynchronous API for use with greenlets or threads
# Support for gevent 0.13 and gevent 1.0
# Support for eventlet
# Support for Zookeeper 3.3 and 3.4 servers
# Integrated testing helpers for Zookeeper clusters
# Pure-Python based implementation of the wire protocol, avoiding all the memory leaks, lacking features, and debugging madness of the C library


from kazoo.client import KazooClient
import logging
from kazoo.client import KazooState

logging.basicConfig()

zk = KazooClient(hosts='127.0.0.1:2181')


def my_listener(state):
    if state == KazooState.LOST:
        # Register somewhere that the session was lost
        print "byebye"
    elif state == KazooState.SUSPENDED:
        # Handle being disconnected from Zookeeper
        pass
    else:
        pass
        # Handle being connected/reconnected to Zookeeper

zk.add_listener(my_listener)

zk.start()
# 创建节点
# ensure_path和create的区别，create可以创建值。
zk.ensure_path("/my/aca")
zk.create('/my/test', b'a value' )

# 读节点
# exists()
# get
# get_children()
if zk.exists("/my/test"):
    data, stat = zk.get("/my/test")
    print("Version: %s, data: %s" % (stat.version, data.decode("utf-8")))

# update node
# 调用set就可以了。
zk.set("/my/test", b"some data")
# 删除节点
zk.delete("/my/aca")
zk.stop()