ZooKeeper之Java API的基本使用以及常见应用场景的实现

记一次zookeeper连接慢的问题和解决方法 1. 环境 服务器:CentOS Linux release 7.9.2009 (Core) zookeeper:3.6.3 java:1.8.0_231 2. 问题 zookeeper客户端连接到zookeeper节点有20秒的延迟,而且每次都是20秒。自己编写客户端进行测试,通过日志看到在某个环节上会出现阻塞,导致延迟。 15:49:12.424 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, con 阅读详情

Java API的基本使用

添加依赖

<dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.7.0</version>
        </dependency>
复制代码

创建ZK连接

@SpringBootTest
public class ZKTest {

    /**
     * 获取zookeeper连接时所需要的服务器连接信息,格式为主机名:端口号;多个服务使用“,”分割
     */
    public final String CONNECT_STRING = "127.0.0.1:2181";

    /**
     * 请求会话超时时长
     */
    public final int SESSIONT_IMEOUT = 5000;

    private ZooKeeper zk;
    
    private String path = "/zk";
    
    private String sonPath = "/zk/son";
    
    private String value = "zk_Data";

    /**
     * 获取ZooKeeper链接
     *
     * @return
     * @throws Exception
     */
    @Before
    public void getZKConnection() throws Exception {
        zk = new ZooKeeper(CONNECT_STRING, SESSIONT_IMEOUT, null);
    }

    /**
     * 关闭ZK连接
     *
     * @throws InterruptedException
     */
    @After
    public void closeZKConnection() throws InterruptedException {
        zk.close();
    }
}
复制代码

创建持久不带序列编号的节点

@Test
    public void createZKNode1() throws Exception {
        // 四个参数: 节点路径,节点数据,权限信息,节点的类型
        zk.create(path, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
    }
复制代码

创建持久带序列编号的节点

@Test
    public void createZKNode2() throws Exception {
        // 四个参数: 节点路径,节点数据,权限信息,节点的类型
        zk.create(path, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL);
        zk.create(path, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL);
        zk.create(path, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT_SEQUENTIAL);
    }
复制代码

创建临时不带序列编号的节点

@Test
    public void createZKNode3() throws Exception {
        // 四个参数: 节点路径,节点数据,权限信息,节点的类型
        zk.create(path, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
    }
复制代码

创建临时带序列编号的节点

@Test
    public void createZKNode4() throws Exception {
        // 四个参数: 节点路径,节点数据,权限信息,节点的类型
        zk.create(path, value.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
        zk.create(path, value.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
        zk.create(path, value.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
    }
复制代码

查看节点数据

@Test
    public void getZNodeData() throws Exception {
        String znodeData = new String(zk.getData(path, false, null));
        System.out.println("查看节点数据:" + znodeData);
    }
复制代码

修改节点数据

@Test
    public void updateZNodeData() throws Exception {
        String value="update_data";
        zk.setData(path, value.getBytes(), -1);
        String newData = new String(zk.getData(path, false, null));
        if (newData.equals(value)) {
            System.out.println("修改节点数据: " + true);
        } else {
            System.out.println("修改节点数据: " + false);
        }
    }
复制代码

判断节点是否存在

@Test
    public void existsZNode() throws Exception {
        boolean existsZNode = (zk.exists(path, false) != null) ? true : false;
        System.out.println("判断节点是否存在: " + existsZNode);
    }
复制代码

查看节点的子节点列表

@Test
    public void getChildrenZNodes() throws Exception {
        zk.create(sonPath, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
        List<String> children = zk.getChildren(path, false);
        for (String child : children) {
            System.out.println("查看节点的子节点列表: " + child);
        }
    }
复制代码

删除节点

@Test
    public void deleteZNode() {
        try {
            zk.delete(path, -1);
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (KeeperException e) {
            e.printStackTrace();
        }
        System.out.println("删除znode节点成功");
    }
复制代码

级联删除节点

@Test
    public void deleteAllNode() throws Exception {
        zk.create(path, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        zk.create(sonPath, value.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        this.deleteAllNode(this.path);
    }

    /**
     * 删除一个节点,不管有有没有任何子节点
     *
     * @return
     * @throws Exception
     */
    public void deleteAllNode(String path) throws Exception {
        if (zk.exists(path, null) == null) {
            System.out.println("该路径 " + path + " 不存在");
        }

        List<String> children = zk.getChildren(path, null);
        if (children.size() == 0) {
            zk.delete(path, -1);
        } else {
            // 删除child路径下的所有子节点,但是并不删除child节点
            for (String child : children) {
                String deletePath = path + "/" + child;
                deleteAllNode(deletePath);
            }
            // 节点所有子节点删除之后,直接当前节点
            zk.delete(path, -1);
        }
    }
复制代码

级联创建节点

@Test
    public void createAllNode() throws Exception {
        this.createAllNode(this.sonPath);
    }

    /**
     * 级联创建任意节点
     *
     * @throws Exception
     */
    public boolean createAllNode(String znodePath) throws Exception {
        // 判断该节点是否存在,如果存在,则不创建
        if (zk.exists(znodePath, null) != null) {
            return false;
        } else {
            try {
                // 直接创建,捕捉异常,根据对应的异常如果是发现没有父节点,那么就创建父节点
                zk.create(znodePath, value.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
            } catch (KeeperException e) {
                // 截取父节点
                String parentPath = znodePath.substring(0, znodePath.lastIndexOf("/"));
                // 创建父节点
                createAllNode(parentPath);
                try {
                    // 父节点创建好后,创建该节点
                    zk.create(znodePath, value.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
                } catch (KeeperException e1) {
                    e1.printStackTrace();
                } catch (InterruptedException e1) {
                    e1.printStackTrace();
                }
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        return true;
    }
复制代码

清空子节点

@Test
    public void clearChildNode() throws Exception {
        this.clearChildNode(this.path);
    }

    /**
     * 清空子节点
     *
     * @throws Exception
     */
    public boolean clearChildNode(String path) throws Exception {
        List<String> children = zk.getChildren(path, null);
        for (String child : children) {
            String childNode = path + "/" + child;
            if (zk.getChildren(childNode, null).size() != 0) {
                clearChildNode(childNode);
            }
            zk.delete(childNode, -1);
        }
        return true;
    }
复制代码

实现节点监听

ZooKeeper可以作为注册中心,原因就在于其可以实现节点监听。

比如:在分布式系统中,一个服务启动后就可以向ZK注册中心的某个节点注册,记录服务器的信息,服务下线则删除节点下记录的服务器信息。

创建监听节点

private String sonPath = "/watch_zk/son";
    
    private String value = "zk_Data";
    
    private String watchPath = "/watch_zk";

    @Test
    public void createWatchNode() throws Exception {
        zk.create(watchPath, value.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
    }
复制代码

实现监听

public class ZKWatch {

    /**
     * 获取zookeeper连接时所需要的服务器连接信息,格式为主机名:端口号;多个服务使用“,”分割
     */
    public static final String CONNECT_STRING = "127.0.0.1:2181";

    /**
     * 请求会话超时时长
     */
    public static final int SESSIONT_IMEOUT = 5000;

    private static ZooKeeper zk;

    public static void main(String[] args) throws IOException, InterruptedException, KeeperException {
        String watchPath = "/watch_zk";
        zk = new ZooKeeper(CONNECT_STRING, SESSIONT_IMEOUT, new Watcher() {
            @Override
            public void process(WatchedEvent event) {
                // 事件类型
                Event.EventType type = event.getType();
                // 节点路径
                String path = event.getPath();
                System.out.println("节点路径 = " + path+" 事件类型 = " + type);

                try {
                    // 注册监听 节点的数据变化
                    // 事件: NodeDeleted, NodeDataChanged
                    // 触发: delete, setData
                    zk.getData(watchPath, true, null);

                    // 注册监听 子节点个数变化
                    // 事件: NodeChildrenChanged, NodeDeleted
                    // 触发: create/delete子节点, delete当前给定节点
                    zk.getChildren(watchPath, true);

                    // 注册监听 该节点的变化(被创建,被删除,数据被改变)
                    // 事件: NodeCreated, NodeDeleted, NodeDataChanged
                    // 触发: create,  delete,  setData
                    zk.exists(watchPath, true);
                } catch (Exception e) {
                    e.printStackTrace();
                }

                // 根据监听的相应的事件类型和节点,编写对应业务逻辑
                if (type == Event.EventType.NodeDeleted && event.getPath().equals(watchPath)) {
                    System.out.println("节点的数据变化---删除");
                } else if (type == Event.EventType.NodeDataChanged) {
                    System.out.println("节点的数据变化---修改");
                } else if (type == Event.EventType.NodeCreated) {
                    System.out.println("节点的数据变化---创建");
                } else if (type == Event.EventType.NodeChildrenChanged) {
                    System.out.println("子节点个数变化---创建/删除");
                } else if (type == Event.EventType.NodeDeleted) {
                    System.out.println("子节点个数变化---删除");
                }

            }
        });

        // 判断监听节点是否,不存在则创建
        if (zk.exists(watchPath, null) == null) {
            zk.create(watchPath, "watchData".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        }

        // 保证watcer线程一直工作
        Thread.sleep(Long.MAX_VALUE);

        // 启动监听就不会关闭zk链接
        // zk.close();
    }
}
复制代码

执行测试

@Test
    public void createZKNode() throws Exception {
        zk.setData(watchPath,value.getBytes(),-1);

        zk.create(sonPath, value.getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        zk.setData(sonPath,value.getBytes(),-1);
        zk.delete(sonPath,-1);
    }
复制代码
15:31:05.201 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=5000 watcher=com.example.demo.zk.ZKWatch$1@23e028a9
15:31:05.206 [main] INFO org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
15:31:05.576 [main] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 1048575 Bytes
15:31:05.618 [main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
15:31:05.624 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.SaslServerPrincipal - Canonicalized address to activate.navicat.com
15:31:05.625 [main-SendThread(127.0.0.1:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server activate.navicat.com/127.0.0.1:2181.
15:31:05.626 [main-SendThread(127.0.0.1:2181)] INFO org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error)
15:31:05.627 [main-SendThread(127.0.0.1:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established, initiating session, client: /127.0.0.1:2955, server: activate.navicat.com/127.0.0.1:2181
15:31:05.628 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Session establishment request sent on activate.navicat.com/127.0.0.1:2181
15:31:05.633 [main-SendThread(127.0.0.1:2181)] INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server activate.navicat.com/127.0.0.1:2181, session id = 0x1000f36f14d001b, negotiated timeout = 5000
节点路径 = null 事件类型 = None
15:31:05.638 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 1,4  replyHeader:: 1,300,0  request:: '/watch_zk,T  response:: #7a6b5f44617461,s{290,295,1661412554041,1661412576797,1,2,0,0,7,0,298} 
15:31:05.642 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 2,8  replyHeader:: 2,300,0  request:: '/watch_zk,T  response:: v{} 
15:31:05.644 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 3,3  replyHeader:: 3,300,0  request:: '/watch_zk,T  response:: s{290,295,1661412554041,1661412576797,1,2,0,0,7,0,298} 
15:31:07.311 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x1000f36f14d001b after 0ms.
15:31:08.977 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x1000f36f14d001b after 0ms.
15:31:09.104 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d001b
15:31:09.105 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeDataChanged path:/watch_zk for session id 0x1000f36f14d001b
节点路径 = /watch_zk 事件类型 = NodeDataChanged
15:31:09.107 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 4,4  replyHeader:: 4,303,0  request:: '/watch_zk,T  response:: #7a6b5f44617461,s{290,303,1661412554041,1661412669092,2,2,0,0,7,0,298} 
15:31:09.108 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 5,8  replyHeader:: 5,303,0  request:: '/watch_zk,T  response:: v{} 
15:31:09.109 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 6,3  replyHeader:: 6,303,0  request:: '/watch_zk,T  response:: s{290,303,1661412554041,1661412669092,2,2,0,0,7,0,298} 
节点的数据变化---修改
15:31:09.113 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d001b
15:31:09.114 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/watch_zk for session id 0x1000f36f14d001b
节点路径 = /watch_zk 事件类型 = NodeChildrenChanged
15:31:09.114 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 7,4  replyHeader:: 7,304,0  request:: '/watch_zk,T  response:: #7a6b5f44617461,s{290,303,1661412554041,1661412669092,2,3,0,0,7,1,304} 
15:31:09.117 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 8,8  replyHeader:: 8,305,0  request:: '/watch_zk,T  response:: v{'son} 
15:31:09.119 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 9,3  replyHeader:: 9,305,0  request:: '/watch_zk,T  response:: s{290,303,1661412554041,1661412669092,2,3,0,0,7,1,304} 
子节点个数变化---创建/删除
15:31:09.119 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d001b
15:31:09.119 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/watch_zk for session id 0x1000f36f14d001b
节点路径 = /watch_zk 事件类型 = NodeChildrenChanged
15:31:09.120 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 10,4  replyHeader:: 10,306,0  request:: '/watch_zk,T  response:: #7a6b5f44617461,s{290,303,1661412554041,1661412669092,2,4,0,0,7,0,306} 
15:31:09.122 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 11,8  replyHeader:: 11,307,0  request:: '/watch_zk,T  response:: v{} 
15:31:09.123 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d001b, packet:: clientPath:null serverPath:null finished:false header:: 12,3  replyHeader:: 12,307,0  request:: '/watch_zk,T  response:: s{290,303,1661412554041,1661412669092,2,4,0,0,7,0,306} 
子节点个数变化---创建/删除
15:31:10.789 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x1000f36f14d001b after 0ms.
复制代码

实现分布式共享锁

分布式业务系统中,可能会存在某种资源,需要被整个系统的各台服务器共享访问,但是只允许一台服务器同时访问,此时就可以使用ZK实现分布式共享锁。

编写共享锁逻辑

public class ZkLock {

	/**
	 * 获取zookeeper连接时所需要的服务器连接信息,格式为主机名:端口号;多个服务使用“,”分割
	 */
	public static final String CONNECT_STRING = "127.0.0.1:2181";

	/**
	 * 请求会话超时时长
	 */
	public static final int SESSIONT_IMEOUT = 5000;
	/**
	 * 定义父节点锁
	 */
	private static final String PARENT_NODE = "/parent_locks";
	/**
	 * 定义客户端节点
	 * 会有多个客户端节点抢着注册,且是有序编号的客户端节点
	 * 谁拿到当前子客户端节点则意外拿到锁
	 */
	private static final String SUB_NODE = "/sub_client";
	/**
	 * ZK连接
	 */
	static ZooKeeper zk = null;
	/**
	 * 当前注册子客户端
	 */
	private static String currentPath = "";

	public static void main(String[] args) throws Exception {
		ZkLock zklock = new ZkLock();
		//  获取zookeeper链接
		zklock.getZookeeperConnect();

		// 父节点是否存在,不存在则创建
		Stat exists = zk.exists(PARENT_NODE, false);
		if (exists == null) {
			zk.create(PARENT_NODE, PARENT_NODE.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
		}

		// 监听父节点
		zk.getChildren(PARENT_NODE, true);

		// 往父节点下注册节点,注册临时节点,好处:当宕机或者断开链接时该节点自动删除
		currentPath = zk.create(PARENT_NODE + SUB_NODE, SUB_NODE.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);

		// 测试故一直监听
		Thread.sleep(Long.MAX_VALUE);
		// 关闭zk链接
		zk.close();
	}

	/**
	 * 获取zookeeper链接,同时进行监听
	 */
	public void getZookeeperConnect() throws Exception {
		zk = new ZooKeeper(CONNECT_STRING, SESSIONT_IMEOUT, new Watcher() {
			@Override
			public void process(WatchedEvent event) {
				// 子节点变化,且是监听的路径
				if (event.getType() == EventType.NodeChildrenChanged && event.getPath().equals(PARENT_NODE)) {
					try {
						// 获取父节点下所有子节点, 并继续监听
						List<String> childrenNodes = zk.getChildren(PARENT_NODE, true);
						// 将创建的子客户端节点进行排序,自然排序,即从小到大
						Collections.sort(childrenNodes);
						System.out.println("当前注册子客户端锁节点:"+currentPath);
						System.out.println("父节点下已注册子客户端节点: "+childrenNodes);
						// 如果当前子客户端节点与父节点下所以子节点客户端经过排序后的第一个子客户端相等,则进行业务处理
						if ((PARENT_NODE + "/" + childrenNodes.get(0)).equals(currentPath)) {
							System.out.println("从父节点下获取排序最小节点:"+ childrenNodes.get(0)+" 执行业务");
							// 业务处理
							handleBusiness(currentPath);
						}else{
							System.out.println("注册子客户端节点: "+childrenNodes.stream().skip(1).collect(Collectors.toList())+"获取锁失败...");
						}
					} catch (Exception e) {
						e.printStackTrace();
					}
				}
			}
		});
	}

	public void handleBusiness(String deletePath) throws Exception {
		System.out.println("业务处理开始....");
		// 模拟业务耗时
		Thread.sleep(new Random().nextInt(5000));
		// 业务完成删除当前子客户端节点
		zk.delete(deletePath, -1);
		System.out.println("业务处理结束,删除锁节点:"+deletePath);

		// 给与一定时间,再次打乱服务器抢子客户端节点锁的顺序
		Thread.sleep(new Random().nextInt(4000));
		// 抢资源锁
		currentPath = zk.create(PARENT_NODE + SUB_NODE, SUB_NODE.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL);
		System.out.println("子客户端节点获取锁:"+currentPath);
	}
}

复制代码

执行测试

同时启动3个线程执行上述代码。

17:06:35.439 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x1000f36f14d0033 after 0ms.
17:06:35.573 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d0033
17:06:35.573 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/parent_locks for session id 0x1000f36f14d0033
17:06:35.573 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d0033, packet:: clientPath:null serverPath:null finished:false header:: 11,8  replyHeader:: 11,827,0  request:: '/parent_locks,T  response:: v{'sub_client0000000233,'sub_client0000000231,'sub_client0000000232} 
当前注册子客户端锁节点:/parent_locks/sub_client0000000232
父节点下已注册子客户端节点: [sub_client0000000231, sub_client0000000232, sub_client0000000233]
注册子客户端节点: [sub_client0000000232, sub_client0000000233]获取锁失败...
17:06:37.018 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d0033
17:06:37.019 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/parent_locks for session id 0x1000f36f14d0033
17:06:37.019 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x1000f36f14d0033 after 0ms.
17:06:37.019 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d0033, packet:: clientPath:null serverPath:null finished:false header:: 12,8  replyHeader:: 12,828,0  request:: '/parent_locks,T  response:: v{'sub_client0000000233,'sub_client0000000232} 
当前注册子客户端锁节点:/parent_locks/sub_client0000000232
父节点下已注册子客户端节点: [sub_client0000000232, sub_client0000000233]
从父节点下获取排序最小节点:sub_client0000000232 执行业务
业务处理开始....
17:06:37.228 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d0033
17:06:37.228 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/parent_locks for session id 0x1000f36f14d0033
17:06:37.228 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d0033, packet:: clientPath:null serverPath:null finished:false header:: 13,2  replyHeader:: 13,829,0  request:: '/parent_locks/sub_client0000000232,-1  response:: null
业务处理结束,删除锁节点:/parent_locks/sub_client0000000232
17:06:38.893 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x1000f36f14d0033 after 0ms.
17:06:39.329 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d0033, packet:: clientPath:null serverPath:null finished:false header:: 14,1  replyHeader:: 14,830,0  request:: '/parent_locks/sub_client,#2f7375625f636c69656e74,v{s{31,s{'world,'anyone}}},3  response:: '/parent_locks/sub_client0000000234 
子客户端节点获取锁:/parent_locks/sub_client0000000234
17:06:39.330 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d0033, packet:: clientPath:null serverPath:null finished:false header:: 15,8  replyHeader:: 15,830,0  request:: '/parent_locks,T  response:: v{'sub_client0000000233,'sub_client0000000234} 
当前注册子客户端锁节点:/parent_locks/sub_client0000000234
父节点下已注册子客户端节点: [sub_client0000000233, sub_client0000000234]
注册子客户端节点: [sub_client0000000234]获取锁失败...
17:06:40.597 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d0033
17:06:40.598 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/parent_locks for session id 0x1000f36f14d0033
17:06:40.598 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x1000f36f14d0033 after 0ms.
17:06:40.598 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000f36f14d0033, packet:: clientPath:null serverPath:null finished:false header:: 16,8  replyHeader:: 16,831,0  request:: '/parent_locks,T  response:: v{'sub_client0000000235,'sub_client0000000233,'sub_client0000000234} 
当前注册子客户端锁节点:/parent_locks/sub_client0000000234
父节点下已注册子客户端节点: [sub_client0000000233, sub_client0000000234, sub_client0000000235]
注册子客户端节点: [sub_client0000000234, sub_client0000000235]获取锁失败...
17:06:41.496 [main-SendThread(127.0.0.1:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x1000f36f14d0033
17:06:41.496 [main-SendThread(127.0.0.1:2181)]
复制代码

实现配置功能

实现客户端对配置信息进行 添加 修改 删除的操作

实现逻辑

public class DataNodeConfig {

    /**
     * 获取zookeeper连接时所需要的服务器连接信息,格式为主机名:端口号;多个服务使用“,”分割
     */
    public static final String CONNECT_STRING = "node01:2181,node02:2182,node03:2183";
    /**
     * 请求会话超时时长
     */
    public static final int SESSIONT_IMEOUT = 5000;
    /**
     * 配置节点
     */
    private static final String PARENT_NODE = "/config";
    static List<String> oldChildNodeList = null;
    static ZooKeeper zk = null;

    public static void main(String[] args) throws Exception {
        zk = new ZooKeeper(CONNECT_STRING, SESSIONT_IMEOUT, new Watcher() {
            @Override
            public void process(WatchedEvent event) {
                EventType type = event.getType();
                String path = event.getPath();
                KeeperState state = event.getState();
                /**
                 * 排除获取连接时的触发: 即type:None path: null
                 */
                if (state.getIntValue() == 3 && path != null) {
                    // 子节点的列表信息
                    List<String> newChildNodeList = null;
                    try {
                        newChildNodeList = zk.getChildren(PARENT_NODE, true);
                    } catch (KeeperException e1) {
                        e1.printStackTrace();
                    } catch (InterruptedException e1) {
                        e1.printStackTrace();
                    }

                    /**
                     * 虽然此处响应的逻辑有可能是 添加 或者删除, 但是经过判断之后,如果是删除,不做任何处理
                     * 如果添加,那么做对应的业务处理
                     */
                    if (path.equals(PARENT_NODE) && type == EventType.NodeChildrenChanged) {
                        // 添加
                        if (oldChildNodeList.size() < newChildNodeList.size()) {
                            List<String> newList = newChildNodeList.stream().filter(s -> !oldChildNodeList.contains(s)).collect(Collectors.toList());
                            String newAdd = newList.get(0);
                            byte[] data = null;
                            try {
                                data = zk.getData(PARENT_NODE + "/" + newAdd, true, null);
                            } catch (KeeperException e) {
                                e.printStackTrace();
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            }
                            System.out.println("新曾一项配置: key=" + newAdd + ", value=" + new String(data));
                        }

                        /**
                         * 父节点下的一个子节点的删除变化事件
                         */
                    } else {
                        for (String child : oldChildNodeList) {
                            String childPath = PARENT_NODE + "/" + child;
                            if (path.equals(childPath) && type == EventType.NodeDeleted) {
                                List<String> finalNewChildNodeList = newChildNodeList;
                                List<String> newList = oldChildNodeList.stream().filter(s -> !finalNewChildNodeList.contains(s)).collect(Collectors.toList());
                                String delete = newList.get(0);
                                System.out.println("删除一项配置: key=" + delete);
                            }
                        }
                    }

                    /**
                     * 父节点下的一个子节点的数据修改变化事件
                     */
                    for (String child : oldChildNodeList) {
                        String childPath = PARENT_NODE + "/" + child;
                        if (path.equals(childPath) && type == EventType.NodeDataChanged) {
                            byte[] data = null;
                            try {
                                data = zk.getData(childPath, true, null);
                            } catch (KeeperException e) {
                                e.printStackTrace();
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            }
                            String newValue = new String(data);
                            System.out.println("修改一项配置: key=" + child + ", newValue=" + newValue);
                            break;
                        }
                    }
                    /**
                     * 新老子节点列表的迭代
                     */
                    oldChildNodeList = newChildNodeList;
                }
            }
        });

        Stat exists = zk.exists(PARENT_NODE, null);
        if (exists == null) {
            zk.create(PARENT_NODE, "".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        }

        /**
         * 添加监听
         *
         * 1、/config节点的NodeChildrenChanged
         *
         * 2、/config节点下的子节点的NodeDataChanged
         */
        oldChildNodeList = zk.getChildren(PARENT_NODE, true);
        for (String child : oldChildNodeList) {
            String childPath = PARENT_NODE + "/" + child;
            // 添加子节点的 NodeDataChanged
            zk.getData(childPath, true, null);
        }

        Thread.sleep(Long.MAX_VALUE);
    }
}

复制代码

执行测试

public class Client {

    /**
     * 获取zookeeper连接时所需要的服务器连接信息,格式为主机名:端口号;多个服务使用“,”分割
     */
    public static final String CONNECT_STRING = "node01:2181,node02:2182,node03:2183";
    /**
     * 请求会话超时时长
     */
    public static final int SESSIONT_IMEOUT = 5000;
    /**
     * 配置节点
     */
    private static final String PARENT_NODE = "/config";

    public static void main(String[] args) throws Exception {
        /**
         * 获取zookeeper连接
         */
        ZooKeeper zk = new ZooKeeper(CONNECT_STRING, SESSIONT_IMEOUT, null);
        /**
         * 判断/config节点是否存在,不存在则创建
         */
        Stat exists = zk.exists(PARENT_NODE, null);
        if (exists == null) {
            zk.create(PARENT_NODE, "".getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        }

        /**
         * 实现 添加 修改 删除
         */
        String path = PARENT_NODE + "/" + "key";
        zk.create(path, "value".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);

        Thread.sleep(5 * 1000);
        zk.setData(path, "value".getBytes(), -1);

        Thread.sleep(5 * 1000);
        zk.delete(path, -1);

        Thread.sleep(100 * 1000);
        zk.close();
    }
}
复制代码
22:41:21.989 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010ac8dab0004, packet:: clientPath:null serverPath:null finished:false header:: 3,8  replyHeader:: 3,17179869263,0  request:: '/config,T  response:: v{'key} 
22:41:21.997 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010ac8dab0004, packet:: clientPath:null serverPath:null finished:false header:: 4,4  replyHeader:: 4,17179869263,0  request:: '/config/key,T  response:: #76616c7565,s{17179869263,17179869263,1661438481952,1661438481952,0,0,0,0,5,0,17179869263} 
新曾一项配置: key=key, value=value
22:41:23.664 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
22:41:25.330 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
22:41:26.960 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x20010ac8dab0004
22:41:26.960 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeDataChanged path:/config/key for session id 0x20010ac8dab0004
22:41:26.960 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
22:41:26.960 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010ac8dab0004, packet:: clientPath:null serverPath:null finished:false header:: 5,8  replyHeader:: 5,17179869264,0  request:: '/config,T  response:: v{'key} 
22:41:26.961 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010ac8dab0004, packet:: clientPath:null serverPath:null finished:false header:: 6,4  replyHeader:: 6,17179869264,0  request:: '/config/key,T  response:: #76616c7565,s{17179869263,17179869264,1661438481952,1661438486958,1,0,0,0,5,0,17179869263} 
修改一项配置: key=key, newValue=value
22:41:28.628 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
22:41:30.294 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
22:41:31.960 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
22:41:31.967 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x20010ac8dab0004
22:41:31.967 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeDeleted path:/config/key for session id 0x20010ac8dab0004
22:41:31.967 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x20010ac8dab0004
22:41:31.967 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/config for session id 0x20010ac8dab0004
22:41:31.967 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010ac8dab0004, packet:: clientPath:null serverPath:null finished:false header:: 7,8  replyHeader:: 7,17179869265,0  request:: '/config,T  response:: v{} 
删除一项配置: key=key
22:41:31.969 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010ac8dab0004, packet:: clientPath:null serverPath:null finished:false header:: 8,8  replyHeader:: 8,17179869265,0  request:: '/config,T  response:: v{} 
22:41:33.635 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
22:41:35.301 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010ac8dab0004 after 0ms.
复制代码

实现同步队列

实现逻辑

public class SyncQueueClient {

    /**
     * 获取zookeeper连接时所需要的服务器连接信息,格式为主机名:端口号;多个服务使用“,”分割
     */
    public static final String CONNECT_STRING = "node01:2181,node02:2182,node03:2183";
    /**
     * 请求会话超时时长
     */
    public static final int SESSIONT_IMEOUT = 5000;
    /**
     * 根节点
     */
    private static final String PARENT_NODE = "/syncQueue";
    /**
     * 需要队列元素个数
     */
    private static final int NEED_QUEUE = 5;

    private static ZooKeeper zk = null;

    private static int count = 0;

    public static void main(String[] args) throws Exception {
        // 获取zookeeper链接
        zk = new ZooKeeper(CONNECT_STRING, SESSIONT_IMEOUT, new Watcher() {

            @Override
            public void process(WatchedEvent event) {
                String path = event.getPath();
                EventType et = event.getType();
                // 跳过初始化监听事件
                if (path == null) {
                    return;
                }

                // 根节点下子节点数量变化
                if (path.equals(PARENT_NODE) && et == EventType.NodeChildrenChanged) {
                    // 判断队列元素是否达到一定数量
                    try {
                        List<String> children = zk.getChildren(PARENT_NODE, true);
                        int queueNumber = children.size();
                        if (queueNumber == NEED_QUEUE) {
                            for (String child : children) {
                                consume(child);
                            }
                        } else if (queueNumber < NEED_QUEUE) {
                            if (count == NEED_QUEUE) {
                                produce(true);
                            } else {
                                produce(false);
                            }
                        }
                        count = queueNumber;
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            }
        });

        // 判断父节点是否存在,不存在则创建
        Stat exists = zk.exists(PARENT_NODE, false);
        if (exists == null) {
            zk.create(PARENT_NODE, PARENT_NODE.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
        }

        // 监听根节点
        zk.getChildren(PARENT_NODE, true);

        Thread.sleep(1000);
        produce(true);

        Thread.sleep(Long.MAX_VALUE);
        zk.close();
    }

    /**
     * 首先,我们的这个程序会一直监听PARENT_NODE,等待所监听的节点下的队列成员是否全部到达 如果满足条件,该业务方法才会执行
     * 这个条件是:
     * 1、该业务程序收到的某一个节点的某一个事件要满足我们监听的该节点
     * 2、队列的所有成员都达到
     */
    public static void produce(boolean tag) throws InterruptedException, KeeperException {
        String path = PARENT_NODE + "/" + UUID.randomUUID();
        if (tag) {
            System.out.println("队列为空,立即生产元素...");
            zk.create(path, path.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
        } else {
            System.out.println("队列元素不足最低消费数量:" + NEED_QUEUE + ",准备生产...");
            Thread.sleep(2000);
            zk.create(path, path.getBytes(), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
        }


    }

    public static void consume(String path) throws InterruptedException, KeeperException {
        zk.delete(PARENT_NODE + "/" + path, -1);
        System.out.println("消费队列中的元素: " + path);
    }
}
复制代码

执行测试

22:47:42.576 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 2,8  replyHeader:: 2,21474840890,0  request:: '/syncQueue,T  response:: v{} 
队列为空,立即生产元素...
22:47:43.586 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x10015dc638f0008
22:47:43.587 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/syncQueue for session id 0x10015dc638f0008
22:47:43.618 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 3,1  replyHeader:: 3,21474840891,0  request:: '/syncQueue/f5fbb385-5692-4a13-9511-e42826a6cac1,#2f73796e6351756575652f66356662623338352d353639322d346131332d393531312d653432383236613663616331,v{s{31,s{'world,'anyone}}},1  response:: '/syncQueue/f5fbb385-5692-4a13-9511-e42826a6cac1 
22:47:43.618 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x10015dc638f0008 after 34ms.
22:47:43.619 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 4,8  replyHeader:: 4,21474840891,0  request:: '/syncQueue,T  response:: v{'f5fbb385-5692-4a13-9511-e42826a6cac1} 
队列元素不足最低消费数量:5,准备生产...
22:47:45.285 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x10015dc638f0008 after 0ms.
22:47:45.621 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x10015dc638f0008
22:47:45.621 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/syncQueue for session id 0x10015dc638f0008
22:47:45.621 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 5,1  replyHeader:: 5,21474840892,0  request:: '/syncQueue/2f6f7502-cd3c-481e-aa2e-25fc4ddce589,#2f73796e6351756575652f32663666373530322d636433632d343831652d616132652d323566633464646365353839,v{s{31,s{'world,'anyone}}},1  response:: '/syncQueue/2f6f7502-cd3c-481e-aa2e-25fc4ddce589 
22:47:45.622 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 6,8  replyHeader:: 6,21474840892,0  request:: '/syncQueue,T  response:: v{'2f6f7502-cd3c-481e-aa2e-25fc4ddce589,'f5fbb385-5692-4a13-9511-e42826a6cac1} 
队列元素不足最低消费数量:5,准备生产...
22:47:47.289 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x10015dc638f0008 after 0ms.
22:47:47.624 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x10015dc638f0008
22:47:47.624 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/syncQueue for session id 0x10015dc638f0008
22:47:47.625 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 7,1  replyHeader:: 7,21474840893,0  request:: '/syncQueue/67816667-1952-4480-adbb-f5c4766a0998,#2f73796e6351756575652f36373831363636372d313935322d343438302d616462622d663563343736366130393938,v{s{31,s{'world,'anyone}}},1  response:: '/syncQueue/67816667-1952-4480-adbb-f5c4766a0998 
22:47:47.625 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 8,8  replyHeader:: 8,21474840893,0  request:: '/syncQueue,T  response:: v{'2f6f7502-cd3c-481e-aa2e-25fc4ddce589,'f5fbb385-5692-4a13-9511-e42826a6cac1,'67816667-1952-4480-adbb-f5c4766a0998} 
队列元素不足最低消费数量:5,准备生产...
22:47:49.293 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x10015dc638f0008 after 0ms.
22:47:49.628 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x10015dc638f0008
22:47:49.628 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/syncQueue for session id 0x10015dc638f0008
22:47:49.629 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 9,1  replyHeader:: 9,21474840894,0  request:: '/syncQueue/e956635a-259f-4695-8ea4-b7ea8ea8398f,#2f73796e6351756575652f65393536363335612d323539662d343639352d386561342d623765613865613833393866,v{s{31,s{'world,'anyone}}},1  response:: '/syncQueue/e956635a-259f-4695-8ea4-b7ea8ea8398f 
22:47:49.630 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 10,8  replyHeader:: 10,21474840894,0  request:: '/syncQueue,T  response:: v{'2f6f7502-cd3c-481e-aa2e-25fc4ddce589,'f5fbb385-5692-4a13-9511-e42826a6cac1,'67816667-1952-4480-adbb-f5c4766a0998,'e956635a-259f-4695-8ea4-b7ea8ea8398f} 
队列元素不足最低消费数量:5,准备生产...
22:47:51.296 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x10015dc638f0008 after 0ms.
22:47:51.631 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x10015dc638f0008
22:47:51.631 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/syncQueue for session id 0x10015dc638f0008
22:47:51.631 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 11,1  replyHeader:: 11,21474840895,0  request:: '/syncQueue/65bb9cc4-71da-412a-b105-39a4738f0e60,#2f73796e6351756575652f36356262396363342d373164612d343132612d623130352d333961343733386630653630,v{s{31,s{'world,'anyone}}},1  response:: '/syncQueue/65bb9cc4-71da-412a-b105-39a4738f0e60 
22:47:51.632 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 12,8  replyHeader:: 12,21474840895,0  request:: '/syncQueue,T  response:: v{'2f6f7502-cd3c-481e-aa2e-25fc4ddce589,'f5fbb385-5692-4a13-9511-e42826a6cac1,'65bb9cc4-71da-412a-b105-39a4738f0e60,'67816667-1952-4480-adbb-f5c4766a0998,'e956635a-259f-4695-8ea4-b7ea8ea8398f} 
22:47:51.635 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x10015dc638f0008
22:47:51.635 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/syncQueue for session id 0x10015dc638f0008
22:47:51.635 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 13,2  replyHeader:: 13,21474840896,0  request:: '/syncQueue/2f6f7502-cd3c-481e-aa2e-25fc4ddce589,-1  response:: null
消费队列中的元素: 2f6f7502-cd3c-481e-aa2e-25fc4ddce589
22:47:51.636 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 14,2  replyHeader:: 14,21474840897,0  request:: '/syncQueue/f5fbb385-5692-4a13-9511-e42826a6cac1,-1  response:: null
消费队列中的元素: f5fbb385-5692-4a13-9511-e42826a6cac1
22:47:51.638 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 15,2  replyHeader:: 15,21474840898,0  request:: '/syncQueue/65bb9cc4-71da-412a-b105-39a4738f0e60,-1  response:: null
消费队列中的元素: 65bb9cc4-71da-412a-b105-39a4738f0e60
22:47:51.639 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 16,2  replyHeader:: 16,21474840899,0  request:: '/syncQueue/67816667-1952-4480-adbb-f5c4766a0998,-1  response:: null
消费队列中的元素: 67816667-1952-4480-adbb-f5c4766a0998
22:47:51.641 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 17,2  replyHeader:: 17,21474840900,0  request:: '/syncQueue/e956635a-259f-4695-8ea4-b7ea8ea8398f,-1  response:: null
消费队列中的元素: e956635a-259f-4695-8ea4-b7ea8ea8398f
22:47:51.641 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 18,8  replyHeader:: 18,21474840900,0  request:: '/syncQueue,T  response:: v{} 
队列为空,立即生产元素...
22:47:51.642 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x10015dc638f0008
22:47:51.642 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/syncQueue for session id 0x10015dc638f0008
22:47:51.643 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 19,1  replyHeader:: 19,21474840901,0  request:: '/syncQueue/717427e7-6a2b-45bb-a2dd-c1b03f5f9669,#2f73796e6351756575652f37313734323765372d366132622d343562622d613264642d633162303366356639363639,v{s{31,s{'world,'anyone}}},1  response:: '/syncQueue/717427e7-6a2b-45bb-a2dd-c1b03f5f9669 
22:47:51.643 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x10015dc638f0008, packet:: clientPath:null serverPath:null finished:false header:: 20,8  replyHeader:: 20,21474840901,0  request:: '/syncQueue,T  response:: v{'717427e7-6a2b-45bb-a2dd-c1b03f5f9669} 
队列元素不足最低消费数量:5,准备生产...
22:47:53.309 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x10015dc638f0008 after 0ms.
复制代码

实现集群高可用

高可用实现

public class MasterHA {

    private static ZooKeeper zk = null;
    /**
     * 获取zookeeper连接时所需要的服务器连接信息,格式为主机名:端口号;多个服务使用“,”分割
     */
    public static final String CONNECT_STRING = "node01:2181,node02:2182,node03:2183";

    /**
     * 节点node01 node02 node03
     */
    private static final String HOSTNAME = "node01";

    /**
     * 请求会话超时时长
     */
    public static final int SESSIONT_IMEOUT = 5000;
    /**
     * 集群根节点
     */
    private static final String PARENT = "/cluster_ha";
    /**
     * 主节点根
     */
    private static final String ACTIVE = PARENT + "/active";
    /**
     * 从节点根
     */
    private static final String STANDBY = PARENT + "/standby";

    /**
     * 主节点
     */
    private static final String activeMasterPath = ACTIVE + "/" + HOSTNAME;
    /**
     * 从节点
     */
    private static final String standByMasterPath = STANDBY + "/" + HOSTNAME;
    /**
     * 锁节点
     */
    private static final String LOCK = PARENT + "/lock";

    /**
     * 临时节点类型
     */
    private static final CreateMode CME = CreateMode.EPHEMERAL;
    /**
     * 持久节点类型
     */
    private static final CreateMode CMP = CreateMode.PERSISTENT;

    public static void main(String[] args) throws Exception {
        System.out.println("-------------------------1.创建连接-------------------------");
        zk = new ZooKeeper(CONNECT_STRING, SESSIONT_IMEOUT, new Watcher() {
            @Override
            public void process(WatchedEvent event) {
                String path = event.getPath();
                EventType type = event.getType();
                if (path == null) {
                    return;
                }

                // 是ACTIVE根节点,且该节点下的主节点被删除
                if (path.equals(ACTIVE) && type == EventType.NodeChildrenChanged) {
                    System.out.println("-------------------------4.ACTIVE根节点下的主节点被删除-------------------------");
                    // active节点下的active master节点被删除后,自己就参与竞选active
                    if (getChildrenNumber(ACTIVE) == 0) {
                        System.out.println("-------------------------4-1." + HOSTNAME + " 准备参与竞选-------------------------");
                        // 注册一把独占锁,多个standby角色,谁注册成功,谁就应该切换成为active状态
                        try {
                            zk.exists(LOCK, true);
                        } catch (KeeperException e) {
                            e.printStackTrace();
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                        // 创建节点, 数据存储对应节点信息
                        createZNode(LOCK, HOSTNAME, CME, "lock");
                    } else {
                        System.out.println("-------------------------4-2.ACTIVE根节点下的主节点刚被创建,放弃任何操作-------------------------");
                    }

                    // 做到循环监听
                    try {
                        zk.getChildren(ACTIVE, true);
                    } catch (KeeperException e) {
                        e.printStackTrace();
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }

                } else if (path.equals(LOCK) && type == EventType.NodeCreated) {
                    System.out.println("-------------------------5.锁被创建-------------------------");
                    // 获取节点数据
                    String nodeData = null;
                    try {
                        byte[] data = zk.getData(LOCK, false, null);
                        nodeData = new String(data);
                    } catch (Exception e) {
                    }

                    // 判断当前节点是否是自身创建,如果是就切换自己的状态为active
                    if (nodeData.equals(HOSTNAME)) {
                        createZNode(activeMasterPath, HOSTNAME, CME, null);
                        if (exists(standByMasterPath)) {
                            System.out.println("-------------------------5-1.当前节点" + HOSTNAME + "持有锁,切换状态为主节点-------------------------");
                            deleteZNode(standByMasterPath);
                        } else {
                            System.out.println("-------------------------5-2." + HOSTNAME + "成为主节点-------------------------");
                        }
                    } else {
                        System.out.println("-------------------------5-3.当前节点" + HOSTNAME + "未持有锁,放弃任务操作-------------------------");
                    }

                }
            }
        });


        System.out.println("-------------------------2.创建PARENT ACTIVE  STANDBY等根节点-------------------------");
        if (!exists(PARENT)) {
            createZNode(PARENT, PARENT, CMP, null);
        }
        if (!exists(ACTIVE)) {
            createZNode(ACTIVE, ACTIVE, CMP, null);
        }
        if (!exists(STANDBY)) {
            createZNode(STANDBY, STANDBY, CMP, null);
        }

        System.out.println("-------------------------3.判断ACTIVE节点下是否有子节点(主节点)-------------------------");
        // 判断ACTIVE节点下是否有子节点, 如果有则必定有主节点存在,没有则注册一把锁, 让自己竞选主节点
        if (getChildrenNumber(ACTIVE) == 0) {
            System.out.println("-------------------------3-1.没有子节点(主节点),准备争抢创建锁-------------------------");
            // 注册监听
            zk.exists(LOCK, true);
            // 创建争抢锁
            createZNode(LOCK, HOSTNAME, CME, null);
        } else {
            System.out.println("-------------------------3-2.存在子节点(主节点)," + HOSTNAME + "自动创建成为从节点-------------------------");
            createZNode(standByMasterPath, HOSTNAME, CME, null);
            // 注册监听, 监听active下子节点个数变化
            zk.getChildren(ACTIVE, true);
        }

        // 阻塞一直运行
        Thread.sleep(Long.MAX_VALUE);
    }

    private static void deleteZNode(String standbymasterpath) {
        try {
            zk.delete(standbymasterpath, -1);
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (KeeperException e) {
            e.printStackTrace();
        }
    }

    /**
     * 获取子节点数量
     *
     * @param path
     * @return
     */
    public static int getChildrenNumber(String path) {
        int number = 0;
        try {
            number = zk.getChildren(path, null).size();
        } catch (KeeperException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        return number;
    }

    /**
     * 判断节点是否存在
     */
    public static boolean exists(String path) {
        Stat exists = null;
        try {
            exists = zk.exists(path, null);
        } catch (KeeperException e) {
        } catch (InterruptedException e) {
        }
        if (exists == null) {
            return false;
        } else {
            return true;
        }
    }

    /**
     * 创建节点
     */
    public static void createZNode(String path, String data, CreateMode cm, String tag) {
        try {
            zk.create(path, data.getBytes(), Ids.OPEN_ACL_UNSAFE, cm);
        } catch (Exception e) {
            System.out.println("节点: " + path + " 创建失败或者节点已经存在");
            if (tag.equals("lock")) {
                System.out.println(HOSTNAME + " 未抢到锁,等待....");
            }
        }
    }
}

复制代码

启动node01

-------------------------1.创建连接-------------------------
21:39:42.899 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 2021-03-17 09:46 UTC
21:39:43.351 [main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
-------------------------2.创建PARENT ACTIVE  STANDBY等根节点-------------------------
21:39:43.357 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.SaslServerPrincipal - Canonicalized address to activate.navicat.com
21:39:43.359 [main-SendThread(node02:2182)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server node02/127.0.0.1:2182.
21:39:43.381 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0003, packet:: clientPath:null serverPath:null finished:false header:: 3,3  replyHeader:: 3,8589934695,0  request:: '/cluster_ha/standby,F  response:: s{8589934621,8589934621,1661429857799,1661429857799,0,22,0,0,19,0,8589934694} 
-------------------------3.判断ACTIVE节点下是否有子节点(主节点)-------------------------
21:39:43.384 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0003, packet:: clientPath:null serverPath:null finished:false header:: 4,8  replyHeader:: 4,8589934695,0  request:: '/cluster_ha/active,F  response:: v{} 
-------------------------3-1.没有子节点(主节点),准备争抢创建锁-------------------------
21:39:43.386 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0003, packet:: clientPath:null serverPath:null finished:false header:: 5,3  replyHeader:: 5,8589934695,-101  request:: '/cluster_ha/lock,T  response::  
21:39:43.393 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x20010446d6e0003
21:39:43.394 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeCreated path:/cluster_ha/lock for session id 0x20010446d6e0003
-------------------------5.锁被创建-------------------------
21:39:43.395 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0003, packet:: clientPath:null serverPath:null finished:false header:: 6,1  replyHeader:: 6,8589934696,0  request:: '/cluster_ha/lock,#6e6f64653031,v{s{31,s{'world,'anyone}}},1  response:: '/cluster_ha/lock 
21:39:43.397 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0003, packet:: clientPath:null serverPath:null finished:false header:: 7,4  replyHeader:: 7,8589934696,0  request:: '/cluster_ha/lock,F  response:: #6e6f64653031,s{8589934696,8589934696,1661434783392,1661434783392,0,0,0,144133074155601923,6,0,8589934696} 
21:39:43.399 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0003, packet:: clientPath:null serverPath:null finished:false header:: 8,1  replyHeader:: 8,8589934697,0  request:: '/cluster_ha/active/node01,#6e6f64653031,v{s{31,s{'world,'anyone}}},1  response:: '/cluster_ha/active/node01 
21:39:43.400 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0003, packet:: clientPath:null serverPath:null finished:false header:: 9,3  replyHeader:: 9,8589934697,-101  request:: '/cluster_ha/standby/node01,F  response::  
-------------------------5-2.node01成为主节点-------------------------
21:39:45.066 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010446d6e0003 after 0ms.
21:39:46.733 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010446d6e0003 after 0ms.
复制代码

启动node02

-------------------------1.创建连接-------------------------
21:13:27.102 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 2021-03-17 09:46 UTC
21:41:43.282 [main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
-------------------------2.创建PARENT ACTIVE  STANDBY等根节点-------------------------
21:41:43.286 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.SaslServerPrincipal - Canonicalized address to activate.navicat.com
21:41:43.288 [main-SendThread(node02:2182)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server node02/127.0.0.1:2182.
21:41:43.309 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 3,3  replyHeader:: 3,8589934698,0  request:: '/cluster_ha/standby,F  response:: s{8589934621,8589934621,1661429857799,1661429857799,0,22,0,0,19,0,8589934694} 
-------------------------3.判断ACTIVE节点下是否有子节点(主节点)-------------------------
21:41:43.311 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 4,8  replyHeader:: 4,8589934698,0  request:: '/cluster_ha/active,F  response:: v{'node01} 
-------------------------3-2.存在子节点(主节点),node02自动创建成为从节点-------------------------
21:41:43.318 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 5,1  replyHeader:: 5,8589934699,0  request:: '/cluster_ha/standby/node02,#6e6f64653032,v{s{31,s{'world,'anyone}}},1  response:: '/cluster_ha/standby/node02 
21:41:43.319 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 6,8  replyHeader:: 6,8589934699,0  request:: '/cluster_ha/active,T  response:: v{'node01} 
21:41:44.991 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010446d6e0004 after 0ms.
复制代码

启动node03

-------------------------1.创建连接-------------------------
21:13:33.784 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.7.0-e3704b390a6697bfdf4b0bef79e3da7a4f6bac4b, built on 2021-03-17 09:46 UTC
21:42:26.606 [main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
-------------------------2.创建PARENT ACTIVE  STANDBY等根节点-------------------------
21:42:26.610 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.SaslServerPrincipal - Canonicalized address to activate.navicat.com
21:42:26.611 [main-SendThread(node01:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server node01/127.0.0.1:2181.
21:42:26.631 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 3,3  replyHeader:: 3,8589934700,0  request:: '/cluster_ha/standby,F  response:: s{8589934621,8589934621,1661429857799,1661429857799,0,23,0,0,19,1,8589934699} 
-------------------------3.判断ACTIVE节点下是否有子节点(主节点)-------------------------
21:42:26.634 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 4,8  replyHeader:: 4,8589934700,0  request:: '/cluster_ha/active,F  response:: v{'node01} 
-------------------------3-2.存在子节点(主节点),node03自动创建成为从节点-------------------------
21:42:26.640 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 5,1  replyHeader:: 5,8589934701,0  request:: '/cluster_ha/standby/node03,#6e6f64653033,v{s{31,s{'world,'anyone}}},1  response:: '/cluster_ha/standby/node03 
21:42:26.641 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 6,8  replyHeader:: 6,8589934701,0  request:: '/cluster_ha/active,T  response:: v{'node01} 
21:42:28.314 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x100104d28f30009 after 0ms.
复制代码

停止node01

node02执行日志

21:43:20.353 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x20010446d6e0004
21:43:20.354 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/cluster_ha/active for session id 0x20010446d6e0004
-------------------------4.ACTIVE根节点下的主节点被删除-------------------------
21:43:20.357 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 7,8  replyHeader:: 7,8589934702,0  request:: '/cluster_ha/active,F  response:: v{} 
-------------------------4-1.node02 准备参与竞选-------------------------
21:43:20.357 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 8,3  replyHeader:: 8,8589934702,-101  request:: '/cluster_ha/lock,T  response::  
21:43:20.359 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x20010446d6e0004
21:43:20.359 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeCreated path:/cluster_ha/lock for session id 0x20010446d6e0004
21:43:20.360 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 9,1  replyHeader:: 9,8589934704,-110  request:: '/cluster_ha/lock,#6e6f64653032,v{s{31,s{'world,'anyone}}},1  response::  
节点: /cluster_ha/lock 创建失败或者节点已经存在
node02 未抢到锁,等待....
21:43:20.373 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 10,8  replyHeader:: 10,8589934706,0  request:: '/cluster_ha/active,T  response:: v{'node03} 
-------------------------5.锁被创建-------------------------
21:43:20.375 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x20010446d6e0004, packet:: clientPath:null serverPath:null finished:false header:: 11,4  replyHeader:: 11,8589934706,0  request:: '/cluster_ha/lock,F  response:: #6e6f64653033,s{8589934703,8589934703,1661435000358,1661435000358,0,0,0,72075517623468041,6,0,8589934703} 
-------------------------5-3.当前节点node02未持有锁,放弃任务操作-------------------------
21:43:22.042 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010446d6e0004 after 0ms.
21:43:23.708 [main-SendThread(node02:2182)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x20010446d6e0004 after 0ms.
复制代码

node03执行日志

21:43:20.354 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/cluster_ha/active for session id 0x100104d28f30009
-------------------------4.ACTIVE根节点下的主节点被删除-------------------------
21:43:20.357 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 7,8  replyHeader:: 7,8589934702,0  request:: '/cluster_ha/active,F  response:: v{} 
-------------------------4-1.node03 准备参与竞选-------------------------
21:43:20.357 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 8,3  replyHeader:: 8,8589934702,-101  request:: '/cluster_ha/lock,T  response::  
21:43:20.359 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x100104d28f30009
21:43:20.359 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeCreated path:/cluster_ha/lock for session id 0x100104d28f30009
21:43:20.360 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 9,1  replyHeader:: 9,8589934703,0  request:: '/cluster_ha/lock,#6e6f64653033,v{s{31,s{'world,'anyone}}},1  response:: '/cluster_ha/lock 
21:43:20.360 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 10,8  replyHeader:: 10,8589934704,0  request:: '/cluster_ha/active,T  response:: v{} 
-------------------------5.锁被创建-------------------------
21:43:20.363 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 11,4  replyHeader:: 11,8589934704,0  request:: '/cluster_ha/lock,F  response:: #6e6f64653033,s{8589934703,8589934703,1661435000358,1661435000358,0,0,0,72075517623468041,6,0,8589934703} 
21:43:20.365 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification session id: 0x100104d28f30009
21:43:20.365 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/cluster_ha/active for session id 0x100104d28f30009
21:43:20.365 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 12,1  replyHeader:: 12,8589934705,0  request:: '/cluster_ha/active/node03,#6e6f64653033,v{s{31,s{'world,'anyone}}},1  response:: '/cluster_ha/active/node03 
21:43:20.366 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 13,3  replyHeader:: 13,8589934705,0  request:: '/cluster_ha/standby/node03,F  response:: s{8589934701,8589934701,1661434946639,1661434946639,0,0,0,72075517623468041,6,0,8589934701} 
-------------------------5-1.当前节点node03持有锁,切换状态为主节点-------------------------
21:43:20.370 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 14,2  replyHeader:: 14,8589934706,0  request:: '/cluster_ha/standby/node03,-1  response:: null
-------------------------4.ACTIVE根节点下的主节点被删除-------------------------
21:43:20.371 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 15,8  replyHeader:: 15,8589934706,0  request:: '/cluster_ha/active,F  response:: v{'node03} 
-------------------------4-2.ACTIVE根节点下的主节点刚被创建,放弃任何操作-------------------------
21:43:20.372 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x100104d28f30009, packet:: clientPath:null serverPath:null finished:false header:: 16,8  replyHeader:: 16,8589934706,0  request:: '/cluster_ha/active,T  response:: v{'node03} 
21:43:22.040 [main-SendThread(node01:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for session id: 0x100104d28f30009 after 0ms.
复制代码
RocketMQ TLS Client-initiated 重协商攻击(CVE-2011-1473) SSL 重协商攻击(SSL renegotiation attack)是一种安全漏洞攻击,它利用了 SSL/TLS 协议的重协商功能,通过与服务器重新协商密钥,来发起攻击。密码重置:攻击者可以利用 SSL 重协商攻击来重置 SSL/TLS 会话密钥,从而能够窃取会话中的敏感信息,如用户名、密码等。a)则仅允许安全重新协商并限制SSL握手的次数,或者通过添加SSL加速器之类的产品来升级服务器资源。2、可以发现执行脚本都使用到了JAVA_OPT_EXT,可以启动容器时,设置JAVA_OPT_EXT达到目的。 阅读详情

相关推荐

Zookeeper集群及基础API

目录 1、Zookeeper简介 2、Zookeeper概念 2.1 重要概念 2.2 Session会话(心跳检测) 2.3 ZNode数据节点 2.4 ZNode数据版本 2.5 Watcher-事件监听器 2.6 ACL策略 3、Zookeeper集群安装配置 3.1 准备工作 3.2 下载解压 3.3 修改配置 3.4 启动测试 4、Zookeeper客户端操作命令 5、Zookeeper API操作 5.1 引入POM依赖坐标 5.2 创建Seesion会话.

长沙老码农 1602

ZooKeeper之Java API的基本使用以及应用场景的实现

介绍ZooKeeper中Java API的基本使用,以及对Zookeeper常见应用场景的具体实现。

积跬步,至千里。 1058

Zookeeper经典应用场景实战

Zookeeper经典应用场景实战

qq_42439945的博客 1353

zookeeper之java API使用

节点创建 package com.zookeeper; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher; impor...

codingToMaster的博客 363

2、zookeeper基本操作及应用示例(shell、java api、应用场景示例)

本文接上一篇zookeeper的部署与验证。本文从三个方面介绍zookeeper的基本操作及简单的应用场景。

alanchanchn的专栏 8万+

【实战解析】RTL8306MB交换芯片驱动开发中的RMII接口调试陷阱

本文通过一个RTL8306MB交换芯片驱动开发的实战案例,深入剖析了RMII接口调试中的经典陷阱。文章详细描述了因TX_EN与CRS_DV信号线交叉连接错误导致的“静默”故障,并分享了使用逻辑分析仪和示波器进行硬件排查的完整流程与核心技巧,为嵌入式网络驱动开发提供了宝贵的避坑指南。

weixin_29267307的博客 424

zookeeper的java API使用及典型应用场景

目录javaAPI使用原生APIzkclientcurator典型应用场景分布式锁选主服务 javaAPI使用 原生API zkclient curator 典型应用场景 分布式锁 选主服务

陆林烽的博客 394

VBA中如何使用Edge内核Browser?

在微软宣布停止对IE的支持后,许多使用VBA中Web Browser的用户遇到了困境,因为Web Browser使用的IE内核。然而微软并没有给出支持Edge的新版Web Browser,网上许多第三方的库也不够完善。经过调研,libEdge可以较好的和VBA集成,实现在Excel中打开Browser,甚至能够通过执行js代码,获取web页面的信息。项目需要WebView2Loader.dll和LibEdge.dll两个库,区分x86和x64版本。

金木AI 2402

81-java-zookeeper(1) -zookeeper单机/原生api/应用场景

本文架构如图 一.简介 1.功能及用途 1.维护配置信息 2.分布式锁 3.集群管理 4.生成分布式唯一id 2.设计目标及优点 优点 1.高性能 内存,存储数据 2.高可用 集群部署 3.严格顺序访问 生成的全局唯一的id,分布式锁 对比redis redis zookeeper 存储类型 内存 内存 数据类型 k-v 树形结构 性能 高 中 高可用 好 好 目标 数据缓存 分布式辅助功能 应用场景 缓存数据库 分布式锁/配置中心 设计目标 由于zooke

weixin_45449911的博客 259

文华指标源码—区间彩带

这就是当年东方财富软件终端里主打经典的指标、主要揭示主力成本区域。

qq2054129753的博客 895

Python爬虫实战:5分钟抓取东方财富股票数据并保存为CSV

在投资分析与量化研究的日常工作中,获取结构化、可用的金融数据往往是第一步。对于个人投资者和量化入门者而言,如何高效获取A股全市场行情,常常面临网页动态加载、反爬机制与数据整理繁琐等挑战。本文从数据采集的基础概念出发,介绍如何利用Python这一通用编程语言,通过requests库请求财经网站公开的HTTP接口,获取返回的JSON数据,再借助pandas进行字段映射、数据清洗与表格化处理。这种方式相比传统页面解析具有响应快、结构稳定、易于批量操作的显著优势,尤其适合构建个人股票数据池、验证选股策略或进行市场情

weixin_33146151的博客 253

大数据技术应用5-3Zookeeper的Java API操作、应用场景

Zookeeper的Java API操作、应用场景

m0_51679196的博客 256

CTF实战:Wireshark数据包分析从入门到精通(附经典题型解析)

本文详细介绍了Wireshark在CTF比赛中的数据包分析技巧,从基础抓包到高级协议解析,涵盖HTTP流量分析、USB键盘数据还原等实战案例。通过经典题型解析,帮助读者掌握精准过滤、文件提取等核心技能,提升CTF竞赛中的网络流量分析能力。

pz890123的博客 587

Zookeeper Api(java)入门详解与应用场景

关键字:Zookeeper Api(java)入门详解与应用场景 如何使用 Zookeeper 作为一个分布式的服务框架,主要用来解决分布式集群中应用系统的一致性问题,它能提供基于类似于文件系统的目录节点树方式的数据存储,但是 Zookeeper 并不是用来专门存储数据的,它的作用主要是用来维护和监控你存储的数据的状态变化。通过监控这些数据状态的变化,从而可以达到基于数据的集群管理,后...

赵士杰的博客 210

vLLM 多机多卡推理测试与验证(Kubernetes)vLLM 多机多卡推理 Kubernetes 验证

本文记录在两台 L20 机器,每台机器 4 块 L20 显卡(46G)的环境下,使用 vLLM 部署 Qwen2.5-32B-Instruct-GPTQ-Int4 模型并提供多节点多卡推理服务的过程。服务(Serving)模型服务是指将模型部署到生产环境中,通过 API 或其他接口提供预测服务的过程。【分布式推理与服务】(Distributed Inference and Serving)是指在多个机器或设备之间部署和管理机器学习模型,以高效地处理推理请求并满足大规模服务需求。3、启动 vLLM 服务。

流水_鱼鱼的博客 3460

Android平台实现Airplay的方法及源代码

Airplay是由苹果公司推出的一项无线音视频传输技术,它允许用户将音频、视频和照片从iOS设备(如iPhone、iPad)或Mac电脑无线传输到支持Airplay的设备(如Apple TV、智能音箱等)。具体的解码和播放方式取决于你选择的库和技术。当Android设备启动AirplayService服务后,它将监听来自iOS设备的连接请求,并在建立连接后接收、解码和播放音视频数据。在AirplayService类的onStartCommand()方法中,我们需要处理Airplay的请求。

CyberLynxX的博客 1602

Zookeeper应用场景实战一

1. Zookeeper Java客户端实战 ZooKeeper应用的开发主要通过Java客户端API去连接和操作ZooKeeper集群。可供选择的Java客户端API有: ZooKeeper官方的Java客户端API。 第三方的Java客户端API,比如Curator。 ZooKeeper官方的客户端API提供了基本的操作。例如,创建会话、创建节点、读取节点、更新数据、删除节点和检查节点是否存在等。不过,对于实际开发来说,ZooKeeper官方API有一些不足之处,具体如下:

qq_45061342的博客 3万+

Python爬虫实战:高效爬取中国专利数据库的完整指南

在当今信息爆炸的时代,专利数据作为技术创新的重要指标,对企业研发、市场分析和学术研究都具有极高价值。中国作为全球专利申请量最大的国家,其专利数据库包含了海量的技术创新信息。本文将详细介绍如何使用Python最新技术构建一个高效、稳定的中国专利数据库爬虫系统。中国专利数据库主要由国家知识产权局(CNIPA)维护,包含发明专利、实用新型专利和外观设计专利三大类。截至2023年,中国专利数据库已收录超过400万件发明专利和300万件实用新型专利。

2201_76125261的博客 1575

zookeeper:环境搭建、zookeeper基本操作、集群搭建

Zookeeper 1.1、概述 ​ ZooKeeper从字面意思理解,【Zoo - 动物园,Keeper - 管理员】动物园中有很多种动物,这里的动物就可以比作分布式环境下多种多样的服务,而ZooKeeper做的就是管理这些服务。 ​ Apache ZooKeeper的系统为分布式协调是构建分布式应用的高性能服务。 ​ ZooKeeper 本质上是一个分布式的小文件存储系统。提供基于类似于文件系统的目录树方式的数据存储,并且可以对树中的节点进行有效管理。从而用来维护和监控你存储的数据的状态变化。通过监控这

weixin_41942838的博客 586

蓝牙模块HM-10手册

蓝牙模块HM-10手册,该手册对蓝牙模块HM-10进行了详细的介绍和使用方法以及一些AT指令。

jre9下载 jdk9

(Java SE Runtime Environment 9.0) 9.0u175 官方最新版 【x86|x64】

Zookeeper Java API

1. 概述 可以通过Java的AIP作为zookeeper客户端,实现与zookeeper服务器的交互。 客户端应该遵循以下步骤,与服务器进行清晰和干净的交互 连接到zookeeper服务器。zookeeper服务器为客户端分配会话ID。 定期向服务器发送心跳。否则,zookeeper服务器将过期会话ID,客户端需要重新连接。 只要会话ID处于活动状态,就可以获取/设置znode。 所有任务完成后,断开与zookeeper服务器的连接。如果客户端长时间不活动,则 zookeeper服务器将自动断开客户端

weixin_42524843的博客 863

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

zookeeper详解

zookeeper详解 文章目录zookeeper详解回顾:今天任务教学目标第一节:zookeeper概述1.1 什么是zookeeper1.2 zookeeper服务的应用场景1.3 zookeeper集群特性1.4 zookeeper数据结构1.5节点类型1.6 Zookeeper 数据结构特点1.7 如何使用第二节 zookeeper集群的配置2.1 集群安装2.2 集群的启动2.3 集群...

qq_36643786的博客 298

jdk-8u144-linux-x64.tar.gz.

该资源为搭建Linux系统Java环境用,资源的后缀为tar.gz格式的压缩包

一文搞定ZooKeeper常见用法

介绍 本文比较系统的梳理了ZooKeeper的常用知识点,包括文件系统、监听通知机制、单机环境搭建、集群环境搭建、常用操作命令、Java API调用和作为注册中心服务等内容,读完本文,相信能有助于我们对ZooKeeper的概念和用法有一个整体的认识。 目录介绍一、文件系统与监听通知机制1、文件系统2、监听通知机制二、单机环境搭建1、安装JDK2、安装ZooKeeper3、客户端连接ZooKeeper三、集群环境搭建1、角色概念2、集群搭建3、服务启停4、客户端连接四、常用操作命令1、ls命令2、create

攀登Fox的博客 1161

Zookeeper知识点梳理

1:了解Zookeeper的概念2:了解分布式的概念​ ZooKeeper从字面意思理解,【Zoo - 动物园,Keeper - 管理员】动物园中有很多种动物,这里的动物就可以比作分布式环境下多种多样的服务,而ZooKeeper做的就是管理这些服务。​ Apache ZooKeeper的系统为分布式协调是构建分布式应用的高性能服务。​ ZooKeeper 本质上是一个分布式的小文件存储系统。提供基于类似于文件系统的目录树方式的数据存储,并且可以对树中的节点进行有效管理。

qq_44794280的博客 878

zookeeper使用和原理探究(一)

(一)zookeeper使用和原理探究 zookeeper介绍 zookeeper是一个为分布式应用提供一致性服务的软件,它是开源的Hadoop项目中的一个子项目,并且根据google发表的论文来实现的,接下来我们首先来安装使用下这个软件,然后再来探索下其中比较重要一致性算法。   zookeeper安装和使用 zookeeper的安装基本上可以按照 http://hadoop.a

appstore81的专栏 727
上一篇: 如何在Java中加密和解密zip文件?
下一篇: 拿着 GitHub 标星 80k 的阿里性能优化笔记,把项目优化个遍
java小姜在线冲
博客等级 码龄4年 53粉丝 137原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值