BigData-16:zookeeper

本文全面介绍了Apache ZooKeeper,一种集中式服务,用于维护配置信息、命名、提供分布式同步和组服务。文章详细阐述了ZooKeeper的功能、工作机制、存储结构及应用场景,包括集群统一配置管理、统一命名服务等,并提供了详细的安装步骤和客户端操作指南。

1)zookeeper简介
动物管理员
Apache ZooKeeper致力于开发和维护开源服务器,实现高度可靠的分布式协调。
2)什么是ZooKeeper?
ZooKeeper是一种集中式服务,用于维护配置信息,命名,提供分布式同步和提供组服务。所有这
些类型的服务都以分布式应用程序的某种形式使用。每次实施它们都需要做很多工作来修复不可避
免的错误和竞争条件。由于难以实现这些类型的服务,应用程序最初通常会吝啬它们,这使得它们
在变化的情况下变得脆弱并且难以管理。即使正确完成,这些服务的不同实现也会在部署应用程序
时导致管理复杂性。
3)zookeeper功能
(1)存储数据
(2)监听
4)zookeeper工作机制
基于观察者模式设计的分布式服务管理框架
5)zookeeper的存储结构
目录树结构
6)zookeeper应用场景

(1)集群统一配置管理
(2)集群统一命名服务
(3)集群统一管理
(4)服务器的动态上下线感知
(5)负载均衡

7)安装zookeeper集群
(1)下载安装包
(2)上传安装包到Linux
alt+p
(3)解压
tar -zxvf .tar
(4)修改配置文件
vi zoo.cfg
dataDir=/root/hd/zookeeper-3.4.10/zkData
(5)启动zk
bin/zkServer.sh start
(6)查看状态
bin/zkServer.sh status
(7)启动客户端
bin/zkCli.sh
8)完全分布式安装
(1)下载安装包
(2)上传安装包到Linux
alt+p
(3)解压
tar -zxvf .tar
(4)修改配置文件
vi zoo.cfg

dataDir=/root/hd/zookeeper-3.4.10/zkData
###############cluster###############
server.1=hd09-01:2888:3888
server.2=hd09-02:2888:3888
server.3=hd09-03:2888:3888

(5)添加文件myid
$cd zookeeper-3.4.10/zkData
$touch myid
(6)添加内容在myid为1
$ vi myid
1
(7)发送zookeeper文件到其它机器
$ scp -r zookeeper-3.4.10 hd09-02:$PWD
(8)修改myid依次为2 3
(9)修改环境变量
vi /etc/profile

export ZOOKEEPER_HOME=/root/hd/zookeeper3.4.10
export PATH=$PATH:$ZOOKEEPER_HOME/bin

(10)生效环境变量
source /etc/profile
(11)启动zookeeper
zkServer.sh start
(12)查看状态
zkServer.sh status
(13)关闭zookeeper
zkServer.sh stop

9)客户端的命令行操作
(1)启动客户端
zkCli.sh
(2)查看帮助
help
(3)查看当前znode所包含的内容
ls /
(4)创建节点
create /hunterhenshuai 18
(5)创建短暂znode
create -e /itstar hunter
(6)创建带序号znode
create -s /bigdata hunter
(7)创建短暂带序号
create -e -s /bigdata hunter
(8)查看此节点的详细信息
ls2 /
(9)获得节点值监听
get /hunterhenshuai watch
(10)监听路径
ls / watch
(11)修改znode数据
set /hunterhenshuai iiiii
(12)删除节点
delete /hunterhenshuai
(13)递归删除
rmr /delireba
(14)查看节点状态信息
stat /
10)客户端API

# Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # server: port: 12345 servlet: session: timeout: 120m context-path: /dolphinscheduler/ compression: enabled: true mime-types: text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json,application/xml jetty: max-http-form-post-size: 5000000 spring: application: name: api-server banner: charset: UTF-8 jackson: time-zone: UTC date-format: "yyyy-MM-dd HH:mm:ss" servlet: multipart: max-file-size: 1024MB max-request-size: 1024MB messages: basename: i18n/messages datasource: driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://rm-2zer55j9y09b13451.mysql.rds.aliyuncs.com:3306/bigdata_dolphinscheduler?&characterEncoding=UTF-8&allowMultiQueries=true&serverTimezone=GMT%2B8 username: admin password: 1231123 hikari: connection-test-query: select 1 minimum-idle: 5 auto-commit: true validation-timeout: 3000 pool-name: DolphinScheduler maximum-pool-size: 50 connection-timeout: 30000 idle-timeout: 600000 leak-detection-threshold: 0 initialization-fail-timeout: 1 quartz: auto-startup: false job-store-type: jdbc jdbc: initialize-schema: never properties: org.quartz.threadPool.threadPriority: 5 org.quartz.jobStore.isClustered: true # org.quartz.jobStore.class: org.quartz.impl.jdbcjobstore.JobStoreTX org.quartz.scheduler.instanceId: AUTO org.quartz.jobStore.tablePrefix: QRTZ_ org.quartz.jobStore.acquireTriggersWithinLock: true org.quartz.scheduler.instanceName: DolphinScheduler org.quartz.threadPool.class: org.quartz.simpl.SimpleThreadPool org.quartz.jobStore.useProperties: false org.quartz.threadPool.makeThreadsDaemons: true org.quartz.threadPool.threadCount: 25 org.quartz.jobStore.misfireThreshold: 60000 org.quartz.scheduler.makeSchedulerThreadDaemon: true org.quartz.jobStore.driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate org.quartz.jobStore.clusterCheckinInterval: 5000 org.quartz.scheduler.batchTriggerAcquisitionMaxCount: 1 management: endpoints: web: exposure: include: '*' endpoint: health: enabled: true show-details: always health: db: enabled: true defaults: enabled: false metrics: tags: application: ${spring.application.name} registry: type: zookeeper zookeeper: namespace: dolphinscheduler connect-string: master-1-1:2181,master-1-2:2181,master-1-3:2181 retry-policy: base-sleep-time: 60ms max-sleep: 300ms max-retries: 5 session-timeout: 30s connection-timeout: 9s block-until-connected: 600ms digest: ~ audit: enabled: false metrics: enabled: true python-gateway: # Weather enable python gateway server or not. The default value is true. enabled: true # The address of Python gateway server start. Set its value to `0.0.0.0` if your Python API run in different # between Python gateway server. It could be be specific to other address like `127.0.0.1` or `localhost` gateway-server-address: 0.0.0.0 # The port of Python gateway server start. Define which port you could connect to Python gateway server from # Python API side. gateway-server-port: 25333 # The address of Python callback client. python-address: 127.0.0.1 # The port of Python callback client. python-port: 25334 # Close connection of socket server if no other request accept after x milliseconds. Define value is (0 = infinite), # and socket server would never close even though no requests accept connect-timeout: 0 # Close each active connection of socket server if python program not active after x milliseconds. Define value is # (0 = infinite), and socket server would never close even though no requests accept read-timeout: 0 security: authentication: # Authentication types (supported types: PASSWORD,LDAP) type: PASSWORD # IF you set type `LDAP`, below config will be effective ldap: # admin userId user.admin: read-only-admin # ldap server config urls: ldap://ldap.forumsys.com:389/ base.dn: dc=example,dc=com username: cn=read-only-admin,dc=example,dc=com password: password user.identity.attribute: uid user.email.attribute: mail # Traffic control, if you turn on this config, the maximum number of request/s will be limited. # global max request number per second # default tenant-level max request number traffic: control: global-switch: false max-global-qps-rate: 300 tenant-switch: false default-tenant-qps-rate: 10 #customize-tenant-qps-rate: # eg. #tenant1: 11 #tenant2: 20 cloud: enabled: false client-id: "" tenant-id: "" client-secret: "" # Override by profile --- spring: config: activate: on-profile: mysql datasource: driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://rm-2zer55j9y09b13451.mysql.rds.aliyuncs.com:3306/bigdata_dolphinscheduler?&characterEncoding=UTF-8&allowMultiQueries=true&serverTimezone=GMT%2B8 username: admin password: 1231123 quartz: properties: org.quartz.jobStore.driverDelegateClass: org.quartz.impl.jdbcjobstore.StdJDBCDelegate 这是配置
最新发布
06-28
在 DolphinScheduler 3.2.2 启动过程中,出现 `Error creating bean with name 'cloudServiceImpl'` 并伴随 `java.lang.ExceptionInInitializerError` 异常,通常表明 Spring 容器在加载 `CloudServiceImpl` 类时遇到了静态初始化失败的问题。此类错误可能源于类路径配置错误、依赖注入缺失、JDK 版本限制或静态初始化逻辑异常。 ### 静态初始化失败与 `ExceptionInInitializerError` 当某个类的静态代码块(static block)或静态变量初始化过程中抛出异常,Java 虚拟机会抛出 `ExceptionInInitializerError`。这种情况在使用反射访问受保护方法时尤为常见,尤其是在 JDK 9 及以上版本中,由于模块系统的引入,对某些内部 API 的访问受到限制。例如,Spring AOP 或 CGLIB 在尝试通过反射调用 `ClassLoader.defineClass` 方法时可能会触发此问题[^3]。 为解决该问题,可以在启动命令中添加如下 JVM 参数以放宽访问控制: ```bash --add-opens java.base/java.lang=ALL-UNNAMED ``` 将其加入启动脚本的 `JAVA_OPTS` 中,例如: ```bash export JAVA_OPTS="--add-opens java.base/java.lang=ALL-UNNAMED" ``` ### Bean 创建失败与 Spring 上下文配置 根据类似场景中的经验,Bean 初始化失败也可能与 Spring 上下文配置文件的路径设置有关。如果配置文件路径拼写错误或未被正确加载,将导致 Spring 无法扫描到相关 Bean,从而引发依赖注入失败的问题。例如,在 web.xml 中若配置了错误的 classpath 路径: ```xml <context-param> <param-name>contextConfigLocation</param-name> <param-value>classpath:applicationContext-*.xml</param-value> </context-param> ``` 上述配置若路径拼写错误(如 `classpath:` 拼写错误),将导致 Spring 无法识别配置文件位置,进而无法加载 `@Service` 注解的 Bean,最终引发类似的 Bean 创建失败问题[^1]。 ### 依赖注入与类加载问题排查 确保 `cloudServiceImpl` 所需的所有依赖都已正确定义并可注入。检查是否遗漏了以下内容: - 是否正确使用 `@Autowired` 注解标注字段或构造函数 - 对应接口或服务是否已在 Spring 上下文中注册 - 数据源、配置中心等基础设施是否正常连接 此外,日志追踪是定位此类问题的关键。查看完整的日志输出,尤其是 `ExceptionInInitializerError` 的嵌套异常(`Caused by:`)。这通常会揭示更具体的错误根源,例如数据库连接失败、配置读取异常或第三方库冲突。 ---
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

phial03

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值