debezium获取mysql binlog通过kafka实现mysql同步

目标是实现mysql通过binlog的同步

---

---

安装confluent

• 下载地址地址
• 解压tar -zxvf confluent-5.1.2-2.11.tar.gz
• 进入到confluent中
• 快速启动 ZooKeeper,Kafka,Schema Registry,Control Center,Kafka Connect,Kafka REST Proxy,KSQL。
• ./bin/confluent start
  
  如图就是快速启动成功

安装debezium插件

官网地址
debezium-connector-mysql/0.8.1.Final下载连接

• 安装插件Debezium 把解压后的debezium复制到conlfuent安装目录share/java文件中例如/usr/local/hadoop/confluent-5.1.2/share/java/
• 配置好后重新启动conlfuent即可

开启mysql的binlog

• 修改 my.cnf

log-bin=/var/lib/mysql/mysql-bin
server-id=1

重新启动服务

systemctl start mysqld.service

• 登陆后查看

show variables like '%log_bin%'

创建kafka connect连接

curl -i -X POST -H "Accept:application/json" -H  "Content-Type:application/json" http://localhost:8083/connectors/  -d '{
  "name": "demo2",
  "config": {
    "connector.class": "io.debezium.connector.mysql.MySqlConnector",
    "tasks.max": "1",
    "database.hostname": "XXX.XX.XXX.XXX",
    "database.port": "3306",
    "database.user": "root",
    "database.password": "XXXX",
    "database.server.id": "1",
    "database.server.name": "demo2",
    "database.whitelist": "kafka1",
    "decimal.handling.mode": "double",
    "key.converter": "io.confluent.connect.avro.AvroConverter",
    "key.converter.schema.registry.url": "http://localhost:8081",
    "value.converter": "io.confluent.connect.avro.AvroConverter",
    "value.converter.schema.registry.url": "http: