目录
什么是 etcd
etcd 是 CoreOS 团队于 2013 年 6 月发起的开源项目,它的目标是构建一个高可用的分布式键值( key-value )数据库,基于 Go 语言实现。
安装
嫌自己编译麻烦 也以直接下载编译好的,选个最新的。下载后解压,然后放到合适的位置就好
https://github.com/etcd-io/etcd/releases
其实就是2个 bin文件,etcd是服务器,etcdctl是客户端。所谓的安装本质上就上把这2个文件下载后放到 PATH (比如 /usr/bin) 目录中就行了。
顺手分享个windows下使用的。要编辑 计算机 高级系统设置 环境变量 把 ETCDCTL_API 添加进去 值为3
然后把 etcdctl.exe 拷贝到系统目录 或者PATH目录中加上 etcdctl.exe 所在目录
制作开机启动
新建 /opt/config/etcd.yml 内容如下, 192.168.255.130 改成你电脑的ip
name: etcd-1
data-dir: /var/lib/etcd
listen-client-urls: http://0.0.0.0:2379
advertise-client-urls: http://192.168.255.130:2379
listen-peer-urls: http://0.0.0.0:2380
initial-advertise-peer-urls: http://192.168.255.130:2380
initial-cluster: etcd-1=http://192.168.255.130:2380
initial-cluster-token: etcd-cluster-token
initial-cluster-state: new
再新建 /lib/systemd/system/etcd.service 内容如下
[Unit]
Description=Etcd Server
Documentation=https://github.com/coreos/etcd
After=network.target
[Service]
User=root
Type=notify
ExecStart=/usr/local/bin/etcd --config-file=/opt/config/etcd.yml
Restart=on-failure
RestartSec=10s
LimitNOFILE=40000
[Install]
WantedBy=multi-user.target
systemctl 的一些操作
systemctl daemon-reload 重新加载
systemctl start etcd 启动
systemctl status etcd -l 查看状态
systemctl enable etcd 开机启动
systemctl disable etcd 禁止开机启动
systemctl stop etcd 停止
go get github.com/etcd-io/etcd
github.com 域名无法解析~ 后来发现虚拟机里的网络没有打开!
构建
make build
2018年8月29日, etcd源码路径改了。也就是说 之前的源码可以编译通过,之后的源码得mv下目录。
- "github.com/coreos/etcd/version"
+ "go.etcd.io/etcd/version"
把 $GOPATH/github.com/coreos/etcd 目录, mv 到 $GOPATH/go.etcd.io/etcd
构建时,我还遇到个问题,提示如下
etcd on unsupported platform without ETCD_UNSUPPORTED_ARCH=386 set
既然源码在手,就全局搜索下源码。发现了下面这个函数
func checkSupportArch() {
// TODO qualify arm64
if runtime.GOARCH == "amd64" || runtime.GOARCH == "ppc64le" {
return
}
// unsupported arch only configured via environment variable
// so unset here to not parse through flag
defer os.Unsetenv("ETCD_UNSUPPORTED_ARCH")
if env, ok := os.LookupEnv("ETCD_UNSUPPORTED_ARCH"); ok && env == runtime.GOARCH {
fmt.Printf("running etcd on unsupported architecture %q since ETCD_UNSUPPORTED_ARCH is set\n", env)
return
}
fmt.Printf("etcd on unsupported platform without ETCD_UNSUPPORTED_ARCH=%s set\n", runtime.GOARCH)
os.Exit(1)
}
原来我安装 go 的 环境变量 GOARCH 一直写错了,写成了386. 我运行在ubuntu 16.04 64位系统上 应该是 amd64
编辑 /etc/environment 文件 把 386 改成 amd64. 保存 注销系统,再登陆,解决。
生成了下面的文件
~/go/gopath/src/github.com/coreos/etcd/bin $ ls
default.etcd etcd etcdctl
为了方便在各个目录下都可以运行 etcd 和 etcdctl
把它们文件拷贝到系统可执行目录
sudo cp etcd* /usr/local/bin/
制作docker
打开 Makefile 文件,找到了写好的制作docker 命令。 那就拿来用吧
make build-docker-release-master
生成后发现 docker 的 image 名字老长老长,手痒改了个短的,重新编译
build-docker-release-master:
$(info ETCD_VERSION: $(ETCD_VERSION))
cp ./Dockerfile-release ./bin/Dockerfile-release
docker build \
--tag etcd:$(ETCD_VERSION) \
--file ./bin/Dockerfile-release \
./bin
rm -f ./bin/Dockerfile-release
docker run \
--rm \
etcd:$(ETCD_VERSION) \
/bin/sh -c "/usr/local/bin/etcd --version && ETCDCTL_API=3 /usr/local/bin/etcdctl version"
于是生成的image如下:
REPOSITORY TAG IMAGE ID CREATED SIZE
etcd 93be31d 24cb021159ce 23 hours ago 66.1MB
简单使用
命令行下直接运行 etcd 即可
etcd
然后 用客户端 etcdctl 试试。 如下,保存了键 wjs, 值为 "hello world"。 然后取出来看看,值,果然是 "hello world"
~/go/gopath/src/github.com/coreos/etcd $ etcdctl put wjs "hello world"
OK
~/go/gopath/src/github.com/coreos/etcd $ etcdctl get wjs
wjs
hello world
网上好多文章都用的 etcdctl set wjs "hello world"。 set 命令已经没有了,用 put 吧,etcd 变化挺快的啊。
利用docker-compose制作集群
我就一台电脑,又想试试集群。那就用docker 吧
version : "3.6"
services:
node1:
image: etcd:93be31d
volumes:
- node1-data:/etcd-data
expose:
- 2379
- 2380
networks:
my_net:
ipv4_address: 172.16.238.100
environment:
- ETCDCTL_API=3
command:
- /usr/local/bin/etcd
- --data-dir=/etcd-data
- --name
- node1
- --initial-advertise-peer-urls
- http://172.16.238.100:2380
- --listen-peer-urls
- http://0.0.0.0:2380
- --advertise-client-urls
- http://172.16.238.100:2379
- --listen-client-urls
- http://0.0.0.0:2379
- --initial-cluster
- node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node3=http://172.16.238.102:2380
- --initial-cluster-state
- new
- --initial-cluster-token
- docker-etcd
node2:
image: etcd:93be31d
volumes:
- node2-data:/etcd-data
expose:
- 2379
- 2380
networks:
my_net:
ipv4_address: 172.16.238.101
environment:
- ETCDCTL_API=3
command:
- /usr/local/bin/etcd
- --data-dir=/etcd-data
- --name
- node2
- --initial-advertise-peer-urls
- http://172.16.238.101:2380
- --listen-peer-urls
- http://0.0.0.0:2380
- --advertise-client-urls
- http://172.16.238.101:2379
- --listen-client-urls
- http://0.0.0.0:2379
- --initial-cluster
- node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node3=http://172.16.238.102:2380
- --initial-cluster-state
- new
- --initial-cluster-token
- docker-etcd
node3:
image: etcd:93be31d
volumes:
- node3-data:/etcd-data
expose:
- 2379
- 2380
networks:
my_net:
ipv4_address: 172.16.238.102
environment:
- ETCDCTL_API=3
command:
- /usr/local/bin/etcd
- --data-dir=/etcd-data
- --name
- node3
- --initial-advertise-peer-urls
- http://172.16.238.102:2380
- --listen-peer-urls
- http://0.0.0.0:2380
- --advertise-client-urls
- http://172.16.238.102:2379
- --listen-client-urls
- http://0.0.0.0:2379
- --initial-cluster
- node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node3=http://172.16.238.102:2380
- --initial-cluster-state
- new
- --initial-cluster-token
- docker-etcd
volumes:
node1-data:
node2-data:
node3-data:
networks:
my_net:
driver: bridge
ipam:
driver: default
config:
-
subnet: 172.16.238.0/24
然后
docker-compose up
运行起来后,看看, 恩集群跑起来了,蛮健康的。
~/go/gopath/src/github.com/coreos/etcd $ etcdctl --endpoints=172.16.238.100:2379,172.16.238.101:2379,172.16.238.102:2379 endpoint status --write-out=table
+---------------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+---------------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+
| 172.16.238.100:2379 | 422a74f03b622fef | 3.3.0+git | 20 kB | false | 2 | 12 | 12 | |
| 172.16.238.101:2379 | ed635d2a2dbef43d | 3.3.0+git | 20 kB | true | 2 | 12 | 12 | |
| 172.16.238.102:2379 | daf3fd52e3583ff | 3.3.0+git | 20 kB | false | 2 | 12 | 12 | |
+---------------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+
~/go/gopath/src/github.com/coreos/etcd $ etcdctl --endpoints=172.16.238.100:2379,172.16.238.101:2379,172.16.238.102:2379 endpoint health --write-out=table
+---------------------+--------+-------------+-------+
| ENDPOINT | HEALTH | TOOK | ERROR |
+---------------------+--------+-------------+-------+
| 172.16.238.101:2379 | true | 37.75884ms | |
| 172.16.238.100:2379 | true | 39.935792ms | |
| 172.16.238.102:2379 | true | 41.891148ms | |
+---------------------+--------+-------------+-------+
再试试存储值。
我操作 238.100 的etcd,添加了 wjs 值为 “hello world"。
然后在从100 101 102的 etcd 获取 wjs。 和我预想的一样,都返回了 “hello world”。 集群应该正确的
~/gol/etcd $ etcdctl --endpoints=172.16.238.100:2379 put wjs "hello world"
OK
~/gol/etcd $ etcdctl --endpoints=172.16.238.100:2379 get wjs
wjs
hello world
~/gol/etcd $ etcdctl --endpoints=172.16.238.101:2379 get wjs
wjs
hello world
~/gol/etcd $ etcdctl --endpoints=172.16.238.102:2379 get wjs
wjs
hello world
注意点:
1、有的复制尽量复制,自己敲键盘很容易敲错。错了不好找问题。
2、 image: etcd:93be31d 这个image id 很可能是不一样的,需要替换成自己电脑上的
3、volumes:
- node1-data:/etcd-data
这个node1-data 具体 创建在宿主的 /var/lib/docker/volumes 目录。
docker volume ls后可以看到, 这个docker-compose.yml 执行后生成了3个 : etcd_node1-data、 etcd_node2-data 、etcd_node3-data。
~/gol/ubuntu $ docker volume ls
DRIVER VOLUME NAME
local etcd_node1-data
local etcd_node2-data
local etcd_node3-data
local ubuntu_node4-data
如果不小心配置错了 docker-compose.yml 文件,一旦运行过后 docker-compose up 后,这个volume 就被创建了。这样即使再配对了yml文件。还是无法加入到集群中去。可能会受到类似下面这样的提示:
2018-12-20 02:22:36.054228 E | rafthttp: request cluster ID mismatch (got c1bdf48cf7cecbd8 want dbfd87085f2f6f94)
出现这个错误的话,docker volume remove ubuntu_node4-data
然后再来次 docker-compose up 就行了。 还不行的话 就去 /var/lib/docker/volumes 目录手动 rm ubuntu_node4-data -r 掉。
4. 注意前面的 volume 名字,都有etcd 前缀, 那是因为这个 docker-compose.yml 所在的目录叫 etcd。
我可以在同目录下新建 .env 文件 , 然后在下面这个变量中指定名字
COMPOSE_PROJECT_NAME=wjs
也可以在 docker-compose 命令行中用 -p 指定。
-p 指定名字
-p 要出现在 up 前面
docker-compose -p wjs up -d
docker-compose -p wjs stop
docker-compose -p wjs down
一旦指定名字 那么 down stop 等命令时 也得指定
5.
networks:
my_net:
driver: bridge
ipam:
driver: default
config:
-
subnet: 172.16.238.0/24
这一段创建了一个 network 名字 叫 etcd_my_net,同上 etcd_前缀 是因为当前目录叫 etcd。
subnet: 172.16.238.0/24 表明子网掩码是24位的。
~/gol/ubuntu $ docker network ls
NETWORK ID NAME DRIVER SCOPE
42015e6ea887 bridge bridge local
3036c2421aa8 etcd_my_net bridge local
528877d5e07f host host local
4ba426aa9b8b none null local
各个 service 都通过类似下面的命令指定了自己的ip 和 network
networks:
my_net:
ipv4_address: 172.16.238.100
利用docker-compose扩充集群
1. 添加
~/gol/ubuntu $ etcdctl --endpoints=172.16.238.100:2380 member add node4 --peer-urls=http://172.16.238.90:2380
Member f96609339b3aec13 added to cluster dbfd87085f2f6f94
ETCD_NAME="node4"
ETCD_INITIAL_CLUSTER="node3=http://172.16.238.102:2380,node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node4=http://172.16.238.90:2380"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://172.16.238.90:2380"
ETCD_INITIAL_CLUSTER_STATE="existing"
2. 根据添加反馈的信息,配置要添加的 etcd
version : "3.6"
services:
node1:
image: etcd:93be31d
volumes:
- node4-data:/etcd-data
expose:
- 2379
- 2380
networks:
default:
ipv4_address: 172.16.238.90
environment:
- ETCDCTL_API=3
command:
- /usr/local/bin/etcd
- --data-dir=/etcd-data
- --name
- node4
- --initial-advertise-peer-urls
- http://172.16.238.90:2380
- --listen-peer-urls
- http://0.0.0.0:2380
- --advertise-client-urls
- http://172.16.238.90:2379
- --listen-client-urls
- http://0.0.0.0:2379
- --initial-cluster
- node3=http://172.16.238.102:2380,node1=http://172.16.238.100:2380,node2=http://172.16.238.101:2380,node4=http://172.16.238.90:2380
- --initial-cluster-state
- existing
- --initial-cluster-token
- docker-etcd
volumes:
node4-data:
networks:
default:
external:
name: etcd_my_net
3. 运行
docker-compose up
4. 注意点
networks:
default:
external:
name: etcd_my_net
如上可以要加到一个已经创建好的 network中去。
如果不小心把 数据目录删了 毁了。
可能遇到下面这个错误:
panic: tocommit(25) is out of range [lastIndex(0)]. Was the raft log corrupted, truncated, or lost?
那就把 节点 删了,重新添加进去后 再吧。
5. 运行下看看
~/gol/ubuntu $ etcdctl --endpoints=172.16.238.100:2380,172.16.238.101:2380,172.16.238.102:2380,172.16.238.90:2380 endpoint status --write-out=table
+---------------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+
| ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+---------------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+
| 172.16.238.100:2380 | 422a74f03b622fef | 3.3.0+git | 20 kB | true | 4 | 22 | 22 | |
| 172.16.238.101:2380 | ed635d2a2dbef43d | 3.3.0+git | 20 kB | false | 4 | 22 | 22 | |
| 172.16.238.102:2380 | daf3fd52e3583ff | 3.3.0+git | 20 kB | false | 4 | 22 | 22 | |
| 172.16.238.90:2380 | 4b50839972742030 | 3.3.0+git | 20 kB | false | 4 | 22 | 22 | |
+---------------------+------------------+-----------+---------+-----------+-----------+------------+--------------------+--------+
参考链接https://blog.youkuaiyun.com/yunlilang/article/details/79726424