Docker 容器文件存储驱动 Overlay2

码字不易，转载请注明出处！！

---

写在前面

• Docker 文件存储驱动有很多种，比如 overlay2、aufs等，可以通过命令 docker info 查看：

> # docker info
> Storage Driver: overlay2

• 镜像是只读的，而容器是可读可写的（这将体现在）

几个关键目录

docker 默认路径：/var/lib/docker

[root@VM-0-7-centos docker]# tree -L 2
.
├── buildkit
│   ├── cache.db
│   ├── containerdmeta.db
│   ├── content
│   ├── executor
│   ├── metadata_v2.db
│   └── snapshots.db
├── containers
│   ├── bc66c5693bb082c2932929d6be3e351bcd8fd176f5ea9113b5874416ca921fdf
│   └── bdba3fa6c74b828d2971547c339412853928213c2f3a75c76de58bc88d8c5d1e
├── image
│   └── overlay2
├── network
│   └── files
├── overlay2
│   ├── 020a1d83c7a2880a5aeaaeefae78ba3229a6acdc45284b8ce9690afbfa709d07
│   ├── 025d66d4c070a7ef461939653fe229b57fdb2c9525e44db2f23b9a7ba11d3cee
│   ├── 0794172ce57701a8fba81c3fcd2c7048502ae4dc237ea5d5793d7a7b2abb1f05
│   ├── 1e97ea0c744e8ffc92a7427c05f32d1b99210fd8868c2d6f0777c693cc68bc66
│   ├── 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d
│   ├── 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d-init
│   ├── 3bc3a1c925a62dee8153595be93b57affe59bc7660aa7ad6b4def2531dce7b76
│   ├── 3bc3a1c925a62dee8153595be93b57affe59bc7660aa7ad6b4def2531dce7b76-init
│   ├── 432352c81670f9b6e2f7634bc5812452cd9c15bfaf7d880fd6a81436477f8c4a
│   ├── 685986bdc2572b46b3e034d81d454a3d9b90b0db544a0308881586154f156fd9
│   ├── 68768e41e7e485a0de695ff2272d378b0309a0d9d5695d80ddb35f51aaf9a236
│   ├── 6c3e399bfbd458d9aef157ee99045fef6a84d936e21a3f4f6e248310b432af59
│   ├── 6fe1babc4c723e81afcdd41ad9d6d0ddc4471974ce32861b8cf5da7f8f255a55
│   ├── 7bfca74231e537d58afa6adc31453f1f3e3a796ee80026b88490fca3ed9b4e6e
│   ├── 88f3a80280da0c23b7d44a8f943703722262cc29dcd64d481f2050ade4aa9aa4
│   ├── b4fa632bcd08c5050e6691e3df80016c9391dc1f48ba8a00b0024c8d1fd66187
│   ├── be8a908c293a4f380a99a4726b0d2f29a11e7aa8e05f089bdef4c88bfe2cc968
│   ├── c5131b6ee23d74fe73057ca566b2d318062a2832d83e44157b28741a8e0e672a
│   ├── cc9f0182b0710b8eda8c64d434eeb7c775a53664223338fc03495d75992db042
│   ├── d7bd582337f9687319331465ce30e052e0c7e4cfc685d3af02643ca2a137de06
│   ├── f3a64c9738393c0c8b0f7d1397b3108a357286646f92c901a413f6411e2df808
│   └── l
├── plugins
│   ├── storage
│   └── tmp
├── runtimes
├── swarm
├── tmp
├── trust
└── volumes
    ├── backingFsBlockDev
    └── metadata.db

• containers：容器信息，文件名与容器id一一对应
• images：镜像信息
• overlay2：虚拟文件系统信息

从一个容器开始查找

1. 查看容器信息

[root@VM-0-7-centos docker]# docker ps | grep mysql
bdba3fa6c74b   mysql:latest   "docker-entrypoint.s…"   3 days ago    Up 3 days   33060/tcp, 0.0.0.0:13307->3306/tcp, :::13307->3306/tcp   mysql

得到容器ID：bdba3fa6c74b

2. 进入 image/overlay2/layerdb/mounts，找到这个容器的目录

[root@VM-0-7-centos mounts]# pwd
/var/lib/docker/image/overlay2/layerdb/mounts
[root@VM-0-7-centos mounts]# ls
bc66c5693bb082c2932929d6be3e351bcd8fd176f5ea9113b5874416ca921fdf  bdba3fa6c74b828d2971547c339412853928213c2f3a75c76de58bc88d8c5d1e

3. 查看这个容器目录下的 init-id 和 mount-id 内容

[root@VM-0-7-centos mounts]# cd bdba3fa6c74b828d2971547c339412853928213c2f3a75c76de58bc88d8c5d1e/
[root@VM-0-7-centos bdba3fa6c74b828d2971547c339412853928213c2f3a75c76de58bc88d8c5d1e]# ls
init-id  mount-id  parent
[root@VM-0-7-centos bdba3fa6c74b828d2971547c339412853928213c2f3a75c76de58bc88d8c5d1e]# cat init-id 
2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d-init
[root@VM-0-7-centos bdba3fa6c74b828d2971547c339412853928213c2f3a75c76de58bc88d8c5d1e]# cat mount-id 
2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d

得到的两串值对应的是overlay2目录下的文件名

4. 找到容器所挂载的虚拟目录

[root@VM-0-7-centos overlay2]# cd /var/lib/docker/overlay2
[root@VM-0-7-centos overlay2]# cd 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# ls
diff  link  lower  merged  work

diff：容器临时创建的可读可写层
link：记录当前容器所挂载的可读写层信息，它的内容对应的是/var/lib/docker/overlay2/l下的目录名，该路径的目录都是链接文件，最后还是会指向上面提到的这个diff文件
lower：记录当前容器的只读层（通常是镜像层）的信息，会有很多层，对应的也是/var/lib/docker/overlay2/l下的目录名
merged：字面意思，可读可写的diff层和只读的镜像层合并的结果，可以看到diff目录下有的东西，merged也会有

[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# ls diff/
logs  run  tmp
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# ls merged/
bin  boot  dev  docker-entrypoint-initdb.d  entrypoint.sh  etc  home  lib  lib64  logs  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# cat diff/run/mysqld/mysqld.pid
1
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# cat merged/run/mysqld/mysqld.pid 
1

我们也可以进入容器内部，会发现内容和merged下的内容是一致的

[root@VM-0-7-centos merged]# docker exec -it mysql /bin/bash
root@bdba3fa6c74b:/# ls
bin  boot  dev  docker-entrypoint-initdb.d  entrypoint.sh  etc  home  lib  lib64  logs  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

所以，我们在diff、merged或者容器三者之一所做的操作，都会同时作用在另外两者，因为他们都是相互挂载的：

[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# echo 'oysq' > diff/root/a.txt
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# echo 'oysq' > merged/root/b.txt
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# docker exec -it mysql /bin/bash
root@bdba3fa6c74b:/# echo 'oysq' > root/c.txt
root@bdba3fa6c74b:/# ls root/
a.txt  b.txt  c.txt
root@bdba3fa6c74b:/# exit
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# ls diff/root/
a.txt  b.txt  c.txt
[root@VM-0-7-centos 2d62d796656b32086ca6105f7ab3f97d630e6fda28dd2e023e5dd6cce161849d]# ls merged/root/
a.txt  b.txt  c.txt

但是！！当我们启动容器的时候，把某个路径 bind （挂载）到宿主机时，容器内的这个路径将不再和diff、merged关联，比如本文的例子，通过docker inspect mysql发现，容器的/logs目录被挂载到宿主机的/usr/local/mysql/logs目录下：

"Mounts": [
            {
                "Type": "bind",
                "Source": "/usr/local/mysql/conf",
                "Destination": "/etc/mysql/conf.d",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/usr/local/mysql/logs",
                "Destination": "/logs",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/usr/local/mysql/data",
                "Destination": "/var/lib/mysql",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],

因此，容器内的/logs和宿主机的/usr/local/mysql/logs会相互修改，而对diff和merged内的修改，虽然会在diff和merged之间相互影响（因为diff是挂载在merged的），但是对容器已经没有作用了

5. 重启、删除容器

• 重启容器：diff和merged都不会受到影响
• 删除容器：diff和merged都会被初始化为镜像的可读写层，而bind的目录不会受影响，这也是为什么容器要把需要持久化的数据bind到宿主机下