背景：
简明：
内网环境harbor1.5.0升级要1.9
要点：
（1）harbor数据库在v1.6版本做了很大的升级。数据库从原来的MySQL升级为postgresql数据库。根据官方文档的解释，旧版本的harbor升级到最新的v1.9,需先将harbor升级到v1.6,再升级到更高的版本。
（2）从v1.8.0开始，Harbor的配置已更改为.yml文件，迁移器将配置文件从harbor.cfg转换为harbor.yml。
（3）从v1.6.0开始，Harbor会在启动时自动尝试迁移数据库，因此如果从v1.6.0或更高版本升级，则无需调用迁移器工具来迁移数据库，只需要更新cfg文件即可。
（4）从v1.7.0开始，支持Helm Chart部署。
方案：
先从1.5升级到1.6，再从1.6升级到1.7，然后升级到1.9（按照官方文档的升级路线）
开始
一、harbor1.5.0升级到harbor1.6.3（既要升级db也要升级cfg）
（1）停下在运行的harbor
[root@bogon ~]# cd harbor
[root@bogon harbor]# ls
common docker-compose.notary.yml ha harbor.v1.5.0.tar.gz LICENSE prepare
docker-compose.clair.yml docker-compose.yml harbor.cfg install.sh NOTICE
[root@bogon harbor]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
717ef68de036 vmware/harbor-jobservice:v1.5.0 "/harbor/start.sh" 40 minutes ago Up 40 minutes harbor-jobservice
ee95661aa905 vmware/nginx-photon:v1.5.0 "nginx -g 'daemon ..." 40 minutes ago Up 38 minutes (healthy) 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp nginx
85aa0167d66a vmware/harbor-ui:v1.5.0 "/harbor/start.sh" 40 minutes ago Up 40 minutes (healthy) harbor-ui
cb256cbf09ed vmware/clair-photon:v2.0.1-v1.5.0 "/docker-entrypoin..." 40 minutes ago Up 40 minutes (healthy) 6060-6061/tcp clair
e3aa51500bdc vmware/postgresql-photon:v1.5.0 "/entrypoint.sh po..." 40 minutes ago Up 40 minutes (healthy) 5432/tcp clair-db
dce4338d4071 vmware/registry-photon:v2.6.2-v1.5.0 "/entrypoint.sh se..." 40 minutes ago Up 40 minutes (healthy) 5000/tcp registry
43e6fbebc160 vmware/harbor-db:v1.5.0 "/usr/local/bin/do..." 40 minutes ago Up 40 minutes (healthy) 3306/tcp harbor-db
eca25eea5b18 vmware/harbor-adminserver:v1.5.0 "/harbor/start.sh" 40 minutes ago Up 40 minutes (healthy) harbor-adminserver
d68e94c20695 vmware/redis-photon:v1.5.0 "docker-entrypoint..." 40 minutes ago Up 40 minutes 6379/tcp redis
280da2ce97a9 vmware/harbor-log:v1.5.0 "/bin/sh -c /usr/l..." 40 minutes ago Up 40 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log
[root@bogon harbor]#
[root@bogon harbor]# docker-compose -f docker-compose.yml -f docker-compose.clair.yml down -v
（2）备份harbor目录
[root@bogon harbor]# cd ..
[root@bogon lanjian]# cp -r harbor harbor1.5.0-bak
（3）下载harbor 1.6.3镜像、数据迁移工具，以及1.5.0数据迁移工具
Get the lastest Harbor release package from Github
[root@localhost lanjian]# docker pull goharbor/harbor-migrator:v1.6.0
Trying to pull repository docker.io/goharbor/harbor-migrator ...
v1.6.0: Pulling from docker.io/goharbor/harbor-migrator
51be32cd3c9d: Pull complete
7e261d84cd9e: Pull complete
f52c9d206429: Pull complete
e399eeb55d79: Pull complete
Digest: sha256:a8398de33c60bba7ad93ddcf977791e187e8b8553214f5589c749959bd265712
Status: Downloaded newer image for docker.io/goharbor/harbor-migrator:v1.6.0
[root@localhost lanjian]# docker images |grep harbor
docker.io/goharbor/harbor-migrator v1.6.3 dac56e1d921f 11 months ago 799 MB
docker.io/goharbor/harbor-migrator v1.6.0 22775c4e4066 14 months ago 803 MB
vmware/harbor-log v1.5.0 62bb6b8350d9 18 months ago 200 MB
vmware/harbor-jobservice v1.5.0 aca9fd2e867f 18 months ago 194 MB
vmware/harbor-ui v1.5.0 1055166068d0 18 months ago 212 MB
vmware/harbor-adminserver v1.5.0 019bc4544829 18 months ago 183 MB
vmware/harbor-db v1.5.0 82354dcf564f 18 months ago 526 MB
vmware/harbor-migrator v1.5.0 466c57ab0dc3 18 months ago 1.16 GB
（4）备份cfg和db
[root@localhost harbor]# docker run -it --rm -e DB_USR=root -e DB_PWD=root123 -v /data/database:/var/lib/mysql -v /home/lanjian/harbor/harbor.cfg:/harbor-migration/harbor-cfg/harbor.cfg -v /home/lanjian/backup:/harbor-migration/backup goharbor/harbor-migrator:v1.6.3 backup
（5）升级数据库框架（将MySQL框架升级为postgresql）并同步数据
[root@localhost harbor]# docker run -it --rm -e DB_USR=root -e DB_PWD=root123 -v /data/database:/var/lib/mysql -v /home/lanjian/harbor/harbor.cfg:/harbor-migration/harbor-cfg/harbor.cfg goharbor/harbor-migrator:v1.6.0 up
（6）升级harbor镜像及cfg并拷贝过去
[root@localhost lanjian]# mkdir harbor-1.6.3
[root@localhost lanjian]# cd 下载
[root@localhost 下载]# tar zxvf harbor-offline-installer-v1.6.3.tgz -C ../harbor1.6.3/
[root@localhost 下载]# cd ../harbor1.6.3/
[root@localhost harbor1.6.3]# ls
harbor
[root@localhost harbor1.6.3]# cd harbor/
[root@localhost harbor]# ls
common docker-compose.clair.yml docker-compose.yml harbor.cfg install.sh NOTICE prepare
docker-compose.chartmuseum.yml docker-compose.notary.yml ha harbor.v1.6.3.tar.gz LICENSE open_source_license
[root@localhost harbor]# docker run -it --rm -v /root/backup/harbor.cfg:/harbor-migration/harbor-cfg/harbor.cfg goharbor/harbor-migrator:v1.6.0 --cfg up
[root@localhost backup]# cp harbor.cfg /home/lanjian/harbor/
cp：是否覆盖"/home/lanjian/harbor/harbor.cfg"？ yes
[root@localhost backup]#
（7）启动harbor1.6.3并验证
[root@localhost harbor]# ./install.sh
验证下版本是否完成升级，以及用户、镜像是否都正常
二、harbor1.6.3升级到1.7.0（相当于只需要升级cfg文件不用管db）
（1）同样的，停下正在运行的harbor
[root@localhost lanjian]# cd harbor
[root@localhost harbor]# docker-compose -f docker-compose.yml -f docker-compose.clair.yml down
（2）备份数据
[root@localhost lanjian]# mkdir harbor1.6.0-bak
[root@localhost lanjian]# mv harbor harbor1.6.0-bak/harbor
[root@localhost lanjian]# cp -r /data/database harbor1.6.0-bak/
（3）下载harbor v1.7.0离线安装包解压和migrate容器镜像
[root@localhost 下载]# tar zxvf harbor-offline-installer-v1.7.0.tgz
[root@localhost 下载]# mv harbor ../harbor
[root@localhost lanjian]# docker pull goharbor/harbor-migrator:v1.7.0
（4）升级harbor.cfg
官方文档：NOTE: The ${harbor_cfg} will be overwritten, you must move it to your installation directory after migration.
即是在备份目录里直接升级cfg文件的，然后再拷贝到新的harbor目录下，所以为防止意外，先对harbor.cfg再备份一次后再升级
PS：这里挂载volume时需要注意宿主机和容器都需要绝对路径，不然会报错，宿主机虽然可以相对路径，但其实所谓的相对路径指的是/var/lib/docker/volumes/，与宿主机的当前目录无关。
[root@localhost harbor]# cd ../harbor1.6.0-bak/
[root@localhost harbor]# cp harbor.cfg harbor.cfg.bak
[root@localhost harbor]# docker run -it --rm -v /home/lanjian/harbor1.6.0-bak/harbor/harbor.cfg:/harbor-migration/harbor-cfg/harbor.cfg docker.io/goharbor/harbor-migrator:v1.7.0 --cfg up
Please backup before upgrade,
Enter y to continue updating or n to abort: y
The path of the migrated harbor.cfg is not set, the input file will be overwritten.
input version: 1.6.0, migrator chain: ['1.7.0']
migrating to version 1.7.0
Written new values to /harbor-migration/harbor-cfg/harbor.cfg
我们看到当前cfg文件已经被修改了，不是1.6的了，将其拷贝到1.7的安装路径下面即可
[root@localhost harbor]# cat harbor.cfg|grep version
#This attribute is for migrator to detect the version of the .cfg file, DO NOT MODIFY!
_version = 1.7.0
#Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.
[root@localhost harbor]# mv harbor.cfg /home/lanjian/harbor/
（5）启动harbor v1.7.0并打开UI确认升级完成情况
[root@localhost harbor]# cd /home/lanjian/harbor
[root@localhost harbor]# ./install.sh
如果需要安装clair则带上参数 --with-clair哈
三、harbor 1.7升级到1.9（升级cfg为yml）
特别注意两点：
With the introduction of storage and artifact quotas in version 1.9.0, migration from 1.7.x and 1.8.x might take a few minutes. This is because the core walks through all blobs in the registry and populates the database with information about the layers and artifacts in projects.
因为引入了storage and artifact quotas，这次升级会比较费时间，核心会遍历仓库和数据库中的层信息
With the introduction of storage and artifact quotas in version 1.9.0, replication between version 1.9.0 and a previous version of Harbor does not work. You must upgrade all Harbor nodes to 1.9.0 if you have configured replication between them.
因为引入了storage and artifact quotas，所以同步复制只能再1.9的版本之间进行，如果其中一个仓库低于1.9则无法进行同步
（1）停止正在运行的harbor
[root@bogon harbor]# docker-compose -f docker-compose.yml -f docker-compose.clair.yml down -v
（2）备份数据
[root@bogon lanjian]# mkdir harbor1.7.0-bak
[root@bogon lanjian]# mv harbor harbor1.7.0-bak/harbor
[root@bogon lanjian]# cp -r /data/database harbor1.7.0-bak/
（3）下载harbor 1.9.0安装包解压及migrator镜像
[root@bogon lanjian]# docker pull goharbor/harbor-migrator:v1.9.0
[root@bogon 下载]# tar zxvf harbor-offline-installer-v1.9.0.tgz -C /home/lanjian/
[root@bogon lanjian]# cd harbor
[root@bogon harbor]# ls
harbor.v1.9.0.tar.gz harbor.yml install.sh LICENSE prepare
（4）升级cfg为yml文件
[root@bogon harbor]# docker run -it --rm -v /home/lanjian/harbor1.7.0-bak/harbor/harbor.cfg:/harbor-migration/harbor-cfg/harbor.yml -v /home/lanjian/harbor/harbor.yml:/harbor-migration/harbor-cfg-out/harbor.yml docker.io/goharbor/harbor-migrator:v1.9.0 --cfg up
发现有报错：jinja2.exceptions.UndefinedError: ‘chart’ is undefined
Please backup before upgrade,
Enter y to continue updating or n to abort: y
Command for config file migration: python ./cfg/run.py --input /harbor-migration/harbor-cfg/harbor.yml --output /harbor-migration/harbor-cfg-out/harbor.yml
input version: 1.7.0, migrator chain: ['1.8.0', '1.9.0']
migrating to version 1.8.0
migrating to version 1.9.0
Traceback (most recent call last):
File "./cfg/run.py", line 70, in <module>
main()
File "./cfg/run.py", line 46, in main
m.migrate(curr_input_path, curr_output_path)
File "/harbor-migration/cfg/migrator_1_9_0/__init__.py", line 21, in migrate
f.write(tpl.render(**config_dict))
File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/harbor-migration/cfg/migrator_1_9_0/harbor.yml.jinja", line 122, in top-level template code
absolute_url: {{ chart.absolute_url if chart.absolute_url == 'enabled' else 'disabled' }}
File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 430, in getattr
return getattr(obj, attribute)
jinja2.exceptions.UndefinedError: 'chart' is undefined
通过查看issue 9174以及issue 9146：
发现很多人都报了这个错误，原因时由于 1.7.x 版本的 harbar.cfg 中不存在下面的相关的配置（在1.8版本开始在模板中使用了chart，且为必须项Required Parameters for Harbor Deployment）：
chart:
absolute_url: disabled
解决方案有两个：
（1）先升级到1.8.x版本，并在harbor.yml中加入上述chart配置。
（2）替换官方的migrator镜像为issue中大神thatsk提供的镜像。
为了避免麻烦，我直接选择方案二，替换掉官方的同步迁移镜像
[root@bogon harbor]# docker pull thatsk/harbor-migrator:v1.9.0
[root@bogon harbor]# docker run -it --rm -v /home/lanjian/harbor1.7.0-bak/harbor/harbor.cfg:/harbor-migration/harbor-cfg/harbor.yml -v /home/lanjian/harbor/harbor.yml:/harbor-migration/harbor-cfg-out/harbor.yml docker.io/thatsk/harbor-migrator:v1.9.0 --cfg up
Please backup before upgrade,
Enter y to continue updating or n to abort: y
Command for config file migration: python ./cfg/run.py --input /harbor-migration/harbor-cfg/harbor.yml --output /harbor-migration/harbor-cfg-out/harbor.yml
input version: 1.7.0, migrator chain: ['1.8.0', '1.9.0']
migrating to version 1.8.0
migrating to version 1.9.0
Written new values to /harbor-migration/harbor-cfg-out/harbor.yml
升级成功
（5）启动harbor并检查UI
[root@bogon harbor]# ./install.sh --with-clair
[Step 0]: checking installation environment ...
✖ Need to upgrade docker package to 17.06.0+.
[root@bogon harbor]# docker --version
Docker version 1.13.1, build b2f74b2/1.13.1
提示docker版本太低，需要升级，那么就升吧，步骤：
先停止docker，删除旧版本，然后安装yum管理工具，然后添加国内镜像源，查看版本，安装指定的docker版本以及docker-compose版本：
[root@bogon bin]# systemctl stop docker
[root@bogon bin]# rpm -qa | grep docker
docker-1.13.1-94.gitb2f74b2.el7.centos.x86_64
docker-client-1.13.1-94.gitb2f74b2.el7.centos.x86_64
docker-common-1.13.1-94.gitb2f74b2.el7.centos.x86_64
[root@bogon bin]# yum remove docker-common-1.13.1-94.gitb2f74b2.el7.centos.x86_64
[root@bogon bin]# yum install -y yum-utils
[root@bogon bin]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
已加载插件：fastestmirror, langpacks
adding repo from: https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
grabbing file https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
[root@bogon bin]# yum list docker-ce --showduplicates|sort -r
已加载插件：fastestmirror, langpacks
可安装的软件包
* updates: mirrors.neusoft.edu.cn
Loading mirror speeds from cached hostfile
* extras: mirrors.neusoft.edu.cn
* epel: mirror.lzu.edu.cn
docker-ce.x86_64 3:19.03.5-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.4-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.3-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.2-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.1-3.el7 docker-ce-stable
docker-ce.x86_64 3:19.03.0-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.9-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.8-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.7-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.6-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.5-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.4-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.3-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.2-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.1-3.el7 docker-ce-stable
docker-ce.x86_64 3:18.09.0-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.3.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.2.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.06.0.ce-3.el7 docker-ce-stable
docker-ce.x86_64 18.03.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 18.03.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.12.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.12.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.09.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.09.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.06.2.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.06.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.06.0.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.3.ce-1.el7 docker-ce-stable
docker-ce.x86_64 17.03.2.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.1.ce-1.el7.centos docker-ce-stable
docker-ce.x86_64 17.03.0.ce-1.el7.centos docker-ce-stable
* base: mirrors.nju.edu.cn
[root@bogon bin]# yum install docker-ce.x86_64.18.03.0.ce-1.el7.centos
[root@bogon bin]# systemctl start docker
[root@bogon bin]# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
[root@bogon bin]# docker version
Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:09:15 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:13:03 2018
OS/Arch: linux/amd64
Experimental: false
PS：这里又趟坑了，在yum-config-manager时报错：
File "/bin/yum-config-manager", line 135
except yum.Errors.RepoError, e:
^
SyntaxError: invalid syntax
这时python升级为3.x后的问题，yum-config-manager还是用Python2写的，所以我们需要替换掉这个文件的Python版本：
[root@bogon bin]# head -n 1 /bin/yum-config-manager
#!/usr/bin/python -tt
[root@bogon bin]# ll /usr/bin/ |grep -i pytho
-rwxr-xr-x. 1 root root 11312 11月 14 2018 abrt-action-analyze-python
lrwxrwxrwx 1 root root 16 9月 29 23:10 python -> /usr/bin/python3
lrwxrwxrwx 1 root root 9 4月 27 2019 python2 -> python2.7
-rwxr-xr-x 1 root root 7216 4月 9 2019 python2.7
lrwxrwxrwx. 1 root root 24 3月 4 2019 python3 -> /usr/local/bin/python3.6
lrwxrwxrwx 1 root root 24 3月 14 2019 python_backup_20190314 -> /etc/alternatives/python
lrwxrwxrwx. 1 root root 7 3月 3 2019 python_old -> python2
[root@bogon bin]# vim /bin/yum-config-manager
[root@bogon bin]# head -n 1 /bin/yum-config-manager
#!/usr/bin/python2 -tt
继续升级docker-compose，这个简单些，可以通过pip安装：
[root@bogon harbor]# rm /usr/local/bin/docker-compose
[root@bogon harbor]# pip install docker-compose==1.18.0
[root@bogon harbor]# docker-compose version
docker-compose version 1.18.0, build 8dd22a9
docker-py version: 2.7.0
CPython version: 3.6.0
OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017
好了，我们终于可以继续启动harbor了：
[root@bogon harbor]# ./install.sh --with-clair
...
...
✔ ----Harbor has been installed and started successfully.----
Now you should be able to visit the admin portal at http://192.168.137.7.
For more details, please visit https://github.com/goharbor/harbor .
完成，检查下UI、镜像和log，升级成功！经历了那么多版本，终于完成了，奈斯！
扫一扫
1304
到【灌水乐园】发言
