本文档详细介绍了如何在多台服务器上部署Ceph存储集群，包括服务器规格设定、环境配置、yum源设置、服务安装、集群配置、OSD创建、监控及管理节点扩展，以及Ceph Dashboard的部署和与P8S的接入。
文章目录
服务器规格
| ip | 主机名 | 服务器规格 |
|---|---|---|
| 11.0.1.46 | node1 | 2c4G80G20G |
| 11.0.1.47 | node2 | 2c4G80G20G |
| 11.0.1.48 | node3 | 2c4G80G20G |
配置基础环境
所有节点
# 关闭 selinux 和防火墙
#配置主机名映射
cat >> /etc/hosts <<eof
11.0.1.46 node1
11.0.1.47 node2
11.0.1.48 node3
eof
# 配置 yum repo
rm -rf /etc/yum.repos.d/*
# 修改主机名 hostnamectl set-hostname node1/node2/node3
# 配置时间同步
# 在 node1 配置
# 将默认四条改成这个
server 11.0.1.46 iburst
# 添加以下两条
allow 11.0.1.0/24
local stratum 10
systemctl restart chronyd
# node2 node3 配置
# 将默认四条改成这个
server 11.0.1.46 iburst
systemctl restart chronyd
# 在node2/ node3 查看时间是否 同步 看Last 是否又数据显示 如果 显示 是0 则 时间没有同步
chronyc sources
210 Number of sources = 1
MS Name/IP address Stratum Poll Reach LastRx Last sample
===============================================================================
^* node1 10 6 377 199 +384ns[-3742ns] +/- 8529ns
配置yum rep源和免密
在node1节点配置
cd /etc/yum.repos.d/
# 配置 centos7源
wget -O CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
#epel 仓库源
wget -O epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
# 配置 ceph repo
cat > ceph << eof
[ceph-nautilus]
name=ceph-nautilus
baseurl=http://mirrors.aliyun.com/ceph/rpm-nautilus/el7/x86_64/
enabled=1
gpgcheck=0
[ceph-nautilus-noarch]
name=ceph-nautilus-noarch
baseurl=http://mirrors.aliyun.com/ceph/rpm-nautilus/el7/noarch/
enabled=1
gpgcheck=0
eof
# 配置免密
ssh-keygen # 一直按回车就可以了
ssh-copy-id -i /root/.ssh/id_rsa.pub root@node1
ssh-copy-id -i /root/.ssh/id_rsa.pub root@node2
ssh-copy-id -i /root/.ssh/id_rsa.pub root@node3
在node2/node3节点配置
# 配置ceph repo
cat >> /etc/yum.repos.d/ceph.repo << EOF
[ceph]
name=ceph
baseurl=ftp://11.0.1.46/ceph/
gpgcheck=0
EOF
yum clean all
yum makecache
安装服务
在 node1 配置
yum install -y ceph-deploy
三个节点安装
yum install ceph -y
配置集群
在 node1 配置
mkdir ceph-cluster
cd ceph-cluster
ceph-deploy new --cluster-network 11.0.1.0/24 --public-network 11.0.1.0/24 node1
-----
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (2.0.1): /usr/bin/ceph-deploy new --cluster-network 11.0.1.0/24 --public-network 11.0.1.0/24 node1
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] func : <function new at 0x7fb3fce91de8>
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fb3fc60b560>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] ssh_copykey : True
[ceph_deploy.cli][INFO ] mon : ['node1']
[ceph_deploy.cli][INFO ] public_network : 11.0.1.0/24
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] cluster_network : 11.0.1.0/24
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] fsid : None
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[node1][DEBUG ] connected to host: node1
[node1][DEBUG ] detect platform information from remote host
[node1][DEBUG ] detect machine type
[node1][DEBUG ] find the location of an executable
[node1][INFO ] Running command: /usr/sbin/ip link show
[node1][INFO ] Running command: /usr/sbin/ip addr show
[node1][DEBUG ] IP addresses found: [u'11.0.1.46']
[ceph_deploy.new][DEBUG ] Resolving host node1
[ceph_deploy.new][DEBUG ] Monitor node1 at 11.0.1.46
[ceph_deploy.new][DEBUG ] Monitor initial members are ['node1']
[ceph_deploy.new][DEBUG ] Monitor addrs are [u'11.0.1.46']
[ceph_deploy.new][DEBUG ] Creating a random mon key...
[ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
-----
安装ceph ceph-radosgw
node1节点
ceph-deploy install --no-adjust-repos node1 node2 node3
-------------------------------------------------------------------------------------------------
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (2.0.1): /usr/bin/ceph-deploy install --no-adjust-repos node1 node2 node3
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] testing : None
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7fcc6afa0d40>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] dev_commit : None
[ceph_deploy.cli][INFO ] install_mds : False
[ceph_deploy.cli][INFO ] stable : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] adjust_repos : False
[ceph_deploy.cli][INFO ] func : <function install at 0x7fcc6bc895f0>
[ceph_deploy.cli][INFO ] install_mgr : False
[ceph_deploy.cli][INFO ] install_all : False
[ceph_deploy.cli][INFO ] repo : False
[ceph_deploy.cli][INFO ] host : ['node1', 'node2', 'node3']
[ceph_deploy.cli][INFO ] install_rgw : False
[ceph_deploy.cli][INFO ] install_tests : False
[ceph_deploy.cli][INFO ] repo_url : None
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] install_osd : False
[ceph_deploy.cli][INFO ] version_kind : stable
[ceph_deploy.cli][INFO ] install_common : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] dev : master
[ceph_deploy.cli][INFO ] nogpgcheck : False
[ceph_deploy.cli][INFO ] local_mirror : None
[ceph_deploy.cli][INFO ] release : None
[ceph_deploy.cli][INFO ] install_mon : False
[ceph_deploy.cli][INFO ] gpg_url : None
[ceph_deploy.install][DEBUG ] Installing stable version mimic on cluster ceph hosts node1 node2 node3
[ceph_deploy.install][DEBUG ] Detecting platform for host node1 ...
[node1][DEBUG ] connected to host: node1
[node1][DEBUG ] detect platform information from remote host
[node1][DEBUG ] detect machine type
[ceph_deploy.install][INFO ] Distro info: CentOS Linux 7.9.2009 Core
[node1][INFO ] installing Ceph on node1
[node1][INFO ] Running command: yum clean all
[node1][DEBUG ] Loaded plugins: fastestmirror
[node1][DEBUG ] Cleaning repos: centos ceph
[node1][DEBUG ] Cleaning up list of fastest mirrors
[node1][DEBUG ] Other repos take up 38 M of disk space (use --verbose for details)
[node1][INFO ] Running command: yum -y install ceph ceph-radosgw
[node1][DEBUG ] Loaded plugins: fastestmirror
[node1][DEBUG ] Determining fastest mirrors
[node1][DEBUG ] Package 2:ceph-14.2.22-0.el7.x86_64 already installed and latest version
[node1][DEBUG ] Package 2:ceph-radosgw-14.2.22-0.el7.x86_64 already installed and latest version
[node1][DEBUG ] Nothing to do
[node1][INFO ] Running command: ceph --version
[node1][DEBUG ] ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable)
[ceph_deploy.install][DEBUG ] Detecting platform for host node2 ...
[node2][DEBUG ] connected to host: node2
[node2][DEBUG ] detect platform information from remote host
[node2][DEBUG ] detect machine type
[ceph_deploy.install][INFO ] Distro info: CentOS Linux 7.9.2009 Core
[node2][INFO ] installing Ceph on node2
[node2][INFO ] Running command: yum clean all
[node2][DEBUG ] Loaded plugins: fastestmirror
[node2][DEBUG ] Cleaning repos: ceph
[node2][DEBUG ] Cleaning up list of fastest mirrors
[node2][DEBUG ] Other repos take up 51 M of disk space (use --verbose for details)
[node2][INFO ] Running command: yum -y install ceph ceph-radosgw
[node2][DEBUG ] Loaded plugins: fastestmirror
[node2][DEBUG ] Determining fastest mirrors
[node2][DEBUG ] Package 2:ceph-14.2.22-0.el7.x86_64 already installed and latest version
[node2][DEBUG ] Package 2:ceph-radosgw-14.2.22-0.el7.x86_64 already installed and latest version
[node2][DEBUG ] Nothing to do
[node2][INFO ] Running command: ceph --version
[node2][DEBUG ] ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable)
[ceph_deploy.install][DEBUG ] Detecting platform for host node3 ...
[node3][DEBUG ] connected to host: node3
[node3][DEBUG ] detect platform information from remote host
[node3][DEBUG ] detect machine type
[ceph_deploy.install][INFO ] Distro info: CentOS Linux 7.9.2009 Core
[node3][INFO ] installing Ceph on node3
[node3][INFO ] Running command: yum clean all
[node3][DEBUG ] Loaded plugins: fastestmirror
[node3][DEBUG ] Cleaning repos: ceph
[node3][DEBUG ] Cleaning up list of fastest mirrors
[node3][DEBUG ] Other repos take up 51 M of disk space (use --verbose for details)
[node3][INFO ] Running command: yum -y install ceph ceph-radosgw
[node3][DEBUG ] Loaded plugins: fastestmirror
[node3][DEBUG ] Determining fastest mirrors
[node3][DEBUG ] Package 2:ceph-14.2.22-0.el7.x86_64 already installed and latest version
[node3][DEBUG ] Package 2:ceph-radosgw-14.2.22-0.el7.x86_64 already installed and latest version
[node3][DEBUG ] Nothing to do
[node3][INFO ] Running command: ceph --version
[node3][DEBUG ] ceph version 14.2.22 (ca74598065096e6fcbd8433c8779a2be0c889351) nautilus (stable)
-------------------------------------------------------------------------------------------------
配置初始MON节点，并收集所有密钥(node1节点)
ceph-deploy mon create-initial
---------------------------------
[ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (<
最低0.47元/天 解锁文章
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
