mixboot

CentOS本机配置为时间源

ceph 数据恢复和回填速度 重建osd 加快数据恢复

ceph 删除 osd 重新添加 osd down 重建

health: HEALTH_WARN 32 slow ops, oldest one blocked for 4107 sec, daemons [mon.k093,mon.k094,mon.k095] have slow ops.# ceph health detailHEALTH_WARN 32 slow ops, oldest one blocked for 4422 sec, daemons [mon.k093,mon.k094,mon.k095] have slow

apt install -y ceph-deployceph-deploy purge hostnameceph-deploy purge `hostname`umount /var/lib/ceph/osd/*umount /var/lib/ceph/osd/*rm -rf /var/lib/ceph/osd/*rm -rf /var/lib/ceph/mon/*rm -rf /var/lib/ceph/mds/*rm -rf /var/lib/ceph/bootstrap-mds

ceph 在线修改参数修改OSD心跳OSD设置维护模式osd_heartbeat_intervalosd_heartbeat_gracemon_osd_min_down_reporters修改OSD心跳OSD 抖动严重ceph -vceph version 13.2.10OSD设置维护模式ceph osd set nooutosd_heartbeat_interval同一个PG的OSD 才会相互心跳检测默认值ceph daemon mon.k213 config show | g

ceph-deploy 安装报错1.ceph-deploy 安装报错2.wget 没有正常安装1.ceph-deploy 安装报错# ceph-deploy install k021[k021][INFO ] installing Ceph on k021[k021][INFO ] using custom repository location: http://mirrors.163.com/ceph/debian-mimic/[k021][INFO ] Running command:

# ceph -sclock skew detected on mon# ceph health detailclock skew 0.0573631s > max 0.05s (latency 0.000725198s)ubuntuapt install ntpsec -y参考：NTP

health: HEALTH_WARN Long heartbeat ping times on back interface seen, longest is 22355.097 msec Long heartbeat ping times on front interface seen, longest is 22418.068 msec# ceph health detailHEALTH_WARN Long heartbeat ping t...

ceph HEALTH_ERR 1 scrub errorsceph health detailceph pg repairpg repair 需要等待几分钟ceph health detail# ceph health detailHEALTH_ERR 1 scrub errors; Possible data damage: 1 pg inconsistentOSD_SCRUB_ERRORS 1 scrub errorsPG_DAMAGED Possible data damage: 1 pg

ceph HEALTH_WARN slow opsHEALTH_WARN重启monHEALTH_WARN health: HEALTH_WARN 2 slow ops, oldest one blocked for 116154 sec, mon.k093 has slow opsceph health detail# ceph health detailHEALTH_WARN 2 slow ops, oldest one blocked for 116369 s

ceph 调整 PG 平衡ceph mgr module ls# ceph mgr module ls{ "enabled_modules": [ "balancer", "crash", "dashboard", "iostat", "restful", "status" ], "disabled_modules": [ { "name":

ceph version# ceph -vceph version 13.2.10 mimic (stable)osd max backfills# ceph daemon osd.0 config show | grep osd_max_backfills 一个OSD允许的最大backfills数 "osd_max_backfills": "1", # ceph daemon osd.0 config show | grep osd_recovery_max_active 同一时间内活

too few PGs per OSD 21 < min 30ceph versionceph version# ceph -vceph version 13.2.10 mimic (stable)

Ubuntu 18.04.5 安装 Ceph mimic 13.2.10 CephFS 自动挂载1，CephFS 内核挂载2，CephFS 开机自动挂载1，CephFS 内核挂载查看key，保存到/etc/ceph/admin.secretceph auth export client.admin查看monceph mon dump内核挂载mount -t ceph 192.168.1.3:6789,192.168.1.4:6789,192.168.1.5:6789:/ /ce

报错[ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directoryceph-deploy new --public-network 192.168.1.0/24 node1 node2 node3ceph-deploy --overwrite-conf mon create-initial参考：ceph-deploy部署ceph集群...

# ceph --versionceph version 12.2.13 luminous (stable)# ceph osd tree0 hdd 7.27739 osd.0 up 1.00000 1.00000# ceph osd out 0marked out osd.0.# ceph osd tree 0 hdd 7.27739 osd.0 up 0 1.00000等待数据迁移# ceph

# ceph --versionceph version 12.2.13 luminous (stable)ceph osd tree

ceph fuse 挂载1，ceph fuse 挂载1，ceph fuse 挂载# mkdir -p /etc/ceph# scp root@192.168.1.100:/etc/ceph/ceph.conf /etc/ceph/ceph.conf# scp root@192.168.1.100:/etc/ceph/ceph.client.admin.keyring /etc/ceph/ceph.client.admin.keyring# mkdir /cephfs/# apt inst

systemctl status ceph-mon@k02.servicesystemctl restart ceph-mon@k02.service

# ceph mgr module enable dashboardvim ceph.conf 增加配置[mgr]mgr modules = dashboard默认监听端口7000# netstat -antp | grep 7000tcp6 0 0 :::7000 :::* LISTEN 104863/ceph-mgr参考：CEPH仪表板Ceph可视化管理dashboa

Ceph 运维手册

Ubuntu 18.04.4 安装 ceph1，安装`ceph-deploy`2，`ceph-deploy`节点配置免密钥登陆到其它节点3，ceph搭建~~3.1，清除ceph配置~~3.2，创建集群1，安装ceph-deploy# wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -OK# echo deb https://download.ceph.com/debian-luminous/ $(

最新版本v12.2.5 LuminousCEPH RELEASES

ubuntu 18.04 搭建ceph luminous 12.2.12 Ceph 文件系统1，ubuntu 18.04 搭建ceph luminous 12.2.12 Ceph 文件系统2，挂载 CEPH 文件系统1，ubuntu 18.04 搭建ceph luminous 12.2.12 Ceph 文件系统# ceph -s cluster: id: 5df0ea22-...

ceph luminous 12.2.12 HEALTH_WARN1，ceph luminous 12.2.12 HEALTH_WARN1，ceph luminous 12.2.12 HEALTH_WARN health: HEALTH_WARN application not enabled on 1 pool(s)# ceph health detail...

ceph 搭建 ubuntu 18.04 块设备 创建1，ceph 搭建 ubuntu 18.04 块设备 创建1，ceph 搭建 ubuntu 18.04 块设备 创建参考：ceph 搭建 ubuntu 18.04 osd 创建

ceph 搭建1，ubuntu搭建ceph1，ubuntu搭建ceph参考：Ceph文档Github Ceph

1. 重启monitor# systemctl status vcfs-mon@node1systemctl status vcfs-mon[tab自动补全]systemctl status vcfs-mon@主机名

1. ceph 文件数# getfattr -d -m ".*" /cephceph.dir.entries="1"ceph.dir.files="1" #文件数统计为1ceph.dir.rbytes="0"ceph.dir.rctime="0.090"ceph.dir.rentries="1"ceph.dir.rfiles="0"ceph.dir.rsubdirs="

getfattr -d -m ‘.*’ /集群目录

1. 内核挂载# mkdir fattr# mount -t ceph 10.0.0.244,10.0.0.246,10.0.0.249:/ /fattr -o vcfs_quota=32. 客户端fuse挂载usage: vcfs-fuse [-m mon-ip-addr:mon-port] <mount point> [OPTIONS] # mkdir fattr_fuse# vcfs-

1. 安装fio这里写代码片参考： 1.块设备性能测试

1. 查看PG# vcfs pg stat# vcfs pg dump2. pg映射OSD位置# vcfs pg map 1.1ffosdmap e99 pg 1.1ff (1.1ff) -> up [8,5] acting [8,5]#【8,5】指osd.8和osd.5 2副本，osd.8为主副本参考： 1.ceph运维常用指令

1. rados命令查看存储池# rados lspools# ceph osd pool ls# ceph osd lspools查看存储池使用情况# rados df 存储池使用情况# rados mkpool testpool 新建存储池# rados create testob

ceph. 创建一个小于4M的文件/test.txthello world2. 创建存储池# ceph osd pool create test_pool_ceph 1283. 上传对象# rados -p test_pool_ceph put object_1 /test.txt# rados -p test_pool_ceph lsobject_14. 查看对象的pg map# ceph

dmsetup remove_all

#cat /etc/ntp.confdriftfile /var/lib/ntp/drift#系统时间与BIOS事件的偏差记录 restrict default nomodify #default 指所有的IP#restrict 控制权限#nomodify 客户端不能更改 服务端的时间参数，但是客户端可以通过服务端校队时间restrict 127.0.0.1 logfile /var/log/n

1. ceph扩展1.1 ceph基准测试集群的读、写、延迟、负载各节点硬件性能测试：磁盘、网络1.2 磁盘性能基准单个磁盘的性能一个OSD节点中所有磁盘的性能单个磁盘写能力#conv=fsync 不是每写一次都要同步一次，在dd命令结束前同步data和metadata（写入数度快） #oflag=dsync 每写一次都要等到这一次写入磁盘，才能进行下一次写（写入速度慢）2. 测试写

1. Ceph安装状态查询#ceph -s#ceph -status2. 集群健康状态ceph -w3. Ceph monitor 仲裁状态ceph