ceph对接nfs-ganesha 安装配置手顺

1. 概要

nfs-ganesha开始支持ceph，对接cephfs,rgw.

 

 

1. 服务端安装和配置

1. 配置yum源

[nfs-ganesha] name=nfs-ganesha baseurl=http://us-west.ceph.com/nfs-ganesha/rpm-V2.7-stable/mimic/x86_64/ enabled=1

现在nfs-ganesha最新版本是v2.8,安装出现兼容性问题

 

1. 安装软件

#yum install libcephfs2 nfs-ganesha nfs-ganesha-ceph

 

1. 配置ganesha.conf

EXPORT { # Export Id (mandatory, each EXPORT must have a unique Export_Id) Export_Id = 1;   # Use NFSv4 Protocols = 4;   # NFSv4 does not allow UDP transport Transports = TCP;   # Path into the cephfs tree. For now, FSAL_CEPH doesn't support # having more than one file system per running ganesha daemon. # # Note that FSAL_CEPH does not support subtree checking, so there is # no way to validate that a filehandle presented by a client is # reachable via an exported subtree. # # For that reason, we just export "/" here. Path = /tmp;   # Pseudo Path (required for NFS v4) # The pseudoroot path. This is where the export will appear in the # NFS pseudoroot namespace. Pseudo = /tmp;   # Required for access (default is None) # Could use CLIENT blocks instead Access_Type = RW;      Squash = no_root_squash; SecType = sys;   # Exporting FSAL FSAL {   Name = CEPH;   } }

 

一个nfs-ganesha只能export一个目录,所以要想export多个目录的就要启动多个nfs-ganesha服务

1. Path:指的是cephfs那边目录
2. Pseudo:虚拟到挂载的点的目录

如果实现一个cephfs到处多个nfs,可以在cephfs的根目录下创建多个目录,分别在Path中指定,然后在不同节点上启动nfs-ganesha服务.

 

1. 启动服务

 

# systemctl restart nfs-ganesha.service

 

1. 客户端安装和配置

 

1. 客户端

# yum install nfs-utils

 

1. 拷贝ceph的配置文件到客户端
2. 创建挂载点

#mkdir -p /mnt/nfs-ganesha

 

1. 挂载

#mount -vv -t nfs -o nfsvers=4.1,proto=tcp 172.16.34.149:/ /mnt/nfs-ganesha/

1. 确认挂载

# nfsiostat   172.16.34.149:/ mounted on /mnt/nfs-ganesha:      op/s rpc bklog    0.06    0.00 read:            ops/s    kB/s   kB/op retrans avg RTT (ms) avg exe (ms)   0.000   0.000   0.000       0 (0.0%)   0.000   0.000 write:           ops/s    kB/s   kB/op retrans avg RTT (ms) avg exe (ms)   0.000   0.000   0.000       0 (0.0%)   0.000   0.000

 

# ls tmp [root@ceph-1 nfs-ganesha]# ls -R .: tmp   ./tmp:

 

1. quota设置

 

可以在cephfs中创建多个目录,然后设置每个目录的quota,设置之前先通过cephfs-fuse或者mount挂载到本地

ceph-fuse -m 172.16.34.243:/  /mnt/cephfs/

 

设置quota

 

#setfattr -n ceph.quota.max_bytes -v 100000000 tmp    #限制 100 MB #setfattr -n ceph.quota.max_files -v 10000 tmp        # 限制文件数

 

Quota查看

getfattr -n ceph.quota.max_bytes /some/dir getfattr -n ceph.quota.max_files /some/dir

 

删除quota

 

setfattr -n ceph.quota.max_bytes -v 0 /some/dir setfattr -n ceph.quota.max_files -v 0 /some/dir