linux环境iscsi共享存储服务配置

配置 iSCSI 服务端

配置前准备

准备三台服务器,192.168.106.22作为iscsi服务端挂载存储,服务端挂载了3块10g硬盘,6块15g硬盘,192.168.119.20和192.168.119.21配置为客户端远程访问服务端存储,两台客户端服务器准备搭建oracle11g rac集群,所以需要共同访问服务端九块硬盘,从而实现配置多路径的需求。

主机名称IP地址
iscsi服务端 info192.168.106.22/23
iscsi客户端 jcyb1192.168.119.20
iscsi客户端 jcyb2192.168.119.21

安装 iSCSI 服务端程序

登录服务端下载软件包,配置yum软件仓库步骤省略
[root@info ~]# yum -y install targetd targetcli
启动target服务并设置为开机自启动
[root@info ~]# systemctl start targetd
[root@info ~]# systemctl enable targetd

[root@info ~]# systemctl start targetd
[root@info ~]# systemctl enable targetd
Created symlink from /etc/systemd/system/multi-user.target.wants/targetd.service to /usr/lib/systemd/system/targetd.service.
[root@info ~]#

配置 iSCSI 服务端共享资源

targetcli 是用于管理 iSCSI 服务端存储资源的专用配置命令,执行 targetcli 命令进入交互式的配置界面,进入/backstores/block目录 ,/backstores/block 是iSCSI 服务端配置共享设备的位置,执行如下命令,将本服务器挂载的硬盘重新命名为disk*作为共享资源加入到“资源池”中。

[root@info ~]# targetcli
targetcli shell version 2.1.fb46
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
  | o- block .................................................................................................. [Storage Objects: 0]
  | o- fileio ................................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................................ [Targets: 0]
  o- loopback ......................................................................................................... [Targets: 0]
/> 
/> cd backstores/block

/backstores/block> create name=disk1 dev=/dev/sdc
Created block storage object disk1 using /dev/sdc.
/backstores/block> create name=disk2 dev=/dev/sdd
Created block storage object disk2 using /dev/sdd.
/backstores/block> create name=disk3 dev=/dev/sde
Created block storage object disk3 using /dev/sde.
/backstores/block> create name=disk4 dev=/dev/sdf
Created block storage object disk4 using /dev/sdf.
/backstores/block> create name=disk5 dev=/dev/sdg
Created block storage object disk5 using /dev/sdg.
/backstores/block> create name=disk6 dev=/dev/sdh
Created block storage object disk6 using /dev/sdh.
/backstores/block> create name=disk7 dev=/dev/sdi
Created block storage object disk7 using /dev/sdi.
/backstores/block> create name=disk8 dev=/dev/sdj
Created block storage object disk8 using /dev/sdj.
/backstores/block> create name=disk9 dev=/dev/sdk
Created block storage object disk9 using /dev/sdk.
/backstores/block> ls
o- block ...................................................................................................... [Storage Objects: 9]
  o- disk1 ............................................................................. [/dev/sdc (10.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk2 ............................................................................. [/dev/sdd (10.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk3 ............................................................................. [/dev/sde (10.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk4 ............................................................................. [/dev/sdf (15.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk5 ............................................................................. [/dev/sdg (15.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk6 ............................................................................. [/dev/sdh (15.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk7 ............................................................................. [/dev/sdi (15.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk8 ............................................................................. [/dev/sdj (15.0GiB) write-thru deactivated]
  | o- alua ....................................................................................................... [ALUA Groups: 1]
  |   o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
  o- disk9 ............................................................................. [/dev/sdk (15.0GiB) write-thru deactivated]
    o- alua ....................................................................................................... [ALUA Groups: 1]
      o- default_tg_pt_gp ........................................................................... [ALUA state: Active/optimized]
/backstores/block>

创建 iSCSI target 名称及配置共享资源

进入/iscsi目录,创建iSCSI target(是由系统自动生成描述共享资源的唯一字符串),系统在生成这个 target 名称后,还会在/iscsi 参数目录中创建一个与其字符串同名的新“目录”用来存放共享资源。我们需要把前面加入到 iSCSI 共享资源池中的硬盘设备添加到这个新目录中,这样用户在登录 iSCSI 服务端后,即可默认使用这硬盘设备提供的共享存储资源了。如下所示,输入create创建target(iqn.2003-01.org.linux-iscsi.info.x8664:sn.3fe5490fa075.)后,进入该目录后存在一个与其相同名称的目录。

/> cd /iscsi
/iscsi> ls
o- iscsi .............................................................................................................. [Targets: 0]
/iscsi> create
Created target iqn.2003-01.org.linux-iscsi.info.x8664:sn.3fe5490fa075.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/iscsi> ls
o- iscsi .............................................................................................................. [Targets: 1]
  o- iqn.2003-01.org.linux-iscsi.info.x8664:sn.3fe5490fa075 .............................................................. [TPGs: 1]
    o- tpg1 ................................................................................................. [no-gen-acls, no-auth]
      o- acls ............................................................................................................ [ACLs: 0]
      o- luns ............................................................................................................ [LUNs: 0]
      o- portals ...................................................................................................... [Portals: 1]
        o- 0.0.0.0:3260 ....................................................................................................... [OK]
/iscsi> 
/iscsi> cd iqn.2003-01.org.linux-iscsi.info.x8664:sn.3fe5490fa075
/iscsi/iqn.20....3fe5490fa075> ls
o- iqn.2003-01.org.linux-iscsi.info.x8664:sn.3fe5490fa075 ................................................................ [TPGs: 1]
  o- tpg1 ................................................................................................... [no-gen-acls, no-auth]
    o- acls .............................................................................................................. [ACLs: 0]
    o- luns .............................................................................................................. [LUNs: 0]
    o- portals ........................................................................................................ [Portals: 1]
      o- 0.0.0.0:3260 ......................................................................................................... [OK]
/iscsi/iqn.20....3fe5490fa075>

这里选择手工创建target,指定target名称(iqn.2024-04.com.info:racdisk)

/iscsi> create iqn.2024-04.com.info:racdisk
Created target iqn.2024-04.com.info:racdisk.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/iscsi> ls
o- iscsi .............................................................................................................. [Targets: 2]
  
  o- iqn.2024-04.com.info:racdisk ........................................................................................ [TPGs: 1]
    o- tpg1 ................................................................................................. [no-gen-acls, no-auth]
      o- acls ............................................................................................................ [ACLs: 0]
      o- luns ............................................................................................................ [LUNs: 0]
      o- portals ...................................................................................................... [Portals: 1]
        o- 0.0.0.0:3260 ....................................................................................................... [OK]
/iscsi>

进入到/tpg1/luns目录创建lun*并与资源池中硬盘绑定
create lun=lun1 storage_object=/backstores/block/disk1
create lun=lun2 storage_object=/backstores/block/disk2
create lun=lun3 storage_object=/backstores/block/disk3
create lun=lun4 storage_object=/backstores/block/disk4
create lun=lun5 storage_object=/backstores/block/disk5
create lun=lun6 storage_object=/backstores/block/disk6
create lun=lun7 storage_object=/backstores/block/disk7
create lun=lun8 storage_object=/backstores/block/disk8
create lun=lun9 storage_object=/backstores/block/disk9

/iscsi> cd iqn.2024-04.com.info:racdisk/tpg1/luns
/iscsi/iqn.20...isk/tpg1/luns> ls
o- luns .................................................................................................................. [LUNs: 0]
/iscsi/iqn.20...isk/tpg1/luns>
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun1 storage_object=/backstores/block/disk1
Created LUN 1.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun2 storage_object=/backstores/block/disk2
Created LUN 2.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun3 storage_object=/backstores/block/disk3
Created LUN 3.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun4 storage_object=/backstores/block/disk4
Created LUN 4.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun5 storage_object=/backstores/block/disk5
Created LUN 5.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun6 storage_object=/backstores/block/disk6
Created LUN 6.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun7 storage_object=/backstores/block/disk7
Created LUN 7.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun8 storage_object=/backstores/block/disk8
Created LUN 8.
/iscsi/iqn.20...isk/tpg1/luns> create lun=lun9 storage_object=/backstores/block/disk9
Created LUN 9.
/iscsi/iqn.20...isk/tpg1/luns>
/iscsi/iqn.20...isk/tpg1/luns> ls
o- luns .................................................................................................................. [LUNs: 9]
  o- lun1 .............................................................................. [block/disk1 (/dev/sdc) (default_tg_pt_gp)]
  o- lun2 .............................................................................. [block/disk2 (/dev/sdd) (default_tg_pt_gp)]
  o- lun3 .............................................................................. [block/disk3 (/dev/sde) (default_tg_pt_gp)]
  o- lun4 .............................................................................. [block/disk4 (/dev/sdf) (default_tg_pt_gp)]
  o- lun5 .............................................................................. [block/disk5 (/dev/sdg) (default_tg_pt_gp)]
  o- lun6 .............................................................................. [block/disk6 (/dev/sdh) (default_tg_pt_gp)]
  o- lun7 .............................................................................. [block/disk7 (/dev/sdi) (default_tg_pt_gp)]
  o- lun8 .............................................................................. [block/disk8 (/dev/sdj) (default_tg_pt_gp)]
  o- lun9 .............................................................................. [block/disk9 (/dev/sdk) (default_tg_pt_gp)]
/iscsi/iqn.20...isk/tpg1/luns>

设置访问控制列表(ACL)

iSCSI 协议是通过客户端名称进行验证的,也就是说,用户在访问存储共享资源时不需要输入密码,只要 iSCSI 客户端的名称与服务端中设置的访问控制列表中某一名称条目一致即可,因此需要在 iSCSI 服务端的配置文件中写入一串能够验证用户信息的名称。acls 参数目录用于存放能够访问 iSCSI 服务端共享存储资源的客户端名称。

/iscsi> cd iqn.2024-04.com.info:racdisk/tpg1/acls
/iscsi/iqn.20...isk/tpg1/acls> pwd
/iscsi/iqn.2024-04.com.info:racdisk/tpg1/acls
/iscsi/iqn.20...isk/tpg1/acls>
/iscsi/iqn.20...isk/tpg1/acls> 
/iscsi/iqn.20...isk/tpg1/acls> create iqn.2024-04.com.info:rac11g1
Created Node ACL for iqn.2024-04.com.info:rac11g1
Created mapped LUN 9.
Created mapped LUN 8.
Created mapped LUN 7.
Created mapped LUN 6.
Created mapped LUN 5.
Created mapped LUN 4.
Created mapped LUN 3.
Created mapped LUN 2.
Created mapped LUN 1.
/iscsi/iqn.20...isk/tpg1/acls> create iqn.2024-04.com.info:rac11g2
Created Node ACL for iqn.2024-04.com.info:rac11g2
Created mapped LUN 9.
Created mapped LUN 8.
Created mapped LUN 7.
Created mapped LUN 6.
Created mapped LUN 5.
Created mapped LUN 4.
Created mapped LUN 3.
Created mapped LUN 2.
Created mapped LUN 1.
/iscsi/iqn.20...isk/tpg1/acls>

设置 iSCSI 服务端的监听 IP 地址和端口号

生产环境服务器可能有多块网卡,需要我们手动在配置文件中定义iscsi服务端信息,手动定义由哪个ip对外提供共享存储资源,即在 portals 参数目录中写上服务器的 IP 地址,如果监听的IP地址设置为 0.0.0.0 就表示你的这个iSCSI服务器监听在本主机的所有IP地址上,通过任何一个IP地址都可以访问到。

/iscsi/iqn.20.../tpg1/portals> pwd
/iscsi/iqn.2024-04.com.info:racdisk/tpg1/portals
/iscsi/iqn.20.../tpg1/portals> ls
o- portals ............................................................................................................ [Portals: 1]
  o- 0.0.0.0:3260 ............................................................................................................. [OK]
/iscsi/iqn.20.../tpg1/portals>

如果需要修改监听地址需要使用create+ip命令,此刻容易遇到一个问题Could not create NetworkPortal in configFS,由于3260端口被0.0.0.0占用,所以无法直接创建新的ip,需要删除这个自动默认生成的0.0.0.0端口,当前服务器一共两个ip(192.168.106.22、192.168.106.23),所以用这两个ip+端口和0.0.0.0+端口作用是一样的,在确认信息无误后输入 exit 命令来退出配置,最后重启 iSCSI 服务端程序,再设置 firewalld 防火墙策略,使其放行 3260/tcp 端口号的流量。

/iscsi/iqn.20.../tpg1/portals> create 192.168.106.22 3260
Using default IP port 3260
Could not create NetworkPortal in configFS
/iscsi/iqn.20.../tpg1/portals>
/iscsi/iqn.20.../tpg1/portals> delete 0.0.0.0 3260
Deleted network portal 0.0.0.0:3260
/iscsi/iqn.20.../tpg1/portals> create 192.168.106.22
Using default IP port 3260
Created network portal 192.168.106.22:3260.
/iscsi/iqn.20.../tpg1/portals> ls
o- portals ............................................................................................................ [Portals: 1]
  o- 192.168.106.22:3260 ...................................................................................................... [OK]
/iscsi/iqn.20.../tpg1/portals> create 192.168.106.23
Using default IP port 3260
Created network portal 192.168.106.23:3260.
/iscsi/iqn.20.../tpg1/portals> ls
o- portals ............................................................................................................ [Portals: 2]
  o- 192.168.106.22:3260 ...................................................................................................... [OK]
  o- 192.168.106.23:3260 ...................................................................................................... [OK]
/iscsi/iqn.20.../tpg1/portals>
/iscsi/iqn.20.../tpg1/portals> exit        #退出保存
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
[root@info ~]#

客户端配置

安装客户端服务程序

登录192.168.119.20和192.168.119.21,分别安装客户端服务程序 initiator,yum源配置略。
[root@jcyb1 ~]# yum install iscsi-initiator-utils
[root@jcyb1 ~]# systemctl start iscsid
[root@jcyb1 ~]# systemctl enable iscsid
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.
[root@jcyb1 ~]#

修改客户端配置文件

iSCSI 协议是通过客户端的名称来进行验证,而该名称也是 iSCSI 客户端的唯一标识,而且必须与服务端配置文件中访问控制列表中的信息一致,否则客户端在尝试访问存储共享设备时,系统会弹出验证失败的保存信息,然后重启客户端 iscsid 服务程序。
192.168.119.20配置如下:

[root@jcyb1 ~]# vi /etc/iscsi/initiatorname.iscsi
[root@jcyb1 ~]# cat /etc/iscsi/initiatorname.iscsi
#InitiatorName=iqn.1994-05.com.redhat:b1f52ec35b
InitiatorName=iqn.2024-04.com.info:rac11g1
[root@jcyb1 ~]# systemctl restart iscsid

192.168.119.21配置如下:

[root@jcyb2 ~]# vi /etc/iscsi/initiatorname.iscsi
[root@jcyb2 ~]# cat /etc/iscsi/initiatorname.iscsi
#InitiatorName=iqn.1994-05.com.redhat:c813d8a9d73d
InitiatorName=iqn.2024-04.com.info:rac11g2
[root@jcyb2 ~]#systemctl restart iscsid

检索服务端可用共享存储资源

iscsiadm 是用于管理、查询、插入、更新或删除 iSCSI数据库配置文件的命令行工具,用户需要先使用这个工具扫描发现远程 iSCSI 服务端,然后查看找到的服务端上有哪些可用的共享存储资源。其中,-m discovery 参数的目的是扫描并发现可用的存储资源,-t st 参数为执行扫描操作的类型,-p 192.168.106.22 参数为 iSCSI 服务端的IP 地址:iscsiadm -m discovery -t st -p IP,以192.168.119.20为例。

[root@jcyb1 ~]# iscsiadm -m discovery -t st -p 192.168.106.22
192.168.106.22:3260,1 iqn.2024-04.com.info:racdisk
[root@jcyb1 ~]# iscsiadm -m discovery -t st -p 192.168.106.23
192.168.106.23:3260,1 iqn.2024-04.com.info:racdisk
[root@jcyb1 ~]#

登录 iSCSI 服务端

-m node 参数为将客户端所在主机作为一台节点服务器,-T iqn.2024-04.com.info:racdisk 参数为要使用的存储资源,-p 192.168.106.22 参数依然为 iSCSI 服务端的 IP 地址,最后使用–login 或-l 参数进行登录验证(以192.168.106.22为例):iscsiadm -m node -T iqn.2024-04.com.info:racdisk -p 192.168.106.22 -l
[root@jcyb1 ~]# iscsiadm -m node -T iqn.2024-04.com.info:racdisk -p 192.168.106.22 -l
Logging in to [iface: default, target: iqn.2024-04.com.info:racdisk, portal: 192.168.106.22,3260] (multiple)
Login to [iface: default, target: iqn.2024-04.com.info:racdisk, portal: 192.168.106.22,3260] successful.
[root@jcyb1 ~]# iscsiadm -m node -T iqn.2024-04.com.info:racdisk -p 192.168.106.23 -l
Logging in to [iface: default, target: iqn.2024-04.com.info:racdisk, portal: 192.168.106.23,3260] (multiple)
Login to [iface: default, target: iqn.2024-04.com.info:racdisk, portal: 192.168.106.23,3260] successful.
[root@jcyb1 ~]#

验证共享存储挂载

成功登录iscsi服务端之后会在客户端显示服务端挂载的共享资源存储,客户端扫描服务端两个ip,所以同一个存储资源也被识别了两遍,挡服务端其中一个网卡出现异常时,仍然可以通过另一个网卡进行通信。如图所示,两台客户端机器均挂载成功。
在这里插入图片描述

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值