Creating Cluster-Wide Target Groups for ALUA

http://www.high-availability.com/faqs/index.php?action=artikel&cat=8&id=56&artlang=en

NMV is currently unable to create the cluster-wide target groups necessary for allowing client-side multipathing software to take full advantage of COMSTAR Asymmetric Logical Unit Access support provided by the new HA Cluster STMF Proxy service.

To add this functionality to NexentaStor at present, the target groups once created must be fully populated with the remaining target mode ports as per the instructions contained within this guide.

This guide assumes two NexentaStor appliances, nextest3 & nextest4, each with two Fibre Channel HBA ports configured in target mode for COMSTAR Asymmetric Logical Unit Access support.

The target group will need to be created initially on one NexentaStor appliance using NMV, this effectively registers the target group name with NMS on both the local and remote appliance, allowing the remaining target ports be added manually.

When creating the target group, select only a single target port on the local appliance even if multiple target ports exist.

On nextest3, create the target group tg01 containing a single target port as follows:

 

Verify the target group tg01 has also been created on nextest4 containing one of its own target ports as follows:

 

 

Once the target group has been created and registered with NMS, the remaining target ports need to be manually added to the target group tg01 on both appliances, this procedure requires accessing the underlying Bash shell from within NMC as follows:

?

nmc@nextest3:/$ option expert_mode=1 nmc@nextest3:/$ !bash   You are about to enter the Unix ("raw") shell and execute low-level Unix command(s). Warning: using low-level Unix commands is not recommended! Execute? Yes   root@nextest3:/volumes#

With the HA Cluster STMF Proxy service running, the stmfadm list-target –v command will return all Fibre Channel ports in target mode on both the local and remote appliance. The Fibre Channel ports in target mode on the remote appliance are easily identified by the Provider Name of pppt.

The output from the command on nextest3 is as follows:

?

root@nextest3:/volumes# stmfadm list-target -v   Target: wwn.500110A00018700E Operational Status: Online Provider Name : pppt remote target port 1 on nextest4 Alias : - Protocol : Fibre Channel Sessions : 0 Target: wwn.500110A00018700C Operational Status: Online Provider Name : pppt remote target port 0 on nextest4 Alias : - Protocol : Fibre Channel Sessions : 0 Target: wwn.2100001B320F034D Operational Status: Online Provider Name : qlt local target port 0 on nextest3 Alias : qlt0,0 Protocol : Fibre Channel Sessions : 0 Target: wwn.2101001B322F034D Operational Status: Online Provider Name : qlt local target port 1 on nextest3 Alias : qlt1,0 Protocol : Fibre Channel Sessions : 0

To view the current member of the target group tg01 on nextest3 run the following command:

?

root@nextest3:/volumes# stmfadm list-tg -v Target Group: tg01 Member: wwn.2100001B320F034D

To allow the remaining target ports to be added to the target group, the target ports currently outside the target group first need to be offlined to allow them to be added to the target group.

On nextest3, the commands are as follows:

?

stmfadm offline-target wwn.2101001B322F034D stmfadm offline-target wwn.500110A00018700C stmfadm offline-target wwn.500110A00018700E

The remaining target ports can then be added into the target group with the following commands:

?

stmfadm add-tg-member -g tg01 wwn.2101001B322F034D stmfadm add-tg-member -g tg01 wwn.500110A00018700C stmfadm add-tg-member -g tg01 wwn.500110A00018700E

Once added, the target ports need to be onlined again to allow them to receive SCSI I/O commands for the Logical Units they serve as follows:

?

stmfadm online-target wwn.2101001B322F034D stmfadm online-target wwn.500110A00018700C stmfadm online-target wwn.500110A00018700E

Access the underlying Bash shell from within NMC on nextest4 as follows:

?

nmc@nextest4:/$ option expert_mode=1 nmc@nextest4:/$ !bash You are about to enter the Unix ("raw") shell and execute low-level Unix command(s). Warning: using low-level Unix commands is not recommended! Execute? Yes root@nextest4:/volumes#

The corresponding output from the stmfadm list-target –v command on nextest4 is as follows:

?

root@nextest4:/volumes# stmfadm list-target -v Target: wwn.2101001B322F034D Operational Status: Online Provider Name : pppt remote target port 1 on nextest3 Alias : - Protocol : Fibre Channel Sessions : 0 Target: wwn.2100001B320F034D Operational Status: Online Provider Name : pppt remote target port 0 on nextest3 Alias : - Protocol : Fibre Channel Sessions : 0 Target: wwn.500110A00018700C Operational Status: Online Provider Name : qlt local target port 0 on nextest4 Alias : qlt0,0 Protocol : Fibre Channel Sessions : 0 Target: wwn.500110A00018700E Operational Status: Online Provider Name : qlt local target port 1 on nextest4 Alias : qlt1,0 Protocol : Fibre Channel Sessions : 0

To view the current member of the target group tg01 on nextest4 run the following command:

?

root@nextest4:/volumes# stmfadm list-tg -v Target Group: tg01 Member: wwn.500110A00018700C

To allow the remaining target ports to be added to the target group, the target ports currently outside the target group first need to be offlined to allow them to be added to the target group.

On nextest4, the commands are as follows:

?

stmfadm offline-target wwn.500110A00018700E stmfadm offline-target wwn.2100001B322F034D stmfadm offline-target wwn.2101001B322F034D

The remaining target ports can then be added into the target group with the following commands:

?

stmfadm add-tg-member -g tg01 wwn.500110A00018700E stmfadm add-tg-member -g tg01 wwn.2100001B322F034D stmfadm add-tg-member -g tg01 wwn.2101001B322F034D

Once added, the target ports need to be onlined again to allow them to receive SCSI I/O commands for the Logical Units they serve as follows:

?

stmfadm online-target wwn.500110A00018700E stmfadm online-target wwn.2100001B322F034D stmfadm online-target wwn.2101001B322F034D

Once completed, the target group tg01 on both nextest3 & nextest4 will contain all four target ports on both the local and remote appliance as follows:

?

root@nextest3:/volumes# stmfadm list-tg -v Target Group: tg01 Member: wwn.2100001B320F034D Member: wwn.2101001B322F034D Member: wwn.500110A00018700C Member: wwn.500110A00018700E

It is now possible within NMV to create mappings to Logical Units using the cluster-wide target group tg01.