linux multi on,About issues for multipathd on Oracle Linux 7.5

Hello, experts

Recently I've deployed successfully Oracle RAC 18c for Linux x86_64 on my VirtualBox 5.2.18 on Oracle Linux 7.5 (only test environment).

At that time I used ASMFD to initialize those shared disks (not using multipathd to bind those disks),

[[email protected] ~]# export ORACLE_HOME=/u01/app/18.0.0/grid[[email protected] ~]# export PATH=$PATH:$ORACLE_HOME/bin[[email protected] ~]# export ORACLE_BASE=/tmp[[email protected] ~]# /u01/app/18.0.0/grid/bin/asmcmd afd_label CRSDG1 /dev/sdb --init/u01/app/18.0.0/grid/bin/asmcmd afd_label CRSDG2 /dev/sdc --init/u01/app/18.0.0/grid/bin/asmcmd afd_label CRSDG3 /dev/sdd --init/u01/app/18.0.0/grid/bin/asmcmd afd_label MGMT /dev/sde --init/u01/app/18.0.0/grid/bin/asmcmd afd_label DATA1 /dev/sdf --init/u01/app/18.0.0/grid/bin/asmcmd afd_label DATA2 /dev/sdg --init/u01/app/18.0.0/grid/bin/asmcmd afd_label ARCH /dev/sdh --init/u01/app/18.0.0/grid/bin/asmcmd afd_label FTRA /dev/sdi –-init[[email protected] ~]# cd /dev/oracleafd/disks/[[email protected] disks]# ls -lrhttotal 32K-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 CRSDG3-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 CRSDG2-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 CRSDG1-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 ARCH-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 MGMT-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 FTRA-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 DATA2-rw-rw-r-- 1 grid oinstall 9 Sep  7 10:07 DATA1

Afterwards I get ready to add another disk "sdj" to use multipathd to bind it, the following are some steps,

(1) halt two nodes RAC,

(2) using "VBoxManage" to create another disk and changing it to be shareable and providing it to two nodes,

# VBoxManage createhd -filename /root/"VirtualBox VMs"/asm_disks/data3.vdi   -size 10240 -format VDI -variant Fixed0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%Medium created. UUID: fc7d2707-616e-49e0-bcff-24540ffffaf1# VBoxManage modifyhd  /root/"VirtualBox VMs"/asm_disks/data3.vdi    --type shareable# # VBoxManage storageattach adccrac1 --storagectl "SATA" --port 12 --device 0 --type hdd --medium /root/"VirtualBox VMs"/asm_disks/data3.vdi   --mtype shareable# # VBoxManage storageattach adccrac2 --storagectl "SATA" --port 12 --device 0 --type hdd --medium /root/"VirtualBox VMs"/asm_disks/data3.vdi   --mtype shareable#

(3) startup two nodes RAC,

(4) configure multipathd service,

# ssh [email protected]'s password: Last login: Thu Sep  6 16:52:16 2018 from gateway[[email protected] ~]# rpm -qa | grep device-mapperdevice-mapper-event-libs-1.02.146-4.0.1.el7.x86_64device-mapper-multipath-libs-0.4.9-119.el7.x86_64device-mapper-1.02.146-4.0.1.el7.x86_64device-mapper-libs-1.02.146-4.0.1.el7.x86_64device-mapper-event-1.02.146-4.0.1.el7.x86_64device-mapper-multipath-0.4.9-119.el7.x86_64device-mapper-persistent-data-0.7.3-3.el7.x86_64[[email protected] ~]# systemctl status multipathd● multipathd.service - Device-Mapper Multipath Device Controller   Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)   Active: inactive (dead)Condition: start condition failed at Thu 2018-09-06 17:11:58 CST; 15h ago           ConditionPathExists=/etc/multipath.conf was not met           [[email protected] ~]# ls -lrht /etc/multipath.confls: cannot access /etc/multipath.conf: No such file or directory[[email protected] ~]# mpathconf --helpusage: /usr/sbin/mpathconf Commands:Enable: --enable Disable: --disableOnly allow certain wwids (instead of enable): --allow Set user_friendly_names (Default y): --user_friendly_names Set find_multipaths (Default y): --find_multipaths Load the dm-multipath modules on enable (Default y): --with_module start/stop/reload multipathd (Default n): --with_multipathd select output file (Default /etc/multipath.conf): --outfile [[email protected] ~]# mpathconf --enable[[email protected] ~]# [[email protected] ~]# ls -lrht /etc/multipathmultipath/      multipath.conf  [[email protected] ~]# ls -lrht /etc/multipath.conf -rw------- 1 root root 2.4K Sep  7 08:55 /etc/multipath.conf

[[email protected] ~]# /usr/lib/udev/scsi_id -gud /dev/sdj1ATA_VBOX_HARDDISK_VBfc7d2707-f1faff0f

[[email protected] ~]# vi /etc/multipath.confmultipaths {        multipath {                wwid                    1ATA_VBOX_HARDDISK_VBfc7d2707-f1faff0f                alias                   mpathdata        }}

[[email protected] ~]# systemctl start multipathd.service[[email protected] ~]# systemctl status multipathd.service● multipathd.service - Device-Mapper Multipath Device Controller   Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)   Active: active (running) since Fri 2018-09-07 09:58:15 CST; 2s ago  Process: 10670 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS)  Process: 10666 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS)  Process: 10664 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS) Main PID: 10672 (multipathd)   CGroup: /system.slice/multipathd.service           └─10672 /sbin/multipathdSep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio4: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio5: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio6: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio7: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio8: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vio9: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: asm/.asm_ctl_vmb: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: ofsctl: HDIO_GETGEO failed with 25Sep 07 09:58:15 adccrac1 multipathd[10672]: oracleafd/admin: HDIO_GETGEO failed with 25Sep 07 09:58:16 adccrac1 multipathd[10672]: path checkers start up

At this very moment as you can see above it seems to be some errors, next to I observe the directory "/dev/mapper",

[[email protected] ~]# cd /dev/mapper/[[email protected] mapper]# ls -lrhttotal 0crw------- 1 root root 10, 236 Sep  7 10:07 control

There is no disk on "/dev/mapper", I re-scan disks via this command "multipath -v2",

[[email protected] ~]# multipath -v2Sep 07 10:25:55 | asm/.asm_ctl_spec: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg0: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg1: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg2: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg3: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg4: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg5: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg6: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg7: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vbg8: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vdbg: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio0: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio1: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio10: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio11: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio12: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio13: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio14: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio15: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio16: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio17: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio18: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio19: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio2: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio20: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio21: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio22: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio23: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio24: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio25: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio26: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio27: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio28: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio29: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio3: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio30: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio31: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio4: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio5: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio6: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio7: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio8: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vio9: HDIO_GETGEO failed with 25Sep 07 10:25:55 | asm/.asm_ctl_vmb: HDIO_GETGEO failed with 25Sep 07 10:25:55 | ofsctl: HDIO_GETGEO failed with 25Sep 07 10:25:55 | oracleafd/admin: HDIO_GETGEO failed with 25

Could you help me for troubleshooting this weird issue?

Thanks in advance.

Best Regards

Quanwen Zhao