Oracle RAC集群增加新共享硬盘并使用AFD的式增加新ASM磁盘组

本文详细介绍了在 Oracle RAC 集群环境中增加新共享硬盘的过程,并使用 AFD 方式创建新的 ASM 磁盘组。包括新增硬盘、配置 udev 规则、使用 AFD 创建磁盘组及表空间的完整步骤。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

Oracle RAC集群增加新共享硬盘并使用AFD的式增加新ASM磁盘组

Oracle RAC集群增加新共享硬盘并使用AFD的式增加新ASM磁盘组

此实验是在ASM共享磁盘的RAC环境下增加新共享硬盘并新建ASM磁盘组创建相应的表空间和表数据进行测试过程。参考资料包括了https://www.oracle.com/cn/technical-resources/article/database/asmfd.html Oracle ASM Filter Driver相关资料。

具体实验记录如下:

实验环境背景

数据库集群版本

[grid@rac2 ~]$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [19.0.0.0.0]

ASM磁盘开启的文件

ASMCMD> lsof
DB_Name  Instance_Name  Path                                                                           
+ASM     +ASM2          +ORS.255.1038877513                                                                                        
racdb    racdb2         +DATA/RACDB/86B637B62FE07A65E053F706E80A27CA/DATAFILE/sysaux.266.1038939581    
racdb    racdb2         +DATA/RACDB/86B637B62FE07A65E053F706E80A27CA/DATAFILE/system.265.1038939581    
racdb    racdb2         +DATA/RACDB/86B637B62FE07A65E053F706E80A27CA/DATAFILE/undotbs1.267.1038939581  
racdb    racdb2         +DATA/RACDB/A4582AEF3E8210F9E0533C23100A3339/TEMPFILE/temp.268.1038939611      
racdb    racdb2         +DATA/RACDB/CONTROLFILE/current.261.1038938883                                 
racdb    racdb2         +DATA/RACDB/DATAFILE/sysaux.258.1038938777                                     
racdb    racdb2         +DATA/RACDB/DATAFILE/system.257.1038938723                                     
racdb    racdb2         +DATA/RACDB/DATAFILE/undotbs1.259.1038938803                                   
racdb    racdb2         +DATA/RACDB/DATAFILE/undotbs2.269.1038940101                                   
racdb    racdb2         +DATA/RACDB/DATAFILE/users.260.1038938803                                      
racdb    racdb2         +DATA/RACDB/ONLINELOG/group_1.262.1038938887                                   
racdb    racdb2         +DATA/RACDB/ONLINELOG/group_2.263.1038938887                                   
racdb    racdb2         +DATA/RACDB/ONLINELOG/group_3.270.1038940635                                   
racdb    racdb2         +DATA/RACDB/ONLINELOG/group_4.271.1038940637                                   
racdb    racdb2         +DATA/RACDB/TEMPFILE/temp.264.1038938901

ASM客户端信息

ASMCMD> lsct
DB_Name  Status     Software_Version  Compatible_version  Instance_Name  Disk_Group 
+ASM     CONNECTED        19.0.0.0.0          19.0.0.0.0  +ASM2          ORS        
+ASM     CONNECTED        19.0.0.0.0          19.0.0.0.0  +ASM2          DATA       
_OCR     CONNECTED                 -                   -  rac2           ORS        
racdb    CONNECTED        19.0.0.0.0          19.0.0.0.0  racdb2         DATA 

实验步骤

  1. 新增共享硬盘

    本次新增硬盘到添加ASM新的磁盘的整个过程和创建搭建数据库时的ASM磁盘组类似,最后会有尝试使用ASMFD的新功能(这是一个可以取代 ASMLIB 和 udev 设置的新功能)直接绑定新硬盘然后加入数据库来使用。

    disk1

    节点1:新增硬盘9,容量为50G

    disk2

    节点2:使用节点1的硬盘9,容量为50G
  2. 使用udev配置Asm磁盘(两节点都做)

    虚拟机在要之前配置共享存储时在各自的虚拟机安装路径中的vmx文件中添加disk.enableUUID = "TRUE"才会显示UUID。udev是一种工具,它能够根据系统中的硬件设备的状态动态更新设备文件,包括设备文件的创建,删除等。设备文件通常放在/dev目录下。使用udev后,在/dev目录下就只包含系统中真正存在的设备。

     [root@rac1 ~]# fdisk -l    //查看新增硬盘
     Disk /dev/sdi: 53.7 GB, 53687091200 bytes, 104857600 sectors
     Units = sectors of 1 * 512 = 512 bytes
     Sector size (logical/physical): 512 bytes / 512 bytes
     I/O size (minimum/optimal): 512 bytes / 512 bytes
     //按步骤增加磁盘的分区1
     root@rac1 ~]# fdisk /dev/sdi
     Welcome to fdisk (util-linux 2.23.2).
     
     Changes will remain in memory only, until you decide to write them.
     Be careful before using the write command.
     
     Device does not contain a recognized partition table
     Building a new DOS disklabel with disk identifier 0x9a414875.
      
     Command (m for help): n
     Partition type:
        p   primary (0 primary, 0 extended, 4 free)
        e   extended
     Select (default p): 
     Using default response p
     Partition number (1-4, default 1): 
     First sector (2048-104857599, default 2048): 
     Using default value 2048
     Last sector, +sectors or +size{K,M,G} (2048-104857599, default 104857599): 
     Using default value 104857599
     Partition 1 of type Linux and of size 50 GiB is set
     
     Command (m for help): w
     The partition table has been altered!
     
     Calling ioctl() to re-read partition table.
     Syncing disks.
    

    两节点分别fdisk -l ,显示都一致

     Disk /dev/sdi: 53.7 GB, 53687091200 bytes, 104857600 sectors
     Units = sectors of 1 * 512 = 512 bytes
     Sector size (logical/physical): 512 bytes / 512 bytes
     I/O size (minimum/optimal): 512 bytes / 512 bytes
     Disk label type: dos
     Disk identifier: 0x9a414875
     
        Device Boot      Start         End      Blocks   Id  System
     /dev/sdi1            2048   104857599    52427776   83  Linux
    

    但是节点2没有sdi1 且没有UUID显示

     [root@rac2 ~]# partprobe /dev/sd
     sda   sda1  sda2  sdb   sdb1  sdc   sdc1  sdd   sdd1  sde   sde1  sdf   sdf1  sdg   sdg1  sdh   sdh1  sdi   
     [root@rac2 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdi1
     [root@rac2 ~]# 
     
     
     [root@rac2 ~]# partprobe /dev/sdi  //通知内核sdi盘变化 
     [root@rac2 ~]# /usr/lib/udev/scsi_id -g -u -d /dev/sdi1         //可以出现UUID代表正常
     36000c295d9c2913ec150e329c3465337
    

    新增物理盘为ASM磁盘组成员,节点1:

     [root@rac1 ~]# vi sdi1.sh
     #!/bin/bash
      for i in i ;
      do
      echo "KERNEL==\"sd?1\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d /dev/\$parent\", RESULT==\"`/usr/lib/udev/scsi_id -g -u -d /dev/sd\$i`\", SYMLINK+=\"asm-disk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""
      done
     [root@rac1 ~]# ./sdi1.sh 
     KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="36000c295d9c2913ec150e329c3465337", SYMLINK+="asm-diski", OWNER="grid", GROUP="asmadmin", MODE="0660"
     
     
     [root@rac1 ~]# vi /etc/udev/rules.d/99-oracle-asmdevices.rules  //新增上述结果
     [root@rac1 ~]# udevadm control --reload
     [root@rac1 ~]# udevadm trigger
     [root@rac1 ~]# ls -l /dev/sd?1  
     brw-rw---- 1 root disk     8,   1 May 13 11:37 /dev/sda1
     brw-rw---- 1 grid asmadmin 8,  17 May 13 11:37 /dev/sdb1
     brw-rw---- 1 grid asmadmin 8,  33 May 13 11:37 /dev/sdc1
     brw-rw---- 1 grid asmadmin 8,  49 May 13 11:37 /dev/sdd1
     brw-rw---- 1 grid asmadmin 8,  65 May 13 11:37 /dev/sde1
     brw-rw---- 1 grid asmadmin 8,  81 May 13 11:37 /dev/sdf1
     brw-rw---- 1 grid asmadmin 8,  97 May 13 11:37 /dev/sdg1
     brw-rw---- 1 root disk     8, 113 May 13 11:37 /dev/sdh1
     brw-rw---- 1 grid asmadmin 8, 129 May 13 11:37 /dev/sdi1
    

    节点2:

     [root@rac2 ~]# vi /etc/udev/rules.d/99-oracle-asmdevices.rules  //增加上述结果
     [root@rac2 ~]# udevadm control --reload
     [root@rac2 ~]# udevadm trigger
     [root@rac2 ~]# ls -l /dev/sd?1
     brw-rw---- 1 root disk     8,   1 May 13 11:51 /dev/sda1
     brw-rw---- 1 grid asmadmin 8,  17 May 13 11:51 /dev/sdb1
     brw-rw---- 1 grid asmadmin 8,  33 May 13 11:51 /dev/sdc1
     brw-rw---- 1 grid asmadmin 8,  49 May 13 11:51 /dev/sdd1
     brw-rw---- 1 grid asmadmin 8,  65 May 13 11:51 /dev/sde1
     brw-rw---- 1 grid asmadmin 8,  81 May 13 11:51 /dev/sdf1
     brw-rw---- 1 grid asmadmin 8,  97 May 13 11:51 /dev/sdg1
     brw-rw---- 1 root disk     8, 113 May 13 11:51 /dev/sdh1
     brw-rw---- 1 grid asmadmin 8, 129 May 13 11:51 /dev/sdi1
    
  3. 查看现有ASM磁盘组的情况

     [grid@rac1 ~]$ sqlplus  / as sysasm
     
     SQL*Plus: Release 19.0.0.0.0 - Production on Wed May 13 13:00:57 2020
     Version 19.3.0.0.0
     
     Copyright (c) 1982, 2019, Oracle.  All rights reserved.
     
     
     Connected to:
     Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
     Version 19.3.0.0.0
     
     SQL> show parameter asm_diskstring;   //查看磁盘组支持的模式
     
     NAME                                 TYPE        VALUE
     ------------------------------------ ----------- ------------------------------
     asm_diskstring                       string      /dev/asm*, AFD:*
     
     
     SQL> set lin 200
     SQL> col path for a40
     
     SQL> select group_number,disk_number,total_mb,name,failgroup,path from v$asm_disk order by 1;
     
     GROUP_NUMBER DISK_NUMBER   TOTAL_MB NAME                           FAILGROUP                      PATH
     ------------ ----------- ---------- ------------------------------ ------------------------------ ----------------------------------------
                0           0          0                                                               /dev/asm-diskf
                0           1          0                                                               /dev/asm-diskg
                0           2          0                                                               /dev/asm-diskc
                0           3          0                                                               /dev/asm-diske
                0           4          0                                                               /dev/asm-diskd
                0           5          0                                                               /dev/asm-diskb
                0           6          0                                                               /dev/asm-diski
                1           0      20476 DATA1                          DATA1                          AFD:DATA1
                1           1      20476 DATA2                          DATA2                          AFD:DATA2
                2           0      20476 FRA1                           FRA1                           AFD:FRA1
                3           0       1020 ORS1                           ORS1                           AFD:ORS1
     
     GROUP_NUMBER DISK_NUMBER   TOTAL_MB NAME                           FAILGROUP                      PATH
     ------------ ----------- ---------- ------------------------------ ------------------------------ ----------------------------------------
                3           1       1020 ORS2                           ORS2                           AFD:ORS2
                3           2       1020 ORS3                           ORS3                           AFD:ORS3
     
     13 rows selected.
    

    可以从以上内容看到支持的有AFD模式的磁盘组并且所有的资源和数据都是写在此类磁盘组的上面,上述的磁盘组都是通过grid集群图形化界面或ASMCA磁盘管理软件界面创建的,创建后默认的模式是就是AFD。之前手动设置的udev rules文件通过查资料其实都可以不用做,以下方式使用shell命令行的形式新建AFD模式的磁盘组。

  4. 直接路径的方式创建ASM磁盘组

    先直接使用/dev/asm-diski直接路径的方式创建

     SQL> create diskgroup bak_test external redundancy    disk  '/dev/asm-diski' name baktest  attribute 'au_size'='4M','compatible.asm' = '19.0.0.0', 'compatible.rdbms' = '19.0.0.0';
     
     Diskgroup created.
    
     //查看新创建的磁盘组
     SQL> select group_number,disk_number,total_mb,name,failgroup,path from v$asm_disk order by 1;
     
     GROUP_NUMBER DISK_NUMBER   TOTAL_MB NAME                           FAILGROUP                      PATH
     ------------ ----------- ---------- ------------------------------ ------------------------------ ----------------------------------------
                0           0          0                                                               /dev/asm-diskf
                0           1          0                                                               /dev/asm-diskg
                0           2          0                                                               /dev/asm-diskc
                0           3          0                                                               /dev/asm-diske
                0           4          0                                                               /dev/asm-diskd
                0           5          0                                                               /dev/asm-diskb
                1           0      20476 DATA1                          DATA1                          AFD:DATA1
                1           1      20476 DATA2                          DATA2                          AFD:DATA2
                2           0      20476 FRA1                           FRA1                           AFD:FRA1
                3           0       1020 ORS1                           ORS1                           AFD:ORS1
                3           1       1020 ORS2                           ORS2                           AFD:ORS2
     
     GROUP_NUMBER DISK_NUMBER   TOTAL_MB NAME                           FAILGROUP                      PATH
     ------------ ----------- ---------- ------------------------------ ------------------------------ ----------------------------------------
                3           2       1020 ORS3                           ORS3                           AFD:ORS3
                4           0      51196 BAKTEST                        BAKTEST                        /dev/asm-diski    //这里可以看到模式是直接路径的方式
     
     13 rows selected.
    

    上述最后一行的信息可以看到新增磁盘组的路径是直接路径的方式,此方式和默认的方式都不同,应修改为AFD的模式。修改模式如下:

     SQL> select name,state from v$asm_diskgroup;   //查看现有磁盘组STATE信息
     
     NAME                           STATE
     ------------------------------ -----------
     DATA                           MOUNTED
     FRA                            MOUNTED
     ORS                            MOUNTED
     BAK_TEST                       MOUNTED
     
     SQL> select name,path from v$asm_disk_stat;  //查案磁盘组路径信息
     
     NAME                           PATH
     ------------------------------ ----------------------------------------
     BAKTEST                        /dev/asm-diski
     ORS1                           AFD:ORS1
     ORS2                           AFD:ORS2
     ORS3                           AFD:ORS3
     DATA1                          AFD:DATA1
     DATA2                          AFD:DATA2
     FRA1                           AFD:FRA1
     
     7 rows selected.
    
    
     SQL> alter diskgroup BAK_TEST dismount;  //dismount磁盘组
     
     Diskgroup altered.
     
     SQL> select name,state from v$asm_diskgroup;
     
     NAME                           STATE
     ------------------------------ -----------
     BAK_TEST                       DISMOUNTED
     DATA                           MOUNTED
     FRA                            MOUNTED
     ORS                            MOUNTED
    
    
     SQL> drop diskgroup BAKTEST;   //删除磁盘组失败报错
     drop diskgroup BAKTEST
     *
     ERROR at line 1:
     ORA-15039: diskgroup not dropped
     ORA-15073: diskgroup BAKTEST is mounted by another ASM instance
    
     切换至节点2:
     [grid@rac2 ~]$ asmcmd
     ASMCMD> umount BAKTEST
     ASMCMD> lsdg
    
     SQL> drop diskgroup BAKTEST;  //删除成功
     
     Diskgroup dropped.
    
  5. AFD方式创建新磁盘组

     ASMCMD> afd_state   //查看AFD状态
     ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'rac1'
     ASMCMD> afd_label  baktest01 /dev/asm-diski
     ASMCMD> afd_lsdsk
     --------------------------------------------------------------------------------
     Label                     Filtering   Path
     ================================================================================
     BAKTEST01                   ENABLED   /dev/asm-diski
     DATA1                       ENABLED   /dev/asm-diske
     DATA2                       ENABLED   /dev/asm-diskf
     FRA1                        ENABLED   /dev/asm-diskg
     ORS1                        ENABLED   /dev/asm-diskb
     ORS2                        ENABLED   /dev/asm-diskc
     ORS3                        ENABLED   /dev/asm-diskd
    
    
     SQL> create diskgroup bak_afdtest external redundancy    disk  'AFD:BAKTEST01'  //在AFD模式上创建磁盘组
     SQL> alter diskgroup BAK_AFDTEST  mount;  //两节点分别mount
     SQL> CREATE tablespace AFD_test datafile '+BAK_AFDTEST' SIZE 100M;
     SQL> CREATE TABLE t_afd (n NUMBER) tablespace AFD_test;
     SQL> INSERT INTO t_afd VALUES(1);
     SQL> commit;
     
     Commit complete.
    
     SQL> select * from t_afd;  //查询信息正常
     
              N
     ----------
              1
    

后续

后续会继续整理相关AFD相关的知识,包括直接创建AFD硬盘不生成ASM磁盘组。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值