vware server+redhat5.4+asm+rac

一、使用vware server创建虚拟机rac01,镜像文件为redhat5.4

安装redhat，关闭防火墙和selinux服务，开启ftp、ssh、vncserver服务等；配置IP地址(eth0为公用IP地址、eth1为私有IP地址)

 二、安装vmware tools工具

cd "/media/VMware Tools"

cp VMwareTools-8.4.6-385536.tar.gz /tmp
cd /tmp
tar xvzf VMwareTools-8.4.6-385536.tar.gz
cd /tmp/vmware-tools-distrib
./vmware-installl.pl

一直回车
在此过程中设置分辨率
设置好分辨率，就安装完成了（其他选项可以按照默认，回车即可），

备注：安装vmware-tool后，重启后就可以设置共享文件夹了，就能与windows交换文件，并且你的鼠标可以自由切换了，
再也不用借助“ctrl+alt”了。

三、创建oracle和gird用户、密码、用户组；创建目录、目录属主、目录权限

创建组

groupadd   oinstall 
groupadd   asmadmin  
groupadd   asmdba
groupadd   asmoper 
groupadd   dba
groupadd   oper

 创建用户

useradd -g oinstall -G  asmadmin,asmdba,asmoper  grid
useradd -g oinstall -G dba,oper,asmdba oracle

创建密码

passwd oracle

passwd grid

 创建软件目录并授权限

mkdir -p /app/product/grid/crs

mkdir  /app/product/grid/11.2.0

chown -R grid:oinstall /app/product/grid/crs
chown -R grid:oinstall /app/product/grid/11.2.0
chmod -R 775 /app/product/grid/crs
chmod -R 775 /app/product/grid/11.2.0

mkdir  /app/product/oraInventory 

chown -R grid:oinstall /app/product/oraInventory
chmod -R 775 /app/product/oraInventory  

mkdir  /app/product/oracle
mkdir  -p /app/product/oracle/11.2.0/db_1
chown -R oracle:oinstall /app/product/oracle
chmod -R 775  /app/product/oracle

四、配置grid、oracle用户的shell限制

su - root 

vi /etc/security/limits.conf

添加：

grid    soft    nproc 2047

grid    hard    nproc 16384

grid    soft    nofile  1024

grid    hard    nofile  65536

 

oracle  soft    nproc   2047

oracle  hard    nproc   16384

oracle  soft    nofile  1024

oracle  hard    nofile 65536

使limits.conf 文件配置生效，添加以下内容到/etc/pam.d/login文件中

session required    pam_limits.so

分别编辑两用户配置文件

vi /etc/profile

if [ $USER = "grid" ]||[ $USER = "oracle" ];then
      if [ $SHELL = "/bin/ksh" ];then
         ulimit -p 16384 

         ulimit -n 65536
      else

         ulimit -u 16384 -n 65536

      fi

      umask 022
fi

五、修改linux内核参数

vi /etc/sysctl.conf

kernel.shmmax = 536870912

kernel.shmall = 2097152

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

fs.aio-max-nr=1048576

fs.file-max = 6815744

net.ipv4.ip_local_port_range=9000 65500

net.core.rmem_default=262144

net.core.rmem_max=4194304

net.core.wmem_default=262144

net.core.wmem_max=1048586

使修改生效sysctl  -p

 

六、关闭ntp服务

service ntpd stop

关闭ntp服务的开机自启动

chkconfig ntpd off

重命名ntp服务的配置文件

mv /etc/ntp.conf  /etc/ntp.conf.bak

 备注：oracle11g有自己的时钟同步机制，所以取消linux系统自带的ntp服务

七、创建oraInst.loc

创建oraInst.loc

touch /etc/oraInst.loc

编辑oraInst.loc文件

vi /etc/oraInst.loc

inventory_loc=/app/product/oraInventory
inst_group=oinstall

备注：创建oraInst.loc是为了避免grid安装过程中报以下错误：

八、编辑grid、oracle用户环境变量文件

编辑grid用户的.bash_profile

ORACLE_SID=+ASM1; export ORACLE_SID

ORACLE_BASE=/app/product/grid/crs; export ORACLE_BASE

ORACLE_HOME=/app/product/grid/11.2.0; export ORACLE_HOME

PATH=$PATH:$ORACLE_HOME/bin
PATH=$PATH:$HOME/bin
export PATH

使立即生效  source  .bash_profile

编辑oracle用户的.bash_profile

ORACLE_SID=racdb01; export ORACLE_SID
ORACLE_BASE=/app/product/oracle; export ORACLE_BASE
ORACLE_HOME=$ORACLE_BASE/11.2.0/db_1; export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin
PATH=$PATH:$HOME/bin
export PATH

使立即生效  source  .bash_profile

 九、修改主机名、域名和IP地址

vi  /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=yes

HOSTNAME=racdb01

 vi   /etc/hosts

127.0.0.1               racdb01 localhost.localdomain localhost
::1             localhost6.localdomain6 localhost6
192.168.1.101           racdb01
192.168.1.102          racdb01-vip
10.1.1.11               racdb01-priv

192.168.1.104            racdb02
192.168.1.105          racdb02-vip
10.1.1.12               racdb02-priv


192.168.1.106           cluster-scan

vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
BOOTPROTO=none
BROADCAST=192.168.1.255
HWADDR=00:0C:29:3D:33:2C
IPADDR=192.168.1.101
IPV6INIT=yes
IPV6_AUTOCONF=yes
NETMASK=255.255.255.0
NETWORK=192.168.1.0
ONBOOT=yes
TYPE=Ethernet
PEERDNS=yes
USERCTL=no

vi /etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1
BOOTPROTO=none
ONBOOT=yes
BROADCAST=10.1.1.255
HWADDR=00:0c:29:3d:33:36
IPADDR=10.1.1.11
IPV6INIT=yes
IPV6_AUTOCONF=yes
NETMASK=255.255.255.0
NETWORK=10.1.1.0
ONBOOT=yes
TYPE=Ethernet
PEERDNS=yes
USERCTL=no
 

重启网络服务，使IP生效

service network rest

十、安装oracle rac 所需的软件包

进入redhat5.4镜像目录

cd  "/media/RHEL_5.4 x86_64 DVD"

cd Server

 rpm -Uvh binutils-2.*
rpm -Uvh elfutils-libelf-0.*
rpm -Uvh glibc-2.*
rpm -Uvh glibc-common-2.*
rpm -Uvh libaio-0.*
rpm -Uvh libgcc-4.*
rpm -Uvh libstdc++-4.*
rpm -Uvh make-3.*
rpm -Uvh compat-libstdc++-33*
rpm -Uvh elfutils-libelf-devel-*
rpm -Uvh glibc-headers*
rpm -Uvh glibc-devel-2.*
rpm -Uvh libgomp*
rpm -Uvh gcc-4.*
rpm -Uvh gcc-c++-4.*
rpm -Uvh libaio-devel-0.*
rpm -Uvh libstdc++-devel-4.*
rpm -Uvh unixODBC-2.*
rpm -Uvh unixODBC-devel-2.*
rpm -Uvh sysstat-7.*

以下是软件包的安装过程 

[root@racdb01 Server]# rpm -Uvh binutils-2.*
warning: binutils-2.17.50.0.6-12.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package binutils-2.17.50.0.6-12.el5.x86_64 is already installed
[root@racdb01 Server]#

[root@racdb01 Server]# rpm -Uvh elfutils-libelf-0.*
warning: elfutils-libelf-0.137-3.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package elfutils-libelf-0.137-3.el5.x86_64 is already installed
        package elfutils-libelf-0.137-3.el5.i386 is already installed
[root@racdb01 Server]# rpm -Uvh glibc-2.*
warning: glibc-2.5-42.i686.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package glibc-2.5-42.x86_64 is already installed
        package glibc-2.5-42.i686 is already installed
[root@racdb01 Server]# rpm -Uvh glibc-common-2.*
warning: glibc-common-2.5-42.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package glibc-common-2.5-42.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh libaio-0.*
warning: libaio-0.3.106-3.2.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package libaio-0.3.106-3.2.x86_64 is already installed
        package libaio-0.3.106-3.2.i386 is already installed
[root@racdb01 Server]# rpm -Uvh libgcc-4.*
warning: libgcc-4.1.2-46.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package libgcc-4.1.2-46.el5.x86_64 is already installed
        package libgcc-4.1.2-46.el5.i386 is already installed

[root@racdb01 Server]# rpm -Uvh libstdc++-4.*
warning: libstdc++-4.1.2-46.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package libstdc++-4.1.2-46.el5.x86_64 is already installed
        package libstdc++-4.1.2-46.el5.i386 is already installed
[root@racdb01 Server]# rpm -Uvh make-3.*
warning: make-3.81-3.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package make-3.81-3.el5.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh compat-libstdc++-33*
warning: compat-libstdc++-33-3.2.3-61.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package compat-libstdc++-33-3.2.3-61.x86_64 is already installed
        package compat-libstdc++-33-3.2.3-61.i386 is already installed

[root@racdb01 Server]# rpm -Uvh elfutils-libelf-devel-*
warning: elfutils-libelf-devel-0.137-3.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package elfutils-libelf-devel-static-0.137-3.el5.x86_64 is already installed
        package elfutils-libelf-devel-0.137-3.el5.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh glibc-headers*
warning: glibc-headers-2.5-42.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package glibc-headers-2.5-42.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh glibc-devel-2.*
warning: glibc-devel-2.5-42.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package glibc-devel-2.5-42.x86_64 is already installed
        package glibc-devel-2.5-42.i386 is already installed

[root@racdb01 Server]# rpm -Uvh libgomp*
warning: libgomp-4.4.0-6.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package libgomp-4.4.0-6.el5.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh gcc-4.*
warning: gcc-4.1.2-46.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package gcc-4.1.2-46.el5.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh gcc-c++-4.*
warning: gcc-c++-4.1.2-46.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package gcc-c++-4.1.2-46.el5.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh libaio-devel-0.*
warning: libaio-devel-0.3.106-3.2.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
   1:libaio-devel           ########################################### [ 50%]
   2:libaio-devel           ########################################### [100%]

[root@racdb01 Server]# rpm -Uvh libstdc++-devel-4.*
warning: libstdc++-devel-4.1.2-46.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
        package libstdc++-devel-4.1.2-46.el5.x86_64 is already installed
[root@racdb01 Server]# rpm -Uvh unixODBC-2.*
warning: unixODBC-2.2.11-7.1.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
   1:unixODBC               ########################################### [ 50%]
   2:unixODBC               ########################################### [100%]
[root@racdb01 Server]# rpm -Uvh unixODBC-devel-2.*
warning: unixODBC-devel-2.2.11-7.1.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
   1:unixODBC-devel         ########################################### [ 50%]
   2:unixODBC-devel         ########################################### [100%]
[root@racdb01 Server]# rpm -Uvh sysstat-7.*
warning: sysstat-7.0.2-3.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing...                ########################################### [100%]
   1:sysstat                ########################################### [100%]

 

十一、ASM软件安装配置、创建ASM磁盘

查看内核信息

[idnms@racdb02 ~]$ uname -a
Linux racdb02 2.6.18-164.el5 #1 SMP Tue Aug 18 15:51:48 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

根据内核信息下载对应ASMlib包

oracleasm-support-2.1.8-1.el5.x86_64.rpm
oracleasm-2.6.18-164.15.1.el5-2.0.5-1.el5.x86_64.rpm
oracleasmlib-2.0.4-1.el5.x86_64.rpm

按照以下顺序安装ASMlib包

[root@racdb01 tmp]# rpm -ivh oracleasm-support-2.1.8-1.el5.x86_64.rpm
warning: oracleasm-support-2.1.8-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...                ########################################### [100%]
   1:oracleasm-support      ########################################### [100%]

 

[root@racdb02 tmp]# rpm -ivh oracleasm-2.6.18-164.el5-2.0.5-1.el5.x86_64.rpm
warning: oracleasm-2.6.18-164.el5-2.0.5-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...                ########################################### [100%]
   1:oracleasm-2.6.18-164.el########################################### [100%]

[root@racdb01 tmp]# rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm
warning: oracleasmlib-2.0.4-1.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 1e5e0159
Preparing...                ########################################### [100%]
   1:oracleasmlib           ########################################### [100%]

查看安装情况

[idnms@racdb02 ~]$ rpm -qa|grep oracleasm

oracleasmlib-2.0.4-1.el5
oracleasm-2.6.18-164.el5-2.0.5-1.el5
oracleasm-support-2.1.8-1.el5

配置ASMlib

[root@racdb02 tmp]# service oracleasm configure
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface [grid]: grid
Default group to own the driver interface [asmadmin]: asmadmin
Start Oracle ASM library driver on boot (y/n) [y]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]

为虚拟机添加三块虚拟硬盘，选择磁盘的时候要依次是SCSI 1:1,SCSI 1:2，SCSI 1:3；

分别对新添加的三个硬盘/dev/sdb、/dev/sdc、/dev/sdd进行分区(只分区，不用格式化和挂载)

创建ASM磁盘

[root@racdb01 ~]# service oracleasm createdisk CRS01 /dev/sdc1
arking disk "CRS01" as an ASM disk:                       [  OK  ]

[root@racdb01 ~]# service oracleasm createdisk CRS02 /dev/sdd1
Marking disk "CRS02" as an ASM disk:                       [  OK  ]

[root@racdb01 ~]# service oracleasm createdisk CRS03 /dev/sde1
Marking disk "CRS03" as an ASM disk:                       [  OK  ]

查看ASM磁盘可用性

[root@racdb01 ~]# service oracleasm listdisks
CRS01
CRS02
CRS03

十二、使用vmware server复制虚拟机

关闭当前虚拟机rac01

创建rac02文件夹：\Virtual Machines\rac02

复制rac01目录下的所有文件到rac02目录下，将rac02目录下所有包含rac01的文件名改为rac02

修改D:\Virtual Machines\rac02\rac02.vmx文件，将rac01修改为rac02

 添加虚拟机rac02到inventory

十三、修改rac02虚拟机配置

修改主机名

修改/etc/hosts文件

修改eth0和eth1的IP信息

重启网络接口

修改grid和oracle环境变量

gird用户下修改ORACLE_SID=+ASM2;         修改生效：source .bash_profile

oracle用户下修改ORACLE_SID=racdb02;     修改生效：source .bash_profile

十四、配置ssh等效性

每个节点上都执行：
su - grid
mkdir .ssh
chmod 700 .ssh
ssh-keygen -t  rsa（一直回车）
ssh-keygen -t  dsa（一直回车）

只在racdb01上执行：

切换到grid用户
[grid@racdb01 .ssh]$ pwd
/home/grid/.ssh
ssh racdb01 cat /home/grid/.ssh/id_rsa.pub >>authorized_keys
ssh racdb01 cat  /home/grid/.ssh/id_dsa.pub >>authorized_keys
ssh racdb02 cat  /home/grid/.ssh/id_rsa.pub >>authorized_keys
ssh racdb02 cat  /home/grid/.ssh/id_dsa.pub >>authorized_keys
将生成的文件拷贝到racdb02上
scp authorized_keys  racdb02:/home/grid/.ssh/

 测试等效性

每个节点上执行:
ssh racdb01 date
ssh racdb02 date
ssh racdb01-priv date
ssh racdb02-priv date

十五、安装用于 Linux 的 cvuqdisk 程序包

两个 Oracle RAC 节点上安装操作系统程序包 cvuqdisk。如果没有 cvuqdisk，集群验证实用程序就无法发现共享磁盘，当运行（手动运行或在 Oracle Grid Infrastructure 安装结束时自动运行）集群验证实用程序时，您会收到这样的错误消息：“Package cvuqdisk not installed”。使用适用于您的硬件体系结构（例如，x86_64 或 i386）的 cvuqdisk RPM。

cvuqdisk RPM 包含在linux.x64_11gR2_grid.zip的 rpm 目录中。

ftp上传linux.x64_11gR2_grid.zip并解压

设置环境变量 CVUQDISK_GRP，使其指向作为 cvuqdisk 的所有者所在的组：

每个节点上都执行：

[root@racdb01 rpm]# export   CVUQDISK_GRP=oinstall
[root@racdb01 rpm]# rpm -ihv cvuqdisk-1.0.7-1.rpm
Preparing...                ########################################### [100%]
   1:cvuqdisk               ########################################### [100%]

十六、CVU 验证 Oracle 集群件要求及硬件和操作系统设置

验证前准备工作：（如果硬盘配置满足需求，此步骤可省略）

每个节点上执行：

因grid软件目录和oracle软件目录空间不足，为其增加新分区、格式化、挂载

备注：grid软件目录和oracle软件目录可查看各自.bash_profile文件

因/tmp空间不足，为其增加新分区、格式化、挂载;设置/tmp目录的权限为777

增加虚拟机rac01、rac02的内存各为1.5G

SWAP分区增加到2G

只在racdb01上执行以下命令：

[grid@racdb01 grid]$ pwd
/home/grid/grid

查看linux.x64_11gR2_grid.zip解压文件
[grid@racdb01 grid]$ ls -lt
总计 40
-rw-r--r--  1 grid oinstall 4228 2009-08-18 welcome.html
drwxr-xr-x  9 grid oinstall 4096 2009-08-17 doc
drwxrwxr-x  2 grid oinstall 4096 2009-08-15 sshsetup
drwxrwxr-x  2 grid oinstall 4096 2009-08-15 response
drwxr-xr-x 14 grid oinstall 4096 2009-08-15 stage
drwxr-xr-x  4 grid oinstall 4096 2009-08-15 install
drwxrwxr-x  2 grid oinstall 4096 2009-08-15 rpm
-rwxr-xr-x  1 grid oinstall 3227 2009-08-15 runInstaller
-rwxrwxr-x  1 grid oinstall 3795 2009-01-29 runcluvfy.sh

验证 Oracle 集群件要求：

./runcluvfy.sh stage -pre crsinst -n racdb01,racdb02 -fixup -verbose

忽略下列错误：

检查: 组 "dba" 中用户 "grid" 的成员资格
  节点名               用户存在          组存在           组中的用户         注释              
  ----------------  ------------  ------------  ------------  ----------------
  racdb02           是             是             否             失败              
  racdb01           是             是             否             失败              
结果:组 "dba" 中用户 "grid" 的成员资格检查失败

验证硬件和操作系统设置（在grid软件目录里运行）
./runcluvfy.sh stage -post hwos -n racdb01,racdb02 -verbose

十七、安装grid软件

使用 xmanager远程连接redhat配置：

修改配置文件：/usr/share/gdm/default.conf

[xdmcp]段

enable=true //257行

Port=177//284行

[security]段

AllowRemoteRoot =true //214行

修改/etc/inittab

文件最后一行为x:5:respawn:/usr/sbin/gdm

重启gdm

gdm-restart

使用xmanager工具连接redhat

[grid@racdb01 grid]$ ls -lt
总计 40
-rw-r--r--  1 grid oinstall 4228 2009-08-18 welcome.html
drwxr-xr-x  9 grid oinstall 4096 2009-08-17 doc
drwxrwxr-x  2 grid oinstall 4096 2009-08-15 sshsetup
drwxrwxr-x  2 grid oinstall 4096 2009-08-15 response
drwxr-xr-x 14 grid oinstall 4096 2009-08-15 stage
drwxr-xr-x  4 grid oinstall 4096 2009-08-15 install
drwxrwxr-x  2 grid oinstall 4096 2009-08-15 rpm
-rwxr-xr-x  1 grid oinstall 3227 2009-08-15 runInstaller
-rwxrwxr-x  1 grid oinstall 3795 2009-01-29 runcluvfy.sh
[grid@racdb01 grid]$ ./runInstaller 

SCAN名称要和/etc/hosts文件中的一致

设置grid用户密码，点击设置在两节点之间建立无口令的SSH连接

设置网络接口

选择自动存储管理，设置SYSASM口令

创建ASM磁盘组，定义磁盘组名，冗余为普通，向磁盘组添加磁盘

定义产品清单目录

先决条件检查