Oracle Real Application Cluster(RAC): September 2012

Oracle 11gR1(11.1.0.6.0) Real Application Cluster Installation (RAC)

Installation Steps on Linux

Installation Prerequisite:

Perform below steps in all nodes as a root user.

Step 1: Install below RPM's as a root user.

binutils-2.17.50.0.6

compat-libstdc++-33-3.2.3

compat-libstdc++-33-3.2.3 (32 bit)

elfutils-libelf-0.125

elfutils-libelf-devel-0.125

gcc-4.1.1

gcc-c++-4.1.1

glibc-2.5-12

glibc-2.5-12 (32 bit)

glibc-common-2.5

glibc-devel-2.5

glibc-devel-2.5-12 (32 bit)

libaio-0.3.106

libaio-0.3.106 (32 bit)

libaio-devel-0.3.106

libgcc-4.1.1

libgcc-4.1.1 (32 bit)

libstdc++-4.1.1

libstdc++-4.1.1 (32 bit)

libstdc++-devel 4.1.1

make-3.81

sysstat-7.0.0

To install RPM use below command as a root user.

rpm -Uvh <rpm_name> or

rpm -ivh <rpm_name>

Step 2: The "/etc/hosts" file must contain the following information.

##################################################

### LocalHost

##################################################

127.0.0.1 localhost.localdomain localhost

##################################################

### Public IP Address

##################################################

152.168.1.50 krac1.dbprod.com krac1

152.168.1.51 krac2.dbprod.com krac2

##################################################

### Private IP Address

##################################################

192.168.1.50 krac1-priv.dbprod.com krac1-priv

192.168.1.51 krac2-priv.dbprod.com krac2-priv

##################################################

### VIP Address

##################################################

152.168.1.60 krac1-vip.dbprod.com krac1-vip

152.168.1.61 krac2-vip.dbprod.com krac2-vip

Step 3: Add the following lines to the "/etc/sysctl.conf" file.

kernel.shmall = 2097152

kernel.shmmax = 2147483648

kernel.shmmni = 4096

kernel.sem = 250 32000 100 128

fs.file-max = 65536

net.ipv4.ip_local_port_range = 1024 65500

net.core.rmem_default = 262144

net.core.rmem_max = 4194304

net.core.wmem_default = 262144

net.core.wmem_max = 1048576

Step 4: Run the following command to change the current kernel parameters.

/sbin/sysctl -p

Step 5: Configuration of the Hangcheck-timer Module

on Kernel 2.4 enter a command similar to the following to start the module on that node:

/sbin/insmod hangcheck-timer hangcheck_tick=1 hangcheck_margin=10 hangcheck_reboot=1

on Kernel 2.6 enter a command similar to the following to start the module located in the directories of the current kernel version:

/sbin/insmod /lib/modules/`uname -r`/kernel/drivers/char/hangcheck-timer.ko

hangcheck_tick=1 hangcheck_margin=10 hangcheck_reboot=1

Add the above command into /etc/rc.d/rc.local file.

Step 6: Create the new groups and users.

[root@krac1 ~]# groupadd -g 501 oinstall

[root@krac1 ~]# groupadd -g 502 dba

[root@krac1 ~]# groupadd -g 503 oper

[root@krac1 ~]# groupadd -g 504 asmadmin

[root@krac1 ~]# useradd -u 501 -c "Oracle Clusterware Owner" -g oinstall crs

[root@krac1 ~]# useradd -u 502 -c "Oracle Software Owner" -g oinstall -G dba,oper,asmadmin oracle

[root@krac1 ~]# passwd crs

Changing password for user crs.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

[root@krac1 ~]# passwd oracle

Changing password for user oracle.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

[root@krac1 ~]#

Step 7: Add the following lines to the "/etc/security/limits.conf" file.

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

crs soft nproc 2047

crs hard nproc 16384

crs soft nofile 1024

crs hard nofile 65536

step 8 : Add the following lines to the "/etc/pam.d/login" file, if it does not already exist.

session required /lib/security/pam_limits.so

Step 9 : Add the following lines to the "/etc/profile" file.

if [[ $USER = "oracle" ] || [ $USER = "crs" ]]; then

if [ $SHELL = "/bin/ksh" ]; then

ulimit -p 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

umask 022

Step 10 : Configuring SSH on Cluster Member Nodes

su - crs

[crs@krac1 ~]$ mkdir ~/.ssh

[crs@krac1 ~]$ chmod 700 ~/.ssh

[crs@krac1 ~]$ cd ~/.ssh

[crs@krac1 .ssh]$/usr/bin/ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/crs/.ssh/id_rsa): <ENTER>

Enter passphrase (empty for no passphrase): <ENTER>

Enter same passphrase again:

Your identification has been saved in /home/crs/.ssh/id_rsa.

Your public key has been saved in /home/crs/.ssh/id_rsa.pub.

The key fingerprint is:

73:49:56:a3:ed:e4:4e:b2:66:35:32:c6:d1:7d:e1:17 crs@krac1.dbprod.com

[crs@krac1 ~]$ /usr/bin/ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/crs/.ssh/id_dsa):

Enter passphrase (empty for no passphrase): <ENTER>

Enter same passphrase again: <ENTER>

Your identification has been saved in /home/crs/.ssh/id_dsa.

Your public key has been saved in /home/crs/.ssh/id_dsa.pub.

The key fingerprint is:

66:64:8c:2f:44:a4:c2:bd:a5:4e:46:a8:dc:d6:1b:11 crs@krac1.dbprod.com

Note: Repeat above steps in all nodes.

[crs@krac1 .ssh]$ touch authorized_keys

[crs@krac1 .ssh]$ cat *.pub > authorized_keys

[crs@krac1 .ssh]$ ssh krac2 "cat ~/.ssh/*.pub >> authorized_keys #Repeat for all nodes

[crs@krac1 .ssh]$ scp authorized_keys crs@krac2:~/.ssh/. #Repeat for all nodes

crs@krac2's password:

authorized_keys 100% 1012 1.0KB/s 00:00

su - oracle

[oracle@krac1 ~]$ mkdir ~/.ssh

[oracle@krac1 ~]$ chmod 700 ~/.ssh

[oracle@krac1 ~]$ /usr/bin/ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_rsa):

Enter passphrase (empty for no passphrase): <ENTER>

Enter same passphrase again: <ENTER>

Your identification has been saved in /home/oracle/.ssh/id_rsa.

Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.

The key fingerprint is:

39:3a:5b:82:e2:56:9c:72:4b:7c:ee:6c:50:2c:f9:5e oracle@krac2.dbprod.com

[oracle@krac2 ~]$ /usr/bin/ssh-keygen -t dsa

Generating public/private dsa key pair.

Enter file in which to save the key (/home/oracle/.ssh/id_dsa):

Enter passphrase (empty for no passphrase): <ENTER>

Enter same passphrase again: <ENTER>

Your identification has been saved in /home/oracle/.ssh/id_dsa.

Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.

The key fingerprint is:

b0:4d:a9:0c:0d:1b:f3:77:f6:fc:1a:71:17:10:35:4f oracle@krac2.dbprod.com

Note: Repeat above steps in all nodes.

[oracle@krac1 .ssh]$ cat *.pub > authorized_keys

[oracle@krac1 .ssh]$ ssh krac2 "cat ~/.ssh/*.pub >> authorized_keys #Repeat all nodes

[oracle@krac1 .ssh]$ scp authorized_keys krac2:~/.ssh/. #Repeat all nodes

The authenticity of host 'krac1 (152.168.1.50)' can't be established.

RSA key fingerprint is e4:b5:5e:ab:5c:0b:a3:f4:20:1a:1f:3f:ad:04:50:f1.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'krac1,152.168.1.50' (RSA) to the list of known hosts.

oracle@krac2's password:

authorized_keys 100% 1012 1.0KB/s 00:00

The "authorized_keys" file on both servers now contains the public keys generated on all RAC nodes.

Step 11 : Enabling SSH User Equivalency on Cluster Member Nodes

ssh krac1 date

ssh krac2 date

ssh krac1.dbprod.com date

ssh krac2.dbprod.com date

ssh krac1-priv date

ssh krac2-priv date

ssh krac1-priv.dbprod.com date

ssh krac2-priv.dbprod.com date

You should now be able to SSH and SCP between servers without entering passwords.

Step 14: Create the directories in which the Oracle software will be installed.

[root@krac1 ~]# mkdir /u01/crs11g

[root@krac1 ~]# chown root:oinstall /u01/crs11g/

[root@krac1 ~]# chmod 775 /u01/crs11g

[root@krac1 ~]# mkdir -p /u02/app/oracle/product/11.1.0/db_1

[root@krac1 ~]# mkdir -p /u02/app/oracle/product/11.1.0/asm_1

[root@krac1 ~]# chown -R oracle:oinstall /u02/app/oracle

[root@krac1 ~]# chmod -R 775 /u02/app/oracle

Step 15 : Prepare the shared storage for Oracle RAC

# cd /dev

# ls sd*

sda sda1 sda2 sdb sdc sdd sde sdf sdg

Use the "fdisk" command to partition the disks sdb to sdg. The following output shows the

expected fdisk output for the sdb disk.

[root@krac1 ~]# fdisk /dev/sdb

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only,

until you decide to write them. After that, of course, the previous

content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/sdc: 1073 MB, 1073741824 bytes

255 heads, 63 sectors/track, 130 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System

Command (m for help): n

Command action

e extended

p primary partition (1-4)

Partition number (1-4): 1

First cylinder (1-130, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-130, default 130):

Using default value 130

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

Load the updated block device partition tables by running the following on ALL servers participating in the cluster:

[root@krac2 ~]# /sbin/partprobe /dev/sdb

[root@krac2 ~]# /sbin/partprobe /dev/sdc

[root@krac2 ~]# /sbin/partprobe /dev/sdd

[root@krac2 ~]# /sbin/partprobe /dev/sde

[root@krac2 ~]# /sbin/partprobe /dev/sdf

[root@krac2 ~]# /sbin/partprobe /dev/sdg

Step 17 : Edit the "/etc/sysconfig/rawdevices" file, adding the following lines.

/dev/raw/raw1 /dev/sdb1

/dev/raw/raw2 /dev/sdc1

/dev/raw/raw3 /dev/sdd1

/dev/raw/raw4 /dev/sde1

/dev/raw/raw5 /dev/sdf1

Step 18 : Run the following commands and add them the "/etc/rc.d/rc.local" file.

chown root:oinstall /dev/raw/raw1

chown crs:oinstall /dev/raw/raw2

chown oracle:oinstall /dev/raw/raw3

chown oracle:oinstall /dev/raw/raw4

chown oracle:oinstall /dev/raw/raw5

chown oracle:oinstall /dev/raw/raw6

chmod 640 /dev/raw/raw1

chmod 640 /dev/raw/raw2

chmod 640 /dev/raw/raw3

chmod 640 /dev/raw/raw4

chmod 640 /dev/raw/raw5

chmod 640 /dev/raw/raw6

Step 19: Cluster Verification Utility stage check

[crs@krac1 clusterware]$ ./runcluvfy.sh stage -pre crsinst -n krac1,krac2 -verbose

Performing pre-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "krac1"

Destination Node Reachable?

------------------------------------ ------------------------

krac1 yes

krac2 yes

Result: Node reachability check passed from node "krac1".

Checking user equivalence...

Check: User equivalence for user "crs"

Node Name Comment

------------------------------------ ------------------------

krac2 passed

krac1 passed

Result: User equivalence check passed for user "crs".

Checking administrative privileges...

Check: Existence of user "crs"

Node Name User Exists Comment

------------ ------------------------ ------------------------

krac2 yes passed

krac1 yes passed

Result: User existence check passed for "crs".

Check: Existence of group "oinstall"

Node Name Status Group ID

------------ ------------------------ ------------------------

krac2 exists 501

krac1 exists 501

Result: Group existence check passed for "oinstall".

Check: Membership of user "crs" in group "oinstall" [as Primary]

Node Name User Exists Group Exists User in Group Primary Comment

---------------- ------------ ------------ ------------ ------------ ------------

krac2 yes yes yes yes passed

krac1 yes yes yes yes passed

Result: Membership check for user "crs" in group "oinstall" [as Primary] passed.

Administrative privileges check passed.

Checking node connectivity...

Interface information for node "krac2"

Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address

---------------- ------------ ------------ ------------ ------------ ------------

eth0 152.168.1.51 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:F7

eth1 192.168.1.51 192.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:01

Interface information for node "krac1"

Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address

---------------- ------------ ------------ ------------ ------------ ------------

eth0 152.168.1.50 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:43

eth1 192.168.1.50 192.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:4D

Check: Node connectivity of subnet "152.168.1.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

krac2:eth0 krac1:eth0 yes

Result: Node connectivity check passed for subnet "152.168.1.0" with node(s) krac2,krac1.

Check: Node connectivity of subnet "192.168.1.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

krac2:eth1 krac1:eth1 yes

Result: Node connectivity check passed for subnet "192.168.1.0" with node(s) krac2,krac1.

Interfaces found on subnet "152.168.1.0" that are likely candidates for VIP:

krac2 eth0:152.168.1.51

krac1 eth0:152.168.1.50

Interfaces found on subnet "192.168.1.0" that are likely candidates for VIP:

krac2 eth1:192.168.1.51

krac1 eth1:192.168.1.50

WARNING:

Could not find a suitable set of interfaces for the private interconnect.

Result: Node connectivity check passed.

Checking system requirements for 'crs'...

Check: Total memory

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 1.98GB (2075468KB) 1GB (1048576KB) passed

krac1 1.98GB (2075468KB) 1GB (1048576KB) passed

Result: Total memory check passed.

Check: Free disk space in "/tmp" dir

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 6.62GB (6946444KB) 400MB (409600KB) passed

krac1 6.52GB (6833792KB) 400MB (409600KB) passed

Result: Free disk space check passed.

Check: Swap space

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 3.42GB (3582484KB) 1.5GB (1572864KB) passed

krac1 3.42GB (3582484KB) 1.5GB (1572864KB) passed

Result: Swap space check passed.

Check: System architecture

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 i686 i686 passed

krac1 i686 i686 passed

Result: System architecture check passed.

Check: Kernel version

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 2.6.18-194.el5 2.6.9 passed

krac1 2.6.18-194.el5 2.6.9 passed

Result: Kernel version check passed.

Check: Package existence for "make-3.81"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 make-3.81-3.el5 passed

krac1 make-3.81-3.el5 passed

Result: Package existence check passed for "make-3.81".

Check: Package existence for "binutils-2.17.50.0.6"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 binutils-2.17.50.0.6-14.el5 passed

krac1 binutils-2.17.50.0.6-14.el5 passed

Result: Package existence check passed for "binutils-2.17.50.0.6".

Check: Package existence for "gcc-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 gcc-4.1.2-48.el5 passed

krac1 gcc-4.1.2-48.el5 passed

Result: Package existence check passed for "gcc-4.1.1".

Check: Package existence for "libaio-0.3.106"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libaio-0.3.106-5 passed

krac1 libaio-0.3.106-5 passed

Result: Package existence check passed for "libaio-0.3.106".

Check: Package existence for "libaio-devel-0.3.106"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libaio-devel-0.3.106-5 passed

krac1 libaio-devel-0.3.106-5 passed

Result: Package existence check passed for "libaio-devel-0.3.106".

Check: Package existence for "libstdc++-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libstdc++-4.1.2-48.el5 passed

krac1 libstdc++-4.1.2-48.el5 passed

Result: Package existence check passed for "libstdc++-4.1.1".

Check: Package existence for "elfutils-libelf-devel-0.125"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 elfutils-libelf-devel-0.137-3.el5 passed

krac1 elfutils-libelf-devel-0.137-3.el5 passed

Result: Package existence check passed for "elfutils-libelf-devel-0.125".

Check: Package existence for "sysstat-7.0.0"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 sysstat-7.0.2-3.el5 passed

krac1 sysstat-7.0.2-3.el5 passed

Result: Package existence check passed for "sysstat-7.0.0".

Check: Package existence for "compat-libstdc++-33-3.2.3"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 compat-libstdc++-33-3.2.3-61 passed

krac1 compat-libstdc++-33-3.2.3-61 passed

Result: Package existence check passed for "compat-libstdc++-33-3.2.3".

Check: Package existence for "libgcc-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libgcc-4.1.2-48.el5 passed

krac1 libgcc-4.1.2-48.el5 passed

Result: Package existence check passed for "libgcc-4.1.1".

Check: Package existence for "libstdc++-devel-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libstdc++-devel-4.1.2-48.el5 passed

krac1 libstdc++-devel-4.1.2-48.el5 passed

Result: Package existence check passed for "libstdc++-devel-4.1.1".

Check: Package existence for "unixODBC-2.2.11"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 unixODBC-2.2.11-7.1 passed

krac1 unixODBC-2.2.11-7.1 passed

Result: Package existence check passed for "unixODBC-2.2.11".

Check: Package existence for "unixODBC-devel-2.2.11"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 unixODBC-devel-2.2.11-7.1 passed

krac1 unixODBC-devel-2.2.11-7.1 passed

Result: Package existence check passed for "unixODBC-devel-2.2.11".

Check: Package existence for "glibc-2.5-12"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 glibc-2.5-49 passed

krac1 glibc-2.5-49 passed

Result: Package existence check passed for "glibc-2.5-12".

Check: Group existence for "dba"

Node Name Status Comment

------------ ------------------------ ------------------------

krac2 exists passed

krac1 exists passed

Result: Group existence check passed for "dba".

Check: Group existence for "oinstall"

Node Name Status Comment

------------ ------------------------ ------------------------

krac2 exists passed

krac1 exists passed

Result: Group existence check passed for "oinstall".

Check: User existence for "nobody"

Node Name Status Comment

------------ ------------------------ ------------------------

krac2 exists passed

krac1 exists passed

Result: User existence check passed for "nobody".

System requirement passed for 'crs'

Pre-check for cluster services setup was successful.

Oracle Clusterware Install

su - crs

./runInstaller

Action : Click "Next"

Action : Entry the full path of the inventory directory, Operating system group name and click "Next"

Action:

Confirm the Cluster Name selected is acceptable

Confirm the Details for the current node are OK.

The defaults are:

Public Node Name : must resolve via hosts and or DNS to the public IP address and must be live
Private Node Name : must resolve via hosts to the interconnect IP address and must be live
Virtual Host Name : must resolve via hosts and or DNS to a new IP address and must not be live
If these are not correct select the node entry and click Edit... to modify OR Click Add... to add more nodes to the cluster

Actions : Enter the new node details

Public Node Name : must resolve via hosts and or DNS to the public IP address and must be live
Private Node Name : must resolve via hosts to the interconnect IP address and must be live
Virtual Host Name : must resolve via hosts and or DNS to a new IP address and must not be live
Click OK to return to the node list for the cluster

Action : Click "Next".

Actions : Select the Interface Type : Public radio button and Click "OK"

Actions :

Select the External radio button
Enter the device to be used for the OCR
Click Next

Actions :

Select the External radio button
Enter the device to be used for the vote disk

Action : Click "Install"

#orainstRoot.sh and root.sh script output on rac1

[root@krac1 ~]# /u02/app/oracle/oraInventory/orainstRoot.sh

Changing permissions of /u02/app/oracle/oraInventory to 770.

Changing groupname of /u02/app/oracle/oraInventory to oinstall.

The execution of the script is complete

[root@krac1 ~]# /u01/crs11g/root.sh

Checking to see if Oracle CRS stack is already configured

/etc/oracle does not exist. Creating it now.

Setting the permissions on OCR backup directory

Setting up Network socket directories

Oracle Cluster Registry configuration upgraded successfully

Successfully accumulated necessary OCR keys.

Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.

node <nodenumber>: <nodename> <private interconnect name> <hostname>

node 1: krac1 krac1-priv krac1

node 2: krac2 krac2-priv krac2

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Now formatting voting device: /dev/raw/raw2

Format of 1 voting devices complete.

/etc/profile: line 65: ulimit: open files: cannot modify limit: Operation not permitted

Startup will be queued to init within 30 seconds.

Adding daemons to inittab

Expecting the CRS daemons to be up within 600 seconds.

Cluster Synchronization Services is active on these nodes.

krac1

Cluster Synchronization Services is inactive on these nodes.

krac2

Local node checking complete. Run root.sh on remaining nodes to start CRS daemons.

[root@krac1 ~]#

#orainstRoot.sh and root.sh script output on rac2

[root@krac2 ~]# /u02/app/oracle/oraInventory/orainstRoot.sh

Changing permissions of /u02/app/oracle/oraInventory to 770.

Changing groupname of /u02/app/oracle/oraInventory to oinstall.

The execution of the script is complete

[root@krac2 ~]# /u01/crs11g/root.sh

Checking to see if Oracle CRS stack is already configured

/etc/oracle does not exist. Creating it now.

Setting the permissions on OCR backup directory

Setting up Network socket directories

Oracle Cluster Registry configuration upgraded successfully

clscfg: EXISTING configuration version 4 detected.

clscfg: version 4 is 11 Release 1.

Successfully accumulated necessary OCR keys.

Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.

node <nodenumber>: <nodename> <private interconnect name> <hostname>

node 1: krac1 krac1-priv krac1

node 2: krac2 krac2-priv krac2

clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.

-force is destructive and will destroy any previous cluster

configuration.

Oracle Cluster Registry for cluster has already been initialized

/etc/profile: line 65: ulimit: open files: cannot modify limit: Operation not permitted

Startup will be queued to init within 30 seconds.

Adding daemons to inittab

Expecting the CRS daemons to be up within 600 seconds.

Cluster Synchronization Services is active on these nodes.

krac1

krac2

Cluster Synchronization Services is active on all the nodes.

Waiting for the Oracle CRSD and EVMD to start

Oracle CRS stack installed and running under init(1M)

Running vipca(silent) for configuring nodeapps

Creating VIP application resource on (2) nodes...

Creating GSD application resource on (2) nodes...

Creating ONS application resource on (2) nodes...

Starting VIP application resource on (2) nodes...

Starting GSD application resource on (2) nodes...

Starting ONS application resource on (2) nodes...

Done.

Action : None required.

Action : Click Exit.

[crs@krac1 ~]$ crs_stat -t -v

Name Type R/RA F/FT Target State Host

----------------------------------------------------------------------

ora.krac1.gsd application 0/5 0/0 ONLINE ONLINE krac1

ora.krac1.ons application 0/3 0/0 ONLINE ONLINE krac1

ora.krac1.vip application 0/0 0/0 ONLINE ONLINE krac1

ora.krac2.gsd application 0/5 0/0 ONLINE ONLINE krac2

ora.krac2.ons application 0/3 0/0 ONLINE ONLINE krac2

ora.krac2.vip application 0/0 0/0 ONLINE ONLINE krac2

[crs@krac1 ~]$

Post CUV Check for clusterware post installation.

[crs@krac1 ~]$ cluvfy stage -post crsinst -n krac1,krac2 -verbose

Performing post-checks for cluster services setup

Checking node reachability...

Check: Node reachability from node "krac1"

Destination Node Reachable?

------------------------------------ ------------------------

krac1 yes

krac2 yes

Result: Node reachability check passed from node "krac1".

Checking user equivalence...

Check: User equivalence for user "crs"

Node Name Comment

------------------------------------ ------------------------

krac2 passed

krac1 passed

Result: User equivalence check passed for user "crs".

Checking Cluster manager integrity...

Checking CSS daemon...

Node Name Status

------------------------------------ ------------------------

krac2 running

krac1 running

Result: Daemon status check passed for "CSS daemon".

Cluster manager integrity check passed.

Checking cluster integrity...

Node Name

------------------------------------

krac1

krac2

Cluster integrity check passed

Checking OCR integrity...

Checking the absence of a non-clustered configuration...

All nodes free of non-clustered, local-only configurations.

Uniqueness check for OCR device passed.

Checking the version of OCR...

OCR of correct Version "2" exists.

Checking data integrity of OCR...

Data integrity check for OCR passed.

OCR integrity check passed.

Checking CRS integrity...

Checking daemon liveness...

Check: Liveness for "CRS daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "CRS daemon".

Checking daemon liveness...

Check: Liveness for "CSS daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "CSS daemon".

Checking daemon liveness...

Check: Liveness for "EVM daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons

Node Name CRS daemon CSS daemon EVM daemon

------------ ------------------------ ------------------------ ----------

krac2 yes yes yes

krac1 yes yes yes

Checking CRS health...

Check: Health of CRS

Node Name CRS OK?

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: CRS health check passed.

CRS integrity check passed.

Checking node application existence...

Checking existence of VIP node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 yes exists passed

krac1 yes exists passed

Result: Check passed.

Checking existence of ONS node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 no exists passed

krac1 no exists passed

Result: Check passed.

Checking existence of GSD node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 no exists passed

krac1 no exists passed

Result: Check passed.

Post-check for cluster services setup was successful.

Oracle ASM Home Software Install.

CVU check before creating ASM instance.

Verify the cluster is configured correctly for an instance creation.

[oracle@krac1 clusterware]$ ./runcluvfy.sh stage -pre dbinst -n krac1,krac2 -verbose

Performing pre-checks for database installation

Checking node reachability...

Check: Node reachability from node "krac1"

Destination Node Reachable?

------------------------------------ ------------------------

krac1 yes

krac2 yes

Result: Node reachability check passed from node "krac1".

Checking user equivalence...

Check: User equivalence for user "oracle"

Node Name Comment

------------------------------------ ------------------------

krac2 passed

krac1 passed

Result: User equivalence check passed for user "oracle".

Checking administrative privileges...

Check: Existence of user "oracle"

Node Name User Exists Comment

------------ ------------------------ ------------------------

krac2 yes passed

krac1 yes passed

Result: User existence check passed for "oracle".

Check: Existence of group "oinstall"

Node Name Status Group ID

------------ ------------------------ ------------------------

krac2 exists 501

krac1 exists 501

Result: Group existence check passed for "oinstall".

Check: Membership of user "oracle" in group "oinstall" [as Primary]

Node Name User Exists Group Exists User in Group Primary Comment

---------------- ------------ ------------ ------------ ------------ ------------

krac2 yes yes yes yes passed

krac1 yes yes yes yes passed

Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed.

Check: Existence of group "dba"

Node Name Status Group ID

------------ ------------------------ ------------------------

krac2 exists 502

krac1 exists 502

Result: Group existence check passed for "dba".

Check: Membership of user "oracle" in group "dba"

Node Name User Exists Group Exists User in Group Primary Comment

---------------- ------------ ------------ ------------ ------------ ------------

krac2 yes yes yes no passed

krac1 yes yes yes no passed

Result: Membership check for user "oracle" in group "dba" passed.

Administrative privileges check passed.

Checking node connectivity...

Interface information for node "krac2"

Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address

---------------- ------------ ------------ ------------ ------------ ------------

eth0 152.168.1.51 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:F7

eth0 152.168.1.61 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:F7

eth1 192.168.1.51 192.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:01

Interface information for node "krac1"

Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address

---------------- ------------ ------------ ------------ ------------ ------------

eth0 152.168.1.50 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:43

eth0 152.168.1.60 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:43

eth1 192.168.1.50 192.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:4D

Check: Node connectivity of subnet "152.168.1.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

krac2:eth0 krac2:eth0 yes

krac2:eth0 krac1:eth0 yes

krac1:eth0 krac1:eth0 yes

Result: Node connectivity check passed for subnet "152.168.1.0" with node(s) krac2,krac1.

Check: Node connectivity of subnet "192.168.1.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

krac2:eth1 krac1:eth1 yes

Result: Node connectivity check passed for subnet "192.168.1.0" with node(s) krac2,krac1.

Interfaces found on subnet "152.168.1.0" that are likely candidates for VIP:

krac2 eth0:152.168.1.51 eth0:152.168.1.61

krac1 eth0:152.168.1.50 eth0:152.168.1.60

Interfaces found on subnet "192.168.1.0" that are likely candidates for VIP:

krac2 eth1:192.168.1.51

krac1 eth1:192.168.1.50

WARNING:

Could not find a suitable set of interfaces for the private interconnect.

Result: Node connectivity check passed.

Checking system requirements for 'database'...

Check: Total memory

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 1.98GB (2075468KB) 1GB (1048576KB) passed

krac1 1.98GB (2075468KB) 1GB (1048576KB) passed

Result: Total memory check passed.

Check: Free disk space in "/tmp" dir

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 6.47GB (6787332KB) 400MB (409600KB) passed

krac1 6.51GB (6827884KB) 400MB (409600KB) passed

Result: Free disk space check passed.

Check: Swap space

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 3.42GB (3582484KB) 1.5GB (1572864KB) passed

krac1 3.42GB (3582484KB) 1.5GB (1572864KB) passed

Result: Swap space check passed.

Check: System architecture

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 i686 i686 passed

krac1 i686 i686 passed

Result: System architecture check passed.

Check: Kernel version

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 2.6.18-194.el5 2.6.18 passed

krac1 2.6.18-194.el5 2.6.18 passed

Result: Kernel version check passed.

Check: Package existence for "make-3.81"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 make-3.81-3.el5 passed

krac1 make-3.81-3.el5 passed

Result: Package existence check passed for "make-3.81".

Check: Package existence for "binutils-2.17.50.0.6"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 binutils-2.17.50.0.6-14.el5 passed

krac1 binutils-2.17.50.0.6-14.el5 passed

Result: Package existence check passed for "binutils-2.17.50.0.6".

Check: Package existence for "gcc-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 gcc-4.1.2-48.el5 passed

krac1 gcc-4.1.2-48.el5 passed

Result: Package existence check passed for "gcc-4.1.1".

Check: Package existence for "libaio-0.3.106"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libaio-0.3.106-5 passed

krac1 libaio-0.3.106-5 passed

Result: Package existence check passed for "libaio-0.3.106".

Check: Package existence for "libaio-devel-0.3.106"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libaio-devel-0.3.106-5 passed

krac1 libaio-devel-0.3.106-5 passed

Result: Package existence check passed for "libaio-devel-0.3.106".

Check: Package existence for "libstdc++-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libstdc++-4.1.2-48.el5 passed

krac1 libstdc++-4.1.2-48.el5 passed

Result: Package existence check passed for "libstdc++-4.1.1".

Check: Package existence for "elfutils-libelf-devel-0.125"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 elfutils-libelf-devel-0.137-3.el5 passed

krac1 elfutils-libelf-devel-0.137-3.el5 passed

Result: Package existence check passed for "elfutils-libelf-devel-0.125".

Check: Package existence for "sysstat-7.0.0"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 sysstat-7.0.2-3.el5 passed

krac1 sysstat-7.0.2-3.el5 passed

Result: Package existence check passed for "sysstat-7.0.0".

Check: Package existence for "compat-libstdc++-33-3.2.3"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 compat-libstdc++-33-3.2.3-61 passed

krac1 compat-libstdc++-33-3.2.3-61 passed

Result: Package existence check passed for "compat-libstdc++-33-3.2.3".

Check: Package existence for "libgcc-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libgcc-4.1.2-48.el5 passed

krac1 libgcc-4.1.2-48.el5 passed

Result: Package existence check passed for "libgcc-4.1.1".

Check: Package existence for "libstdc++-devel-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libstdc++-devel-4.1.2-48.el5 passed

krac1 libstdc++-devel-4.1.2-48.el5 passed

Result: Package existence check passed for "libstdc++-devel-4.1.1".

Check: Package existence for "unixODBC-2.2.11"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 unixODBC-2.2.11-7.1 passed

krac1 unixODBC-2.2.11-7.1 passed

Result: Package existence check passed for "unixODBC-2.2.11".

Check: Package existence for "unixODBC-devel-2.2.11"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 unixODBC-devel-2.2.11-7.1 passed

krac1 unixODBC-devel-2.2.11-7.1 passed

Result: Package existence check passed for "unixODBC-devel-2.2.11".

Check: Package existence for "glibc-2.5-12"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 glibc-2.5-49 passed

krac1 glibc-2.5-49 passed

Result: Package existence check passed for "glibc-2.5-12".

Check: Kernel parameter for "semmsl"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 250 250 passed

krac1 250 250 passed

Result: Kernel parameter check passed for "semmsl".

Check: Kernel parameter for "semmns"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 32000 32000 passed

krac1 32000 32000 passed

Result: Kernel parameter check passed for "semmns".

Check: Kernel parameter for "semopm"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 100 100 passed

krac1 100 100 passed

Result: Kernel parameter check passed for "semopm".

Check: Kernel parameter for "semmni"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 128 128 passed

krac1 128 128 passed

Result: Kernel parameter check passed for "semmni".

Check: Kernel parameter for "shmmax"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 2147483648 536870912 passed

krac1 2147483648 536870912 passed

Result: Kernel parameter check passed for "shmmax".

Check: Kernel parameter for "shmall"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 2097152 2097152 passed

krac1 2097152 2097152 passed

Result: Kernel parameter check passed for "shmall".

Check: Kernel parameter for "shmmni"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 4096 4096 passed

krac1 4096 4096 passed

Result: Kernel parameter check passed for "shmmni".

Check: Kernel parameter for "file-max"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 65536 65536 passed

krac1 65536 65536 passed

Result: Kernel parameter check passed for "file-max".

Check: Kernel parameter for "rmem_default"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 4194304 4194304 passed

krac1 4194304 4194304 passed

Result: Kernel parameter check passed for "rmem_default".

Check: Kernel parameter for "rmem_max"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 4194304 4194304 passed

krac1 4194304 4194304 passed

Result: Kernel parameter check passed for "rmem_max".

Check: Kernel parameter for "wmem_default"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 262144 262144 passed

krac1 262144 262144 passed

Result: Kernel parameter check passed for "wmem_default".

Check: Kernel parameter for "wmem_max"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 1048576 262144 passed

krac1 1048576 262144 passed

Result: Kernel parameter check passed for "wmem_max".

Check: Group existence for "dba"

Node Name Status Comment

------------ ------------------------ ------------------------

krac2 exists passed

krac1 exists passed

Result: Group existence check passed for "dba".

Check: User existence for "nobody"

Node Name Status Comment

------------ ------------------------ ------------------------

krac2 exists passed

krac1 exists passed

Result: User existence check passed for "nobody".

System requirement passed for 'database'

Checking CRS integrity...

Checking daemon liveness...

Check: Liveness for "CRS daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "CRS daemon".

Checking daemon liveness...

Check: Liveness for "CSS daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "CSS daemon".

Checking daemon liveness...

Check: Liveness for "EVM daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons

Node Name CRS daemon CSS daemon EVM daemon

------------ ------------------------ ------------------------ ----------

krac2 yes yes yes

krac1 yes yes yes

Checking CRS health...

Check: Health of CRS

Node Name CRS OK?

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: CRS health check passed.

CRS integrity check passed.

Checking node application existence...

Checking existence of VIP node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 yes exists passed

krac1 yes exists passed

Result: Check passed.

Checking existence of ONS node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 no exists passed

krac1 no exists passed

Result: Check passed.

Checking existence of GSD node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 no exists passed

krac1 no exists passed

Result: Check passed.

Pre-check for database installation was successful.

[oracle@krac1 clusterware]$

Oracle ASM Home Software Install.

su - oracle

./runInstaller

Action : Select the Enterprise Edition radio button and click "Next"

Action : Confirm entries are OK and Click "Next"

Actions : Click "Select All" and Click "Next"

Action : None Required click "Next"

Action : Select Install Software Only ration button and click "Next"

Action : Review summary and click "Install".

Action : None Required.

[root@krac1 ~]# /u02/app/oracle/product/11.1.0/asm_1/root.sh

Running Oracle 11g root.sh script...

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u02/app/oracle/product/11.1.0/asm_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to /usr/local/bin ...

Copying oraenv to /usr/local/bin ...

Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root.sh script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

[root@krac1 ~]# ssh krac2 /u02/app/oracle/product/11.1.0/asm_1/root.sh

root@krac2's password:

Running Oracle 11g root.sh script...

The following environment variables are set as:

ORACLE_OWNER= oracle

ORACLE_HOME= /u02/app/oracle/product/11.1.0/asm_1

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to /usr/local/bin ...

Copying oraenv to /usr/local/bin ...

Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root.sh script.

Now product-specific root actions will be performed.

Finished product-specific root actions.

[root@krac1 ~]#

Action : Click "Exit"

Oracle RAC Database Listener Creation

cd $ORA_ASM_HOME/bin/

./netca

Action : Select "Cluster Configuration" ration button and click "Next"

Action : Click Select all nodes and "click Next"

Action : Select Listener Configuration Ratio button and click "Next"

Action : Select Add Ratio button and click "Next"

Action : Click "Next"

Action : Ensure the Selected Protocols list includes TCP and click "Next"

Action : Ensure the Use the Standard port number of 1521 radio button is set and click "Next"

Action : Select No ratio button and click "Next"

Action : Click "Next"

Oracle ASM Instance and diskgroup Creation

cd $ORA_ASM_HOME/bin/

./dbca

Action : Select Configure Automatic Storage Management ration button and click "Next"

Action : Click Select All and click "Next"

Action : Enter the SYS Password and click "Next"

Action: Click "OK"

Action : None required.

Action: Click Create New

Action : Ensure the Disk Group Name, Redundancy, select disks and click "OK"

Action : None required.

Repeat the same steps for FRA Disk group creation.

Action: Click "Finish"

Oracle RAC Database Home Software Install

[oracle@krac1 clusterware]$ ./runcluvfy.sh stage -pre dbinst -n krac1,krac2 -verbose

Performing pre-checks for database installation

Checking node reachability...

Check: Node reachability from node "krac1"

Destination Node Reachable?

------------------------------------ ------------------------

krac1 yes

krac2 yes

Result: Node reachability check passed from node "krac1".

Checking user equivalence...

Check: User equivalence for user "oracle"

Node Name Comment

------------------------------------ ------------------------

krac2 passed

krac1 passed

Result: User equivalence check passed for user "oracle".

Checking administrative privileges...

Check: Existence of user "oracle"

Node Name User Exists Comment

------------ ------------------------ ------------------------

krac2 yes passed

krac1 yes passed

Result: User existence check passed for "oracle".

Check: Existence of group "oinstall"

Node Name Status Group ID

------------ ------------------------ ------------------------

krac2 exists 501

krac1 exists 501

Result: Group existence check passed for "oinstall".

Check: Membership of user "oracle" in group "oinstall" [as Primary]

Node Name User Exists Group Exists User in Group Primary Comment

---------------- ------------ ------------ ------------ ------------ ------------

krac2 yes yes yes yes passed

krac1 yes yes yes yes passed

Result: Membership check for user "oracle" in group "oinstall" [as Primary] passed.

Check: Existence of group "dba"

Node Name Status Group ID

------------ ------------------------ ------------------------

krac2 exists 502

krac1 exists 502

Result: Group existence check passed for "dba".

Check: Membership of user "oracle" in group "dba"

Node Name User Exists Group Exists User in Group Primary Comment

---------------- ------------ ------------ ------------ ------------ ------------

krac2 yes yes yes no passed

krac1 yes yes yes no passed

Result: Membership check for user "oracle" in group "dba" passed.

Administrative privileges check passed.

Checking node connectivity...

Interface information for node "krac2"

Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address

---------------- ------------ ------------ ------------ ------------ ------------

eth0 152.168.1.51 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:F7

eth0 152.168.1.61 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:F7

eth1 192.168.1.51 192.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:F7:59:01

Interface information for node "krac1"

Interface Name IP Address Subnet Subnet Gateway Default Gateway Hardware Address

---------------- ------------ ------------ ------------ ------------ ------------

eth0 152.168.1.50 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:43

eth0 152.168.1.60 152.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:43

eth1 192.168.1.50 192.168.1.0 0.0.0.0 192.168.1.1 00:0C:29:71:EE:4D

Check: Node connectivity of subnet "152.168.1.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

krac2:eth0 krac2:eth0 yes

krac2:eth0 krac1:eth0 yes

krac1:eth0 krac1:eth0 yes

Result: Node connectivity check passed for subnet "152.168.1.0" with node(s) krac2,krac1.

Check: Node connectivity of subnet "192.168.1.0"

Source Destination Connected?

------------------------------ ------------------------------ ----------------

krac2:eth1 krac1:eth1 yes

Result: Node connectivity check passed for subnet "192.168.1.0" with node(s) krac2,krac1.

Interfaces found on subnet "152.168.1.0" that are likely candidates for VIP:

krac2 eth0:152.168.1.51 eth0:152.168.1.61

krac1 eth0:152.168.1.50 eth0:152.168.1.60

Interfaces found on subnet "192.168.1.0" that are likely candidates for VIP:

krac2 eth1:192.168.1.51

krac1 eth1:192.168.1.50

WARNING:

Could not find a suitable set of interfaces for the private interconnect.

Result: Node connectivity check passed.

Checking system requirements for 'database'...

Check: Total memory

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 1.98GB (2075468KB) 1GB (1048576KB) passed

krac1 1.98GB (2075468KB) 1GB (1048576KB) passed

Result: Total memory check passed.

Check: Free disk space in "/tmp" dir

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 6.47GB (6787332KB) 400MB (409600KB) passed

krac1 6.51GB (6827884KB) 400MB (409600KB) passed

Result: Free disk space check passed.

Check: Swap space

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 3.42GB (3582484KB) 1.5GB (1572864KB) passed

krac1 3.42GB (3582484KB) 1.5GB (1572864KB) passed

Result: Swap space check passed.

Check: System architecture

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 i686 i686 passed

krac1 i686 i686 passed

Result: System architecture check passed.

Check: Kernel version

Node Name Available Required Comment

------------ ------------------------ ------------------------ ----------

krac2 2.6.18-194.el5 2.6.18 passed

krac1 2.6.18-194.el5 2.6.18 passed

Result: Kernel version check passed.

Check: Package existence for "make-3.81"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 make-3.81-3.el5 passed

krac1 make-3.81-3.el5 passed

Result: Package existence check passed for "make-3.81".

Check: Package existence for "binutils-2.17.50.0.6"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 binutils-2.17.50.0.6-14.el5 passed

krac1 binutils-2.17.50.0.6-14.el5 passed

Result: Package existence check passed for "binutils-2.17.50.0.6".

Check: Package existence for "gcc-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 gcc-4.1.2-48.el5 passed

krac1 gcc-4.1.2-48.el5 passed

Result: Package existence check passed for "gcc-4.1.1".

Check: Package existence for "libaio-0.3.106"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libaio-0.3.106-5 passed

krac1 libaio-0.3.106-5 passed

Result: Package existence check passed for "libaio-0.3.106".

Check: Package existence for "libaio-devel-0.3.106"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libaio-devel-0.3.106-5 passed

krac1 libaio-devel-0.3.106-5 passed

Result: Package existence check passed for "libaio-devel-0.3.106".

Check: Package existence for "libstdc++-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libstdc++-4.1.2-48.el5 passed

krac1 libstdc++-4.1.2-48.el5 passed

Result: Package existence check passed for "libstdc++-4.1.1".

Check: Package existence for "elfutils-libelf-devel-0.125"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 elfutils-libelf-devel-0.137-3.el5 passed

krac1 elfutils-libelf-devel-0.137-3.el5 passed

Result: Package existence check passed for "elfutils-libelf-devel-0.125".

Check: Package existence for "sysstat-7.0.0"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 sysstat-7.0.2-3.el5 passed

krac1 sysstat-7.0.2-3.el5 passed

Result: Package existence check passed for "sysstat-7.0.0".

Check: Package existence for "compat-libstdc++-33-3.2.3"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 compat-libstdc++-33-3.2.3-61 passed

krac1 compat-libstdc++-33-3.2.3-61 passed

Result: Package existence check passed for "compat-libstdc++-33-3.2.3".

Check: Package existence for "libgcc-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libgcc-4.1.2-48.el5 passed

krac1 libgcc-4.1.2-48.el5 passed

Result: Package existence check passed for "libgcc-4.1.1".

Check: Package existence for "libstdc++-devel-4.1.1"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 libstdc++-devel-4.1.2-48.el5 passed

krac1 libstdc++-devel-4.1.2-48.el5 passed

Result: Package existence check passed for "libstdc++-devel-4.1.1".

Check: Package existence for "unixODBC-2.2.11"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 unixODBC-2.2.11-7.1 passed

krac1 unixODBC-2.2.11-7.1 passed

Result: Package existence check passed for "unixODBC-2.2.11".

Check: Package existence for "unixODBC-devel-2.2.11"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 unixODBC-devel-2.2.11-7.1 passed

krac1 unixODBC-devel-2.2.11-7.1 passed

Result: Package existence check passed for "unixODBC-devel-2.2.11".

Check: Package existence for "glibc-2.5-12"

Node Name Status Comment

------------------------------ ------------------------------ ----------------

krac2 glibc-2.5-49 passed

krac1 glibc-2.5-49 passed

Result: Package existence check passed for "glibc-2.5-12".

Check: Kernel parameter for "semmsl"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 250 250 passed

krac1 250 250 passed

Result: Kernel parameter check passed for "semmsl".

Check: Kernel parameter for "semmns"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 32000 32000 passed

krac1 32000 32000 passed

Result: Kernel parameter check passed for "semmns".

Check: Kernel parameter for "semopm"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 100 100 passed

krac1 100 100 passed

Result: Kernel parameter check passed for "semopm".

Check: Kernel parameter for "semmni"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 128 128 passed

krac1 128 128 passed

Result: Kernel parameter check passed for "semmni".

Check: Kernel parameter for "shmmax"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 2147483648 536870912 passed

krac1 2147483648 536870912 passed

Result: Kernel parameter check passed for "shmmax".

Check: Kernel parameter for "shmall"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 2097152 2097152 passed

krac1 2097152 2097152 passed

Result: Kernel parameter check passed for "shmall".

Check: Kernel parameter for "shmmni"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 4096 4096 passed

krac1 4096 4096 passed

Result: Kernel parameter check passed for "shmmni".

Check: Kernel parameter for "file-max"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 65536 65536 passed

krac1 65536 65536 passed

Result: Kernel parameter check passed for "file-max".

Check: Kernel parameter for "rmem_default"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 4194304 4194304 passed

krac1 4194304 4194304 passed

Result: Kernel parameter check passed for "rmem_default".

Check: Kernel parameter for "rmem_max"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 4194304 4194304 passed

krac1 4194304 4194304 passed

Result: Kernel parameter check passed for "rmem_max".

Check: Kernel parameter for "wmem_default"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 262144 262144 passed

krac1 262144 262144 passed

Result: Kernel parameter check passed for "wmem_default".

Check: Kernel parameter for "wmem_max"

Node Name Configured Required Comment

------------ ------------------------ ------------------------ ----------

krac2 1048576 262144 passed

krac1 1048576 262144 passed

Result: Kernel parameter check passed for "wmem_max".

Check: Group existence for "dba"

Node Name Status Comment

------------ ------------------------ ------------------------

krac2 exists passed

krac1 exists passed

Result: Group existence check passed for "dba".

Check: User existence for "nobody"

Node Name Status Comment

------------ ------------------------ ------------------------

krac2 exists passed

krac1 exists passed

Result: User existence check passed for "nobody".

System requirement passed for 'database'

Checking CRS integrity...

Checking daemon liveness...

Check: Liveness for "CRS daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "CRS daemon".

Checking daemon liveness...

Check: Liveness for "CSS daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "CSS daemon".

Checking daemon liveness...

Check: Liveness for "EVM daemon"

Node Name Running

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: Liveness check passed for "EVM daemon".

Liveness of all the daemons

Node Name CRS daemon CSS daemon EVM daemon

------------ ------------------------ ------------------------ ----------

krac2 yes yes yes

krac1 yes yes yes

Checking CRS health...

Check: Health of CRS

Node Name CRS OK?

------------------------------------ ------------------------

krac2 yes

krac1 yes

Result: CRS health check passed.

CRS integrity check passed.

Checking node application existence...

Checking existence of VIP node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 yes exists passed

krac1 yes exists passed

Result: Check passed.

Checking existence of ONS node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 no exists passed

krac1 no exists passed

Result: Check passed.

Checking existence of GSD node application

Node Name Required Status Comment

------------ ------------------------ ------------------------ ----------

krac2 no exists passed

krac1 no exists passed

Result: Check passed.

Pre-check for database installation was successful.

[oracle@krac1 clusterware]$

su - oracle

./runInstaller

Action: Select Enterprise Edition ration button and click "Next"

Action : Ensure Oracle Base, Name, Path and click "Next"

Action : Nothing required click "Next".

Action : Select All and click "Next"

Action: Select Install Software Only ration button.

Action : Click "Install".

Action : Run root.sh script as a root and click "OK"

Action : Click "Exit"

Use dbca to create the RAC database

su - oracle

cd $ORACLE_HOME/bin

./dbca

Action: Select Real Application Clusters Databases ration button and click "Next".

Action : Select Create a Database ration button and click "Next".

Action: Select All and click "Next"

Action : Select General Purpose or Transaction Processing and click "Next"

Action : Enter the Database name and click "Next"

Action: Select Enterprise Manager Configuration and click "Next"

Action : Enter Administrative Password and click "Next"

Action: Select Automatic Memory Management(ASM) and click "OK"

Action: Select ASM Disk Group and click "Next"

Action : Ensure Disk group name for Database files and click "Next"

Action: Ensure Flash Recovery Area Disk Group and Click "Next"

Action : Ensure Sample Schema and Click "Next"

Action : Nothing Required.

Action : Ensure Recommended one click "Next"

Action : Select Automatic Maintenance Enable Task ration button and click "Next"

Action : Nothing Required.

Action : Click "Finish".

Action : Click "OK"
Once the database creation is complete you are presented with the following screen.

Action : "Exit"

[oracle@krac1 ~]$ crs_stat -t -v

Name Type R/RA F/FT Target State Host

----------------------------------------------------------------------

ora....L1.inst application 0/5 0/0 ONLINE ONLINE krac1

ora....L2.inst application 0/5 0/0 ONLINE ONLINE krac2

ora.ORCL.db application 0/0 0/1 ONLINE ONLINE krac1

ora....SM1.asm application 0/5 0/0 ONLINE ONLINE krac1

ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE krac1

ora.krac1.gsd application 0/5 0/0 ONLINE ONLINE krac1

ora.krac1.ons application 0/3 0/0 ONLINE ONLINE krac1

ora.krac1.vip application 0/0 0/0 ONLINE ONLINE krac1

ora....SM2.asm application 0/5 0/0 ONLINE ONLINE krac2

ora....C2.lsnr application 0/5 0/0 ONLINE ONLINE krac2

ora.krac2.gsd application 0/5 0/0 ONLINE ONLINE krac2

ora.krac2.ons application 0/3 0/0 ONLINE ONLINE krac2

ora.krac2.vip application 0/0 0/0 ONLINE ONLINE krac2

[oracle@krac1 ~]$ sqlplus

SQL*Plus: Release 11.1.0.6.0 - Production on Wed Sep 26 00:43:35 2012

Enter user-name: sys@orcl1 as sysdba

Enter password:

Connected to:

Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production

With the Partitioning, Real Application Clusters, OLAP, Data Mining

and Real Application Testing options

SQL> col instance_name format a15

SQL> col host_name format a25

SQL> select instance_name,host_name from gv$instance;

INSTANCE_NAME HOST_NAME

--------------- -------------------------

ORCL1 krac1.dbprod.com

ORCL2 krac2.dbprod.com

SQL> select instance_name,host_name from v$instance;

INSTANCE_NAME HOST_NAME

--------------- -------------------------

ORCL1 krac1.dbprod.com

SQL> connect sys@orcl2 as sysdba

Enter password:

Connected.

SQL> select instance_name,host_name from gv$instance;

INSTANCE_NAME HOST_NAME

--------------- -------------------------

ORCL2 krac2.dbprod.com

ORCL1 krac1.dbprod.com

SQL> select instance_name,host_name from v$instance;

INSTANCE_NAME HOST_NAME

--------------- -------------------------

ORCL2 krac2.dbprod.com

SQL> connect sys@orcl as sysdba

Enter password:

Connected.

SQL> select instance_name,host_name from v$instance;

INSTANCE_NAME HOST_NAME

--------------- -------------------------

ORCL1 krac1.dbprod.com

SQL> connect sys@orcl as sysdba

Enter password:

Connected.

SQL> select instance_name,host_name from v$instance;

INSTANCE_NAME HOST_NAME

--------------- -------------------------

ORCL2 krac2.dbprod.com

SQL>

Sample LISTENER.ORA and TNSNAMES.ORA file.

krac1

# listener.ora.krac1 Network Configuration File: /u02/app/oracle/product/11.1.0/db_1/network/admin/listener.ora.krac1

# Generated by Oracle configuration tools.

LISTENER_KRAC1 =

(DESCRIPTION_LIST =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = krac1-vip)(PORT = 1521)(IP = FIRST))

(ADDRESS = (PROTOCOL = TCP)(HOST = 152.168.1.50)(PORT = 1521)(IP = FIRST))

)

krac2

# listener.ora.krac2 Network Configuration File: /u02/app/oracle/product/11.1.0/db_1/network/admin/listener.ora.krac2

# Generated by Oracle configuration tools.

LISTENER_KRAC2 =

(DESCRIPTION_LIST =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = krac2-vip)(PORT = 1521)(IP = FIRST))

(ADDRESS = (PROTOCOL = TCP)(HOST = 152.168.1.51)(PORT = 1521)(IP = FIRST))

)

TNSNAMES.ORA.

# tnsnames.ora Network Configuration File: /u02/app/oracle/product/11.1.0/db_1/network/admin/tnsnames.ora

# Generated by Oracle configuration tools.

LISTENERS_ORCL =

(ADDRESS_LIST =

(ADDRESS = (PROTOCOL = TCP)(HOST = krac1-vip)(PORT = 1521))

(ADDRESS = (PROTOCOL = TCP)(HOST = krac2-vip)(PORT = 1521))

)

ORCL2 =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = krac2-vip)(PORT = 1521))

(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = ORCL.DBPROD.COM)

(INSTANCE_NAME = ORCL2)

)

ORCL1 =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = krac1-vip)(PORT = 1521))

(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = ORCL.DBPROD.COM)

(INSTANCE_NAME = ORCL1)

)

ORCL =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = krac1-vip)(PORT = 1521))

(ADDRESS = (PROTOCOL = TCP)(HOST = krac2-vip)(PORT = 1521))

(LOAD_BALANCE = yes)

(CONNECT_DATA =

(SERVER = DEDICATED)

(SERVICE_NAME = ORCL.DBPROD.COM)

)

---------------------------------------------------------------------------------------------------------------------------

Hope! This helps...

Regards,

Kavin.

BE THE BEST!!! BE WITH THE BEST !!!.

Oracle Real Application Cluster(RAC)

Pages

Tuesday, 25 September 2012

Oracle 11g Release 1 RAC Installation Steps On Linux

About Me