Showing posts with label RAC & ASM. Show all posts
Showing posts with label RAC & ASM. Show all posts

Wednesday, August 14, 2013

Different types of Failover mechanism in Oracle

FAILOVER: 
In the context of Oracle Net, failover refers to the mechanism of switching over to an alternate resource when connection to the primary resource gets terminated due to any reason. Connection failure may be broadly categorized as:
Those that occur while making the initial connection. (Connect Time Failover)
Those that occur after a connection has been successfully established. (TAF)

The first category of connection failure can be dealt with comparatively easily. If your attempt to connect to an instance fails, you can make the connection attempt again, but this time to a backup instance. As long as you have backup instances configured, you can continue trying to connect, until a successful connection is established. This process is technically termed as Connect Time Failover.

Connect Time Failover.
The connect time failover feature allows client to connet to another listener if the initial connection to the first listener fails.Multiple listener locations are specified in the client tnsnames.ora file. If a connection attempt to the first listener fails , a connection request to the next listener in the list is attempted .You can acheive this feature by adding Multiple listener addresses within an address list of tnsnames.Here is an example of what a tnsnames.ora file looks like with a connect-time failover enabled.

TNS Parameter for Connect time failover is FAILOVER. Default value of this is ON.
(failover=on) is default for ADDRESS_LISTs and DESCRIPTION_LISTs , so we do not need to explicitly specify this parameter to put it on.

Sample for TNS net service alias for Client Side Connect time Failover :-

myrac_failover =  
(DESCRIPTION =  
 (ADDRESS_LIST =  
 (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1522))
  (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1522))  
 )  
 (CONNECT_DATA =  
 (SERVICE_NAME = myrac)  
 )  
 )

Notice the additional entry under the ADDRESS_LIST section. Here two listener are specified .If a connection is unsuccessful  when attempting to connect to the rac1-vip host on port 1521, a connection attempt is made to the rac2-vip host on port 1521 . 

Note:- Entries in the ADDRESS_LIST do not have to be a RAC nodes . The example that i explained here is for two node RAC database. You can also configure connect-time failover with a standby database , so that one of the entries in the list may be a standby database .As long as the defined service(Database) in the ADDRESS_LIST is availabe on the other 
node , client is able to connect to that node.
For example,

failover_norac=
(DESCRIPTION=
(ADDRESS_LIST=
(ADDRESS= (PROTOCOL=TCP) (HOST=prod_db) (PORT=1521))
(ADDRESS= (PROTOCOL=TCP) (HOST=standby_db) (PORT=1521))
(FAILOVER= TRUE)
)
(CONNECT_DATA=
(SERVICE_NAME= orcl)
)
)
The other criteria, of cource , that the address in the list will allow the client to get to the data that is needed by the application. With oracle RAC , we know that data is always the same, as it is the same database. In the case of physical or logical standby database  
whether or not the data is accessible depends upon how these standby database is configured.

Note:- An important issue to be aware of at this point is that Connect Time Failover only works for you, if you are using dynamic registration. 

Transparent application failover (TAF)
Now, let’s look at how TAF works. Unlike connect time failover, which is invoked before the connection is made, TAF comes into play after the connection is made (and then, broken). If the connection is lost while the application is running, Oracle Net will transparently reconnect the application to another instance accessing the same database.

TAF supports two types of failover: SESSION and SELECT. A SESSION failover connection is not over ambitious. It just fails over to a backup instance. All work in progress at that point are irrevocably lost. SELECT failover is more intricate in as much as it enables some type of read only application to failover without losing the work in progress. If a SELECT statement was in progress at the time of the termination of the connection, then as the connection is reestablished to a backup instance, Oracle Net re-executes the SELECT statement and positions the cursor in a manner that the client can seamlessly continue fetching the rows. But that’s about all that TAF has to offer. It doesn’t have any mechanism to recover DML statements that were in progress, when the failure occurred, or even for SELECT statements, you lose global temporary tables, package states and session settings.
TAF supports two failover methods: BASIC and PRECONNECT. In BASIC method, you connect to the backup instance when the primary connection fails. In the PRECONNECT method, you connect to the backup instance at the same time you connect to the primary instance. This has the obvious benefit of having a backup connection available all of the time, thus reducing the time of ‘failover’. But the downside is that you have to pay the extra ‘cost’ in terms of resources spent, of having a backup connection open all the while.
TAF is configured by adding a FAILOVER_MODE parameter to the CONNECT_DATA parameter for a net service name. Below is an example for configuring TAF with two node RAC

Note:-
Specifies a different net service name to be used to establish the backup connection. A backup should be specified when using PRECONNECT to pre-establish connections. Specifying a BACKUP is strongly recommended for BASIC methods; otherwise, reconnection might first attempt the instance that has just failed, adding additional delay until the client reconnects. 
http://oracleinquisitive.blogspot.in/2012/09/transparent-application-failover-taf.html

Tuesday, August 13, 2013

How load balancing works in oracle ?

Load Balancing
Oracle RAC systems provide two types of load balancing for automatic workload management:
* Server load balancing distributes processing workload among Oracle RAC nodes.
* Client load balancing distributes new connections among Oracle RAC nodes so that no one server is overwhelmed with connection requests. For example, when a connection fails over to another node because of hardware failure, client load balancing ensures that the redirected connection requests are distributed among the other nodes in the RAC.
The primary difference between these two methods is that the former method distributes processing and the latter method distributes connection attempts.Lets look in details

Client side load balancing 
Client side load balancing is determined by configuring the tnsnames.ora with multiple address in the description ,with parameter LOAD_BALANCE=yes (can specify 'ON' also) , as in this example .
myrac =
  (DESCRIPTION =
   (LOAD_BALANCE=ON)
   (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1521))
   (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1521))
    (CONNECT_DATA =
     (SERVICE_NAME = myrac)
    )
  )

Client load balancing helps distribute new connections in your environment so that no one server is overwhelmed with connection requests. When client load balancing is enabled, connection attempts are made randomly among RAC nodes. Suppose you have the Oracle RAC environment  with multiple Oracle RAC nodes, A, B, C, and D. Without client load balancing enabled, connection attempts may be front-loaded, meaning that most connection attempts would try Node A first, then Node B, and so on until a connection attempt is successful.This creates a situation where Node A and Node B can become overloaded with connection requests. Thus in client side load balancing the client will randomly pick an address to try ,without regards to the availability of the node or to how heavily loaded the node may be.

Server Load Balancing
Server side load balancing is controlled by two parameters LOCAL_LISTENER and REMOTE_LISTENER.By default LOCAL_LISTENER is set to blank . Setting local_listener parameter is mandatory when using a different port rather than default .LOCAL_LISTENER on each node should point to the listener on that node.Oracle database have the ability to automatically register their 
presence with an existing default listener with the help of pmon , or you must specify the init parameter LOCAL_LISTENER for non default listener .

A remote listener is a listener residing on one computer that redirects connections to a database instance on another computer. Remote listeners are typically used in an Oracle Real Application Clusters (Oracle RAC) environment. You can configure registration to remote listeners, such as in the case of Oracle RAC, for dedicated server or shared server environments. The REMOTE_LISTENER parameter in the spfile should be pointing to a TNSNAMES entry on each server mode, which in turn list the IPs and ports of all available listeners in the cluster. DBCA will set these entries up for you if you use it to create your databases (recommended).

The remote_listener parameter is then set to listeners_myrac in the spfile

*.remote_listener='LISTENERS_MYRAC'

In tnsnames.ora you can see entry like this,

LISTENERS_MYRAC =
  (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip )(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip )(PORT = 1521))
  )

There are two types of server-side load balancing: 
Load Based — Server side load balancing redirects connections by default depending on node load. This id default.
Session Based — Session based load balancing takes into account the number of sessions connected to each node and then distributes the connections to balance the number of sessions across the different nodes.

Note:- Use listener_<instance_name> as the alias name for the local listener, and listeners_<servicename> for the remote listener alias.
The way you set the remote listener changed in rac 11.2 , instead of registering with local listener, the database register with SCAN listener.
This approach is similar to above, but the pain of configuring individual nodes/their VIPs in client tnsnames.ora is eliminated. All we need is a Round Robin enabled DNS name resolution and use the SCAN DNS name in the client tnsnames.ora.If you do not use DBCA, you should set the REMOTE_LISTENER database parameter to scan_name:scan_port. 

Server-side load balancing divides the connection load evenly between all available listeners by determining the total number of connections on each listener, and then distributing new user session connection requests to the least loaded listener(s) based on the total number of sessions already connected. While a bit more complex to implement because it requires configuration of multiple listeners, it most definitely helps to even out connections across all available listeners in a database system.

To implement server-side load balancing, at least two listeners must be configured. Also, the REMOTE_LISTENERS initialization parameter must be added to the database’s PFILE or SPFILE so that the database knows to search out the value provided in that parameter in the database server’s TNSNAMES.ORA configuration file. When server-side load balancing is activated, each listener that contributes a listening endpoint communicates with the other listener(s) via each database instance’s PMON process. Oracle then determines how many user connections each listener is servicing, and it will distribute any new connection requests so that the load is balanced evenly across all servers. The entries in TNSNAMES.ORA direct the listeners to share information about the relative load connectivity.
   

Friday, August 17, 2012

11gR2 RAC Step By Step Configuration On VMWare part5

STAGE 5
Oracle rdbms instance configuration  

Go to Node1 and login as root


[root@racha1 ~]# xhost +

access control disabled, clients can connect from any host
[root@racha1 ~]# su - oracle
[oracle@racha1 ~]$xclock ---> it should return clock display
[oracle@racha1 ~]$ cd /install/disk1
[oracle@racha1 disk1]$ ls  
doc install response rpm runInstaller sshsetup stage welcome.html
[oracle@racha1 disk1]$./runInstaller
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 80 MB. Actual 9852 MB Passed
Checking swap space: must be greater than 150 MB. Actual 3887 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-07-25_08-46-27PM. Please wait ...

Uncheck the security updates checkbox and click the "Next" button.


  



 Accept the "Create and configure a database" option by clicking the "Next" button.





Accept the "Server Class" option by clicking the "Next" button.



  

Make sure both nodes are selected, then click the "Next" button.



   

 Accept the "Typical install" option by clicking the "Next" button.




Enter "/u01/app/oracle/product/11.2.0/dbhome_1" for the software location. The storage type should be set to "Automatic Storage Manager". Enter the appropriate passwords and database name, in this case "RAC.localdomain".


  



Wait for the prerequisite check to complete. If there are any problems either fix them, or check the "Ignore All" checkbox and click the "Next" button.



  

If you are happy with the summary information, click the "Finish" button.



  







Once the software installation is complete the Database Configuration Assistant (DBCA) will start automatically.



  

Once the Database Configuration Assistant (DBCA) has finished, click the "OK" button.



  

When prompted, run the configuration scripts on each node. When the scripts have been run on each node, click the "OK" button.



  


Node 1:
[root@racha1 ~]# /u01/app/oracle/product/11.2.0/dbhome_1/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/11.2.0/dbhome_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n) 
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
[root@racha1 ~]#

Node 2: 
[root@racha2 ~]# /u01/app/oracle/product/11.2.0/dbhome_1/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/oracle/product/11.2.0/dbhome_1
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The file "dbhome" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "oraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
The file "coraenv" already exists in /usr/local/bin. Overwrite it? (y/n)
[n]:
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
Finished product-specific root actions.
[root@racha2 ~]#

Two Node RAC Installation Successfully completed. Click the "Close" button to exit the installer.


Check the Status of the RAC
There are several ways to check the status of the RAC. The srvctl utility shows the current configuration and status of the RAC database.For that you have to set the environment for  GRID infrastructure. I made a grid.env in my oracle home and it contains

[oracle@racha1 ~]$ cat grid.env
export GRID_HOME=/u01/app/11.2.0/grid
export PATH=$ORACLE_HOME/bin:$PATH
export ORACLE_SID=+ASM1
[oracle@racha1 ~]$. grid.env
$ srvctl config database -d RAC
Database unique name: RAC
Database name: RAC
Oracle home: /u01/app/oracle/product/11.2.0/dbhome_1
Oracle user: oracle
Spfile: +DATA/RAC/spfileRAC.ora
Domain: localdomain
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: RAC
Database instances: RAC1,RAC2
Disk Groups: DATA
Services: 
Database is administrator managed
$

$ srvctl status database -d RAC
Instance RAC1 is running on node racha1
Instance RAC2 is running on node racha2
$
The V$ACTIVE_INSTANCES view can also display the current status of the instances. 

Just Check
Node 1:
[oracle@racha1 ~]$. .bash_profile (sourcing the rdbms home environment) 
[oracle@racha1 ~]$ export ORACLE_SID=RAC1
[oracle@racha1 ~]$ sqlplus
SQL*Plus: Release 11.2.0.1.0 Production on Tue Jul 26 13:00:31 2011
Copyright (c) 1982, 2009, Oracle. All rights reserved.
Enter user-name: / as sysdba
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SQL> select name from v$database;
NAME
---------
RAC

SQL> select instance_name from v$instance;
INSTANCE_NAME
----------------
RAC1

SQL> SELECT inst_name FROM v$active_instances;

INST_NAME
-------------------------------------------------------------------
racha1.localdomain:RAC1
racha2.localdomain:RAC2


    
                                         I hope this may help you , enjoy ..!!!
    

Thursday, August 16, 2012

11gR2 RAC Step By Step Configuration On VMWare part4

STAGE 4 
GRID infrastructure configuration  

Before going ahead we have to copy oracle 11g r2 grid and database software , here i copied my softwares through my pen drive.But you may face problem as pen drive formatted with NTFS file system will not recognize in OEL . You can overcome this issue by installing  "fuse-ntfs-3g-2010.5.22-1.el5.rf.x86_64.rpm" , and is available in internet.
I dumped all oracle related software  in /install directory and change its permission to oracle:oinstall

[root@racha1 ~]#mkdir /install
[root@racha1 ~]# chown -R oracle:oinstall /install
[root@racha1 ~]# cd /install
[root@racha1 install]# ls

-r-xr-xr-x 1 root   root     1239269270 Jul 25 01:14 linux.x64_11gR2_database_1of2.zip
-r-xr-xr-x 1 root   root     1111416131 Jul 24 19:04 linux.x64_11gR2_database_2of2.zip
-r-xr-xr-x 1 root   root     1052897657 Jul 25 16:10 oracle11gr2gridforLINUX.zip

[root@racha1 install]# unzip  oracle11gr2gridforLINUX.zip
By unzipping oracle11gr2gridforLINUX.zip you will get one grid directory

[root@racha1 install]# unzip linux.x64_11gR2_database_1of2.zip
By unzipping  linux.x64_11gR2_database_1of2.zip you will get one database direcotory ,         rename it as disk1 
[root@racha1 install]#mv database disk1
[root@racha1 install]# unzip linux.x64_11gR2_database_2of2.zip
By unzipping  linux.x64_11gR2_database_1of2.zip you will get one database direcotory 

Now you have to copy the contents of '/install/database/stage/Components'  to '/install/disk1/stage/Components' directory . If you omit this stage you will face problem during database installation.

 [root@racha1 install]# mv /install/database/stage/Components/* /install/disk1/stage/Components/

SSH Configuration :

Manual SSH configuration not need for Oracle 11gr2, because of During grid infrastructure It will ask SSH setup. But i  suggest to set up ssh manually as we can run runcluvfy.sh before going ahead with actual grid installation

Node 1 :
[root@racha1 ~]# su - oracle
[oracle@racha1 ~]$ mkdir ~/.ssh
[oracle@racha1 ~]$ chmod 700 ~/.ssh
[oracle@racha1 ~]$ /usr/bin/ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
cf:36:68:1b:be:18:51:7d:7a:67:ee:b7:ee:8e:ca:0b oracle@racha1.localdomain

Node 2:
[root@racha2 ~]# su - oracle
[oracle@racha2 ~]$ mkdir ~/.ssh
[oracle@racha2 ~]$ chmod 700 ~/.ssh
[oracle@racha2 ~]$ /usr/bin/ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. 
The key fingerprint is:
7b:69:c1:7d:7e:b1:e8:62:72:1d:3d:ee:9b:d6:26:c6 oracle@racha2.localdomain

Node 1:
[oracle@racha1 ~]$ cd ~/.ssh
[oracle@racha1 .ssh]$ cat id_rsa.pub >> authorized_keys
[oracle@racha1 .ssh]$ scp authorized_keys racha2:/home/oracle/.ssh
The authenticity of host 'racha2 (192.168.2.202)' can't be established.
RSA key fingerprint is fd:dd:51:77:0d:46:69:f8:55:fe:2c:53:77:75:90:e0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'racha2,192.168.2.202' (RSA) to the list of known hosts.
oracle@racha2's password: 
authorized_keys 100% 407 0.4KB/s 00:00

Node 2:
[oracle@racha2 ~]$ cd ~/.ssh
[oracle@racha2 .ssh]$ cat id_rsa.pub >> authorized_keys
[oracle@racha2 .ssh]$ scp authorized_keys racha1:/home/oracle/.ssh
The authenticity of host 'racha1 (192.168.2.201)' can't be established.
RSA key fingerprint is fd:dd:51:77:0d:46:69:f8:55:fe:2c:53:77:75:90:e0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'racha1,192.168.2.201' (RSA) to the list of known hosts.
oracle@racha1's password:
authorized_keys 100% 814 0.8KB/s 00:00
Check below for each node 

Test the ssh connectivity 

Node 1:
[oracle@racha1 .ssh]$ ssh racha2 date
[oracle@racha1 .ssh]$ ssh racha2-priv date
[oracle@racha1 .ssh]$ ssh racha1-priv date

Node 2:
[oracle@racha2 .ssh]$ ssh racha2 date
[oracle@racha2 .ssh]$ ssh racha2-priv date
[oracle@racha2 .ssh]$ ssh racha1-priv date

Prior to 11gR2 we would probably use the "runcluvfy.sh" utility in the clusterware root directory to check the prerequisites have been met. If you are intending to configure SSH connectivity using the installer this check should be omitted as it will always fail. If you want to setup SSH connectivity manually, then once it is done you can run the "runcluvfy.sh" with the following command.


[oracle@racha1 grid]$ ./runcluvfy.sh stage -pre crsinst -n racha1,racha2 -verbose
Performing pre-checks for cluster services setup
Checking node reachability...
Check: Node reachability from node "racha1"
Destination Node Reachable?
------------------------------------ ------------------------
racha1 yes
racha2 yes
Result: Node reachability check passed from node "racha1"
Checking user equivalence...
Check: User equivalence for user "oracle"
Node Name Comment
------------------------------------ ------------------------
racha1 passed
racha2 passed
Result: User equivalence check passed for user "oracle"
---- -- -- - -- -- -- - - ---- --- ---
------ -- -- - -- - -- - ---- - - - -
Starting Clock synchronization checks using Network Time Protocol(NTP)...
NTP Configuration file check started...
Network Time Protocol(NTP) configuration file not found on any of the nodes. Oracle Cluster Time Synchronization Service(CTSS) can be used instead of NTP for time synchronization on the cluster nodes
Result: Clock synchronization check using Network Time Protocol(NTP) passed
Pre-check for cluster services setup was successful.

If you get any failures be sure to correct them before proceeding.

Oracle Grid Infrastructure installation

Note:- It's a good idea to take a snapshot of the virtual machines, so you can repeat the following stages if you run into any problems. To do this, shutdown both virtual machines and copy racha1 , racha2 and shareasm folder 

You only need to install oracle grid software  from any of the two node , Oracle OUI automatically install the software on remaining nodes. 

Install Grid Infrastructure
[root@racha1 ~]# chown -R oracle:oinstall /install/grid
[root@racha1 ~]# xhost +
access control disabled, clients can connect from any host
[root@racha1 ~]# su - oracle
[root@racha1 ~]#xclock  ----> make sure you are getting clock display

[oracle@racha1 ~]$ cd /install/grid
[oracle@racha1 grid]$ ls
doc response runcluvfy.sh sshsetup welcome.html
install rpm runInstaller stage
[oracle@racha1 grid]$ ./runInstaller

Starting Oracle Universal Installer...
Checking Temp space: must be greater than 80 MB. Actual 19061 MB Passed
Checking swap space: must be greater than 150 MB. Actual 4031 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-07-25_10-18-49AM. Please wait ...[oracle@racha1 grid]$

Select the "Install and Configure Grid Infrastructure for a Cluster" option, then click the "Next" button. 

   
   

Select the "Typical Installation" option, then click the "Next" button




On the "Specify Cluster Configuration" screen, click the "Add" button



 Enter the details of the second node in the cluster, then click the "OK" button.



  


Test SSH Connectivity, by proving oracle username and password . Here I was already configured SSH. 




Click the "Identify network interfaces..." button and check the public and private networks are specified correctly. Once you are happy with them, click the "OK" button and the "Next" button 

Enter "/u01/app/11.2.0/grid" as the software location and "Automatic Storage Manager" as the cluster registry storage type. Enter the ASM password and click the "Next" button.

  

Note:- In oracle 11gr2 asm instance are managed by sysasm privilege . So you have to login with "sys as sysasm" to manage ASM instance in 11gR2

Select Disks, If its not show, choose change recovery path and add correct disk path here. These disks are should be oracle permission. Keep the redundancy as external

  

Here we are creating  only one disk group , if you have enough storage you can create one  more disk group .
 Accept the default inventory directory by clicking the "Next" button. 




Wait while the prerequisite checks complete. If you have any issues, either fix them or check the "Ignore All" checkbox and click the "Next" button.

  



When prompted, run the configuration scripts on each node.


  

Node 1:
[root@racha1 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@racha1 ~]# 

Node 2:
[root@racha2 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.  
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@racha2 ~]#

Node 1:
[root@racha1 ~]# /u01/app/11.2.0/grid/root.sh
Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2011-07-25 14:43:57: Parsing the host name
2011-07-25 14:43:57: Checking for super user privileges
2011-07-25 14:43:57: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-2672: Attempting to start 'ora.gipcd' on 'racha1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'racha1'
CRS-2676: Start of 'ora.gipcd' on 'racha1' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racha1'
CRS-2676: Start of 'ora.gpnpd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racha1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racha1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racha1'
CRS-2676: Start of 'ora.diskmon' on 'racha1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'racha1'
CRS-2676: Start of 'ora.ctssd' on 'racha1' succeeded
ASM created and started successfully.
DiskGroup DATA created successfully.
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-2672: Attempting to start 'ora.crsd' on 'racha1'
CRS-2676: Start of 'ora.crsd' on 'racha1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk 0cf9af8190014f43bf110fdde04e726a.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 0cf9af8190014f43bf110fdde04e726a (/dev/oracleasm/disks/DISK1) [DATA]
Located 1 voting disk(s).
CRS-2673: Attempting to stop 'ora.crsd' on 'racha1'
CRS-2677: Stop of 'ora.crsd' on 'racha1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'racha1'
CRS-2677: Stop of 'ora.asm' on 'racha1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'racha1'
CRS-2677: Stop of 'ora.ctssd' on 'racha1' succeeded
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'racha1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'racha1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'racha1'
CRS-2677: Stop of 'ora.cssd' on 'racha1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'racha1'
CRS-2677: Stop of 'ora.gpnpd' on 'racha1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'racha1'
CRS-2677: Stop of 'ora.gipcd' on 'racha1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'racha1'
CRS-2677: Stop of 'ora.mdnsd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.mdnsd' on 'racha1'
CRS-2676: Start of 'ora.mdnsd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'racha1'
CRS-2676: Start of 'ora.gipcd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racha1'
CRS-2676: Start of 'ora.gpnpd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racha1'
CRS-2676: Start of 'ora.cssdmonitor' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racha1'
CRS-2672: Attempting to start 'ora.diskmon' on 'racha1'
CRS-2676: Start of 'ora.diskmon' on 'racha1' succeeded
CRS-2676: Start of 'ora.cssd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'racha1'
CRS-2676: Start of 'ora.ctssd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racha1'
CRS-2676: Start of 'ora.asm' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racha1'
CRS-2676: Start of 'ora.crsd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'racha1'
CRS-2676: Start of 'ora.evmd' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racha1'
CRS-2676: Start of 'ora.asm' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'racha1'
CRS-2676: Start of 'ora.DATA.dg' on 'racha1' succeeded
CRS-2672: Attempting to start 'ora.registry.acfs' on 'racha1'
CRS-2676: Start of 'ora.registry.acfs' on 'racha1' succeeded
racha1 2011/07/25 14:57:56 /u01/app/11.2.0/grid/cdata/racha1/backup_20110725_145756.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 4031 MB Passed
 The inventory pointer is located at /etc/oraInst.loc 
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.
[root@racha1 ~]#  

Note:- while running root.sh oracle will silently load VIPCA and it will assign the virtual IP to the node available from /etc/hosts . You  can see your VIP  got configured as eth0:1

Node 2:
[root@racha2 ~]# /u01/app/11.2.0/grid/root.sh
]Running Oracle 11g root.sh script...
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/11.2.0/grid
Enter the full pathname of the local bin directory: [/usr/local/bin]:
Creating ] directory...
Copying dbhome to ] ...
Copying oraenv to ] ...
Copying coraenv to ] ... 
 Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root.sh script.
Now product-specific root actions will be performed.
2011-07-26 09:35:41: Parsing the host name
2011-07-26 09:35:41: Checking for super user privileges
2011-07-26 09:35:41: User has super user privileges
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
LOCAL ADD MODE
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Adding daemon to inittab
CRS-4123: Oracle High Availability Services has been started.
ohasd is starting
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node racha1, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
CRS-2672: Attempting to start 'ora.mdnsd' on 'racha2'
CRS-2676: Start of 'ora.mdnsd' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'racha2'
CRS-2676: Start of 'ora.gipcd' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'racha2'
CRS-2676: Start of 'ora.gpnpd' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'racha2'
CRS-2676: Start of 'ora.cssdmonitor' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'racha2'
CRS-2672: Attempting to start 'ora.diskmon' on 'racha2'
CRS-2676: Start of 'ora.diskmon' on 'racha2' succeeded
CRS-2676: Start of 'ora.cssd' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'racha2'
CRS-2676: Start of 'ora.ctssd' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'racha2'
CRS-2676: Start of 'ora.drivers.acfs' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'racha2'
CRS-2676: Start of 'ora.asm' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'racha2'
CRS-2676: Start of 'ora.crsd' on 'racha2' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'racha2'
CRS-2676: Start of 'ora.evmd' on 'racha2' succeeded
racha2 2011/07/26 09:41:40 /u01/app/11.2.0/grid/cdata/racha2/backup_20110726_094140.olr
Preparing packages for installation...
cvuqdisk-1.0.7-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Updating inventory properties for clusterware
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB. Actual 4031 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /u01/app/oraInventory
'UpdateNodeList' was successful.
[root@racha2 ~]#

Once the scripts have completed, return to the "Execute Configuration Scripts" screen on RAC1 and click the "OK" button 



NOTE:- We expect the verification phase to fail with an error relating to the SCAN, assuming you are not using DNS. 
INFO: Checking name resolution setup for "rac-scan.localdomain"...
INFO: ERROR:
INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "rac-scan.localdomain"
INFO: ERROR:
INFO: PRVF-4657 : Name resolution setup check for "rac-scan.localdomain" (IP address: 192.168.2.201) failed
INFO: ERROR:
INFO: PRVF-4664 : Found inconsistent name resolution entries for SCAN name "rac-scan.localdomain"
INFO: Verification of SCAN VIP and Listener setup failed

Provided this is the only error, it is safe to ignore this and continue by clicking the "Next" button.Click the "Close" button to exit the installer.




Check Cluster status for both nodes
[oracle@racha1 ~]$ export GRID_HOME=/u01/app/11.2.0/grid

[oracle@racha1 ~]$ export PATH=/u01/app/11.2.0/grid/bin
[oracle@racha1 ~]$ crsctl check cluster -all
**************************************************************
racha1:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

racha2:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online 
**************************************************************  

[oracle@racha1 ~]$ olsnodes
racha1
racha2
[oracle@racha1 ~]$ crs_stat -t -v


Name Type R/RA F/FT Target State Host
----------------------------------------------------------------------
ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE racha1
ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE racha1
ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE racha2

ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE racha1
ora.eons ora.eons.type 0/3 0/ ONLINE ONLINE racha1
ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE
ora....network ora....rk.type 0/5 0/ ONLINE ONLINE racha1
ora.oc4j ora.oc4j.type 0/5 0/0 OFFLINE OFFLINE
ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE racha1
ora.rac.db ora....se.type 0/2 0/1 ONLINE ONLINE racha1
ora....SM1.asm application 0/5 0/0 ONLINE ONLINE racha1
ora....A1.lsnr application 0/5 0/0 ONLINE ONLINE racha1
ora.racha1.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.racha1.ons application 0/3 0/0 ONLINE ONLINE racha1
ora.racha1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE racha1
ora....SM2.asm application 0/5 0/0 ONLINE ONLINE racha2
ora....A2.lsnr application 0/5 0/0 ONLINE ONLINE racha2
ora.racha2.gsd application 0/5 0/0 OFFLINE OFFLINE
ora.racha2.ons application 0/3 0/0 ONLINE ONLINE racha2
ora.racha2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE racha2
ora....ry.acfs ora....fs.type 0/5 0/ ONLINE ONLINE racha1
ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE racha2
[oracle@racha1 ~]$

You  can see gsd services are offline by default in 11g r2

The grid infrastructure installation is now complete. Now we have to install database software..