Note: This note was created for 9i RAC. The 10g Oracle documentation
provides installation instructions for 10g RAC. These instructions
can be found on OTN:
Oracle(r) Real Application Clusters Installation andOracle(r) Real
Application Clusters Installation and Configuration Guide
10g Release 1 (10.1) for AIX-Based Systems, hp HP-UX PA-RISC (64-bit),
hp Tru64 UNIX, Linux, Solaris Operating System (SPARC 64-bit)
Purpose
This document will provide the reader with step-by-step instructions
on how to install a cluster, install Oracle Real Application Clusters
(RAC) (Version 9.2.0.5), and start a cluster database on Linux. For
additional explanation or information on any of these steps, please
see the references listed at the end of this document.
Disclaimer: If there are any errors or issues prior to step 2, please
contact your Linux distributor.
The information contained here is as accurate as possible at the time
of writing.
* 1. Configuring the Cluster Hardware
o 1.1 Minimal Hardware list / System Requirements
+ 1.1.1 Hardware
+ 1.1.2 Software
o 1.2 Installing the Shared Disk Subsystem
o 1.3 Configuring the Cluster Interconnect and Public Network Hardware
* 2. Creating a cluster
o 2.1 UNIX Pre-installation tasks
o 2.2 Configuring the Shared Disks
o 2.3 Run the Oracle Universal Installer to install the
9.2.0.4 ORACM (Oracle Cluster Manager)
o 2.4 Configure the hangcheck-timer
o 2.5 Install Version 10.1.0.2 of the Oracle Universal Installer
o 2.6 Run the 10.1.0.2 Oracle Universal Installer to patch
the Oracle Cluster Manager (ORACM) to 9.2.0.5
o 2.7 Modify the ORACM configuration files to utilize the
hangcheck-timer
o 2.8 Start the ORACM (Oracle Cluster Manager)
* 3. Installing RAC
o 3.1 Install 9.2.0.4 RAC
o 3.2 Patch the RAC Installation to 9.2.0.5
o 3.3 Start the GSD (Global Service Daemon)
o 3.4 Create a RAC Database using the Oracle Database
Configuration Assistant
* 4. Administering Real Application Clusters Instances
* 5. References
1. Configuring the Clusters Hardware<>
1.1 Minimal Hardware list / System Requirements
Please check the RAC/Linux certification matrix for information on
currentlyRAC/Linux certification matrix for information on currently
supported hardware/software.
1.1.1 Hardware
* Requirements:
o Refer to the RAC/Linux certification matrix for
information onRAC/Linux certification matrix for information on
supported configurations. Ensure that the system has at least the
following resources:
- 400 MB in /tmp
- 512 MB of Physical Memory (RAM)
- Three times the amount of Physical Memory for Swap
space (unless the system exceeds 1 GB of Physical Memory, where two
times the amount of Physical Memory for Swap space is sufficient)
An example system disk layout is as follows:-
A sample system disk layout
Slice
Contents
Allocation (in Mbytes)
0
/
2000 or more
1
/boot
64
2
/tmp
1000
3
/usr
3000-7000 depending on operating system and packages installed
4
/var
512 (can be more if required)
5
swap
Three times the amount of Physical Memory for Swap space (unless the
system exceeds 1 GB of Physical Memory, where two times the amount of
Physical Memory for Swap space is sufficient).
6
/home
2000 (can be more if required)
1.1.2 Software
* For RAC on Linux support, consult the operating system vendor
and see the RAC/Linux certification matrix.
* RAC/Linux certification matrix. Make sure you have make and
rsh-server packages installed, check with:
$rpm -q rsh-server make
rsh-server-0.17-5
make-3.79.1-8
If these are not installed, use your favorite package manager to
install them.
1.1.3 Patches
Consult with your operating system vendor to get on the latest patch
version of the kernel.
1.2 Installing the Shared Disk Subsystem
This is highly dependent on the subsystem you have chosen. Please
refer to your hardware documentation for installation and
configuration instructions on Linux. Additional drivers and patches
might be required. In this article we assume that the shared disk
subsystem is correctly installed and that the shared disks are visible
to all nodes in the cluster.
1.3 Configuring the Cluster Interconnect and Public Network Hardware
If not already installed, install host adapters in your cluster nodes.
For the procedure on installing host adapters, see the documentation
that shipped with your host adapters and node hardware.
Each system will have at least an IP address for the public network
and one for the private cluster interconnect. For the public network,
get the addresses from your network manager. For the private
interconnect use 1.1.1.1 , 1.1.1.2 for the first and second node. Make
sure to add all addresses in /etc/hosts.
[oracle@opcbrh1 oracle]$ more /etc/hosts
ex:
9.25.120.143 rac1 #Oracle 9i Rac node 1 - public network
9.25.120.143 rac2 #Oracle 9i Rac node 2 - public network
1.1.1.1 int-rac1 #Oracle 9i Rac node 1 - interconnect
1.1.1.2 int-rac2 #Oracle 9I Rac node 2 - interconnect
Use your favorite tool to configure these adapters. Make sure your
public network is the primary (eth0).
Interprocess communication is an important issue for RAC since cache
fusion transfers buffers between instances using this mechanism. Thus,
networking parameters are important for RAC databases. The values in
the following table are the recommended values. These are NOT the
default on most distributions.
Parameter
Meaning
Value
/proc/sys/net/core/rmem_default
The default setting in bytes of the socket receive buffer
262144
/proc/sys/net/core/rmem_max
The maximum socket receive buffer size in bytes
262144
/proc/sys/net/core/wmem_default
The default setting in bytes of the socket send buffer
262144
/proc/sys/net/core/wmem_max
The maximum socket send buffer size in bytes
262144
You can see these settings with:
$ cat /proc/sys/net/core/rmem_default
Change them with:
$ echo 262144 > /proc/sys/net/core/rmem_default
This will need to be done each time the system boots. Some
distributions already have setup a method for this during boot. On Red
Hat , this can be configured in /etc/sysctl.conf (like :
net.core.rmem_default = 262144).
2. Creating a Cluster
On Linux, the cluster software required to run Real Application
Clusters is included in the Oracle distribution.
The Oracle Cluster Manager (ORACM) installation process includes eight
major tasks.
1. UNIX pre-installation tasks.
2. Configuring the shared disks
3. Run the Oracle Universal Installer to install the 9.2.0.4 ORACM
(Oracle Cluster Manager)
4. Configure the hangcheck-timer.
5. Install version 10.1.0.2 of the Oracle Universal Installer
6. Run the 10.1.0.2 Oracle Universal Installer to patch the Oracle
Cluster Manager (ORACM) to 9.2.0.5
7. Modify the ORACM configuration files to utilize the hangcheck-timer.
8. Start the ORACM (Oracle Cluster Manager)
2.1 UNIX Pre-installation tasks
These steps need to be performed on ALL nodes.
* First, on each node, create the Oracle group. Example:
# groupadd dba -g 501
* Next, make the Oracle user's home directory. Example:
# mkdir -p /u01/home/oracle
* On each node, create the Oracle user. Make sure that the Oracle
user is part of the dba group. Example:
# useradd -c "Oracle Software Owner" -G dba -u 101 -m -d
/u01/home/oracle -s /bin/csh oracle
* On each node, Create a mount point for the Oracle software
installation (at least 2.5 GB, typically /u01). The oracle user
should own this mount point and all of the directories below the mount
point. Example:
# mkdir /u01
# chown -R oracle.dba /u01
# chmod -R ug=rwx,o=rx /u01
* Once this is done, test the permissions on each node to ensure
that the oracle user can write to the new mount points. Example:
# su - oracle
$ touch /u01/test
$ ls -l /u01/test
-rw-rw-r-- 1 oracle dba 0 Aug 15 09:36 /u01/test
* Depending on your Linux distribution, make sure inetd or xinetd
is started on all nodes and that the ftp, telnet, shell and login (or
rsh) services are enabled (see /etc/inetd.conf or /etc/xinetd.conf and
/etc/xinetd.d). Example:
# more /etc/xinetd.d/telnet
# default: on
# description: The telnet server serves telnet sessions; it uses # unencrypted username/password pairs for authentication.
service telnet
{
flags = REUSE
socket_type = stream
wait = no
user = root
server = /usr/sbin/in.telnetd
log_on_failure += USERID
disable = no
}
In this example, disable should be set to 'no'.
* On the node from which you will run the Oracle Universal
Installer, set up user equivalence by adding entries for all nodes in
the cluster, including the local node, to the .rhosts file of the
oracle account, or the /etc/hosts.equiv file.
Sample entries in /etc/hosts.equiv file:
rac1
rac2
int-rac1
int-rac2
* As oracle user, check for user equivalence for the oracle
account by performing a remote copy (rcp) to each node (public and
private) in the cluster. Example:
RAC1:
$ touch /u01/test
$ rcp /u01/test rac2:/u01/test1
$ rcp /u01/test int-rac2:/u01/test2
RAC2:
$ touch /u01/test
$ rcp /u01/test rac1:/u01/test1
$ rcp /u01/test int-rac1:/u01/test2
$ ls /u01/test*
/u01/test /u01/test1 /u01/test2
RAC1:
$ ls /u01/test*
/u01/test /u01/test1 /u01/test2
Note: If you are prompted for a password, you have not given the
oracle account the same attributes on all nodes. You must correct this
because the Oracle Universal Installer cannot use the rcp command to
copy Oracle products to the remote node's directories without user
equivalence.
System Kernel Parameters
Verify operating system kernel parameters are set to appropriate levels:
Kernel Parameter
Setting
Purpose
SHMMAX
2147483648
Maximum allowable size of one shared memory segment.
SHMMIN
1
Minimum allowable size of a single shared memory segment.
SHMMNI
100
Maximum number of shared memory segments in the entire system.
SHMSEG
10
Maximum number of shared memory segments one process can attach.
SEMMNI
100
Maximum number of semaphore sets in the entire system.
SEMMSL
250
Minimum recommended value. SEMMSL should be 10 plus the largest
PROCESSES parameter of any Oracle database on the system.
SEMMNS
1000
Maximum semaphores on the system. This setting is a minimum
recommended value. SEMMNS should be set to the sum of the PROCESSES
parameter for each Oracle database, add the largest one twice, plus
add an additional 10 for each database.
SEMOPM
100
Maximum number of operations per semop call.
You will have to set the correct parameters during system startup, so
include them in your startup script (startoracle_root.sh):
$ export SEMMSL=100
$ export SEMMNS=1000
$ export SEMOPM=100
$ export SEMMNI=100
$ echo $SEMMSL $SEMMNS $SEMOPM $ SEMMNI > /proc/sys/kernel/sem
$ export SHMMAX=2147483648
$ echo $SHMMAX > /proc/sys/kernel/shmmax
Check these with:
$ cat /proc/sys/kernel/sem
$ cat /proc/sys/kernel/shmmax
You might want to increase the maximum number of file handles, include
this in your startup script or use /etc/sysctl.conf :
$ echo 65536 > /proc/sys/fs/file-max
To allow your oracle processes to use these file handles, add the
following to your oracle account login script (ex.: .profile)
$ ulimit -n 65536
Note: This will only allow you to set the soft limit as high as the
hard limit. You might have to increase the hard limit on system level.
This can be done by adding ulimit -Hn 65536 to /etc/initscript. You
will have to reboot the system to make this active. Sample
/etc/initscript:
ulimit -Hn 65536
eval exec "$4"
Establish Oracle environment variables: Set the following Oracle
environment variables:
Environment Variable
Suggested value
ORACLE_HOME
eg /u01/app/oracle/product/920
ORACLE_TERM
xterm
PATH
/u01/app/oracle/product/9.2.0/bin: /usr/ccs/bin:/usr/bin/X11/:/usr/local/bin
and any other items you require in your PATH
DISPLAY
(review Note:153960.1 for detailed information)
Note:153960.1 for detailed information)
TMPDIR
Set a temporary directory path for TMPDIR with at least 100 Mb of free
space to which the OUI has write permission.
ORACLE_SID
Set this to what you will call your database instance. This should be
UNIQUE on each node.
It is best to save these in a .login or .profile file so that you do
not have to set the environment every time you log in.
* Create the directory /var/opt/oracle and set ownership to the
oracle user. Example:
$ mkdir /var/opt/oracle
$ chown oracle.dba /var/opt/oracle
* Set the oracle user's umask to "022" in you ".profile" or
".login" file. Example:
$ umask 022
Note: There is a verification script InstallPrep.sh available which
may beverification script InstallPrep.sh available which may be
downloaded and run prior to the installation of Oracle Real
Application Clusters. This script verifies that the system is
configured correctly according to the Installation Guide. The output
of the script will report any further tasks that need to be performed
before successfully installing Oracle 9.x DataServer (RDBMS). This
script performs the following verifications:-
* ORACLE_HOME Directory Verification
* UNIX User/umask Verification
* UNIX Group Verification
* Memory/Swap Verification
* TMP Space Verification
* Real Application Cluster Option Verification
* Unix Kernel Verification
. ./InstallPrep.sh
You are currently logged on as oracle
Is oracle the unix user that will be installing Oracle Software? y or n
y
Enter the unix group that will be used during the installation
Default: dba
Enter the version of Oracle RDBMS you will be installing
Enter either : 901 OR 920 - Default: 920
920
The rdbms version being installed is 920
Enter Location where you will be installing Oracle
Default: /u01/app/oracle/product/oracle9i
/u01/app/oracle/product/9.2.0
Your Operating System is Linux
Gathering information... Please wait
JDK check is ignored for Linux since it is provided by Oracle
Checking unix user ...
Checking unix umask ...
umask test passed
Checking unix group ...
Unix Group test passed
Checking Memory & Swap...
Memory test passed
/tmp test passed
Checking for a cluster...
Linux Cluster test section has not been implemented yet
No cluster warnings detected
Processing kernel parameters... Please wait
Running Kernel Parameter Report...
Check the report for Kernel parameter verification\n
Completed.
/tmp/Oracle_InstallPrep_Report has been generated
Please review this report and resolve all issues before attempting to
install the Oracle Database Software
Note: If you get an error like this:
InstallPrep.sh: line 45: syntax error near unexpected token `fi'
or
./InstallPrep.sh: Command not found.
Then you need to copy the script into a text file (it will not run if
the file is in binary format).
2.2 Configuring the Shared Disks
For 9.2 Real Application Clusters on Linux, you can use either OCFS
(Oracle Cluster Filesystem), RAW, or NFS (Redhat and Network Appliance
Only) for storage of Oracle database files.
* For more information on setting up OCFS for RAC on Linux, see
the following MetaLink Note:
Note 220178.1 - Installing and setting up ocfs on Linux -
BasicNote 220178.1 - Installing and setting up ocfs on Linux - Basic
Guide
* For more information on setting up RAW for RAC on Linux, see the
following MetaLink Note:
Note 246205.1 - Configuring Raw Devices for Real
ApplicationNote 246205.1 - Configuring Raw Devices for Real
Application Clusters on Linux
* For more information on setting up NFS for RAC on Linux, see the
following MetaLink Note (Steps 1-6):
Note 210889.1 - RAC Installation with a NetApp Filer in Red
HatNote 210889.1 - RAC Installation with a NetApp Filer in Red Hat
Linux Environment
2.3 Run the Oracle Universal Installer to install the 9.2.0.4 ORACM
(Oracle Cluster Manager)
These steps only need to be performed on the node that you are
installing from (typically Node 1).
* If you are using OCFS or NFS for your shared storage, pre-create
the quorum file and srvm file. Example:
# dd if=/dev/zero of=/ocfs/quorum.dbf bs=1M count=20
# dd if=/dev/zero of=/ocfs/srvm.dbf bs=1M count=100
# chown root:dba /ocfs/quorum.dbf
# chmod 664 /ocfs/quorum.dbf
# chown oracle:dba /ocfs/srvm.dbf
# chmod 664 /ocfs/srvm.dbf
* Verify the Environment - Log off and log on as the oracle user
to ensure all environment variables
are set correctly. Use the following command to view them:
% env | more
Note: If you are on Redhat Advanced Server 3.0, you will need to
temporarily use an older gcc for the install:
mv gcc gcc3.2.3
mv g++ g++3.2.3
ln -s /usr/bin/gcc296 /usr/bin/gcc
ln -s /usr/bin/g++296 /usr/bin/g++
You will also need to apply patch 3006854 if on RHAS 3.0
* Before attempting to run the Oracle Universal Installer, verify
that you can successfully run the following command:
% /usr/bin/X11/xclock
* If this does not display a clock on your display screen, please
review the following article:
Note 153960.1 FAQ: X Server testing and troubleshooting
Note 153960.1 FAQ: X Server testing and troubleshooting
* Start the Oracle Universal Installer and install the RDBMS
software - Follow these procedures to use the Oracle Universal
Installer to install the Oracle Cluster Manager software. Oracle9i is
supplied on multiple CD-ROM disks. During the installation process it
is necessary to switch between the CD-ROMS. OUI will manage the
switching between CDs.
Use the following commands to start the installer:
% cd /tmp
% /cdrom/runInstaller
Or cd to /stage/Disk1 and run ./runInstaller
Respond to the installer prompts as shown below:
* At the "Welcome Screen", click Next.
If this is your first install on this machine:
* If the "Inventory Location" screen appears, enter the inventory
location then click OK.
* If the "Unix Group Name" screen appears, enter the unix group
name created in step 2.1 then click Next.
* At this point you may be prompted to run /tmp/orainstRoot.sh.
Run this and click Continue.
* At the "File Locations Screen", verify the destination listed is
your ORACLE_HOME directory. Also enter a NAME to identify this
ORACLE_HOME. The NAME can be anything.
* At the "Available Products Screen", Check "Oracle Cluster
Manager". Click Next.
* At the public node information screen, enter the public node
names and click Next.
* At the private node information screen, enter the interconnect
node names. Click Next.
* Enter the full name of the file or raw device you have created
for the ORACM Quorum disk information. Click Next.
* Press Install at the summary screen.
* You will now briefly get a progress window followed by the end
of installation screen. Click Exit and confirm by clicking Yes.
Note: Create the directory $ORACLE_HOME/oracm/log (as oracle) on the
other nodes if it doesn't exist.
2.4 Configure the hangcheck-timer
These steps need to be performed on ALL nodes.
Some kernel versions include the hangcheck-timer with the kernel. You
can check to see if your kernel contains the hangcheck-timer by
running:
# /sbin/lsmod
Then you will see hangcheck-timer listed. Also verify that
hangcheck-timer is starting in your /etc/rc.local file (on Redhat) or
/etc/init.d/boot.local (on United Linux). If you see hangcheck-timer
listed in lsmod and in the rc.local file or boot.local, you can skip
to section 2.5.
If hangcheck-timer is not listed here and you are not using Redhat
Advanced Server, see the following note for information on obtaining
the hangcheck-timer:
Note 232355.1 - Hangcheck Timer FAQ
Note 232355.1 - Hangcheck Timer FAQ
If you are on Redhat Advanced Server, you can either apply the latest
errata version (> 12) or go to MetaLink - Patches:
Enter 2594820 in the Patch Number field.
Click Go.
Click Download.
Save the file p2594820_20_LINUX.zip to the local disk, such as /tmp.
Unzip the file. The output should be similar to the following:
inflating: hangcheck-timer-2.4.9-e.3-0.4.0-1.i686.rpm
inflating: hangcheck-timer-2.4.9-e.3-enterprise-0.4.0-1.i686.rpm
inflating:
hangcheck-timer-2.4.9-e.3-smp-0.4.0-1.i686.rpm
inflating: README.TXT
Run the uname -a command to identify the RPM that corresponds to the
kernel in use. This will show if the kernel is single CPU, smp, or
enterprise.
The p2594820_20_LINUX.zip file contains four files. The following
describes the files:
hangcheck-timer-2.4.9-e.3-0.4.0-1.i686.rpm is for single CPU machines
hangcheck-timer-2.4.9-e.3-enterprise=0.4.0-1.i686.rpm is for
multi-processor machines with more than 4 GB of RAM
hangcheck-timer-2.4.9-e.3-smp-0.4.0-1.i686.rpm is for multi-processor
machines with 4 GB of RAM or less
The three RPMs will work for the e3 kernels in Red Hat Advanced Server
2.1 gold and the e8 kernels in the latest Red Hat Advanced Server 2.1
errata release. These RPMs are for Red Hat Advanced Server 2.1 kernels
only.
Transfer the relevant hangcheck-timer RPM to the /tmp directory of the
Oracle Real Applications Cluster node.
Log in to the node as the root user.
Change to the /tmp directory.
Run the following command to install the module:
#rpm -ivh hangcheck-timer RPM name
If you have previously installed RAC on this cluster, remove or
disable the mechanism that loads the softdog module at system start
up, if that module is not used by other software on the node. This is
necessary for subsequent steps in the installation process. This step
may require log in as the root user. One method for setting up
previous versions of Oracle Real Applications Clusters involved
loading the softdog module in the /etc/rc.local (on Redhat) or
/etc/init.d/boot.local (on United Linux) file. If this method was
used, then remove or comment out the following line in the file:
/sbin/insmod softdog nowayout=0 soft_noboot=1 soft_margin=60
Append the following line to the /etc/rc.local file (on Redhat) or
/etc/init.d/boot.local (on United Linux):
/sbin/insmod hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
Load the hangcheck-timer kernel module using the following command as root user:
# /sbin/insmod hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
Repeat the above steps on all Oracle Real Applications Clusters nodes
where the kernel module needs to be installed.
Run dmesg after the module is loaded. Note the build number while
running the command. The following is the relevant information output:
build 334adfa62c1a153a41bd68a787fbe0e9
The build number is required when making support calls.
2.5 Install Version 10.1.0.2 of the Oracle Universal Installer
These steps need to be performed on ALL nodes.
Download the 9.2.0.5 patchset from MetaLink - Patches:
Enter 3501955 in the Patch Number field.
Click Go.
Click Download.
* Place the file in a patchset directory on the node you are
installing from. Example:
$ mkdir $ORACLE_HOME/9205
$ cp p3501955_9205_LINUX.zip $ORACLE_HOME/9205
* Unzip the file:
$ cd $ORACLE_HOME/9205
$ unzip p3501955_9205_LINUX.zip
Archive: p3501955_9205_LINUX
inflating: 9205_lnx32_release.cpio
inflating: README.html
inflating: ReleaseNote9205.pdf
* Run CPIO against the file:
$ cpio -idmv < 9205_lnx32_release.cpio
Run the installer from the 9.2.0.5 staging location:
$ cd $ORACLE_HOME/9205/Disk1
$ ./runInstaller
Respond to the installer prompts as shown below:
* At the "Welcome Screen", click Next.
* At the "File Locations Screen", Change the $ORACLE_HOME name
from the dropdown list to the 9.2 $ORACLE_HOME name. Click Next.
* On the "Available Products Screen", Check "Oracle Universal
Installer 10.1.0.2. Click Next.
* Press Install at the summary screen.
* You will now briefly get a progress window followed by the end
of installation screen. Click Exit and confirm by clicking Yes.
Remember to install the 10.1.0.2 Installer on ALL cluster nodes. Note
that you may need to ADD the 9.2 $ORACLE_HOME name on the "File
Locations Screen" for other nodes. It will ask if you want to specify
a non-empty directory, say "Yes".
2.6 Run the 10.1.0.2 Oracle Universal Installer to patch the Oracle
Cluster Manager (ORACM) to 9.2.0.5
These steps only need to be performed on the node that you are
installing from (typically Node 1).
The 10.1.0.2 OUI will use SSH (Secure Shell) if it is configured. If
it is not configured it will use RSH (Remote Shell). If you have SSH
configured on your cluster, test and make sure that you can SSH and
SCP to all nodes of the cluster without being prompted. If you do not
have SSH configured, skip this step and run the installer from
$ORACLE_BASE/oui/bin as noted below.
SSH Test:
As oracle user, check for user equivalence for the oracle account by
performing a secure copy (scp) to each node (public and private) in
the cluster. Example:
RAC1:
$ touch /u01/sshtest
$ scp /u01/sshtest rac2:/u01/sshtest1
$ scp /u01/sshtest int-rac2:/u01/sshtest2
RAC2:
$ touch /u01/sshtest
$ scp /u01/sshtest rac1:/u01/sshtest1
$ scp /u01/sshtest int-rac1:/u01/sshtest2
$ ls /u01/sshtest*
/u01/sshtest /u01/sshtest1 /u01/sshtest2
RAC1:
$ ls /u01/sshtest*
/u01/sshtest /u01/sshtest1 /u01/sshtest2
Run the installer from the 9.2.0.5 oracm staging location:
$ cd $ORACLE_HOME/9205/Disk1/oracm
$ ./runInstaller
Respond to the installer prompts as shown below:
* At the "Welcome Screen", click Next.
* At the "File Locations Screen", make sure the source location is
to the products.xml file in the 9.2.0.5 patchset location under
Disk1/stage. Also verify the destination listed is your ORACLE_HOME
directory. Change the $ORACLE_HOME name from the dropdown list to the
9.2 $ORACLE_HOME name. Click Next.
* At the "Available Products Screen", Check "Oracle9iR2 Cluster
Manager 9.2.0.5.0". Click Next.
* At the public node information screen, enter the public node
names and click Next.
* At the private node information screen, enter the interconnect
node names. Click Next.
* Click Install at the summary screen.
* You will now briefly get a progress window followed by the end
of installation screen. Click Exit and confirm by clicking Yes.
2.7 Modify the ORACM configuration files to utilize the hangcheck-timer.
These steps need to be performed on ALL nodes.
Modify the $ORACLE_HOME/oracm/admin/cmcfg.ora file:
Add the following line:
KernelModuleName=hangcheck-timer
Adjust the value of the MissCount line based on the sum of the
hangcheck_tick and hangcheck_margin values. (> 210)
MissCount=210
Make sure that you can ping each of the names listed in the private
and public node name sections from each node. Example:
$ ping rac2
PING opcbrh2.us.oracle.com (138.1.137.46) from 138.1.137.45 :
56(84) bytes of data.
64 bytes from opcbrh2.us.oracle.com (138.1.137.46): icmp_seq=0
ttl=255 time=1.295 msec
64 bytes from opcbrh2.us.oracle.com (138.1.137.46): icmp_seq=1
ttl=255 time=154 usec
Verify that a valid CmDiskFile line exists in the following format:
CmDiskFile=file or raw device name
In the preceding command, the file or raw device must be valid. If a
file is used but does not exist, then the file will be created if the
base directory exists. If a raw device is used, then the raw device
must exist and have the correct ownership and permissions. Sample
cmcfg.ora file:
ClusterName=Oracle Cluster Manager, version 9i
MissCount=210
PrivateNodeNames=int-rac1 int-rac2
PublicNodeNames=rac1 rac2
ServicePort=9998
CmDiskFile=/u04/quorum.dbf
KernelModuleName=hangcheck-timer
HostName=int-rac1
Note: The cmcfg.ora file should be the same on both nodes with the
exception of the HostName parameter which should be set to the local
(internal) hostname.
Make sure all of these changes have been made to all RAC nodes. More
information on ORACM parameters can be found in the following note:
Note 222746.1 - RAC Linux 9.2: Configuration of cmcfg.ora
andNote 222746.1 - RAC Linux 9.2: Configuration of cmcfg.ora and
ocmargs.ora
Note: At this point it would be a good idea to patch to the latest
ORACM, especially if you have more than 2 nodes. For more information
see:
Note 278156.1 - ORA-29740 or ORA-29702 After Applying 9.2.0.5 Patchset
on RAC / Linux
ORA-29740 or ORA-29702 After Applying 9.2.0.5 Patchset on RAC / Linux
2.8 Start the ORACM (Oracle Cluster Manager)
These steps need to be performed on ALL nodes.
Cd to the $ORACLE_HOME/oracm/bin directory, change to the root user,
and start the ORACM.
$ cd $ORACLE_HOME/oracm/bin
$ su root
# ./ocmstart.sh
oracm &1 >/u01/app/oracle/product/9.2.0/oracm/log/cm.out &
Verify that ORACM is running with the following:
# ps -ef | grep oracm
On RHEL 3.0, add the -m option:
# ps -efm | grep oracm
You should see several oracm threads running. Also verify that the
ORACM version is the same on each node:
# cd $ORACLE_HOME/oracm/log
# head -1 cm.log
oracm, version[ 9.2.0.2.0.49 ] started {Fri May 14 09:22:28 2004 }
3.0 Installing RAC
The Real Application Clusters installation process includes four major tasks.
1. Install 9.2.0.4 RAC.
2. Patch the RAC Installation to 9.2.0.5.
3. Start the GSD.
4. Create and configure your database.
3.1 Install 9.2.0.4 RAC
These steps only need to be performed on the node that you are
installing from (typically Node 1).
Note: Due to bug 3547724, temporarily create a symbolic link /oradata
directory pointing to an oradata directory with space available as
root prior to running the RAC install:
# mkdir /u04/oradata
# chmod 777 /u04/oradata
# ln -s /u04/oradata /oradata
Install 9.2.0.4 RAC into your $ORACLE_HOME by running the installer
from the 9.2.0.4 cd or your original stage location for the 9.2.0.4
install.
Use the following commands to start the installer:
% cd /tmp
% /cdrom/runInstaller
Or cd to /stage/Disk1 and run ./runInstaller
Respond to the installer prompts as shown below:
* At the "Welcome Screen", click Next.
* At the "Cluster Node Selection Screen", make sure that all RAC
nodes are selected.
* At the "File Locations Screen", verify the destination listed is
your ORACLE_HOME directory and that the source directory is pointing
to the products.jar from the 9.2.0.4 cd or staging location.
* At the "Available Products Screen", check "Oracle 9i Database
9.2.0.4". Click Next.
* At the "Installation Types Screen", check "Enterprise Edition"
(or whichever option your prefer), click Next.
* At the "Database Configuration Screen", check "Software Only".
Click Next.
* At the "Shared Configuration File Name Screen", enter the path
of the CFS or NFS srvm file created at the beginning of step 2.3 or
the raw device created for the shared configuration file. Click Next.
* Click Install at the summary screen. Note that some of the
items installed will say "9.2.0.1" for the version, this is normal
because only some items needed to be patched up to 9.2.0.4.
* You will now get a progress window, run root.sh when prompted.
* You will then see the end of installation screen. Click Exit and
confirm by clicking Yes.
Note: You can now remove the /oradata symbolic link:
# rm /oradata
3.2 Patch the RAC Installation to 9.2.0.5
These steps only need to be performed on the node that you are installing from.
Run the installer from the 9.2.0.5 staging location:
$ cd $ORACLE_HOME/9205/Disk1
$ ./runInstaller
Respond to the installer prompts as shown below:
* At the "Welcome Screen", click Next.
* View the "Cluster Node Selection Screen", click Next.
* At the "File Locations Screen", make sure the source location is
to the products.xml file in the 9.2.0.5 patchset location under
Disk1/stage. Also verify the destination listed is your ORACLE_HOME
directory. Change the $ORACLE_HOME name from the dropdown list to the
9.2 $ORACLE_HOME name. Click Next.
* At the "Available Products Screen", Check "Oracle9iR2 PatchSets
9.2.0.5.0". Click Next.
* Click Install at the summary screen.
* You will now get a progress window, run root.sh when prompted.
* You will then see the end of installation screen. Click Exit and
confirm by clicking Yes.
3.3 Start the GSD (Global Service Daemon)
These steps need to be performed on ALL nodes.
Start the GSD on each node with:
% gsdctl start
Successfully started GSD on local node
Then check the status with:
% gsdctl stat
GSD is running on the local node
If the GSD does not stay up, try running 'srvconfig -init -f' from the
OS prompt. If you get a raw device exception error or PRKR-1064 error
then see the following note to troubleshoot:
Note 212631.1 - Resolving PRKR-1064 in a RAC Environment
Note 212631.1 - Resolving PRKR-1064 in a RAC Environment
Note: After confirming that GSD starts, if you are on Redhat Advanced
Server 3.0, restore gcc296:
rm /usr/bin/gcc
mv /usr/bin/gcc3.2.3 /usr/bin/gcc
rm /usr/bin/g++
mv /usr/bin/g++3.2.3 /usr/bin/g++
3.4 Create a RAC Database using the Oracle Database Configuration Assistant
These steps only need to be performed on the node that you are
installing from (typically Node 1).
The Oracle Database Configuration Assistant (DBCA) will create a
database for you. The DBCA creates your database using the optimal
flexible architecture (OFA). This means the DBCA creates your database
files, including the default server parameter file, using standard
file naming and file placement practices. The primary phases of DBCA
processing are:-
* Verify that you correctly configured the shared disks for each
tablespace (for non-cluster file system platforms)
* Create the database
* Configure the Oracle network services
* Start the database instances and listeners
Oracle Corporation recommends that you use the DBCA to create your
database. This is because the DBCA preconfigured databases optimize
your environment to take advantage of Oracle9i features such as the
server parameter file and automatic undo management. The DBCA also
enables you to define arbitrary tablespaces as part of the database
creation process. So even if you have datafile requirements that
differ from those offered in one of the DBCA templates, use the DBCA.
You can also execute user-specified scripts as part of the database
creation process.
Note: Prior to running the DBCA it may be necessary to run the NETCA
tool or to manually set up your network files. To run the NETCA tool
execute the command netca from the $ORACLE_HOME/bin directory. This
will configure the necessary listener names and protocol addresses,
client naming methods, Net service names and Directory server usage.
If you are using OCFS or NFS, launch DBCA with the
-datafileDestination option and point to the shared location where
Oracle datafiles will be stored. Example:
% cd $ORACLE_HOME/bin
% dbca -datafileDestination /ocfs/oradata
If you are using RAW, launch DBCA without the -datafileDestination
option. Example:
% cd $ORACLE_HOME/bin
% dbca
Respond to the DBCA prompts as shown below:
* Choose Oracle Cluster Database option and select Next.
* The Operations page is displayed. Choose the option Create a
Database and click Next.
* The Node Selection page appears. Select the nodes that you want
to configure as part of the RAC database and click Next.
* The Database Templates page is displayed. The templates other
than New Database include datafiles. Choose New Database and then
click Next. Note: The Show Details button provides information on the
database template selected.
* DBCA now displays the Database Identification page. Enter the
Global Database Name and Oracle System Identifier (SID). The Global
Database Name is typically of the form name.domain, for example
mydb.us.oracle.com while the SID is used to uniquely identify an
instance (DBCA should insert a suggested SID, equivalent to name1
where name was entered in the Database Name field). In the RAC case
the SID specified will be used as a prefix for the instance number.
For example, MYDB, would become MYDB1, MYDB2 for instance 1 and 2
respectively.
* The Database Options page is displayed. Select the options you
wish to configure and then choose Next. Note: If you did not choose
New Database from the Database Template page, you will not see this
screen.
* Select the connection options desired from the Database
Connection Options page. Click Next.
* DBCA now displays the Initialization Parameters page. This page
comprises a number of Tab fields. Modify the Memory settings if
desired and then select the File Locations tab to update information
on the Initialization Parameters filename and location. The option
Create persistent initialization parameter file is selected by
default. If you have a cluster file system, then enter a file system
name, otherwise a raw device name for the location of the server
parameter file (spfile) must be entered. The button File Location
Variables… displays variable information. The button All
Initialization Parameters… displays the Initialization Parameters
dialog box. This box presents values for all initialization parameters
and indicates whether they are to be included in the spfile to be
created through the check box, included (Y/N). Instance specific
parameters have an instance value in the instance column. Complete
entries in the All Initialization Parameters page and select Close.
Note: There are a few exceptions to what can be altered via this
screen. Ensure all entries in the Initialization Parameters page are
complete and select Next.
* DBCA now displays the Database Storage Window. This page allows
you to enter file names for each tablespace in your database.
* The Database Creation Options page is displayed. Ensure that the
option Create Database is checked and click Finish.
* The DBCA Summary window is displayed. Review this information
and then click OK. Once you click the OK button and the summary
screen is closed, it may take a few moments for the DBCA progress bar
to start. DBCA then begins to create the database according to the
values specified.
During the database creation process, you may see the following error:
ORA-29807: specified operator does not exist
This is a known issue (bug 2925665). You can click on the "Ignore"
button to continue. Once DBCA has completed database creation,
remember to run the 'prvtxml.plb' script from $ORACLE_HOME/rdbms/admin
independently, as the user SYS. It is also advised to run the
'utlrp.sql' script to ensure that there are no invalid objects in the
database at this time.
A new database now exists. It can be accessed via Oracle SQL*PLUS or
other applications designed to work with an Oracle RAC database.
Additional database configuration best practices can be found in the
following note:
Note 240575.1 - RAC on Linux Best Practices
Note 240575.1 - RAC on Linux Best Practices
4.0 Administering Real Application Clusters Instances
Oracle Corporation recommends that you use SRVCTL to administer your
Real Application Clusters database environment. SRVCTL manages
configuration information that is used by several Oracle tools. For
example, Oracle Enterprise Manager and the Intelligent Agent use the
configuration information that SRVCTL generates to discover and
monitor nodes in your cluster. Before using SRVCTL, ensure that your
Global Services Daemon (GSD) is running after you configure your
database. To use SRVCTL, you must have already created the
configuration information for the database that you want to
administer. You must have done this either by using the Oracle
Database Configuration Assistant (DBCA), or by using the srvctl add
command as described below.
To display the configuration details for, example, databases racdb1/2,
on nodes racnode1/2 with instances racinst1/2 run:-
$ srvctl config
racdb1
racdb2
$ srvctl config -p racdb1 -n racnode1
racnode1 racinst1 /u01/app/oracle/product/9.2.0
$ srvctl status database -d racdb1
Instance racinst1 is running on node racnode1
Instance racinst2 is running on node racnode2
Examples of starting and stopping RAC follow:-
$ srvctl start database -d racdb2
$ srvctl stop database -d racdb2
$ srvctl stop instance -d racdb1 -i racinst2
$ srvctl start instance -d racdb1 -i racinst2
For further information on srvctl and gsdctl see the Oracle9i Real
Application Clusters Administration manual.
5.0 References
* 9.2.0.5 Patch Set Notes
* Tips for Installing and Configuring Oracle9i RealTips for
Installing and Configuring Oracle9i Real Application Clusters on Red
Hat Linux Advanced Server
* Note 201370.1 - LINUX Quick Start Guide - 9.2.0 RDBMSNote
201370.1 - LINUX Quick Start Guide - 9.2.0 RDBMS Installation
* Note 252217.1 - Requirements for Installing Oracle 9iR2 on RHEL3
* Note 240575.1 - RAC on Linux Best Practices
* Note 240575.1 - RAC on Linux Best Practices Note 222746.1 - RAC
Linux 9.2: Configuration of cmcfg.oraNote 222746.1 - RAC Linux 9.2:
Configuration of cmcfg.ora and ocmargs.ora
* Note 212631.1 - Resolving PRKR-1064 in a RAC EnvironmentNote
212631.1 - Resolving PRKR-1064 in a RAC Environment
* Note 220178.1 - Installing and setting up ocfs on Linux -Note
220178.1 - Installing and setting up ocfs on Linux - Basic Guide
* Note 246205.1 - Configuring Raw Devices for RealNote 246205.1 -
Configuring Raw Devices for Real Application Clusters on Linux
* Note 210889.1 - RAC Installation with a NetApp Filer inNote
210889.1 - RAC Installation with a NetApp Filer in Red Hat Linux
Environment
* Note 153960.1 FAQ: X Server testing and troubleshootingNote
153960.1 FAQ: X Server testing and troubleshooting
* Note 232355.1 - Hangcheck Timer FAQ
* Note 232355.1 - Hangcheck Timer FAQ RAC/Linux certification matrix
* RAC/Linux certification matrix Oracle9i Real Application
Clusters AdministrationOracle9i Real Application Clusters
Administration
* Oracle9i Real Application Clusters Concepts
* Oracle9i Real Application Clusters Concepts Oracle9i Real
Application Clusters Deployment andOracle9i Real Application Clusters
Deployment and Performance
* Oracle9i Real Application Clusters Setup andOracle9i Real
Application Clusters Setup and Configuration
* Oracle9i Installation Guide Release 2 for UNIXOracle9i
Installation Guide Release 2 for UNIX Systems: AIX-Based Systems,
Compaq Tru64 UNIX, HP 9000 Series HP-UX, Linux Intel, and Sun Solaris
No comments:
Post a Comment