1-67
Oracle RAC9i(R2) for Sun Cluster Workshop
RAC 개념
RAC 는 하드웨어 클러스터 상에서 운영된다. 클러스터는 단일 시스템으로서 함께 연동되는 독립 서버(노드)그룹이다. 주요 클러스터 컴포넌트는 프로세서 노드, 클러스터 인터커넥트 및 공유 스토리지 서브시스템 등이다. 노드는 데이타를 관리하는 스토리지 서브 시스템과 자원에 대한 액세스를 공유하지만 개별 노드에서 메인 메모리를 물리적으로 공유 하지는 않는다. ORAC 은 개별 노드의 메모리를 결합해 전체 데이타베이스 시스템에 대한 분산 캐시 메모리의 단일 뷰를 제공한다. 노드는 다중 프로세서로 구성될 수 있다. 대표적인 노드 유형은 SMP(Symmetric Multi-Processor) 노드이다.
각 노드는 자체 전용 시스템 메모리 뿐만 아니라 자체 운영체계, 데이타베이스 인스턴스 및 애플리케이션 소프트웨어를 보유하고 있다.
클러스터의 장점
단일 대규모 노드 보다 클러스터를 이용함으로써 실현되는 이점은 다음과 같다.
- 시스템이 원하는 용량으로 확장할 수 있도록 지원하는 용량 계획의 유연성 및 비용 효율성 - 특히 노드 장애 발생 시 클러스터 내 부분적 장애에 대한 폴트 톨로런스
확장성
ORAC 은 용량에 대한 요구가 증가함에 따라 클러스터에 노드를 추가할 수 있는 유연성을 사용자에게 제공함으로써 시스템을 단계적으로 확장해 자본 투자 비용을 절감하고 소규모 단일 노드 시스템을 대규모 노드 시스템으로 교체해야 하는 필요성을 없애도록 했다. 대부분의 경우
시스템을 업그레이드하기 위해 완전히 새로운 대규모 노드를 이용하는 대신 유사하거나 동일한 구성을 보유한 하나 이상의 노드가 클러스터에 추가되기 때문에 이는 보다 쉽고 신속하게 용량 업그레이드 프로세스를 실행할 수 있도록 하게 된다. ORAC 에 구현된 Cache Fusion 기술은 거의 선형적으로 용량을 확장할 수 있도록 지원한다.
고가용성
클러스터 아키텍처의 또 다른 주요 장점은 다중 노드에 의해 제공되는 고유 폴트 톨로런스이다.
물리적인 노드가 독립적으로 운영되기 때문에 하나 이상의 노드에 장애가 발생 하더라도
클러스터의 여타 노드에 영향을 미치지 않는다. 극한 상황의 경우, 클러스터 시스템은 단 하나의 노드 만이 생존하더라도 이용할 수 있기 때문에 클러스터를 토대로 하는 시스템은 고도의 가용성을 실현할 수 있게 된다. 또한 이 아키텍처는 유지보수를 위해 노드 그룹이 오프라인으로 되어 있는 동안 나머지 클러스터가 서비스를 온라인으로 제공할수 있도록 지원한다.
9i (RAC) New Feautes
Cache Fusion
- 모든 node 의 interconnected cache 를 활용한다.
- write/write 충돌 시 필요한 data block 을 remote node 의 cache 에서 바로 요청한 Instance 로 고속 interconect 를 통하여 전송
- Update 작업의 동기화를 위한 Disk I/O 불필요하다.
(이전 버전에서는 Cache 상에 변경되어 있는 Block 을 Disk 에 기록한 후, 다른 Instance 는 해당 Block 을 Disk 에서 Cache 로 다시 읽어 변경 작업을 하는 ‘Ping’ protocol 사용)
Cache Fusion 기법의 개념적 구조
Databas e
buffers
Node A
Databas e
buffers
Node
Data
B
Transfer Reques
t
Databas
e
3-67
Real Application Clusters Installation Overview
■ Oracle Real Application Clusters Validation Matrix
■ RAC 구축
1. Install the operating system-dependent (OSD) clusterware. The OSD clusterware installation process varies according to platform:
1.1 For UNIX, refer to your vendor documentation and to the documentation for the SunCluster UDLM patch.
2. If you are not using a cluster file system, then configure the shared disks
3. Run the OUI to install the Oracle9i Enterprise Edition and the Oracle9i Real Application Clusters software
4. Create and configure your database as described in either:
4.1 Create a RAC Database using the Oracle Database Configuration Assistant(DBCA), or:
4.2 Manually Creating Real Application Clusters Databases 5. Single Database에서 RAC 구현하기 (node 추가하기) 6. Administering Real Application Clusters Instances (옵션)
Cache Fusion을 이용한 경우의 비교
Without Cache Fusion
Block in Local Cache
Block in Remote Cache
Block on Disk With
Cache Fusion
Time to write data block to disk
0.01 msec 1 msec 20 msec 100 msec
Block Access Time(ms)
7. Network 구성 & CTF (Connection Time Failover) vs TAF (Transparent Application Failover)
■ Trouble Shooting
5-67
■ Oracle Real Application Clusters Validation Matrix
Oracle Real Application Clusters is a proven way to scale applications both in terms of number of users and volume of transactions. Additionally, Oracle Real Application Clusters provides proven availability that is a must for any mission-critical application. In this environment, Network Appliance enables these mission-critical applications to scale storage rapidly on an as-needed basis without any unplanned downtime. With its software offerings like Snapshot™, SnapMirror®, and SnapRestore®, Network Appliance provides a proven and
compelling platform that makes the Oracle Real Application Clusters implementation even more unbreakable.
Oracle has created an extensive test suite that validates compatibility of servers and storage with Oracle Real Application Clusters database. NetApp has received the validation for the following products from Oracle and Cluster Software vendors. Following is a matrix of Oracle Real Application Clusters Validation.
■ RAC Inatallation
1. Install the operating system-dependent (OSD) clusterware. The OSD clusterware installation process varies according to platform:
1.1 Sun Cluster
1.1.1 Installation UDLM package
The Sun Cluster install CD's contain the required SC udlm package:-
Package SUNWudlm Sun Cluster Support for Oracle Parallel Server UDLM, (opt) on SunCluster v3 To install use the pkgadd command:-
# pkgadd -d . SUNWudlm
Once installed, Oracle's interface with this, the Oracle UDLM, can be installed.
1.1.1.1 VxVM을 사용할 경우
To install Sun Cluster Support for RAC with VxVM, the following Sun Cluster 3 Agents data services packages need to be installed as superuser (see Sun's Sun Cluster 3 Data Services Installation and Configuration Guide):-
# pkgadd -d . SUNWscucm SUNWudlmr SUNWcvmr SUNWcvm (SUNWudlm will also need to be included unless already installed from the step above)
1.1.2 UDLM package check 및 UDLM patch
Before rebooting the nodes, you must ensure that you have correctly installed and configured the Oracle UDLM software.
The Oracle Unix Distributed Lock Manager (ORCLudlm also known as the Oracle Node Monitor) must be installed. This may be referred to in the Oracle documentation as the "Parallel Server Patch". To check version information on any previously installed dlm package:
$ pkginfo -l ORCLudlm |grep PSTAMP OR
$ pkginfo -l ORCLudlm |grep VERSION
You must apply the following steps to all cluster nodes. The Oracle udlm can be found on Disk1 of the Oracle9i server installation CD-ROM, in the directory opspatch or racpatch in later versions. Oracle patchsets may also ship udlm installs which are again found in the Disk1 racpatch directory of the patchset. A version of the Oracle udlm may also be found on the Sun Cluster CD set but check the Oracle release for the latest applicable version.
The informational files README.udlm & release_notes.334x are located in this directory with version and install
7-67
information. This is the Oracle udlm package for 7.X.X or later on Solaris Operating System (SPARC) and requires any previous versions to be removed prior to installation. Use the latest udlm pacakge as this supports previous versions of Oracle RAC and Parallel Server provided that the same bit-size (32-bit or 64-bit) for Oracle is used throughout.
• Shutdown all existing clients of Oracle Unix Distributed Lock Manager (including all Oracle Parallel Server/RAC instances).
• Become super user.
• Reboot the cluster node in non-cluster mode (replace <node name> with your cluster node name):-
# scswitch -S -h <node name>
# shutdown -g 0 -y ... wait for the ok prompt ok boot -x
• Unpack the file ORCLudlm.tar.Z into a directory:
cd <CD-ROM mount>/opspatch #(or racpatch in later versions) cp ORCLudlm.tar.Z /tmp
cd /tmp
uncompress ORCLudlm.tar.Z tar xvf ORCLudlm.tar
• Install the patch by adding the package as root:
cd /tmp
pkgadd -d . ORCLudlm
The udlm configuration files in SC2.X and SC3.0 are the following:
SC2.X: /etc/opt/SUNWcluster/conf/<default_cluster_name>.ora_cdb SC3.0: /etc/opt/SUNWcluster/conf/udlm.conf
The udlm log files in SC2.X and SC3.0 are the following:
SC2.X: /var/opt/SUNWcluster/dlm_<node_name>/logs/dlm.log SC3.0: /var/cluster/ucmm/dlm_<node_name>/logs/dlm.log
pkgadd will copy a template file, <configuration_file_name>.template, to /etc/opt/SUNWcluster/conf.
• Now that udlm (also referred to as the "Cluster Membership Monitor") is installed, you can start it up by rebooting the cluster node in cluster mode:-
# shutdown -g 0 -y -i 6
2. Configure the shared disks and UNIX preinstallation tasks.
2.1 Configure the shared disks(Raw Device 생성 )
The Oracle instances in the RAC configuration write information to raw devices defined for:
• The control file
• The spfile.ora
• Each datafile
• Each ONLINE redo log file
• Server Manager (SRVM) configuration information : svrctl 기능을 사용할 경우 필요
It is therefore necessary to define raw devices for each of these categories of file. The Oracle Database Configuration Assistant (DBCA) will create a seed database expecting the following configuration:
Create a Raw Device for: File Size Sample name
SYSTEM tablespace 400 MB db_name_raw_system_400
USERS tablespace 120 MB db_name_raw_user_120
TEMP tablespace 100 MB db_name_raw_temp_100
An undo tablespace per instance 500 MB db_name_raw_undo_500
OEMREPO 20 MB db_name_raw_oemrepo_20
INDX tablespace 70 MB db_name_raw_indx_70
TOOLS tablespace 12 MB db_name_raw_tools_12
DRYSYS tablespace 90 MB db_name_raw_dr_90
EXAMPLES tablespace 160 MB db_name_raw_examples_160
First control file 110 MB db_name_raw_control01_110
Second control file 110 MB db_name_raw_control02_110
Two redo log files per instance 120 MB per file db_name_thread_lognumb_120
Spfile.ora 5 MB db_name_raw_spfile_5
Srvmconfig 100 MB db_name_raw_srvmconf_100
※ Srvmconfig file은 svrctl 기능을 사용할 경우 필요하다.
2.1.1 On the node from which you run the Oracle Universal Installer, create an ASCII file identifying the raw volume objects as shown above. The DBCA requires that these objects exist during installation and database creation. When creating the ASCII file content for the objects, name them using the format:
database_object=raw_device_file_path
When you create the ASCII file, separate the database objects from the paths with equals (=) signs as shown in the example below:-
system=/dev/vx/rdsk/oracle_dg/clustdb_raw_system_400m spfile=/dev/vx/rdsk/oracle_dg/clustdb_raw_spfile_5m users=/dev/vx/rdsk/oracle_dg/clustdb_raw_users_120m temp=/dev/vx/rdsk/oracle_dg/clustdb_raw_temp_100m
undotbs1=/dev/vx/rdsk/oracle_dg/clustdb_raw_undotbs1_290m
9-67
undotbs2=/dev/vx/rdsk/oracle_dg/clustdb_raw_undotbs2_290m example=/dev/vx/rdsk/oracle_dg/clustdb_raw_example_30m cwmlite=/dev/vx/rdsk/oracle_dg/clustdb_raw_cwmlite_100m indx=/dev/vx/rdsk/oracle_dg/clustdb_raw_indx_70m
tools=/dev/vx/rdsk/oracle_dg/clustdb_raw_tools_12m drsys=/dev/vx/rdsk/oracle_dg/clustdb_raw_dr_90m
control1=/dev/vx/rdsk/oracle_dg/clustdb_raw_controlfile1_110m control2=/dev/vx/rdsk/oracle_dg/clustdb_raw_controlfile2_110m redo1_1=/dev/vx/rdsk/oracle_dg/clustdb_raw_log11_120m redo1_2=/dev/vx/rdsk/oracle_dg/clustdb_raw_log12_120m redo2_1=/dev/vx/rdsk/oracle_dg/clustdb_raw_log21_120m redo2_2=/dev/vx/rdsk/oracle_dg/clustdb_raw_log22_120m
You must specify that Oracle should use this file to determine the raw device volume names by setting the following environment variable where filename is the name of the ASCII file that contains the entries shown in the example above:
setenv DBCA_RAW_CONFIG filename or
export DBCA_RAW_CONFIG=filename
$ vi /var/opt/oracle/srvConfig.loc
srvconfig_loc=/dev/vx/rdsk/datadg/rac_srvconfig_10m
2.2 UNIX Preinstallation Steps
2.2.1 Group(dba) 생성 (각 node에서 실행) # groupadd –g 900 dba
2.2.2 User(oracle) 생성 (각 node에서 실행)
# useradd -g dba -u 900 -m -d /oracle -s /bin/ksh oracle # passwd oracle
2.2.3 remote shell(rcp, rlogin, rsh) 구성
Select the node from which you will run the Oracle Universal Installer (OUI) and establish user equivalence by adding an entry for all nodes in the cluster, including the local node, to the .rhosts file of the oracle account, or to the /etc/host.equiv file.
2.2.3.1 Set up user equivalence for the oracle account, to enable rsh, rcp, rlogin commands.
/etc/hosts (on all nodes)
127.0.0.1 localhost 61.250.123.231 sc1 loghost 61.250.123.232 sc2
172.16.0.129 inter1 172.16.0.130 inter2 /etc/hosts.equiv file.
Put the list of machines or nodes into hosts.equiv.
/etc/hosts.equiv (on all nodes) +sc1 root
+sc2 root +sc1 oracle +sc2 oracle +inter1 root +inter2 root +inter1 oracle +inter2 oracle .rhosts file.
In the root and oracle users’ home directory, put the list of machines into .rhosts.
[sc1:oracle:/oracle]% cat /oracle/.rhosts +
Note : It is possible, though not advised for security reasons, to put a “+” in the hosts.equiv and .rhosts files.
Test if the user equivalence is correctly set up (node2 is the secondary cluster machine) : Logged on node1 as oracle (then, as root) :
$ rlogin node2 (-> no pwd)
$ rcp /tmp/toto node2:/tmp/toto
$ rsh node2 pwd
2.2.3.2 remote shell test
[sc1:oracle:/oracle]% touch test
[sc1:oracle:/oracle]% rcp test sc2:/oracle [sc2:oracle:/oracle]% rsh sc2 ls -l total 16
drwx--- 2 root root 8192 Mar 5 05:04 lost+found -rw-r--r-- 1 oracle dba 0 Mar 5 11:15 test 2.2.4 System Kernel Parameters(각 node에 설정)
Kernel Parameter Setting Purpose
SHMMAX 4294967295 Maximum allowable size of one shared memory segment (4 Gb) SHMMIN 1 Minimum allowable size of a single shared memory segment.
SHMMNI 100 Maximum number of shared memory segments in the entire system.
SHMSEG 10 Maximum number of shared memory segments one process can attach.
SEMMNI 1024 Maximum number of semaphore sets in the entire system.
SEMMSL 100 Minimum recommended value. SEMMSL should be 10 plus the largest PROCESSES parameter of any Oracle database on the system.
SEMMNS 1024 Maximum semaphores on the system. This setting is a minimum recommended value. SEMMNS should be set to the sum of the
PROCESSES parameter for each Oracle database, add the largest one twice, plus add an additional 10 for each database.
SEMOPM 100 Maximum number of operations per semop call.
SEMVMX 32767 Maximum value of a semaphore.
(swap space) 750 MB Two to four times your system's physical memory size.
/etc/system (on all nodes)
set shmsys:shminfo_shmmax=4294967295 set shmsys:shminfo_shmmin=1
set shmsys:shminfo_shmmni=100 set shmsys:shminfo_shmseg=10
11-67 set semsys:seminfo_semmap=100
set semsys:seminfo_semmni=1024 set semsys:seminfo_semmns=1024 set semsys:seminfo_semmnu=2500 set semsys:seminfo_semmsl=100 set semsys:seminfo_semopm=100 set semsys:seminfo_semume=2500 set semsys:seminfo_semvmx=32767 set msgsys:msginfo_msgmax=16384 set msgsys:msginfo_msgmnb=16384 set msgsys:msginfo_msgmni=2200 set msgsys:msginfo_msgtql=2500 forceload: sys/shmsys
forceload: sys/semsys forceload: sys/msgsys
2.2.5 Establish system environment variables
• Set a local bin directory in the user's PATH, such as /usr/local/bin, or /opt/bin. It is necessary to have execute permissions on this directory.
• Set the DISPLAY variable to point to the system's (from where you will run OUI) IP address, or name, X server, and screen.
• Set a temporary directory path for TMPDIR with at least 20 Mb of free space to which the OUI has write permission.
2.2.5.1 Establish Oracle environment variables: Set the following Oracle environment variables:
Environment Variable Suggested value
ORACLE_BASE eg /u01/app/oracle
ORACLE_HOME eg /u01/app/oracle/product/9201
ORACLE_TERM xterm
NLS_LANG AMERICAN-AMERICA.UTF8 for example ORA_NLS33 $ORACLE_HOME/ocommon/nls/admin/data
PATH Should contain $ORACLE_HOME/bin
CLASSPATH $ORACLE_HOME/JRE:$ORACLE_HOME/jlib \
$ORACLE_HOME/rdbms/jlib: \
$ORACLE_HOME/network/jlib
• Create the directory /var/opt/oracle and set ownership to the oracle user.
• Verify the existence of the file /opt/SUNWcluster/bin/lkmgr. This is used by the OUI to indicate that the installation is being performed on a cluster.
[sc1:oracle:/oracle]% cat /oracle/.profile (on all nodes)
umask 022 set filec set -o trackall set -o vi stty erase ^H
stty -istrip cs8 erase ^H stty -istrip cs8 werase ^W stty -istrip cs8 intr ^C
export PATH=/usr/local/bin/:$PATH
export PATH=/bin:/usr/bin:/usr/sbin:/usr/ccs/bin:/usr/ucb:/usr/ucb/bin:/usr/local/bin:/etc:.
export PATH=/usr/openwin/bin:/usr/j2se/jre/bin:/usr/j2se/jre/lib:/usr/j2se/bin:$PATH export EDITOR=vi
export LANG=ko export ENV=./.kshrc
export PS1=[`hostname`:`whoami`:'$PWD]% ' alias df='df -k'
alias rm='rm -i' alias ll='ls -al'
alias dead='/usr/ucb/ps -aux | more' alias cls='clear'
alias ls='ls -aF'
alias ptd='/usr/platform/sun4u/sbin/prtdiag -v'
#export DISPLAY=61.250.123.158:0.0 export ORACLE_BASE=/oracle/app/oracle
export ORACLE_HOME=$ORACLE_BASE/product/920 export ORACLE_TERM=vt100
export ORACLE_SID=RAC1 # node2(sc2)에서는 “RAC2”로 설정 export TNS_ADMIN=$ORACLE_HOME/network/admin
export
PATH=$PATH:$ORACLE_HOME/bin:$ORACLE_HOME/ctx/bin:/tmp/OPatch:$ORACLE_HOME/Apatch/perl/bin export NLS_LANG=American_America.KO16KSC5601
export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data export
LD_LIBRARY_PATH=$ORACLE_HOME/lib32:$ORACLE_HOME/lib:/usr/lib:/usr/local/lib:$DAMO_INST_HOME export CLASSPATH=$ORACLE_HOME/jre/1.4.2/bin
#export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib:/usr/local/lib export DATE=d`date`+%Y%m%d export LANG=ko
export TMPDIR=/tmp alias oh='cd $ORACLE_HOME'
alias ob='cd $ORACLE_BASE/admin/RAC/bdump' alias ss='sqlplus "/as sysdba"'
13-67
※ Verification Script
(Note: 189256.1)Note: There is a verification script InstallPrep.sh available which may be downloaded and run prior to the installation of Oracle Real Application Clusters. This script verifies that the system is configured correctly according to the Installation Guide. The output of the script will report any further tasks that need to be performed before successfully installing Oracle 9.x DataServer (RDBMS). This script performs the following verifications:-
• ORACLE_HOME Directory Verification
• UNIX User/umask Verification
• UNIX Group Verification
• Memory/Swap Verification
• TMP Space Verification
• Real Application Cluster Option Verification
• Unix Kernel Verification
Unix InstallPrep Script
The Unix InstallPrep Script should be run prior to installing the Oracle 8.0.5 to 9.2 DataServer. This script checks for all known items that will cause the install to fail.
Follow these steps to download and run the script:
1. Change the name of the script to
InstallPrep.sh
2. Ftp the
InstallPrep.sh
script (in ascii format) to the Unix system that you will be installing the Oracle DataServer on.3. Change the permissions of the script to execute for the user running the script:
chmod 777 InstallPrep.sh
4. Log in as the user that will be installing the Oracle software.
5. Copy the InstallPrep.sh script to the /tmp directory and run it from this location.
6. Note: on some Linux platforms you may get the error "bad interpreter", just run the command 'sh InstallPrep.sh' and the try running it. Otherwise, you can try "dos2unix InstallPrep.sh" and then retry it.
7. Answer the first few questions, the output from the script will be written to
/tmp/InstallPrep.out
and the errors written to
/tmp/InstallPrep.err
.8. Resolve any problems found in
/tmp/InstallPrep.err
and then install the Oracle DataServer software.. ./InstallPrep.sh
3 Using the Oracle Universal Installer for Real Application Clusters 3.1 install the Oracle Software
Login as the oracle user
# su – oracle
[sc1:oracle:/Media/9i/Disk1]% export LANG=C [sc1:oracle:/Media/9i/Disk1]% ./runInstaller
• At the OUI Welcome screen, click Next.
15-67
• A prompt will appear for the Inventory Location (if this is the first time that OUI has been run on this system). This is the base directory into which OUI will install files. The Oracle Inventory definition can be found in the file /var/opt/oracle/oraInst.loc. Click OK.
• Unix Group Name
• Verify the UNIX group name of the user who controls the installation of the Oracle9i software. If an instruction to run /tmp/orainstRoot.sh appears, the pre-installation steps were not completed successfully. Typically, the /var/opt/oracle directory does not exist or is not writeable by oracle. Run /tmp/orainstRoot.sh to correct this, forcing Oracle Inventory files, and others, to be written to the ORACLE_HOME directory. Once again this screen only appears the first time Oracle9i products are installed on the system. Click Next.
• Select the other nodes on to which the Oracle RDBMS software will be installed. It is not necessary to select the node on which the OUI is currently running. Click Next.
• Cluster Node Selection : <SHIFT> 키를 누르면서 mouse 로 화면에 보이는 모든 node 들을 선택후에 Next 버튼을 누른다. % 이부분을 실행하면 동시에 각각의 노드에 oracle 이 설치된다.
17-67
• The File Location window will appear. Do NOT change the Source field. The Destination field defaults to the ORACLE_HOME environment variable. Click Next.
• Select the Products to install. In this example, select the Oracle9i Server then click Next.
• Product Language 를 선택한다
• Select the installation type. Choose the Custom option. Click Next.
19-67
• Oracle9i Real Application Cluster 를 체크하고, 기타 필요/불필요한 option 들을 추가/제거한다. click Next.
• Identify the raw partition in to which the Oracle9i Real Application Clusters (RAC) configuration information will be written. It is recommended that this raw partition is a minimum of 100MB in size.
• Raw Device 생성시에 생성한 Srvmconfig 파일명을 입력한다.
• Privileged Operating System Groups dba 를 입력한다.
• An option to Upgrade or Migrate an existing database is presented. Do NOT select the radio button. The Oracle Migration utility is not able to upgrade a RAC database, and will error if selected to do so.
21-67
• Create Database
No 를 선택하고, Next 를 클릭한다.
• The Summary screen will be presented. Confirm that the RAC database software will be installed and then click Install. The OUI will install the Oracle9i software on to the local node, and then copy this information to the other nodes selected.
• Once Install is selected, the OUI will install the Oracle RAC software on to the local node, and then copy software to the other nodes selected earlier. This will take some time. During the installation process, the OUI does not display messages indicating that components are being installed on other nodes - I/O activity may be the only indication that the process is continuing.
• A screen indicating the OUI progress will be displayed
• 99%에서 2 번 node 로 copy 한다.(cpio)
• ps 명령등으로 확인 할 수 있음.
23-67
※ RAC option 제거하기 (manually)
[oracle:/oracle/app/oracle/product/9.2.0/rdbms/lib] make -f ins_rdbms.mk rac_off rm -f /oracle/app/oracle/product/9.2.0/lib/libskgxp9.a
cp /oracle/app/oracle/product/9.2.0/lib//libskgxpd.a /oracle/app/oracle/product/9.2.0/lib/libskgxp9.a rm -f /oracle/app/oracle/product/9.2.0/lib/libskgxn9.a
cp /oracle/app/oracle/product/9.2.0/lib//libskgxns.a /oracle/app/oracle/product/9.2.0/lib/libskgxn9.a /bin/ar -X64 d /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a kcsm.o
/bin/ar -X64 cr /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a /oracle/app/oracle/product/9.2.0/rdbms/lib/ksnkcs.o
[oracle:/oracle/app/oracle/product/9.2.0/rdbms/lib] make -f ins_rdbms.mk ioracle - Linking Oracle
rm -f /oracle/app/oracle/product/9.2.0/rdbms/lib/oracle
ld -b64 -o /oracle/app/oracle/product/9.2.0/rdbms/lib/oracle -L/oracle/app/oracle/product/9.2.0/rdbms/lib/ - L/oracle/app/oracle/product/9.2.0/lib/ -bbigtoc -bnoipath -bI:/oracle/app/oracle/product/9.2.0/lib/ksms.imp /oracle/app/oracle/product/9.2.0/rdbms/lib/opimai.o /oracle/app/oracle/product/9.2.0/rdbms/lib/ssoraed.o /oracle/app/oracle/product/9.2.0/rdbms/lib/ttcsoi.o /oracle/app/oracle/product/9.2.0/lib/nautab.o /oracle/app/oracle/product/9.2.0/lib/naeet.o /oracle/app/oracle/product/9.2.0/lib/naect.o /oracle/app/oracle/product/9.2.0/lib/naedhs.o /oracle/app/oracle/product/9.2.0/rdbms/lib/config.o -lserver9 /oracle/app/oracle/product/9.2.0/lib/libodm9.so -lskgxp9 -lskgxn9 -lclient9 -lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 /oracle/app/oracle/product/9.2.0/rdbms/lib/defopt.o -lknlopt `if /bin/ar -X64 tv /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a | grep xsyeolap.o > /dev/null 2>&1 ; then echo "-loraolap9 - bE:/oracle/app/oracle/product/9.2.0/rdbms/lib/olap.exp -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib/libamd.exp" ; fi` - lslax9 -lpls9 -lplp9 -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib/plsqlncomp.exp -lserver9 -lclient9 -lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lknlopt -lslax9 -lpls9 -lplp9 -ljox9 -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib//oracle.exp - lwwg9 `sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lnro9
`sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lclient9 - lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lwtcserver9 -lmm -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 - lnls9 `sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lnro9
`sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lclient9 - lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lpls9 -ltrace9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 -lnls9 - lclient9 -lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 -lnls9 - lserver9 `if /bin/ar -X64 tv /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null 2>&1 ; then echo " " ; else echo "-lordsdo9"; fi` -lctxc9 -lctx9 -lzx9 -lgx9 -lctx9 -lzx9 -lgx9 -lordimt9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 - lxml9 -lcore9 -lunls9 -lnls9 -lsnls9 -lunls9 -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib//libcorejava.exp -lld -lm `cat /oracle/app/oracle/product/9.2.0/lib/sysliblist` -lm `if [ "\`/usr/bin/uname -v\`" = "4" ]; \
then echo "-bI:/oracle/app/oracle/product/9.2.0/lib/pw-syscall.exp"; fi;` `if /bin/ar -X64 t /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a | grep '^'kcsm.o > /dev/null 2>&1 ; then echo "-lha_gs64_r -lha_em_r";
fi` -locijdbcst9 -lxsd9
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_ADT is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_CHAR is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_DATETIME is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_INDEXED is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_INTERVAL is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_LOB is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_OBJREF is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_OPQ is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_SSCALAR is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_UROWID is already exported.
ld: 0711-415 WARNING: Symbol plzcls is already exported.
ld: 0711-415 WARNING: Symbol plzexe is already exported.
ld: 0711-415 WARNING: Symbol plzopn is already exported.
ld: 0711-415 WARNING: Symbol plzosq is already exported.
ld: 0711-415 WARNING: Symbol pevm_ENTER is already exported.
ld: 0711-415 WARNING: Symbol pevm_MOVX is already exported.
ld: 0711-415 WARNING: Symbol peslcl2 is already exported.
ld: 0711-319 WARNING: Exported symbol not defined: pen_PIPE ld: 0711-319 WARNING: Exported symbol not defined: pevm_MOVCADT ld: 0711-319 WARNING: Exported symbol not defined: pevm_MOVL ld: 0711-319 WARNING: Exported symbol not defined: pevm_MOVS ld: 0711-319 WARNING: Exported symbol not defined: pifi_i
ld: 0711-773 WARNING: Object /oracle/app/oracle/product/9.2.0/lib//libordsdo9.a[mdrcr.o], imported symbol kcbstdbz_
Symbol was expected to be local. Extra instructions are being generated to reference the symbol.
ld: 0711-773 WARNING: Object /oracle/app/oracle/product/9.2.0/lib//libordsdo9.a[mdrt.o], imported symbol kcbstdbz_
Symbol was expected to be local. Extra instructions are being generated to reference the symbol.
ld: 0711-783 WARNING: TOC overflow. TOC size: 87032 Maximum size: 65536 Extra instructions are being generated for each reference to a TOC
symbol if the symbol is in the TOC overflow area.
mv -f /oracle/app/oracle/product/9.2.0/bin/oracle /oracle/app/oracle/product/9.2.0/bin/oracleO mv /oracle/app/oracle/product/9.2.0/rdbms/lib/oracle /oracle/app/oracle/product/9.2.0/bin/oracle chmod 6751 /oracle/app/oracle/product/9.2.0/bin/oracle
※ RAC option 추가하기 (manually)
[oracle:/oracle/app/oracle/product/9.2.0/rdbms/lib] make -f ins_rdbms.mk rac_on rm -f /oracle/app/oracle/product/9.2.0/lib/libskgxp9.a
cp /oracle/app/oracle/product/9.2.0/lib//libskgxpu.a /oracle/app/oracle/product/9.2.0/lib/libskgxp9.a - Use reference SKGXN library
rm -f /oracle/app/oracle/product/9.2.0/lib/libskgxn9.a
cp /oracle/app/oracle/product/9.2.0/lib//libskgxnr.a /oracle/app/oracle/product/9.2.0/lib/libskgxn9.a /bin/ar -X64 d /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a ksnkcs.o
/bin/ar -X64 cr /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a /oracle/app/oracle/product/9.2.0/rdbms/lib/kcsm.o
[oracle:/oracle/app/oracle/product/9.2.0/rdbms/lib] make -f ins_rdbms.mk ioracle - Linking Oracle
rm -f /oracle/app/oracle/product/9.2.0/rdbms/lib/oracle
ld -b64 -o /oracle/app/oracle/product/9.2.0/rdbms/lib/oracle -L/oracle/app/oracle/product/9.2.0/rdbms/lib/ - L/oracle/app/oracle/product/9.2.0/lib/ -bbigtoc -bnoipath -bI:/oracle/app/oracle/product/9.2.0/lib/ksms.imp /oracle/app/oracle/product/9.2.0/rdbms/lib/opimai.o /oracle/app/oracle/product/9.2.0/rdbms/lib/ssoraed.o /oracle/app/oracle/product/9.2.0/rdbms/lib/ttcsoi.o /oracle/app/oracle/product/9.2.0/lib/nautab.o /oracle/app/oracle/product/9.2.0/lib/naeet.o /oracle/app/oracle/product/9.2.0/lib/naect.o /oracle/app/oracle/product/9.2.0/lib/naedhs.o /oracle/app/oracle/product/9.2.0/rdbms/lib/config.o -lserver9 /oracle/app/oracle/product/9.2.0/lib/libodm9.so -lskgxp9 -lskgxn9 -lclient9 -lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 /oracle/app/oracle/product/9.2.0/rdbms/lib/defopt.o -lknlopt `if /bin/ar -X64 tv /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a | grep xsyeolap.o > /dev/null 2>&1 ; then echo "-loraolap9 - bE:/oracle/app/oracle/product/9.2.0/rdbms/lib/olap.exp -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib/libamd.exp" ; fi` - lslax9 -lpls9 -lplp9 -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib/plsqlncomp.exp -lserver9 -lclient9 -lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lknlopt -lslax9 -lpls9 -lplp9 -ljox9 -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib//oracle.exp - lwwg9 `sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lnro9
`sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lclient9 - lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lwtcserver9 -lmm -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 - lnls9 `sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lnro9
`sed -e 's/-ljava//g' /oracle/app/oracle/product/9.2.0/lib/ldflags` -lncrypt9 -lnsgr9 -lnzjs9 -ln9 -lnl9 -lnzjs9 -lclient9 - lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lpls9 -ltrace9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 -lnls9 - lclient9 -lvsn9 -lwtcserver9 -lcommon9 -lgeneric9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 -lxml9 -lcore9 -lunls9 -lnls9 -
25-67
lserver9 `if /bin/ar -X64 tv /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null 2>&1 ; then echo " " ; else echo "-lordsdo9"; fi` -lctxc9 -lctx9 -lzx9 -lgx9 -lctx9 -lzx9 -lgx9 -lordimt9 -lnls9 -lcore9 -lnls9 -lcore9 -lnls9 - lxml9 -lcore9 -lunls9 -lnls9 -lsnls9 -lunls9 -bE:/oracle/app/oracle/product/9.2.0/rdbms/lib//libcorejava.exp -lld -lm `cat /oracle/app/oracle/product/9.2.0/lib/sysliblist` -lm `if [ "\`/usr/bin/uname -v\`" = "4" ]; \
then echo "-bI:/oracle/app/oracle/product/9.2.0/lib/pw-syscall.exp"; fi;` `if /bin/ar -X64 t /oracle/app/oracle/product/9.2.0/rdbms/lib/libknlopt.a | grep '^'kcsm.o > /dev/null 2>&1 ; then echo "-lha_gs64_r -lha_em_r";
fi` -locijdbcst9 -lxsd9
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_ADT is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_CHAR is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_DATETIME is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_INDEXED is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_INTERVAL is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_LOB is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_OBJREF is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_OPQ is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_SSCALAR is already exported.
ld: 0711-415 WARNING: Symbol pevm_INMDH_INDEXED_UROWID is already exported.
ld: 0711-415 WARNING: Symbol plzcls is already exported.
ld: 0711-415 WARNING: Symbol plzexe is already exported.
ld: 0711-415 WARNING: Symbol plzopn is already exported.
ld: 0711-415 WARNING: Symbol plzosq is already exported.
ld: 0711-415 WARNING: Symbol pevm_ENTER is already exported.
ld: 0711-415 WARNING: Symbol pevm_MOVX is already exported.
ld: 0711-415 WARNING: Symbol peslcl2 is already exported.
ld: 0711-319 WARNING: Exported symbol not defined: pen_PIPE ld: 0711-319 WARNING: Exported symbol not defined: pevm_MOVCADT ld: 0711-319 WARNING: Exported symbol not defined: pevm_MOVL ld: 0711-319 WARNING: Exported symbol not defined: pevm_MOVS ld: 0711-319 WARNING: Exported symbol not defined: pifi_i
ld: 0711-773 WARNING: Object /oracle/app/oracle/product/9.2.0/lib//libordsdo9.a[mdrcr.o], imported symbol kcbstdbz_
Symbol was expected to be local. Extra instructions are being generated to reference the symbol.
ld: 0711-773 WARNING: Object /oracle/app/oracle/product/9.2.0/lib//libordsdo9.a[mdrt.o], imported symbol kcbstdbz_
Symbol was expected to be local. Extra instructions are being generated to reference the symbol.
ld: 0711-783 WARNING: TOC overflow. TOC size: 87536 Maximum size: 65536 Extra instructions are being generated for each reference to a TOC
symbol if the symbol is in the TOC overflow area.
mv -f /oracle/app/oracle/product/9.2.0/bin/oracle /oracle/app/oracle/product/9.2.0/bin/oracleO mv /oracle/app/oracle/product/9.2.0/rdbms/lib/oracle /oracle/app/oracle/product/9.2.0/bin/oracle chmod 6751 /oracle/app/oracle/product/9.2.0/bin/oracle
4. Create and configure your database
4.1 Create a RAC Database using the Oracle Database Configuration Assistant(DBCA)
• Verify that you correctly configured the shared disks for each tablespace (for non-cluster file system platforms)
• Create the database
• Configure the Oracle network services
• Start the database instances and listeners
Oracle Corporation recommends that you use the DBCA to create your database. This is because the DBCA preconfigured databases optimize your environment to take advantage of Oracle9i features such as the server parameter file and automatic undo management. The DBCA also enables you to define arbitrary tablespaces as part of the database creation process. So even if you have datafile requirements that differ from those offered in one of the DBCA templates, use the DBCA. You can also execute user-specified scripts as part of the database creation process.
The DBCA and the Oracle Net Configuration Assistant (NETCA) also accurately configure your Real Application Clusters environment for various Oracle high availability features and cluster administration tools.
Note: Prior to running the DBCA it may be necessary to run the NETCA tool or to manually set up your network files. To run the NETCA tool execute the command netca from the $ORACLE_HOME/bin directory. This will configure the necessary listener names and protocol addresses, client naming methods,
Net service names and Directory server usage. Also, it is recommended that the Global Services Daemon (GSD) is started on all nodes prior to running DBCA. To run the GSD execute the command gsd from the
$ORACLE_HOME/bin directory.
• DBCA will launch as part of the installation process, but can be run manually by executing the command dbca from the $ORACLE_HOME/bin directory on UNIX platforms. The RAC Welcome Page displays.
Choose Oracle Cluster Database option and select Next.
27-67
• The Operations page is displayed. Choose the option Create a Database and click Next.
• The Node Selection page appears. Select the nodes that you want to configure as part of the RAC database and click Next. If nodes are missing from the Node Selection then perform clusterware
diagnostics by executing the $ORACLE_HOME/bin/lsnodes -v command and analyzing its output. Refer to your vendor's clusterware documentation if the output indicates that your clusterware is not properly installed. Resolve the problem and then restart the DBCA.
• The Database Templates page is displayed. The templates other than New Database include datafiles.
Choose New Database and then click Next.
• The Show Details button provides information on the database template selected.
29-67
• DBCA now displays the Database Identification page. Enter the Global Database Name and Oracle System Identifier (SID). The Global Database Name is typically of the form name.domain, for example mydb.us.oracle.com while the SID is used to uniquely identify an instance (DBCA should insert a suggested SID, equivalent to name1 where name was entered in the Database Name field). In the RAC case the SID specified will be used as a prefix for the instance number. For example, MYDB, would become MYDB1, MYDB2 for instance 1 and 2 respectively.
• The Database Options page is displayed. Select the options you wish to configure and then choose Next. Note: If you did not choose New Database from the Database Template page, you will not see this screen.
• The Additional database Configurations button displays additional database features. Make sure both are checked and click OK.
• Select the connection options desired from the Database Connection Options page. Note: If you did not choose New Database from the Database Template page, you will not see this screen. Click Next.
31-67
• DBCA now displays the Initialization Parameters page. This page comprises a number of Tab fields.
Modify the Memory settings if desired and then select the File Locations tab to update information on the Initialization Parameters filename and location. Then click Next.
• The option Create persistent initialization parameter file is selected by default. If you have a cluster file system, then enter a file system name, otherwise a raw device name for the location of the server parameter file (spfile) must be entered. Then click Next.
• The button File Location Variables… displays variable information. Click OK.
33-67
• The button All Initialization Parameters… displays the Initialization Parameters dialog box. This box presents values for all initialization parameters and indicates whether they are to be included in the spfile to be created through the check box, included (Y/N). Instance specific parameters have an instance value in the instance column. Complete entries in the All Initialization Parameters page and select Close. Note: There are a few exceptions to what can be altered via this screen. Ensure all entries in the Initialization Parameters page are complete and select Next.
• DBCA now displays the Database Storage Window. This page allows you to enter file names for each tablespace in your database.
• The file names are displayed in the Datafiles folder, but are entered by selecting the Tablespaces icon, and then selecting the tablespace object from the expanded tree. Any names displayed here can be changed. A configuration file can be used, see section 3.2.1, (pointed to by the environment variable DBCA_RAW_CONFIG). Complete the database storage information and click Next.
• The Database Creation Options page is displayed. Ensure that the option Create Database is checked and click Finish.
35-67
• The DBCA Summary window is displayed. Review this information and then click OK.
• Once the Summary screen is closed using the OK option, DBCA begins to create the database according to the values specified.
37-67
4.2 Manually Creating Real Application Clusters Databases
==============================================================
Manual Database Creation steps for Real Application Clusters (137288.1)
==============================================================
Here are the steps to be followed to create a Real Application Clusters database:
1. Make a init<SID>.ora in your $ORACLE_HOME/dbs directory. To simplify, you can copy init.ora to init<SID>.ora and modify the file. Remember that your control file must be pointing to a pre-existing raw device or cluster file system location.
*** Path names, file names, and sizes will need to be modified Example parameter settings for the first instance:
Cluster-Wide Parameters for Database "RAC":
db_block_size=8192 db_cache_size=52428800
background_dump_dest=/oracle/app/oracle/product/9.0.1/rdbms/log core_dump_dest=/oracle/oracle/product/9.0.1/rdbms/log
user_dump_dest=/oracle/oracle/product/9.0.1/rdbms/log timed_statistics=TRUE
control_files=("/dev/RAC/control_01.ctl", "/dev/RAC/control_02.ctl") db_name=RAC
shared_pool_size=52428800 sort_area_size=524288 undo_management=AUTO cluster_database=true
cluster_database_instances=2 remote_listener=LISTENERS_RAC
Instance Specific Parameters for Instance "RAC1":
instance_name=RAC1 instance_number=1
local_listener=LISTENER_RAC1 thread=1
undo_tablespace=UNDOTBS
* The local_listener parameter requires that you first add the listener
address to the TNSNAMES.ORA - remember to do so on both Node 1 and Node 2.
** You can also use an spfile as described in Note 136327.1.
2. Run the following sqlplus command to connect to the database:
sqlplus '/ as sysdba'
3. Startup up the database in NOMOUNT mode:
SQL> startup nomount
4. Create the Database (All raw devices must be pre-created) :
*** Path names, file names, and sizes will need to be modified
CREATE DATABASE <db_name>
CONTROLFILE REUSE MAXDATAFILES 254 MAXINSTANCES 32 MAXLOGHISTORY 100 MAXLOGMEMBERS 5 MAXLOGFILES 64
DATAFILE '/dev/RAC/system_01_400.dbf' SIZE 400M
REUSE AUTOEXTEND ON NEXT 10240K MAXSIZE UNLIMITED UNDO TABLESPACE "UNDOTBS" DATAFILE
'/dev/RAC/undotbs_01_210.dbf' SIZE 200M REUSE NEXT 5120K MAXSIZE UNLIMITED
CHARACTER SET US7ASCII
LOGFILE GROUP 1 ('/dev/RAC/redo1_01_100.dbf') SIZE 100M REUSE, GROUP 2 ('/dev/RAC/redo1_02_100.dbf') SIZE 100M REUSE;
5. Create a Users Tablespace:
*** Path names, file names, and sizes will need to be modified CREATE TABLESPACE "USERS" LOGGING DATAFILE '/dev/RAC/users_01_125.dbf' SIZE 120M REUSE
NEXT 1280K MAXSIZE UNLIMITED EXTENT MANAGEMENT LOCAL;
6. Create a Temporary Tablespace:
*** Path names, file names, and sizes will need to be modified CREATE TEMPORARY TABLESPACE "TEMP" TEMPFILE '/dev/RAC/temp_01_50.dbf' SIZE 40M REUSE
7. Create a 2nd Undo Tablespace:
*** Path names, file names, and sizes will need to be modified CREATE UNDO TABLESPACE "UNDOTBS2" DATAFILE '/dev/RAC/undotbs_02_210.dbf' SIZE 200M REUSE NEXT 5120K MAXSIZE UNLIMITED;
8. Run the necessary scripts to build views, synonyms, etc.:
The primary scripts that you must run are:
i> CATALOG.SQL--creates the views of data dictionary tables and the dynamic performance views
SQL> @?/rdbms/admin/catalog.sql
ii> CATPROC.SQL--establishes the usage of PL/SQL functionality and creates many of the PL/SQL Oracle supplied packages
SQL> @?/rdbms/admin/catproc.sql
iii> CATCLUST.SQL--creates RAC specific views SQL> @?/rdbms/admin/catclust.sql
39-67
9. Edit init<SID>.ora and set appropriate values for the 2nd instance on the 2nd Node:
*** Names may need to be modified
instance_name=RAC2 instance_number=2
local_listener=LISTENER_RAC2 thread=2
undo_tablespace=UNDOTBS2
10. From the first instance, run the following command:
*** Path names, file names, and sizes will need to be modified RAC1-SQL> alter database add logfile thread 2
group 3 ('/dev/RAC/redo2_01_100.dbf') size 100M, group 4 ('/dev/RAC/redo2_02_100.dbf') size 100M;
RAC1-SQL> alter database enable public thread 2;
11. Start the second Instance. (Assuming that your cluster configuration is up and running).
RAC2-SQL> startup
5. Single Database에서 RAC 구현하기 (Node 추가하기)
5.1 shutdown database / stop listener 5.2 network (rsh, rcp, rlogin) 구성 /etc/hosts.equive
+
~$HOME/.rhosts (all node) +
5.2.1 rcp test
[RAC1:/oracle] $ rcp test node2:/oracle [RAC1:/oracle] $ rlogin node2
5.3 rac option 추가(using OUI) 및 node2로 rcp
5.3.1 node1에서 OUI 실행 후, “Available Product Components”에서 RAC 만 추가 설치
5.3.2 rcp 를 이용해서 node1의 product를 node2로 copy 한다.
[RAC1:/oracle] $ rcp –r /oracle/* node2:/oracle 5.4 node1 init<SID>.ora 수정 및 clustered database startup
5.4.1 maxinstance 수를 체크하여 1로 되어있으면, 필요한 node수 이상으로 controlfile을 재생성 한다.
5.4.2 init<SID>.ora 수정
$ORACLE_HOME/initRAC1.ora
################################################################
41-67
# Cluster Database
################################################################
cluster_database_instances=1 #cluster_database_instances=2 cluster_database=true #cluster_database=true
instance_name=RAC1 #instance_name=RAC2 instance_number=1 #instance_number=2
#local_listener=LISTENER_RAC1 ##local_listener=LISTENER_RAC2 thread=1 #thread=2
undo_tablespace=UNDOTBS1 #undo_tablespace=UNDOTBS2
cluster_interconnects = 192.168.2.102 #cluster_interconnects = 192.168.2.202 log_archive_dest=/ARCH/RAC1 #log_archive_dest=/ARCH/RAC2
5.5 node2 (thread 2) 추가(undo tablespace, Online redologfile 생성, archive dest 지정(file system)) 5.5.1 Thread 2 용 redolog 생성
RAC1-SQL> alter database add logfile thread 2 group 4 '/dev/rrac_redo2_01' size 100m;
RAC1-SQL> alter database add logfile thread 2 group 5 '/dev/rrac_redo2_02' size 100m;
RAC1-SQL> alter database add logfile thread 2 group 6 '/dev/rrac_redo2_03' size 100m;
5.5.2 undotbs02 용 Undo tablespace 생성
RAC1-SQL> create undo tablespace undotbs2 datafile '/dev/rrac_undotbs02'
size 500M reuse AUTOEXTEND ON NEXT 5120K MAXSIZE UNLIMITED;
5.5.3 Thread 2 enable
RAC1-SQL> alter database enable thread 2;
5.6 RAC용 Data Dictionary 생성
RAC1-SQL> @?/rdbms/admin/catclust.sql 5.7 node2 Database startup
5.7.1 init<SID>.ora 수정
$ORACLE_HOME/initRAC1.ora
################################################################
# Cluster Database
################################################################
cluster_database_instances=2 #cluster_database_instances=1 cluster_database=true #cluster_database=true
instance_name=RAC2 #instance_name=RAC1 instance_number=2 #instance_number=1
#local_listener=LISTENER_RAC2 ##local_listener=LISTENER_RAC1 thread=2 #thread=1
undo_tablespace=UNDOTBS2 #undo_tablespace=UNDOTBS1
cluster_interconnects = 192.168.2.202 #cluster_interconnects = 192.168.2.102 log_archive_dest=/ARCH/RAC2 #log_archive_dest=/ARCH/RAC1
RAC2-SQL> startup
6. Administering Real Application Clusters Instances
Oracle Corporation recommends that you use SRVCTL to administer your Real Application Clusters database environment. SRVCTL manages configuration information that is used by several Oracle tools. For example, Oracle Enterprise Manager and the Intelligent Agent use the configuration information that SRVCTL generates to discover and monitor nodes in your cluster. Before using SRVCTL, ensure that your Global Services Daemon (GSD) is running after you configure your database. To use SRVCTL, you must have already created the configuration information for the database that you want to administer. You must have done this either by using the Oracle Database Configuration Assistant (DBCA), or by using the srvctl add command as described below.
If this is the first Oracle9i database created on this cluster, then you must initialize the clusterwide SRVM configuration. Firstly, create or edit the file /var/opt/oracle/srvConfig.loc file and add the entry
srvconfig_loc=path_name.where the path name is a small cluster-shared raw volume eg
$ vi /var/opt/oracle/srvConfig.loc
srvconfig_loc=/dev/vx/rdsk/datadg/rac_srvconfig_10m
Then execute the following command to initialize this raw volume (Note: This cannot be run while the gsd is running. Prior to 9i Release 2 you will need to kill the .../jre/1.1.8/bin/... process to stop the gsd from running.
From 9i Release 2 use the gsdctl stop command):-
$ srvconfig -init
The first time you use the SRVCTL Utility to create the configuration, start the Global Services Daemon (GSD) on all nodes so that SRVCTL can access your cluster's configuration information. Then execute the srvctl add command so that Real Application Clusters knows what instances belong to your cluster using the following syntax:-
For Oracle RAC v9.0.1:-
$ gsd
Successfully started the daemon on the local node.
$ srvctl add db -p db_name -o oracle_home Then for each instance enter the command:
$ srvctl add instance -p db_name -i sid -n node
To display the configuration details for, example, databases racdb1/2, on nodes racnode1/2 with instances racinst1/2 run:-
$ srvctl config racdb1
racdb2
$ srvctl config -p racdb1 racnode1 racinst1
racnode2 racinst2
43-67
$ srvctl config -p racdb1 -n racnode1 racnode1 racinst1
Examples of starting and stopping RAC follow:-
$ srvctl start -p racdb1
Instance successfully started on node: racnode2 Listeners successfully started on node: racnode2 Instance successfully started on node: racnode1 Listeners successfully started on node: racnode1
$ srvctl stop -p racdb2
Instance successfully stopped on node: racnode2 Instance successfully stopped on node: racnode1 Listener successfully stopped on node: racnode2 Listener successfully stopped on node: racnode1
$ srvctl stop -p racdb1 -i racinst2 -s inst Instance successfully stopped on node: racnode2
$ srvctl stop -p racdb1 -s inst
PRKO-2035 : Instance is already stopped on node: racnode2 Instance successfully stopped on node: racnode1
For Oracle RAC v9.2.0+:-
$ gsdctl start
Successfully started GSD on the local node.
$ srvctl add database -d db_name -o oracle_home [-m domain_name] [-s spfile]
Then for each instance enter the command:
$ srvctl add instance -d db_name -i sid -n node
To display the configuration details for, example, databases racdb1/2, on nodes racnode1/2 with instances racinst1/2 run:-
$ srvctl config racdb1
racdb2
$ srvctl config -p racdb1 -n racnode1
racnode1 racinst1 /u01/app/oracle/product/9.2.0.1
$ srvctl status database -d racdb1
Instance racinst1 is running on node racnode1 Instance racinst2 is running on node racnode2
Examples of starting and stopping RAC follow:-
$ srvctl start database -d racdb2
$ srvctl stop database -d racdb2
$ srvctl stop instance -d racdb1 -i racinst2
$ srvctl start instance -d racdb1 -i racinst2
$ gsdctl stat
GSD is running on local node
$ gsdctl stop
For further information on srvctl and gsdctl see the Oracle9i Real Application Clusters Administration manual.
45-67
7. Network 구성 & CTF
(Connection Time Failover)vs TAF
(Transparent Application Failover)No. 17563
OPS의 TAF (TRANSPARENT APPLICATION FAILOVER) 개념 및 구성 (8.1이상)
===================================================================
PURPOSE ---
Oracle8 부터는 OPS node 간의 TAF (Transparent Application Fail-over)가 제공된다. 즉 OPS의 한쪽 node에 fail이 발생하여도 해당 node로 접속하여 사용하던 모든 session이 사용하던 session을 잃지 않고 자동으로 정상적인 node로의 재접속이 이루어저 작업이 계속 진행하도록 하는 것이다.
이 문서에는 이 TAF에 대해서 간단히 살펴보고 실제 configuration을 기술한다.
Explanation ---
TAF가 cover하는 fail의 형태에 대한 설명과, TAF 시 지정하는 fail over의 type과 method에 대해서 설명한다.
(1) fail의 형태:
TAF는 다음과 같은 fail에 대해서 모두 TAF가 정상적으로 수행되게 된다.
단 MTS mode에 대해서는 전혀 문제가 없지만, dedicated mode의 경우는 반드시 dynamic registration형 태로 구현이 되어야 정상적으로 TAF가 가능하다.
instance fail: mts의 경우는 문제가 없지만 dedicated mode의 경우는 반드시 dynamic registration 형태로 구성되어야 한다. fail된 instance 측의 listener가 정상적이라 하더라도, dynamic registration에 의해서 instance가 fail되면 listener로부터 deregistration되게 되어 listener 정보를 확인 후 다른 node의 listener로 접속을 시도하게 된다.
그러나 dynamic registration을 사용하지 않게 되면 fail된 instance 쪽의 listener는 fail된 instance 정보를 services로 보여주게 되고 해당 instance와 연결을 시도하면서 ORA-1034:
Oracle not available 오류가 발생하게 되는 것이다.
instance & listener down: listener까지 down되게 되면 문제 발생 후 재접속 시도 시 fail된 쪽의 listener 접속이 실패하게 되고, 다른 node의 listener로 접속이 이루어지게 된다.
node down: node 자체가 down되는 경우에도 TAF는 이루어진다. 단 clinet에 적정한 TCP configuration parameter인 keepalive 의 설정이 요구되어진다.
node fail시 client와 server간의 작업이 진행중이라면 문제가 없지만 만약 server쪽에서 수행 되는 작업이 없는 상태라면 cleint가 node가 down이 되어도 바로 인지할 수가 없다. client에 서 다음 server로의 요청이 이루어지는 순간에 client가 더이상 존재하지 않는 TCP end point 쪽으로 TCP packet을 보내게 되고, server node가 더이상 살아있지 않다는것을 확인하게 되 는데 일반적으로 2,3분이 걸릴수 있다. node가 fail이 된경우 network에 대한 write() function call이 오류를 return하게 되고, 이것을 client가 받은후 failover기능을 호출하게 되는 것이다.
client에서 idle한 상태에서도 server node가 down되었는지를 확인하려면 TCP keepalive를 설 정해야 하며, 이 keepalive를 오라클의 connection에서 사용하려면 TNS service name에서 ENABLE=BROKEN절을 지정해 주어야한다.
DESCRIPTION절에 포함되는 이 ENABLE=BROKEN절에 대한 예제는 아래 구성 예제의 (3)번 tnsnames.ora 구성 부분에서 참조할 수 있다.
이렇게 ENABLE=BROKEN을 지정하면 network쪽 configuration인 keepalive 설정을 이용하게 되는데 이것이 일반적으로는 2 ~ 3시간으로 설정되어 있기 때문에 이값이 적당히 짧아야 TAF에서 의미가 있을 수 있다.
단 이 keepalive time이 너무 짧으면, 그리고 idle한 session이 많은 편이라면 network부하가 매우 증가할 수 있으므로 이 지정에 대해서는 os나 network administrator와 충분히 상의하여 야 한다.
이 keepalive 대한 자세한 내용과 설정 방법은 <bulletin:11323 : SQL*NET DCD(DEAD CONNECTION DETECTION)과 KEEPALIVE의 관계>를 참조한다.
(2) type: session vs. select
session은 유지하고 수행중이던 SQL문장은 모두 fail되는 session type과 DML문장은 rollback되고 select문장은 유지되는 select type이 제공된다.
select type의 경우도 fail된 instance에서만 얻을 수 있는 정보의 경우는 조회수행 도중 다음과 같은 오 류를 발생시키고 중단될 수 있다.
예를 들어 해당 instance에 대한 gv$session으로부터의 조회와 같은것이 그 예이다.
ORA-25401: can not continue fetches (3) method: basic vs. backup
fail발생시 다른 node로 session을 연결하는 basic method와, 미리 다른 node로 backup session을 연결 해 두었다가 fail발생시 사용하는 backup method가 존재한다.
Example ---
TAF설정을 위해서는 init.ora, listener.ora, tnsnames.ora에 설정이 필요하다.
MTS mode에서는 문제가 없기 때문에 여기서는 반드시 dynamic registration으로 설정해야 하는 dedicated방 식을 예로 들었다.
test는 Oracle 8.1.7.4/Sun solaris 2.8에서 수행되었다.
A/B 두 node를 가정한다.
(1)initSID.ora에서
- A node의 initSID.ora service_names=INS1, DB1
local_listener="(address=(protocol=TCP)(host=krtest1)(port=1521))"
- B node의 initSID.ora service_names=INS2, DB1
local_listener="(address=(protocol=TCP)(host=krtest2)(port=1521))"
service_names는 여러개를 지정가능한데, 중요한것은 두 node가 공통으로 사용할 service name한가지는
47-67 반드시 지정하여야 한다.
일반적으로 db_name을 지정하면 된다.
host=부분은 hostname이나 ip address를 지정하면 된다.
(2) listener.ora LISTENER =
(DESCRIPTION = (ADDRESS =
(PROTOCOL = tcp)
(HOST = krtest1)(PORT= 1521)))
B node에서는 krtest1대신 b node의 hostname혹은 ip address를 지정하면 된다
(3) tnsnames.ora은 지정하는 방법이 두가지입니다.
아래에 basic method와 backup method 두 가지 방법에 대한 예를 모두 기술한다.
이중 한가지를 사용하면 되며 backup method의 fail-over시 미리 연결된 session을 사용하므로 시간이 적게 걸릴수 있으나 반대 node에 사용안하는 session을 미리 맺어놓는것에 대한 부하가 있어 서로 장단점이 있 을 수 있다.
두 설정 모두 TAF뿐 아니라 connect time fail-over도 가능한 설정이다.
즉 A node가 fail시 같은 tns service name을 이용하여서 (여기서는 opsbasic 또는 ops1) B node로 접속이 이 루어진다.
address=로 정의된 address절이 위쪽을 먼저 시도하므로 정상적인 상태에서 B node로 접속을 원하는 경우 는 opsbasic의 경우 krtest2를 위쪽에 적고, ops1/ops2의 경우는 ops2를 사용하도록 한다.
여기에서 (enable=broken)설정이 되어 있는데 이것은 client machine에 설정되어 있는 TCP keepalive를 이용 하는 것으로 network부하를 고려하여 설정을 제거할 수 있다.
a. basic method
krtest1의 tnsnames.ora에서는 opsbasic과 ops2에 대해서 설정해두고, krtest2 node에서는 opsbasic과 ops1을 설정한 후, backup=ops2를 backup=ops1으로 수정하면 된다.
opsbasic = (description=
(address_list=
(enable=broken) (load_balance=off) (failover=on)
(address= (protocol=tcp) (host=krtest1) (port=1521)) (address= (protocol=tcp) (host=krtest2) (port=1521)) )
(connect_data =
(service_name=DB1) (failover_mode=
(type=select) (method=basic) (backup=ops2))))
