본문 바로가기

IBM/HACMP

PowerHA6.1 구성 절차

필요 소프트웨어와 버전

The software requirements for PowerHA 6.1 on 10 gR2 are as follows: 

 

SoftwareVersion and other details
AIX

5.3 TL10 SP5 or newer, must include fix for APAR IZ89268

6.1 TL4 SP7 or newer, must include fix for APAR IZ87768  

VIOS 2.2.0.10 FP24 SP1 or newer 
PowerHA6.1 SP3 or newer
RSCT 

For AIX 5.3, RSCT 2.4.12.0 or newer 

For AIX 6.1, RSCT 2.5.4.0 or newer

Oracle 10gR2

Oracle 10gR2 patchset 10.2.0.5 or newer is required.

Patch 6718715 should be installed on  all nodes before installing Oravcle

10.2.0.5 software. Be sure to download the 10gR2 version of <<patch 6718715.>>


The software requirements for PowerHA 6.1 on 11 gR1 are as follows: 

Software Version and other details 
AIX 

5.3 TL10 SP5 or newer, must include fix for APAR IZ89268

6.1 TL4 SP7 or newer, must include fix for APAR IZ87768  

VIOS2.2.0.10 FP24 SP1 or newer 
PowerHA6.1 SP3 or newer 
RSCT 

For AIX 5.3, RSCT 2.4.12.0 or newer

For AIX 6.1, RSCT 2.5.4.0 or newer 

Oracle 11gR1

Oracle 11gR1 patchset 11.1.0.7 or newer is required. 

rootpre.sh <<patch:6718715>> is required when using HACMP 5.4.1 with a fresh install of Oracle  RAC clusterware or when upgrading to Oracle RAC 11gR1. Be sure to download the 11gR1 version of <<patch:6718715>>. 


The software requirements for PowerHA 6.1 on 11 gR2 are as follows: 

Software Version and other details 
AIX

5.3 TL10 SP5 or newer, must include fix for APAR IZ89268 

6.1 TL4 SP7 or newer, must include fix for APAR IZ87768  

VIOS2.2.0.10 FP24 SP1 or newer
PowerHA6.1 SP3 or newer 
RSCT

For AIX 5.3, RSCT 2.4.12.0 or newer 

For AIX 6.1, RSCT 2.5.4.0 or newer

Oracle 11gR2 Oracle RAC 11.2.0.1


오라클 shared storage option

Oracle RAC 10g ~ 11gR1

Storage optionOCR and vote diskOracle SoftwareOracle database files
ASMNoNoYes
GPFSYesYesYes
Raw device (w/ PowerHA)YesNoYes
Local file systemNoYesNo

 

Oracle RAC 11gR2

Storage optionOCR and v ote diskOracle SoftwareOracle database f iles
ASMYesNoYes
GPFSYesYesYes
Raw device (w/ PowerHA)

Yes: Upgrade from previous versions

No: New install with OUI

No

Yes: Upgrade from previous versions

No: New install with OUI

Local file systemNoYesNo


Memory and Paging space Requirements

  • Memory
    1.5GB for Oracle Grid Infrastructure
    2.5GB for Oracle Grid Infrastructure and oracle RAC
  • Swap space
    Minimum : 1.5GB
    System with 2GB of RAM or less : 1.5 times the amount of RAM
    System with 2GB to 16GB of RAM : equal to RAM
    system with more than 16GB of RAM : 16GB

Consideration on OCR and vote devices

  • With normal redundancy
    3 OCR and 3 vote devices are needed
    each device needs at least 280MB in size
  • OCR and vote on raw devices are not supported from 11gR2
    ASM or cluster file system are supported

Operating System Requirements

  • Operating system
    - AIX 7.1 Base
    - AIX 6.1 TL02 SP1
    - AIX 5.3 TL09 SP1
  • Operting system filesets(for AIX6.1)
    openssh
    bos.adt.base
    bos.adt.lib
    bos.adt.lib
    bos.perf.libperfstat
    bos.perf.perfstat
    bos.perf.proctools
    rsct.basic.rte
    rsct.compat.clients.rte
    xlC.aix61.rte 10.1.0.0

Name Resolution

#cat /etc/hosts

127.0.0.1 loopback localhost

130.130.10.115 node15

130.130.10.116 node16

192.168.10.115 node15rac

192.168.10.116 node16rac

130.130.10.215 node15vip

130.130.10.216 node16vip

130.130.10.15 node15scan

  • Scan IP(Single client access name)
    - Host name to provied for all clients connecting to the cluster
    - The SCAN is a domain name registered to at least one and up to three IP addresses, either in the domain name service(DNS) or the Grid Naming Service(GNS)
    - The SCAN eliminates the need to change clients when nodes are added to or removed from the cluster

Configure OS kernel parameters

  • Tuning Virtual Memory Manager
    #vmo -p -o minperm%=3
    #vmo -p -o maxperm%=90
    #vmo -p -o maxclient%=90
    #vmo -p -o lru_file_repage=0
    #vmo -p -o strict_maxclient=1
    #vmo -p -o strict_maxperm=0
  • From AIX6.1, these are all default values

Configure AIX System Environment

Shell LimitItem in limits.confHard Limit
Maximum number of open file descriptorsnofile-1
Maximum number of processes available to a single usermaxuproc16384

/etc/security/limits

default :

fsize = -1

data = -1

core = -1

fsize_hard = -1

cpu_hard = -1

data_hard = -1

stack_hard = 65536

core_hard = -1

Tuning AIX System Environment

  • Configuring Network Tuning Parameters

    ParameterValue
    udp_sendspace65536
    udp_recvspace

    655360

    (udp_sendspace* 10)

    tcp_sendspace65536
    tcp_recvspace65536
    rfc13231
    sb_max4194304
    ipqmaxlen512

#no -p -o udp_sendspace=65536

#no -p -o udp_recvspace=655360

#no -p -o tcp_sendspace=65536

#no -p -o tcp_recvspace=65536

#no -p -o rfc1323=1

#no -p -o sb_max=4194304

#no -r -o ipqmaxlen=512

 

Users and Groups

  • Configuring users
    oragrid : create a user for installing the Oracle Grid Infrastructure software
    oracle : user for installing the Oracle RAC software and creating database

From 11g, run the following command as root

#chuser capabilities=CAP_NUMA_ATTACH, CAP_BYPASS_RAC_VMM, CAP_PROPAGATE oragrid

#chuser capabilities=CAP_NUMA_ATTACH, CAP_BYPASS_RAC_VMM, CAP_PROPAGATE oracle

#lsuser -a capabilities oragrid (checking capabilities for oragrid)

  • Configuring groups
    dba : primary group for the oragrid and oracle user
    hagsuser : Required in PowerHA environment

Network Time Protocol

  • NTP in AIX
  1. 시간 동기화 서버 환경구성
    - edit /etc/ntp.conf
       server 127.127.1.0 prefer 추가(자신을 동기화 함)
    - stop ntpd and restart it with the -x option
       #startsrc -s xntpd -a "-x"
    - edit /etc/rc.tcpip to add the -x option to xntpd on all nodes
        start /usr/sbin/xntpd "$src_running" "-x"
    Note : -x option prevents time from being adjusted backward(Time backward를 막는다. 시간이 늦는 서버의 시간을 동기화 하는 것을 방지)
    - lssrc -ls xntpd 결과 값 확인

    root@v5:/>lssrc -ls xntpd

     

     Program name:    /usr/sbin/xntpd

     

     Version:         3

     

     Leap indicator:  00 (No leap second today.)

     

     Sys peer:        127.127.1.0

     

     Sys stratum:     4

     

     Sys precision:   -16

     

     Debug/Tracing:   DISABLED

     

     Root distance:   0.000000

     

     Root dispersion: 0.010010

     

     Reference ID:    127.127.1.0

     

     Reference time:  c52992e1.7c721000  Tue, Oct 26 2004 22:22:09.486

     

     Broadcast delay: 0.003906 (sec)

     

     Auth delay:      0.000122 (sec)

     

     System flags:    pll monitor filegen

     

     System uptime:   73593 (sec)

     

     Clock stability: 0.000000 (sec)

     

     Clock frequency: 0.000000 (sec)

     

      Peer: 127.127.1.0

     

          flags: (configured)(refclock)(sys peer)(preferred)

     

          stratum:  3, version: 3

     

          our mode: client, his mode: server

     

    Subsystem         Group            PID          Status

     

     xntpd            tcpip            352262       active

  2.  

    시간동기화 Client 환경 구성
    - xntpd는 Server와 Client 간 1000초(16분) 이상의 차이가 발생하면 더 이상 시간을 동기화 하지 않는다. date로 확인 후 시간차가 크면 근접한 시간으로 변경
    - edit /etc/ntp.conf
    server (server_name or ip)
    - edit /etc/rc.tcpip
    start /usr/sbin/xntpd "$src_running""-x" 주석 제거
    - xntpd 기동
    #startsrc -s xntpd -a "-x"
    -#lssrc -ls xntpd 결과값 확인

    root@v4:/> lssrc -ls xntpd

     

     Program name:    /usr/sbin/xntpd

     

     Version:         3

     

     Leap indicator:  11 (Leap indicator is insane.)

     

     Sys peer:        no peer, system is insane

     

     Sys stratum:     16

     

     Sys precision:   -16

     

     Debug/Tracing:   DISABLED

     

    Root distance:   0.000000

     

     Root dispersion: 0.000000

     

     Reference ID:    no refid, system is insane

     

     Reference time:  no reftime, system is insane

     

     Broadcast delay: 0.003906 (sec)

     

     Auth delay:      0.000122 (sec)

     

     System flags:    pll monitor filegen

     

     System uptime:   23 (sec)

     

     Clock stability: 0.000000 (sec)

     

     Clock frequency: 0.000000 (sec)

     

     Peer: v5.kr.ibm.com

     

          flags: (configured)(preferred)

     

          stratum:  4, version: 3

     

          our mode: client, his mode: server

     

    Subsystem         Group            PID     Status

     

     xntpd            tcpip            15002   active
    - ntpq -d로 확인

    root@v4:/> ntpq -p

         remote           refid      st t when   poll  reach   delay   offset    disp

    ========================================================================

    *v5.kr.ibm.com   LOCAL(0)       4 u  35   64     377     1.27   -3.345    0.53
    refid 가 LOCAL로 바뀌고 reach 가 377에 다다르면 정상(6분~7분)

  • Oracle CTSS
     - Installed in active mode if NTP daemon in AIX is not active
     - Started in observer mode if NTP is active
     - In order to confirm that ctssd is active, run the following command
       $crsctl stat resource ora.ctssd -t -init

Asynchronous I/O

  • AIX 6.1 or later
    - AIO device drivers are enabled by default
    - Increase the number of aioservier processes to 65536
       #ioo -p -o aio_maxreqs=65536
  • AIX 5.x
    - rootpre.sh enables AIO device driver
    - use smit aio change the maxreqs parameter

SSH configuration

  • node1 서버에서 /.ssh로 이동하여 #ssh-keygen -t dsa 명령으로 id_dsa, id_dsa.pub 생성
  • id_dsa.pub를 node2 서버의 /.ssh로 authorized_keys라는 이름으로 copy (rcp나 scp를 이용함. 방법은 여러가지)
  • node2 서버에서도 위와 같은 작업 수행
  • ssh로 접속 시 패스워드 물어 보지 않고 바로 접속 가능

 

GPFS 설정

'IBM > HACMP' 카테고리의 다른 글

HACMP_PD교육  (0) 2012.11.09
바뀐LUN사이즈를VG에적용할때  (0) 2012.11.09
PowerHA7.1교육  (0) 2012.11.09
How to synchronize timestamp  (0) 2012.11.09
zero-off-set  (0) 2012.03.24
Understanding_active_and_passive_varyon_in_enhanced_concurrent_mode  (0) 2011.08.24
Fast_disk_takeover  (0) 2011.08.24
Enhanced_concurrent_mode  (0) 2011.08.24
clstat 실행 시 error 날 때 해결책  (0) 2011.07.21
PowerHA 7.1 vs Veritas Cluster erver (VCS)  (0) 2011.07.20