반응형

사전작업
현 상태확인

[oracle@datarac1 grid]$ id
uid=54321(oracle) gid=54322(dba) groups=54322(dba) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[oracle@datarac1 grid]$ env | grep SID
ORACLE_SID=+ASM1
[oracle@datarac1 grid]$ crsctl status res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
               ONLINE  ONLINE       datarac1               STABLE
               ONLINE  ONLINE       datarac2               STABLE
ora.chad
               ONLINE  ONLINE       datarac1               STABLE
               ONLINE  ONLINE       datarac2               STABLE
ora.net1.network
               ONLINE  ONLINE       datarac1               STABLE
               ONLINE  ONLINE       datarac2               STABLE
ora.ons
               ONLINE  ONLINE       datarac1               STABLE
               ONLINE  ONLINE       datarac2               STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
      1        ONLINE  ONLINE       datarac1               STABLE
      2        ONLINE  ONLINE       datarac2               STABLE
      3        ONLINE  OFFLINE                               STABLE
ora.DATA1.dg(ora.asmgroup)
      1        ONLINE  ONLINE       datarac1               STABLE
      2        ONLINE  ONLINE       datarac2               STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       datarac2               STABLE
ora.OCRVOTE.dg(ora.asmgroup)
      1        ONLINE  ONLINE       datarac1               STABLE
      2        ONLINE  ONLINE       datarac2               STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asm(ora.asmgroup)
      1        ONLINE  ONLINE       datarac1               Started,STABLE
      2        ONLINE  ONLINE       datarac2               Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
      1        ONLINE  ONLINE       datarac1               STABLE
      2        ONLINE  ONLINE       datarac2               STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       datarac2               STABLE
ora.datarac.db
      1        ONLINE  ONLINE       datarac1               Open,HOME=/u01/app/o
                                                             racle/product/19.0.0
                                                             /db_1,STABLE
      2        ONLINE  ONLINE       datarac2               Open,HOME=/u01/app/o
                                                             racle/product/19.0.0
                                                             /db_1,STABLE
ora.datarac1.vip
      1        ONLINE  ONLINE       datarac1               STABLE
ora.datarac2.vip
      1        ONLINE  ONLINE       datarac2               STABLE
ora.qosmserver
      1        ONLINE  ONLINE       datarac2               STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       datarac2               STABLE
--------------------------------------------------------------------------------
[oracle@datarac1 grid]$

전체 Node /etc/hosts 파일 내용 추가

[oracle@datarac1 grid]$ cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
#Real
192.168.56.201  datarac1.localdomain  datarac1
192.168.56.202  datarac2.localdomain  datarac2
192.168.56.203  datarac3.localdomain  datarac3
#Interconnect
100.100.100.201 datarac1-priv.localdomain     datarac1-priv
100.100.100.202 datarac2-priv.localdomain     datarac2-priv
100.100.100.203 datarac3-priv.localdomain     datarac3-priv
#VIP
192.168.56.211  datarac1-vip.localdomain      datarac1-vip
192.168.56.212  datarac2-vip.localdomain      datarac2-vip
192.168.56.213  datarac3-vip.localdomain      datarac3-vip
#SCAN
192.168.56.215  datarac-scan.localdomain      datarac-scan
[oracle@datarac1 grid]$

Passwordless SSH Setting

[oracle@datarac1 deinstall]$ id
uid=54321(oracle) gid=54322(dba) groups=54322(dba) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[oracle@datarac1 deinstall]$ env | grep SID
ORACLE_SID=+ASM1
[oracle@datarac1 deinstall]$ pwd
/u01/app/19.0.0/grid/deinstall
[oracle@datarac1 deinstall]$ ls
bootstrap_files.lst  bootstrap.pl  deinstall  deinstall.pl  deinstall.xml  jlib  readme.txt  response  sshUserSetup.sh  utl
[oracle@datarac1 deinstall]$
[oracle@datarac1 deinstall]$
[oracle@datarac1 deinstall]$
[oracle@datarac1 deinstall]$ ./sshUserSetup.sh -user oracle -hosts "datarac1 datarac2 datarac3" -noPromptPassphrase -advanced
The output of this script is also logged into /tmp/sshUserSetup_2023-05-22-16-07-46.log
Hosts are datarac1 datarac2 datarac3
user is oracle
Platform:- Linux
Checking if the remote hosts are reachable
PING datarac1.localdomain (192.168.56.201) 56(84) bytes of data.
64 bytes from datarac1.localdomain (192.168.56.201): icmp_seq=1 ttl=64 time=0.073 ms
64 bytes from datarac1.localdomain (192.168.56.201): icmp_seq=2 ttl=64 time=0.079 ms
64 bytes from datarac1.localdomain (192.168.56.201): icmp_seq=3 ttl=64 time=0.060 ms
64 bytes from datarac1.localdomain (192.168.56.201): icmp_seq=4 ttl=64 time=1.03 ms
64 bytes from datarac1.localdomain (192.168.56.201): icmp_seq=5 ttl=64 time=0.046 ms

--- datarac1.localdomain ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 116ms
rtt min/avg/max/mdev = 0.046/0.257/1.031/0.387 ms
PING datarac2.localdomain (192.168.56.202) 56(84) bytes of data.
64 bytes from datarac2.localdomain (192.168.56.202): icmp_seq=1 ttl=64 time=0.619 ms
64 bytes from datarac2.localdomain (192.168.56.202): icmp_seq=2 ttl=64 time=0.531 ms
64 bytes from datarac2.localdomain (192.168.56.202): icmp_seq=3 ttl=64 time=0.393 ms
64 bytes from datarac2.localdomain (192.168.56.202): icmp_seq=4 ttl=64 time=0.622 ms
64 bytes from datarac2.localdomain (192.168.56.202): icmp_seq=5 ttl=64 time=0.620 ms

--- datarac2.localdomain ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 111ms
rtt min/avg/max/mdev = 0.393/0.557/0.622/0.089 ms
PING datarac3.localdomain (192.168.56.203) 56(84) bytes of data.
64 bytes from datarac3.localdomain (192.168.56.203): icmp_seq=1 ttl=64 time=1.12 ms
64 bytes from datarac3.localdomain (192.168.56.203): icmp_seq=2 ttl=64 time=0.513 ms
64 bytes from datarac3.localdomain (192.168.56.203): icmp_seq=3 ttl=64 time=0.399 ms
64 bytes from datarac3.localdomain (192.168.56.203): icmp_seq=4 ttl=64 time=0.516 ms
64 bytes from datarac3.localdomain (192.168.56.203): icmp_seq=5 ttl=64 time=0.438 ms

--- datarac3.localdomain ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 91ms
rtt min/avg/max/mdev = 0.399/0.596/1.117/0.265 ms
Remote host reachability check succeeded.
The following hosts are reachable: datarac1 datarac2 datarac3.
The following hosts are not reachable: .
All hosts are reachable. Proceeding further...
firsthost datarac1
numhosts 3
The script will setup SSH connectivity from the host datarac1.localdomain to all
the remote hosts. After the script is executed, the user can use SSH to run
commands on the remote hosts or copy files between this host datarac1.localdomain
and the remote hosts without being prompted for passwords or confirmations.

NOTE 1:
As part of the setup procedure, this script will use ssh and scp to copy
files between the local host and the remote hosts. Since the script does not
store passwords, you may be prompted for the passwords during the execution of
the script whenever ssh or scp is invoked.

NOTE 2:
AS PER SSH REQUIREMENTS, THIS SCRIPT WILL SECURE THE USER HOME DIRECTORY
AND THE .ssh DIRECTORY BY REVOKING GROUP AND WORLD WRITE PRIVILEGES TO THESE
directories.

Do you want to continue and let the script make the above mentioned changes (yes/no)?
yes

The user chose yes
User chose to skip passphrase related questions.
Creating .ssh directory on local host, if not present already
Creating authorized_keys file on local host
Changing permissions on authorized_keys to 644 on local host
Creating known_hosts file on local host
Changing permissions on known_hosts to 644 on local host
Creating config file on local host
If a config file exists already at /home/oracle/.ssh/config, it would be backed up to /home/oracle/.ssh/config.backup.
Creating .ssh directory and setting permissions on remote host datarac1
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host datarac1. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host datarac1.
Warning: Permanently added 'datarac1,192.168.56.201' (ECDSA) to the list of known hosts.
oracle@datarac1's password:
Done with creating .ssh directory and setting permissions on remote host datarac1.
Creating .ssh directory and setting permissions on remote host datarac2
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host datarac2. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host datarac2.
Warning: Permanently added 'datarac2,192.168.56.202' (ECDSA) to the list of known hosts.
Done with creating .ssh directory and setting permissions on remote host datarac2.
Creating .ssh directory and setting permissions on remote host datarac3
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host datarac3. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host datarac3.
Warning: Permanently added 'datarac3,192.168.56.203' (ECDSA) to the list of known hosts.
oracle@datarac3's password:
Done with creating .ssh directory and setting permissions on remote host datarac3.
Copying local host public key to the remote host datarac1
The user may be prompted for a password or passphrase here since the script would be using SCP for host datarac1.
oracle@datarac1's password:
Done copying local host public key to the remote host datarac1
Copying local host public key to the remote host datarac2
The user may be prompted for a password or passphrase here since the script would be using SCP for host datarac2.
Done copying local host public key to the remote host datarac2
Copying local host public key to the remote host datarac3
The user may be prompted for a password or passphrase here since the script would be using SCP for host datarac3.
oracle@datarac3's password:
Done copying local host public key to the remote host datarac3
Creating keys on remote host datarac1 if they do not exist already. This is required to setup SSH on host datarac1.

Creating keys on remote host datarac2 if they do not exist already. This is required to setup SSH on host datarac2.

Creating keys on remote host datarac3 if they do not exist already. This is required to setup SSH on host datarac3.
Generating public/private rsa key pair.
Your identification has been saved in .ssh/id_rsa.
Your public key has been saved in .ssh/id_rsa.pub.
The key fingerprint is:
SHA256:6HgRpPpDC5xRNJ/9dUuP3DUBykwkBtyBZiWzed92AA8 oracle@datarac3.localdomain
The key's randomart image is:
+---[RSA 1024]----+
|   .+ o+==oE ... |
|   . = B*.= =   .|
|  . . *o.. +.oo..|
| . +   o.....+.=o|
|  = . o S .. o+.o|
|   + + .    . .  |
|    = o          |
|     o           |
|                 |
+----[SHA256]-----+
Updating authorized_keys file on remote host datarac1
Updating known_hosts file on remote host datarac1
Updating authorized_keys file on remote host datarac2
Updating known_hosts file on remote host datarac2
Updating authorized_keys file on remote host datarac3
Updating known_hosts file on remote host datarac3
cat: /home/oracle/.ssh/known_hosts.tmp: No such file or directory
cat: /home/oracle/.ssh/authorized_keys.tmp: No such file or directory
SSH setup is complete.

------------------------------------------------------------------------
Verifying SSH setup
===================
The script will now run the date command on the remote nodes using ssh
to verify if ssh is setup correctly. IF THE SETUP IS CORRECTLY SETUP,
THERE SHOULD BE NO OUTPUT OTHER THAN THE DATE AND SSH SHOULD NOT ASK FOR
PASSWORDS. If you see any output other than date or are prompted for the
password, ssh is not setup correctly and you will need to resolve the
issue and set up ssh again.
The possible causes for failure could be:
1. The server settings in /etc/ssh/sshd_config file do not allow ssh
for user oracle.
2. The server may have disabled public key based authentication.
3. The client public key on the server may be outdated.
4. ~oracle or ~oracle/.ssh on the remote host may not be owned by oracle.
5. User may not have passed -shared option for shared remote users or
may be passing the -shared option for non-shared remote users.
6. If there is output in addition to the date, but no password is asked,
it may be a security alert shown as part of company policy. Append the
additional text to the <OMS HOME>/sysman/prov/resources/ignoreMessages.txt file.
------------------------------------------------------------------------
--datarac1:--
Running /usr/bin/ssh -x -l oracle datarac1 date to verify SSH connectivity has been setup from local host to datarac1.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Mon May 22 16:08:28 KST 2023
------------------------------------------------------------------------
--datarac2:--
Running /usr/bin/ssh -x -l oracle datarac2 date to verify SSH connectivity has been setup from local host to datarac2.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Mon May 22 16:08:28 KST 2023
------------------------------------------------------------------------
--datarac3:--
Running /usr/bin/ssh -x -l oracle datarac3 date to verify SSH connectivity has been setup from local host to datarac3.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Mon May 22 16:08:29 KST 2023
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from datarac1 to datarac1
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Mon May 22 16:08:30 KST 2023
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from datarac1 to datarac2
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Mon May 22 16:08:30 KST 2023
------------------------------------------------------------------------
------------------------------------------------------------------------
Verifying SSH connectivity has been setup from datarac1 to datarac3
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL.
Mon May 22 16:08:31 KST 2023
------------------------------------------------------------------------
-Verification from complete-
SSH verification complete.
[oracle@datarac1 deinstall]$

추가 node에 대한 Run cluvfy 실행

[oracle@datarac1 grid]$ id
uid=54321(oracle) gid=54322(dba) groups=54322(dba) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[oracle@datarac1 grid]$ env | grep SID
ORACLE_SID=+ASM1
[oracle@datarac1 grid]$ pwd
/u01/app/19.0.0/grid
[oracle@datarac1 grid]$
[oracle@datarac1 grid]$ ./runcluvfy.sh stage -pre nodeadd -help

USAGE:
runcluvfy.sh stage -pre nodeadd -n <node_list>
                                [-vip <vip_list>] [-fixup] [-fixupnoexec]
                                [-method sudo -user <user_name> [-location <dir_path>]|-method root] [-verbose]
                                [-json] [-html]
                                [-save [-savedir <save_dir>]]

<vip_list> is the comma-separated list of virtual IP addresses that will be applied to the comma-separated list of non-domain qualified node names entered.
-json Specifies that the user wants the detailed report to be output in a JSON format. If the -json option is missing, then the detailed report is output in text to standard output format.
-html Specifies that the user wants the detailed report to be output in html format. If a browser recognized by CVU is available on the system, the results will be reported using that browser. Otherwise, the user will have to use -save option to get a copy of the report in html format which can be viewed by the user using a browser of his or her choice. If -html option is missing, then the detailed report is output in text format.
-save Specifies that the user wants the validation reports to be saved for future reference. The reports named cvucheckreport_<timestamp>.(htm|txt|json) as requested are saved in the location of the user's choice if specified using the -savedir option. If the -savedir option has not been specified, then the reports are saved in the default location <CV-HOME>/cv/report.
<node_list> is the comma-separated list of non-domain qualified node names which will be added to the cluster.
-method is the privilege delegation method, 'sudo' or 'root', to be used for root user access.
<user_name> is the user name to access all the nodes with root privileges.
<dir_path> is the full file system path for the 'sudo' executable.

DESCRIPTION:
Performs the appropriate checks on the nodes to be added to the existing cluster, and verifies the integrity of the cluster before the nodes can be added.  If '-fixup' option is specified, on verification failure, fix up operations are performed if feasible.  If '-fixupnoexec' option is specified, on verification failure, the fix up data will be generated and the instruction for manual execution of the generated fix ups will be displayed.

[oracle@datarac1 grid]$
[oracle@datarac1 grid]$
[oracle@datarac1 grid]$ ./runcluvfy.sh stage -pre nodeadd -n datarac3

Performing following verification checks ...

  Physical Memory ...PASSED
  Available Physical Memory ...PASSED
  Swap Size ...PASSED
  Free Space: datarac3:/usr,datarac3:/var,datarac3:/etc,datarac3:/u01/app/19.0.0/grid,datarac3:/sbin,datarac3:/tmp ...PASSED
  Free Space: datarac1:/usr,datarac1:/var,datarac1:/etc,datarac1:/u01/app/19.0.0/grid,datarac1:/sbin,datarac1:/tmp ...PASSED
  User Existence: oracle ...
    Users With Same UID: 54321 ...PASSED
  User Existence: oracle ...PASSED
  User Existence: root ...
    Users With Same UID: 0 ...PASSED
  User Existence: root ...PASSED
  Group Existence: dba ...PASSED
  Group Membership: dba ...PASSED
  Run Level ...PASSED
  Hard Limit: maximum open file descriptors ...PASSED
  Soft Limit: maximum open file descriptors ...PASSED
  Hard Limit: maximum user processes ...PASSED
  Soft Limit: maximum user processes ...PASSED
  Soft Limit: maximum stack size ...FAILED (PRVG-0449)
  Architecture ...PASSED
  OS Kernel Version ...PASSED
  OS Kernel Parameter: semmsl ...PASSED
  OS Kernel Parameter: semmns ...PASSED
  OS Kernel Parameter: semopm ...PASSED
  OS Kernel Parameter: semmni ...PASSED
  OS Kernel Parameter: shmmax ...FAILED (PRVG-1205, PRVG-1201)
  OS Kernel Parameter: shmmni ...PASSED
  OS Kernel Parameter: shmall ...PASSED
  OS Kernel Parameter: file-max ...PASSED
  OS Kernel Parameter: ip_local_port_range ...PASSED
  OS Kernel Parameter: rmem_default ...PASSED
  OS Kernel Parameter: rmem_max ...PASSED
  OS Kernel Parameter: wmem_default ...PASSED
  OS Kernel Parameter: wmem_max ...PASSED
  OS Kernel Parameter: aio-max-nr ...PASSED
  OS Kernel Parameter: panic_on_oops ...PASSED
  Package: kmod-20-21 (x86_64) ...PASSED
  Package: kmod-libs-20-21 (x86_64) ...PASSED
  Package: binutils-2.30-49.0.2 ...PASSED
  Package: libgcc-8.2.1 (x86_64) ...PASSED
  Package: libstdc++-8.2.1 (x86_64) ...PASSED
  Package: sysstat-10.1.5 ...PASSED
  Package: ksh ...PASSED
  Package: make-4.2.1 ...PASSED
  Package: glibc-2.28 (x86_64) ...PASSED
  Package: glibc-devel-2.28 (x86_64) ...PASSED
  Package: libaio-0.3.110 (x86_64) ...PASSED
  Package: nfs-utils-2.3.3-14 ...PASSED
  Package: smartmontools-6.6-3 ...PASSED
  Package: net-tools-2.0-0.51 ...PASSED
  Package: policycoreutils-2.9-3 ...PASSED
  Package: policycoreutils-python-utils-2.9-3 ...PASSED
  Users With Same UID: 0 ...PASSED
  Current Group ID ...PASSED
  Root user consistency ...PASSED
  Package: cvuqdisk-1.0.10-1 ...PASSED
  Node Addition ...
    CRS Integrity ...PASSED
    Clusterware Version Consistency ...PASSED
    '/u01/app/19.0.0/grid' ...PASSED
  Node Addition ...PASSED
  Host name ...PASSED
  Node Connectivity ...
    Hosts File ...PASSED
    Check that maximum (MTU) size packet goes through subnet ...PASSED
    subnet mask consistency for subnet "100.100.100.0" ...PASSED
    subnet mask consistency for subnet "192.168.56.0" ...PASSED
  Node Connectivity ...PASSED
  Multicast or broadcast check ...PASSED
  ASM Integrity ...PASSED
  Device Checks for ASM ...
    Package: cvuqdisk-1.0.10-1 ...PASSED
    ASM device sharedness check ...
      Shared Storage Accessibility:/dev/oracleasm/asm-disk04,/dev/oracleasm/asm-disk05,/dev/oracleasm/asm-disk06,/dev/oracleasm/asm-disk01,/dev/oracleasm/asm-disk02,/dev/oracleasm/asm-disk03 ...PASSED
    ASM device sharedness check ...PASSED
    Access Control List check ...PASSED
  Device Checks for ASM ...PASSED
  Database home availability ...PASSED
  OCR Integrity ...PASSED
  Time zone consistency ...PASSED
  Network Time Protocol (NTP) ...
    '/etc/chrony.conf' ...PASSED
    Daemon 'chronyd' ...PASSED
    NTP daemon or service using UDP port 123 ...PASSED
    chrony daemon is synchronized with at least one external time source ...FAILED (PRVG-13606)
  Network Time Protocol (NTP) ...FAILED (PRVG-13606)
  User Not In Group "root": oracle ...PASSED
  Time offset between nodes ...PASSED
  resolv.conf Integrity ...PASSED
  DNS/NIS name service ...PASSED
  User Equivalence ...PASSED
  Software home: /u01/app/19.0.0/grid ...PASSED
  /dev/shm mounted as temporary file system ...PASSED
  zeroconf check ...FAILED (PRVE-10077)

Pre-check for node addition was unsuccessful.
Checks did not pass for the following nodes:
        datarac3,datarac1


Failures were encountered during execution of CVU verification request "stage -pre nodeadd".

Soft Limit: maximum stack size ...FAILED
datarac3: PRVG-0449 : Proper soft limit for maximum stack size was not found
            on node "datarac3" [Expected >= "10240" ; Found = "8192"].

OS Kernel Parameter: shmmax ...FAILED
datarac3: PRVG-1205 : OS kernel parameter "shmmax" does not have expected
            current value on node "datarac3" [Expected = "4169566208" ;
            Current = "4169564160"; Configured = "4169564160"].
datarac3: PRVG-1201 : OS kernel parameter "shmmax" does not have expected
            configured value on node "datarac3" [Expected = "4169566208" ;
            Current = "4169564160"; Configured = "4169564160"].

Network Time Protocol (NTP) ...FAILED
  chrony daemon is synchronized with at least one external time source ...FAILED
  datarac3: PRVG-13606 : chrony daemon is not synchronized with any external
              time source on node "datarac3".

  datarac1: PRVG-13606 : chrony daemon is not synchronized with any external
              time source on node "datarac1".

zeroconf check ...FAILED
datarac3: PRVE-10077 : NOZEROCONF parameter was not  specified or was not set
            to 'yes' in file "/etc/sysconfig/network" on node
            "datarac3.localdomain"


CVU operation performed:      stage -pre nodeadd
Date:                         May 22, 2023 4:25:54 PM
CVU version:                  19.19.0.0.0 (040623x8664)
Clusterware version:          19.0.0.0.0
CVU home:                     /u01/app/19.0.0/grid
Grid home:                    /u01/app/19.0.0/grid
User:                         oracle
Operating system:             Linux5.4.17-2011.7.4.el8uek.x86_64
[oracle@datarac1 grid]$

 

반응형
반응형

[환 경]
IBM AIX + Oracle DB 11.2.0.4 + PSU 11.2.0.4.201020 + Raw devices

[증 상]
기존 Oracle DB 11.2.0.3에서 Oracle DB 11.2.0.4로 Upgrade시 ORA-17515 Error 메시지와 DB 중지 됨(Raw devices 환경)
ORA-01157: cannot identify/lock data file 20 - see DBWR trace file
ORA-01110: data file 20: '/dev/rhdisk20'
ORA-17503: ksfdopn:1 Failed to open file /dev/rhdisk20
ORA-17515: Creation of clonedb failed using snapshot file

 

[원 인]
Bug 19487099 - AIX: ORA-17503 and ORA-17515 when upgrading and using raw devices (문서 ID 19487099.8)

 

[해결책]
Oracle DB 12cR2 이상으로 Upgrade

[기타사항]
Oracle DB 12cR2 이상으로 Upgrade 하기 전에 Oracle SR을 통해 다른 방법이 있는지 확인 필요 함.

반응형
반응형

[증상]
Oracle RAC ocssd.bin Process를 Kill 하는 경우 전체 Node 또는 일부 Node가 Reboot 발생

[환경]
IBM AIX + Oracle RAC 19c + Veritas CFS

[원인]
Oracle CSS misscount 값이 30, disktimeout 597로 Setting 되어 있음

[해결책]
Oracle CSS misscount 값을 600으로 변경

[기타사항]
Oracle RAC 19c 구성 후 Veritas Relink 단계에서 Oracle CRS가 기동 되어 있어야 함.
만약, Oracle CRS가 중자 상태에서 Veritas Relink를 수행 하면 Oracle CSS misscount 값을 600으로 변경 하라는 안내 메시지가 나옴.

[참고문서]
Oracle Clusterware에서 CSS Timeout 계산 (Doc ID 2158867.1)

반응형
반응형

[환경]

IBM AIX 7.2 + Oracle CRS/DB 11.2.0.4 + 2-Node RAC + ASM + PSU 11.2.0.4.201020

[증 상]

PSU 11.2.0.4.201020 적용 마지막 단계인 rootcrs.pl -patch 실행 하면 약 10분~15분간 대기 하다 crsconfig_lib.pm line 11821 발생 하며 실패

Oracle ASM Instance 기동 되지 않음

[원 인]

BUG 32123313 - SMR FILE KEEPS GETTING CORRUPTED AFTER APPLYING OCT 2020 DBPSU IN AIX7

Bug 32109594 - ASM IS UNABLE TO START WITH SMR CORRUPTED ERROR POST OCT 2020 PSU APPLY

 

[해결방법]

Patch 32109594 적용 전 rootcrs.pl -unlock

Patch 32109594를 $GRID_HOME과 $ORACLE_DB_HOME모두 적용

Patch 32109594 적용 후 rootcrs.pl -patch

[참고문서]

AIX:11.2.0.4 :ASM&DB Instance Not Starting Up Due to SMR Corrupt After OCT 2020 PSU (문서 ID 2732507.1)

 

Oracle MOS를 찾아 보니 IBM AIX 7.2에 PSU 11.2.0.4.201020 이상을 적용하면 발생 하는 것 같다.

반응형
반응형

현 상태 확인

[oracle@ol7 ~]$ id
uid=54321(oracle) gid=54322(dba) groups=54322(dba) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
[oracle@ol7 ~]$
[oracle@ol7 ~]$ env | grep SID
ORACLE_SID=CDBTST
[oracle@ol7 ~]$
[oracle@ol7 ~]$ sqlplus "/as sysdba"

SQL*Plus: Release 19.0.0.0.0 - Production on Thu Oct 6 14:49:50 2022
Version 19.16.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.16.0.0.0

SQL> show con_name

CON_NAME
------------------------------
CDB$ROOT
SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDBTST                         READ WRITE NO
SQL>
SQL> set linesize 150
SQL> col name for a10
SQL> col FILE_NAME for a20
col file_name for a70
SQL> SELECT VP.NAME, CDF.TABLESPACE_NAME, CDF.FILE_NAME
  2  FROM CDB_DATA_FILES CDF, V$PDBS VP
  3  WHERE CDF.CON_ID = VP.CON_ID;

NAME       TABLESPACE_NAME                FILE_NAME
---------- ------------------------------ ----------------------------------------------------------------------
PDBTST     SYSTEM                         /orapdbtst/system/sytem01.dbf
PDBTST     SYSAUX                         /orapdbtst/system/sysaux01.dbf
PDBTST     UNDOTBS1                       /orapdbtst/system/unotodb01.dbf
PDBTST     USERS                          /orapdbtst/system/user01.dbf
PDBTST     TS_DATA                        /orapdbtst/data/ts_data_01.dbf
PDBTST     TS_IDX                         /u01/app/oracle/product/19.0.0.0/dbs/ /orapdbtst/idx/ts_indx_01.dbf  --> 이 것을 변경 하려 함

6 rows selected.

SQL>
SQL> !ls -a /u01/app/oracle/product/19.0.0.0/dbs/
   .  ..  hc_CDBTST.dat  hc_TST.dat  init.ora  lkCDBTST  lkTST  orapwCDBTST  orapwTST  spfileCDBTST.ora  spfileTST.ora

SQL> !ls -a /u01/app/oracle/product/19.0.0.0/dbs/ /orapdbtst/idx/
/orapdbtst/idx/:
.  ..

/u01/app/oracle/product/19.0.0.0/dbs/:
   .  ..  hc_CDBTST.dat  hc_TST.dat  init.ora  lkCDBTST  lkTST  orapwCDBTST  orapwTST  spfileCDBTST.ora  spfileTST.ora

SQL> !ls -a '/u01/app/oracle/product/19.0.0.0/dbs/ /orapdbtst/idx/'
.  ..  ts_indx_01.dbf

SQL>
SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.16.0.0.0
[oracle@ol7 ~]$

DBF File Move

[oracle@ol7 ~]$ sqlplus "/as sysdba"

SQL*Plus: Release 19.0.0.0.0 - Production on Thu Oct 6 14:55:47 2022
Version 19.16.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.


Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.16.0.0.0

SQL> show con_name

CON_NAME
------------------------------
CDB$ROOT
SQL>
SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDBTST                         READ WRITE NO
SQL>
SQL> ALTER SESSION SET CONTAINER=PDBTST;

Session altered.

SQL> show con_name

CON_NAME
------------------------------
PDBTST
SQL>
SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         3 PDBTST                         READ WRITE NO
SQL> select file_name from dba_data_files;

FILE_NAME
--------------------------------------------------------------------------------
/orapdbtst/system/sytem01.dbf
/orapdbtst/system/sysaux01.dbf
/orapdbtst/system/unotodb01.dbf
/orapdbtst/system/user01.dbf
/orapdbtst/data/ts_data_01.dbf
/u01/app/oracle/product/19.0.0.0/dbs/ /orapdbtst/idx/ts_indx_01.dbf

6 rows selected.

SQL> alter database move datafile '/u01/app/oracle/product/19.0.0.0/dbs/ /orapdbtst/idx/ts_indx_01.dbf' to '/orapdbtst/idx/ts_indx_01.dbf';

Database altered.

SQL> !ls -al '/u01/app/oracle/product/19.0.0.0/dbs/ /orapdbtst/idx/ts_indx_01.dbf'
ls: cannot access /u01/app/oracle/product/19.0.0.0/dbs/ /orapdbtst/idx/ts_indx_01.dbf: No such file or directory

SQL> !ls -al /orapdbtst/idx/ts_indx_01.dbf
-rw-r-----. 1 oracle dba 1048584192 Oct  6 15:02 /orapdbtst/idx/ts_indx_01.dbf

SQL>
SQL> select file_name from dba_data_files;

FILE_NAME
--------------------------------------------------------------------------------
/orapdbtst/system/sytem01.dbf
/orapdbtst/system/sysaux01.dbf
/orapdbtst/system/unotodb01.dbf
/orapdbtst/system/user01.dbf
/orapdbtst/data/ts_data_01.dbf
/orapdbtst/idx/ts_indx_01.dbf

6 rows selected.

SQL>
SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.16.0.0.0
[oracle@ol7 ~]$
반응형

+ Recent posts