Monday, May 16, 2016

PSU patching updates (APR 2016) in RAC Database.

Two Node RAC Database with following environments :

1. dcdb01 (Hostname Node1)
2. dcdb02 (Hostname Node2)
3. dcdb (RAC DB Name)
4. dcdb1 (RAC DB Instance 1)
5. dcdb2 (RAC DB Instance 2)
6. /u01/stage/patch/psu_apr_2016/ (Patch staging directory for April 2016)
7. /u01/app/11.2.4/grid (Grid Home)
8. /u01/app/oracle/product/11.2.4/db_1 (RAC DB Home)

The APR2016 PSU patching updates the Grid Infrastucture (GI), Database (RAC), and OJVM components of both the GI and RAC installations. The GI patches are applied first and then the RAC patches are applied. The OJVM patches are applied during both the GI and RAC patching sessions.

The following link for downloading PSU for April 2016.

https://updates.oracle.com/download/22738793.html 

Step 1 to 5 perform in all of rac node if Oracle RAC and GI home are not shared.

1. Download Opatch and move to Oracle RAC Database Home and GI Home.

$ unzip <OPATCH-ZIP> -d <ORACLE_HOME>
$ <ORACLE_HOME>/OPatch/opatch version

unzip /u01/stage/patch/psu_apr_2016/p6880880_112000_Linux-x86-64.zip -d /u01/app/11.2.4/grid/

$opatch version

The version output of the previous command should be 11.2.0.3.6 or later.

2. OCM Configuration perform the following steps from GI Home.

export PATH=$PATH:/u01/app/11.2.0.2/grid/OPatch

export JAVA_HOME=/u01/app/oracle/product/11.2.4/db_1/jdk

$ORACLE_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /u01/stage/patch/psu_apr_2016/ocm.rsp

chmod 775 /u01/stage/patch/psu_apr_2016/ocm.rsp

3. Validation of Oracle Inventory for Oracle RAC Database Home and GI Home. This should run from grid and oracle user.

$ <ORACLE_HOME>/OPatch/opatch lsinventory -detail -oh <ORACLE_HOME>

$ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME

4. Download and Unzip the Patch. The  directory should be empty and not /tmp

Unzip the patch as grid home owner.

cd <UNZIPPED_PATCH_LOCATION>

unzip /u01/stage/patch/psu_apr_2016/p22738793_112040_Linux-x86-64.zip

5. Stop EM Agent Processes Prior to Patching and Prior to Rolling Back the Patch

As the Oracle RAC database home owner execute:

$ <ORACLE_HOME>/bin/emctl stop dbconsole

emctl stop dbconsole

6. From the Oracle database home, make sure to stop the Oracle RAC databases running on all nodes. As the database home owner execute:

$ <ORACLE_HOME>/bin/srvctl stop database –d <db-unique-name>

srvctl stop database –d dcdb

7. On the 1st node, apply the patch to the GI Home using the opatch auto command. As root user, execute the following command:

# opatch auto <UNZIPPED_PATCH_LOCATION>/22738793 -ocmrf <ocm response file>

opatch auto /u01/stage/patch/psu_apr_2016/22738793 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp

8. On the 1st node, apply the patch to the Database home using the opatch auto command. Since the Database home is shared, this operation will patch the Database home across the cluster. Note that a USM only patch cannot be applied to a database home. As root user, execute the following command:

# opatch auto <UNZIPPED_PATCH_LOCATION>/22738793 -oh <DATABASE_HOME> -ocmrf <ocm response file>

 opatch auto /u01/stage/patch/psu_apr_2016/22738793 -oh /u01/app/oracle/product/11.2.4/db_1 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp

9. On the 1st node only, restart the Oracle instance, which you have previously stopped in Step 1. As the database home owner execute:

$ <ORACLE_HOME>/bin/srvctl start instance –d <db-unique-name> -n <nodename>

srvctl start instance -d dcdb -n dcdb01

10. On the 2nd node, apply the patch to GI Home using the opatch auto command. As root user, execute the following command:

# opatch auto <UNZIPPED_PATCH_LOCATION>/22738793 -oh <GI_HOME> -ocmrf <ocm response file>

opatch auto /u01/stage/patch/psu_apr_2016/22738793 -oh /u01/app/11.2.4/grid -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp

11. On the 2st node, apply the patch to the Database home using the opatch auto command.

# opatch auto <UNZIPPED_PATCH_LOCATION>/22738793 -oh <DATABASE_HOME> -ocmrf <ocm response file>

opatch auto /u01/stage/patch/psu_apr_2016/22738793 -oh /u01/app/oracle/product/11.2.4/db_1 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp

12. On the 2nd node only, restart the Oracle instance, which you have previously stopped in Step 1. As the database home owner execute:

$ <ORACLE_HOME>/bin/srvctl start instance –d <db-unique-name> -n <nodename>

srvctl start instance -d dcdb -n dcdb02

#####################   Loading Modified SQL Files into the Database   #####################

1. load modified SQL files into the database in Oracle RAC environment, perform these steps on only one node.

cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> SHUTDOWN
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> QUIT

The catbundle.sql execution is reflected in the dba_registry_history view by a row associated with bundle series PSU.

2. If the OJVM PSU was applied for a previous GI PSU patch, you may see invalid Java classes after execution of the catbundle.sql script in the previous step. If this is the case, run utlrp.sql to re-validate these Java classes.

cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @utlrp.sql

3. Check the following log files in $ORACLE_BASE/cfgtoollogs/catbundle for any errors:

catbundle_PSU_<database SID>_APPLY_<TIMESTAMP>.log
catbundle_PSU_<database SID>_GENERATE_<TIMESTAMP>.log
where TIMESTAMP is of the form YYYYMMMDD_HH_MM_SS.

#####################   Upgrade Oracle Recovery Manager Catalog   #####################

If you are using the Oracle Recovery Manager, the catalog needs to be upgraded. Enter the following command to upgrade it:

$ rman catalog username/password@alias
RMAN> UPGRADE CATALOG;

#####################   Post install steps for OJVM   #####################

The following steps load modified SQL files into the database. For an Oracle RAC environment, perform these steps on only one node.

Install the SQL portion of the patch by running the following command for a single instance environment.

cd $ORACLE_HOME/sqlpatch/22674697
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> SHUTDOWN
SQL> STARTUP
SQL> alter system set cluster_database=false scope=spfile;
SQL> SHUTDOWN
SQL> STARTUP UPGRADE
SQL> @postinstall.sql
SQL> alter system set cluster_database=true scope=spfile;
SQL> SHUTDOWN
SQL> STARTUP

After installing the SQL portion of the patch, some packages could become INVALID. This will get recompiled upon access or you can run utlrp.sql to get them back into a VALID state.

cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @utlrp.sql



##################### Log ####################
[grid@dcdb01 ~]$ cd /u01/stage/patch/psu_apr_2016/
[grid@dcdb01 psu_apr_2016]$ ls -ltr
total 774564
-rw-r--r-- 1 oracle oinstall  52733900 May 12 17:05 p6880880_112000_Linux-x86-64.zip
-rw-r--r-- 1 oracle oinstall 740284621 May 13 01:06 p22738793_112040_Linux-x86-64.zip
[grid@dcdb01 ~]$ unzip /u01/stage/patch/psu_apr_2016/p6880880_112000_Linux-x86-64.zip -d /u01/app/11.2.4/grid/
Archive:  /u01/stage/patch/psu_apr_2016/p6880880_112000_Linux-x86-64.zip
  inflating: /u01/app/11.2.4/grid/OPatch/opatchdiag
  inflating: /u01/app/11.2.4/grid/OPatch/opatch.pl
  .....
[grid@dcdb02 ~]$ cd /u01/stage/patch/psu_apr_2016/
[grid@dcdb02 psu_apr_2016]$ ls -ltr
total 774564
-rw-r--r-- 1 oracle oinstall  52733900 May 12 17:05 p6880880_112000_Linux-x86-64.zip
-rw-r--r-- 1 oracle oinstall 740284621 May 13 01:06 p22738793_112040_Linux-x86-64.zip
[grid@dcdb02 ~]$ unzip /u01/stage/patch/psu_apr_2016/p6880880_112000_Linux-x86-64.zip -d /u01/app/11.2.4/grid/
Archive:  /u01/stage/patch/psu_apr_2016/p6880880_112000_Linux-x86-64.zip
  inflating: /u01/app/11.2.4/grid/OPatch/opatchdiag
  inflating: /u01/app/11.2.4/grid/OPatch/opatch.pl
  .....
[grid@dcdb01 ~]$ export JAVA_HOME=/u01/app/oracle/product/11.2.4/db_1/jdk
[grid@dcdb01 ~]$ $ORACLE_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /u01/stage/patch/psu_apr_2016/ocm.rsp
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
The OCM configuration response file (/u01/stage/patch/psu_apr_2016/ocm.rsp) was successfully created.
[grid@dcdb01 ~]$ chmod 775 /u01/stage/patch/psu_apr_2016/ocm.rsp
[grid@dcdb01 ~]$

[grid@dcdb02 ~]$ export JAVA_HOME=/u01/app/oracle/product/11.2.4/db_1/jdk
[grid@dcdb02 ~]$ $ORACLE_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /u01/stage/patch/psu_apr_2016/ocm.rsp
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
The OCM configuration response file (/u01/stage/patch/psu_apr_2016/ocm.rsp) was successfully created.
[grid@dcdb02 ~]$ chmod 775 /u01/stage/patch/psu_apr_2016/ocm.rsp

[grid@dcdb01 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.12
Copyright (c) 2016, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/11.2.4/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/11.2.4/grid/oraInst.loc
OPatch version    : 11.2.0.3.12
OUI version       : 11.2.0.4.0
Log file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/opatch2016-05-15_17-34-08PM_1.log

Lsinventory Output file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/lsinv/lsinventory2016-05-15_17-34-08PM.txt

--------------------------------------------------------------------------------
Local Machine Information::
Hostname: sbldcdb02.spftl.com
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1):

Oracle Grid Infrastructure 11g                                       11.2.0.4.0
There are 1 products installed in this Oracle Home.


Installed Products (89):

Agent Required Support Files                                         10.2.0.4.5
Assistant Common Files                                               11.2.0.4.0
Automatic Storage Management Assistant                               11.2.0.4.0
Bali Share                                                           1.1.18.0.0
Buildtools Common Files                                              11.2.0.4.0
Character Set Migration Utility                                      11.2.0.4.0
Cluster Ready Services Files                                         11.2.0.4.0
Cluster Verification Utility Common Files                            11.2.0.4.0
Cluster Verification Utility Files                                   11.2.0.4.0
Database SQL Scripts                                                 11.2.0.4.0
Deinstallation Tool                                                  11.2.0.4.0
Enterprise Manager Common Core Files                                 10.2.0.4.5
Enterprise Manager Common Files                                      10.2.0.4.5
Enterprise Manager plugin Common Files                               11.2.0.4.0
Expat libraries                                                       2.0.1.0.1
HAS Common Files                                                     11.2.0.4.0
HAS Files for DB                                                     11.2.0.4.0
Installation Common Files                                            11.2.0.4.0
Installation Plugin Files                                            11.2.0.4.0
Installer SDK Component                                              11.2.0.4.0
Java Development Kit                                                1.5.0.51.10
LDAP Required Support Files                                          11.2.0.4.0
OLAP SQL Scripts                                                     11.2.0.4.0
Oracle Advanced Security                                             11.2.0.4.0
Oracle Clusterware RDBMS Files                                       11.2.0.4.0
Oracle Configuration Manager Deconfiguration                         10.3.1.0.0
Oracle Containers for Java                                           11.2.0.4.0
Oracle Core Required Support Files                                   11.2.0.4.0
Oracle Database 11g                                                  11.2.0.4.0
Oracle Database 11g Multimedia Files                                 11.2.0.4.0
Oracle Database Deconfiguration                                      11.2.0.4.0
Oracle Database User Interface                                       2.2.13.0.0
Oracle Database Utilities                                            11.2.0.4.0
Oracle DBCA Deconfiguration                                          11.2.0.4.0
Oracle Extended Windowing Toolkit                                    3.4.47.0.0
Oracle Globalization Support                                         11.2.0.4.0
Oracle Globalization Support                                         11.2.0.4.0
Oracle Grid Infrastructure 11g                                       11.2.0.4.0
Oracle Help For Java                                                  4.2.9.0.0
Oracle Ice Browser                                                    5.2.3.6.0
Oracle Internet Directory Client                                     11.2.0.4.0
Oracle Java Client                                                   11.2.0.4.0
Oracle JDBC/OCI Instant Client                                       11.2.0.4.0
Oracle JDBC/THIN Interfaces                                          11.2.0.4.0
Oracle JFC Extended Windowing Toolkit                                4.2.36.0.0
Oracle JVM                                                           11.2.0.4.0
Oracle LDAP administration                                           11.2.0.4.0
Oracle Locale Builder                                                11.2.0.4.0
Oracle Multimedia                                                    11.2.0.4.0
Oracle Multimedia Client Option                                      11.2.0.4.0
Oracle Multimedia Java Advanced Imaging                              11.2.0.4.0
Oracle Multimedia Locator                                            11.2.0.4.0
Oracle Multimedia Locator RDBMS Files                                11.2.0.4.0
Oracle Net                                                           11.2.0.4.0
Oracle Net Listener                                                  11.2.0.4.0
Oracle Net Required Support Files                                    11.2.0.4.0
Oracle Netca Client                                                  11.2.0.4.0
Oracle Notification Service                                          11.2.0.3.0
Oracle Notification Service (eONS)                                   11.2.0.4.0
Oracle One-Off Patch Installer                                       11.2.0.3.4
Oracle Quality of Service Management (Client)                        11.2.0.4.0
Oracle Quality of Service Management (Server)                        11.2.0.4.0
Oracle RAC Deconfiguration                                           11.2.0.4.0
Oracle RAC Required Support Files-HAS                                11.2.0.4.0
Oracle Recovery Manager                                              11.2.0.4.0
Oracle Security Developer Tools                                      11.2.0.4.0
Oracle Text Required Support Files                                   11.2.0.4.0
Oracle Universal Installer                                           11.2.0.4.0
Oracle USM Deconfiguration                                           11.2.0.4.0
Oracle Wallet Manager                                                11.2.0.4.0
Parser Generator Required Support Files                              11.2.0.4.0
Perl Interpreter                                                     5.10.0.0.2
Perl Modules                                                         5.10.0.0.1
PL/SQL                                                               11.2.0.4.0
PL/SQL Embedded Gateway                                              11.2.0.4.0
Platform Required Support Files                                      11.2.0.4.0
Precompiler Required Support Files                                   11.2.0.4.0
RDBMS Required Support Files                                         11.2.0.4.0
RDBMS Required Support Files for Instant Client                      11.2.0.4.0
RDBMS Required Support Files Runtime                                 11.2.0.4.0
Required Support Files                                               11.2.0.4.0
Secure Socket Layer                                                  11.2.0.4.0
SQL*Plus                                                             11.2.0.4.0
SQL*Plus Files for Instant Client                                    11.2.0.4.0
SQL*Plus Required Support Files                                      11.2.0.4.0
SSL Required Support Files for InstantClient                         11.2.0.4.0
Universal Storage Manager Files                                      11.2.0.4.0
XDK Required Support Files                                           11.2.0.4.0
XML Parser for Java                                                  11.2.0.4.0
There are 89 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


Rac system comprising of multiple nodes
  Local node = dcdb01
  Remote node = dcdb02

--------------------------------------------------------------------------------

OPatch succeeded.

[grid@dcdb02 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.12
Copyright (c) 2016, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/11.2.4/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/11.2.4/grid/oraInst.loc
OPatch version    : 11.2.0.3.12
OUI version       : 11.2.0.4.0
Log file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/opatch2016-05-15_17-34-08PM_1.log

Lsinventory Output file location : /u01/app/11.2.4/grid/cfgtoollogs/opatch/lsinv/lsinventory2016-05-15_17-34-08PM.txt

--------------------------------------------------------------------------------
Local Machine Information::
Hostname: sbldcdb02.spftl.com
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1):

Oracle Grid Infrastructure 11g                                       11.2.0.4.0
There are 1 products installed in this Oracle Home.


Installed Products (89):

Agent Required Support Files                                         10.2.0.4.5
Assistant Common Files                                               11.2.0.4.0
Automatic Storage Management Assistant                               11.2.0.4.0
Bali Share                                                           1.1.18.0.0
Buildtools Common Files                                              11.2.0.4.0
Character Set Migration Utility                                      11.2.0.4.0
Cluster Ready Services Files                                         11.2.0.4.0
Cluster Verification Utility Common Files                            11.2.0.4.0
Cluster Verification Utility Files                                   11.2.0.4.0
Database SQL Scripts                                                 11.2.0.4.0
Deinstallation Tool                                                  11.2.0.4.0
Enterprise Manager Common Core Files                                 10.2.0.4.5
Enterprise Manager Common Files                                      10.2.0.4.5
Enterprise Manager plugin Common Files                               11.2.0.4.0
Expat libraries                                                       2.0.1.0.1
HAS Common Files                                                     11.2.0.4.0
HAS Files for DB                                                     11.2.0.4.0
Installation Common Files                                            11.2.0.4.0
Installation Plugin Files                                            11.2.0.4.0
Installer SDK Component                                              11.2.0.4.0
Java Development Kit                                                1.5.0.51.10
LDAP Required Support Files                                          11.2.0.4.0
OLAP SQL Scripts                                                     11.2.0.4.0
Oracle Advanced Security                                             11.2.0.4.0
Oracle Clusterware RDBMS Files                                       11.2.0.4.0
Oracle Configuration Manager Deconfiguration                         10.3.1.0.0
Oracle Containers for Java                                           11.2.0.4.0
Oracle Core Required Support Files                                   11.2.0.4.0
Oracle Database 11g                                                  11.2.0.4.0
Oracle Database 11g Multimedia Files                                 11.2.0.4.0
Oracle Database Deconfiguration                                      11.2.0.4.0
Oracle Database User Interface                                       2.2.13.0.0
Oracle Database Utilities                                            11.2.0.4.0
Oracle DBCA Deconfiguration                                          11.2.0.4.0
Oracle Extended Windowing Toolkit                                    3.4.47.0.0
Oracle Globalization Support                                         11.2.0.4.0
Oracle Globalization Support                                         11.2.0.4.0
Oracle Grid Infrastructure 11g                                       11.2.0.4.0
Oracle Help For Java                                                  4.2.9.0.0
Oracle Ice Browser                                                    5.2.3.6.0
Oracle Internet Directory Client                                     11.2.0.4.0
Oracle Java Client                                                   11.2.0.4.0
Oracle JDBC/OCI Instant Client                                       11.2.0.4.0
Oracle JDBC/THIN Interfaces                                          11.2.0.4.0
Oracle JFC Extended Windowing Toolkit                                4.2.36.0.0
Oracle JVM                                                           11.2.0.4.0
Oracle LDAP administration                                           11.2.0.4.0
Oracle Locale Builder                                                11.2.0.4.0
Oracle Multimedia                                                    11.2.0.4.0
Oracle Multimedia Client Option                                      11.2.0.4.0
Oracle Multimedia Java Advanced Imaging                              11.2.0.4.0
Oracle Multimedia Locator                                            11.2.0.4.0
Oracle Multimedia Locator RDBMS Files                                11.2.0.4.0
Oracle Net                                                           11.2.0.4.0
Oracle Net Listener                                                  11.2.0.4.0
Oracle Net Required Support Files                                    11.2.0.4.0
Oracle Netca Client                                                  11.2.0.4.0
Oracle Notification Service                                          11.2.0.3.0
Oracle Notification Service (eONS)                                   11.2.0.4.0
Oracle One-Off Patch Installer                                       11.2.0.3.4
Oracle Quality of Service Management (Client)                        11.2.0.4.0
Oracle Quality of Service Management (Server)                        11.2.0.4.0
Oracle RAC Deconfiguration                                           11.2.0.4.0
Oracle RAC Required Support Files-HAS                                11.2.0.4.0
Oracle Recovery Manager                                              11.2.0.4.0
Oracle Security Developer Tools                                      11.2.0.4.0
Oracle Text Required Support Files                                   11.2.0.4.0
Oracle Universal Installer                                           11.2.0.4.0
Oracle USM Deconfiguration                                           11.2.0.4.0
Oracle Wallet Manager                                                11.2.0.4.0
Parser Generator Required Support Files                              11.2.0.4.0
Perl Interpreter                                                     5.10.0.0.2
Perl Modules                                                         5.10.0.0.1
PL/SQL                                                               11.2.0.4.0
PL/SQL Embedded Gateway                                              11.2.0.4.0
Platform Required Support Files                                      11.2.0.4.0
Precompiler Required Support Files                                   11.2.0.4.0
RDBMS Required Support Files                                         11.2.0.4.0
RDBMS Required Support Files for Instant Client                      11.2.0.4.0
RDBMS Required Support Files Runtime                                 11.2.0.4.0
Required Support Files                                               11.2.0.4.0
Secure Socket Layer                                                  11.2.0.4.0
SQL*Plus                                                             11.2.0.4.0
SQL*Plus Files for Instant Client                                    11.2.0.4.0
SQL*Plus Required Support Files                                      11.2.0.4.0
SSL Required Support Files for InstantClient                         11.2.0.4.0
Universal Storage Manager Files                                      11.2.0.4.0
XDK Required Support Files                                           11.2.0.4.0
XML Parser for Java                                                  11.2.0.4.0
There are 89 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


Rac system comprising of multiple nodes
  Local node = dcdb02
  Remote node = dcdb01

--------------------------------------------------------------------------------

OPatch succeeded.

[grid@dcdb01 ~]$
[grid@dcdb01 ~]$ cd /u01/stage/patch/psu_apr_2016/
[grid@dcdb01 psu_apr_2016]$ unzip p22738793_112040_Linux-x86-64.zip
Archive:  p22738793_112040_Linux-x86-64.zip
   creating: 22738793/
   creating: 22738793/22674697/
   creating: 22738793/22674697/etc/
   creating: 22738793/22674697/etc/config/
  inflating: 22738793/22674697/etc/config/deploy.xml
  inflating: 22738793/22674697/etc/config/inventory.xml
  inflating: 22738793/22674697/etc/config/actions.xml
   creating: 22738793/22674697/etc/xml/
.....
[grid@dcdb02 ~]$ cd /u01/stage/patch/psu_apr_2016/
[grid@dcdb02 psu_apr_2016]$ unzip p22738793_112040_Linux-x86-64.zip
Archive:  p22738793_112040_Linux-x86-64.zip
   creating: 22738793/
   creating: 22738793/22674697/
   creating: 22738793/22674697/etc/
   creating: 22738793/22674697/etc/config/
  inflating: 22738793/22674697/etc/config/deploy.xml
  inflating: 22738793/22674697/etc/config/inventory.xml
  inflating: 22738793/22674697/etc/config/actions.xml
   creating: 22738793/22674697/etc/xml/
.....

[oracle@dcdb01 ~]$ emctl stop dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://dcdb01.dc.com:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.
[oracle@dcdb01 ~]$

[oracle@dcdb02 ~]$ emctl stop dbconsole
Oracle Enterprise Manager 11g Database Control Release 11.2.0.4.0
Copyright (c) 1996, 2013 Oracle Corporation.  All rights reserved.
https://dcdb01.dc.com:1158/em/console/aboutApplication
Stopping Oracle Enterprise Manager 11g Database Control ...
 ...  Stopped.
[oracle@dcdb02 ~]$
[oracle@dcdb01 oracle]# srvctl stop database -d dcdb
[oracle@dcdb01 oracle]# srvctl status database -d dcdb
Instance dcdb1 is not running on node dcdb01
Instance dcdb1 is not running on node dcdb02
[oracle@dcdb01 ~]$ su grid
Password:
[grid@dcdb01 oracle]$ cd
[grid@dcdb01 ~]$
[grid@dcdb01 ~]$ . ./.bash_profile
[grid@dcdb01 ~]$ su root
Password:
[root@dcdb01 grid]#
[root@dcdb01 grid]# echo $ORACLE_HOME
/u01/app/11.2.4/grid
[root@dcdb01 grid]#
[root@dcdb01 ~]# opatch auto /u01/stage/patch/psu_apr_2016/22738793 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp
Executing /u01/app/11.2.4/grid/perl/bin/perl /u01/app/11.2.4/grid/OPatch/crs/patch11203.pl -patchdir /u01/stage/patch/psu_apr_2016 -patchn 22738793 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp -paramfile /u01/app/11.2.4/grid/crs/install/crsconfig_params

This is the main log file: /u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_13-33-15.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_13-33-15.report.log

2016-05-15 13:33:15: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.4/grid/crs/install/crsconfig_params

Stopping RAC /u01/app/oracle/product/11.2.4/db_1 ...
Stopped RAC /u01/app/oracle/product/11.2.4/db_1 successfully

patch /u01/stage/patch/psu_apr_2016/22738793/22502456  apply successful for home  /u01/app/oracle/product/11.2.4/db_1
patch /u01/stage/patch/psu_apr_2016/22738793/22502549/custom/server/22502549  apply successful for home  /u01/app/oracle/product/11.2.4/db_1
patch /u01/stage/patch/psu_apr_2016/22738793/22674697  apply successful for home  /u01/app/oracle/product/11.2.4/db_1

Stopping CRS...
Stopped CRS successfully

patch /u01/stage/patch/psu_apr_2016/22738793/22502456  apply successful for home  /u01/app/11.2.4/grid
patch /u01/stage/patch/psu_apr_2016/22738793/22502549  apply successful for home  /u01/app/11.2.4/grid
patch /u01/stage/patch/psu_apr_2016/22738793/22502505  apply successful for home  /u01/app/11.2.4/grid

Starting CRS...
Installing Trace File Analyzer
CRS-4123: Oracle High Availability Services has been started.

Starting RAC /u01/app/oracle/product/11.2.4/db_1 ...
Started RAC /u01/app/oracle/product/11.2.4/db_1 successfully

opatch auto succeeded.
[root@dcdb01 ~]#
[root@dcdb01 ~]# cd /home/oracle/
[root@dcdb01 oracle]# . ./.bash_profile
[root@dcdb01 oracle]#  echo $ORACLE_HOME
/u01/app/oracle/product/11.2.4/db_1
[root@dcdb01 oracle]#
[root@dcdb01 oracle]# opatch auto /u01/stage/patch/psu_apr_2016/22738793 -oh /u01/app/oracle/product/11.2.4/db_1 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp
Executing /u01/app/11.2.4/grid/perl/bin/perl /u01/app/11.2.4/grid/OPatch/crs/patch11203.pl -patchdir /u01/stage/patch/psu_apr_2016 -patchn 22738793 -oh /u01/app/oracle/product/11.2.4/db_1 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp -paramfile /u01/app/11.2.4/grid/crs/install/crsconfig_params

This is the main log file: /u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_13-51-45.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_13-51-45.report.log

2016-05-15 13:51:45: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.4/grid/crs/install/crsconfig_params

Stopping RAC /u01/app/oracle/product/11.2.4/db_1 ...
Stopped RAC /u01/app/oracle/product/11.2.4/db_1 successfully

patch /u01/stage/patch/psu_apr_2016/22738793/22502456  apply successful for home  /u01/app/oracle/product/11.2.4/db_1
patch /u01/stage/patch/psu_apr_2016/22738793/22502549/custom/server/22502549  apply successful for home  /u01/app/oracle/product/11.2.4/db_1
patch /u01/stage/patch/psu_apr_2016/22738793/22674697  apply successful for home  /u01/app/oracle/product/11.2.4/db_1

Starting RAC /u01/app/oracle/product/11.2.4/db_1 ...
Started RAC /u01/app/oracle/product/11.2.4/db_1 successfully

opatch auto succeeded.
[root@dcdb01 oracle]#
[root@dcdb01 oracle]# su oracle
[oracle@dcdb01 ~]$ srvctl start instance -d dcdb -n dcdb01
[oracle@dcdb01 ~]$
[oracle@dcdb01 ~]$ opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.12
Copyright (c) 2016, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/11.2.4/db_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/11.2.4/db_1/oraInst.loc
OPatch version    : 11.2.0.3.12
OUI version       : 11.2.0.4.0
Log file location : /u01/app/oracle/product/11.2.4/db_1/cfgtoollogs/opatch/opatch2016-05-15_13-57-42PM_1.log

Lsinventory Output file location : /u01/app/oracle/product/11.2.4/db_1/cfgtoollogs/opatch/lsinv/lsinventory2016-05-15_13-57-42PM.txt

--------------------------------------------------------------------------------
Local Machine Information::
Hostname: dcdb01.dc.com
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1):

Oracle Database 11g                                                  11.2.0.4.0
There are 1 products installed in this Oracle Home.


Interim patches (3) :

Patch  22674697     : applied on Sun May 15 13:37:40 BDT 2016
Unique Patch ID:  20038581
Patch description:  "OJVM PATCH SET UPDATE 11.2.0.4.160419"
   Created on 17 Mar 2016, 07:51:04 hrs PST8PDT
   Bugs fixed:
     19058059, 18933818, 19176885, 17201047, 19007266, 19554117, 14774730
     17285560, 19153980, 21911849, 18166577, 22670385, 19187988, 18458318
     19374518, 19006757, 17056813, 21811517, 19909862, 19223010, 22118835
     19895326, 22253904, 20408829, 19852360, 17804361, 21047766, 19231857
     17528315, 21566944

Patch  22502549     : applied on Sun May 15 13:37:12 BDT 2016
Unique Patch ID:  19880366
Patch description:  "OCW Patch Set Update : 11.2.0.4.160419 (22502549)"
   Created on 11 Mar 2016, 01:34:41 hrs PST8PDT
   Bugs fixed:
     19270660, 18328800, 18508710, 18691572, 20038451, 21251192, 21232394
     20365005, 17750548, 17387214, 17617807, 14497275, 20219458, 17733927
     18180541, 18962892, 17292250, 17378618, 16759171, 20110156, 17843489
     17065496, 13991403, 17273020, 17155238, 20012766, 21245437, 18261183
     18053580, 20218012, 17013634, 17886392, 20995001, 17039197, 17947785
     16317771, 10052729, 22353346, 20340620, 16237657, 20317221, 15917869
     18199185, 18399991, 20186278, 17374271, 18024089, 16849642, 20746251
     20246071, 14270845, 20552947, 18882642, 18414137, 17001914, 17927970
     14378120, 16346413, 15986647, 18068871, 21222147, 18143836, 16206997
     21982225, 19168690, 20235511, 18343490, 21875360, 16613232, 19276791
     17722664, 20440643, 12928658, 18952577, 18520351, 16249829, 18226143
     16076412, 18265482, 18229842, 17172091, 20676340, 17818075, 20091753
     18231837, 14373486, 20136892, 17483479, 20551654, 18120545, 18729166
     13843841, 21225209, 17405302, 18709496, 18330979, 18744838, 17087371
     20531190, 14525998, 18187697, 20598625, 14385860, 18348155, 19479503
     12928592, 17516024, 18370031, 17764053, 19272663, 17551223, 14671408
     18272135, 14207615, 21255373, 17500165, 18875012, 14769643, 18464784
     19558324, 18848125, 19241857, 14851828, 17955615, 20315294, 14693336
     16284825, 17352230, 20014326, 17238586, 17089344, 17405605, 21327402
     17531342, 19398098, 17159489, 17640316, 13823394, 16543190, 22024217
     17983675, 20795241, 17598201, 17481314, 16281493, 18346135, 15986311
     19601468, 17208793, 18700935, 18999857, 14076173, 18428146, 18352845
     17435488, 20408163, 17592037, 18352846, 19616601, 17391726, 17387779
     14777968, 15851860, 16206882, 20141091, 21113068, 20175174, 17305100
     15832129, 19885321, 16901346, 17985714, 18536826, 17780903, 18752378
     18946768, 16876500, 16875342, 17769597, 19955755, 16429265, 18336452
     17273003, 17209968, 16988311, 20094984, 19319357, 17059927, 17046460
     18053631, 16867761, 18774591, 21442094, 20235486, 19359787, 15869775
     19642566, 17447588, 16798862, 15920201

Patch  22502456     : applied on Sun May 15 13:36:11 BDT 2016
Unique Patch ID:  19856194
Patch description:  "Database Patch Set Update : 11.2.0.4.160419 (22502456)"
   Created on 21 Mar 2016, 11:49:22 hrs
Sub-patch  21948347; "Database Patch Set Update : 11.2.0.4.160119 (21948347)"
Sub-patch  21352635; "Database Patch Set Update : 11.2.0.4.8 (21352635)"
Sub-patch  20760982; "Database Patch Set Update : 11.2.0.4.7 (20760982)"
Sub-patch  20299013; "Database Patch Set Update : 11.2.0.4.6 (20299013)"
Sub-patch  19769489; "Database Patch Set Update : 11.2.0.4.5 (19769489)"
Sub-patch  19121551; "Database Patch Set Update : 11.2.0.4.4 (19121551)"
Sub-patch  18522509; "Database Patch Set Update : 11.2.0.4.3 (18522509)"
Sub-patch  18031668; "Database Patch Set Update : 11.2.0.4.2 (18031668)"
Sub-patch  17478514; "Database Patch Set Update : 11.2.0.4.1 (17478514)"
   Bugs fixed:
     17288409, 21051852, 17811429, 18607546, 17205719, 20506699, 17816865
     17922254, 17754782, 16934803, 13364795, 17311728, 17441661, 17284817
     16992075, 17446237, 14015842, 19972569, 21756677, 21538558, 20925795
     17449815, 17375354, 19463897, 13866822, 17982555, 17235750, 17478514
     18317531, 14338435, 18235390, 20803583, 13944971, 20142975, 17811789
     16929165, 18704244, 20506706, 17546973, 20334344, 14054676, 17088068
     17346091, 18264060, 17343514, 21538567, 19680952, 18471685, 19211724
     13951456, 21847223, 16315398, 18744139, 16850630, 19049453, 18673304
     17883081, 19915271, 18641419, 18262334, 17006183, 16065166, 18277454
     16833527, 10136473, 18051556, 17865671, 17852463, 18554871, 17853498
     18334586, 17551709, 17588480, 19827973, 17344412, 17842825, 18828868
     17025461, 11883252, 13609098, 17239687, 17602269, 19197175, 22195457
     18316692, 17313525, 12611721, 19544839, 18964939, 17600719, 18191164
     19393542, 17571306, 18482502, 20777150, 19466309, 17040527, 17165204
     18098207, 16785708, 17465741, 17174582, 16180763, 16777840, 12982566
     19463893, 22195465, 16875449, 12816846, 17237521, 19358317, 17811438
     17811447, 21983325, 17945983, 18762750, 16912439, 17184721, 18061914
     17282229, 18331850, 18202441, 17082359, 18723434, 21972320, 19554106
     14034426, 18339044, 19458377, 17752995, 20448824, 17891943, 17258090
     17767676, 16668584, 18384391, 17040764, 17381384, 15913355, 18356166
     14084247, 20596234, 20506715, 21756661, 13853126, 18203837, 14245531
     21756699, 16043574, 22195441, 17848897, 17877323, 21453153, 17468141
     20861693, 17786518, 17912217, 17037130, 18155762, 16956380, 17478145
     17394950, 18641461, 18189036, 18619917, 17027426, 21352646, 16268425
     22195492, 19584068, 18436307, 17265217, 17634921, 13498382, 21526048
     19258504, 20004087, 17443671, 22195485, 18000422, 20004021, 22321756
     17571039, 21067387, 16344544, 18009564, 14354737, 21286665, 18135678
     18614015, 20441797, 18362222, 17835048, 16472716, 17936109, 17050888
     17325413, 14010183, 18747196, 17761775, 16721594, 17082983, 20067212
     21179898, 17302277, 18084625, 15990359, 18203835, 17297939, 17811456
     16731148, 21168487, 13829543, 17215560, 14133975, 17694209, 17385178
     18091059, 8322815, 17586955, 17201159, 17655634, 18331812, 19730508
     18868646, 17648596, 16220077, 16069901, 17348614, 17393915, 17274537
     17957017, 18096714, 17308789, 18436647, 14285317, 19289642, 14764829
     18328509, 17622427, 22195477, 16943711, 22502493, 14368995, 17346671
     18996843, 17783588, 21343838, 16618694, 17672719, 18856999, 18783224
     17851160, 17546761, 17798953, 18273830, 22092979, 16596890, 19972566
     16384983, 17726838, 17360606, 22321741, 13645875, 18199537, 16542886
     21787056, 17889549, 14565184, 17071721, 17610798, 20299015, 21343897
     22893153, 20657441, 17397545, 18230522, 16360112, 19769489, 12905058
     18641451, 12747740, 18430495, 17016369, 17042658, 14602788, 17551063
     19972568, 21517440, 18508861, 19788842, 14657740, 17332800, 13837378
     19972564, 17186905, 18315328, 19699191, 17437634, 22353199, 18093615
     19006849, 19013183, 17296856, 18674024, 17232014, 16855292, 17762296
     14692762, 21051840, 17705023, 19121551, 21330264, 19854503, 21868720
     19309466, 18681862, 18554763, 20558005, 17390160, 18456514, 16306373
     13955826, 18139690, 17501491, 17752121, 21668627, 17299889, 17889583
     18673325, 19721304, 18293054, 17242746, 17951233, 17649265, 18094246
     19615136, 17011832, 16870214, 17477958, 18522509, 20631274, 16091637
     17323222, 16595641, 16524926, 18228645, 18282562, 17596908, 17156148
     18031668, 16494615, 22683225, 17545847, 17655240, 17614134, 13558557
     17341326, 17891946, 17716305, 16392068, 19271443, 21351877, 18092127
     18440047, 17614227, 14106803, 16903536, 18973907, 18673342, 19032867
     17389192, 17612828, 16194160, 17006570, 17721717, 17390431, 17570240
     16863422, 18325460, 19727057, 16422541, 19972570, 17267114, 18244962
     21538485, 18765602, 18203838, 16198143, 17246576, 14829250, 17835627
     18247991, 14458214, 21051862, 16692232, 17786278, 17227277, 16042673
     16314254, 16228604, 16837842, 17393683, 17787259, 20331945, 20074391
     15861775, 16399083, 18018515, 22683212, 18260550, 21051858, 17036973
     16613964, 17080436, 16579084, 18384537, 18280813, 20296213, 16901385
     15979965, 18441944, 16450169, 9756271, 17892268, 11733603, 16285691
     17587063, 21343775, 16538760, 18180390, 18193833, 21387964, 21051833
     17238511, 17824637, 16571443, 18306996, 14852021, 18674047, 17853456
     12364061, 22195448



Rac system comprising of multiple nodes
  Local node = dcdb01
  Remote node = dcdb02

--------------------------------------------------------------------------------

OPatch succeeded.
[oracle@dcdb01 ~]$

[root@dcdb01 oracle]# ssh grid@dcdb02
grid@dcdb02's password:
Last login: Mon May  1 15:40:54 2016 from X.X.X.X
[grid@dcdb02 ~]$
[root@dcdb02 grid]# opatch auto /u01/stage/patch/psu_apr_2016/22738793 -oh /u01/app/11.2.4/grid -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp
Executing /u01/app/11.2.4/grid/perl/bin/perl /u01/app/11.2.4/grid/OPatch/crs/patch11203.pl -patchdir /u01/stage/patch/psu_apr_2016 -patchn 22738793 -oh /u01/app/11.2.4/grid -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp -paramfile /u01/app/11.2.4/grid/crs/install/crsconfig_params

This is the main log file: /u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_17-39-47.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_17-39-47.report.log

2016-05-15 17:39:47: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.4/grid/crs/install/crsconfig_params

Stopping CRS...
Stopped CRS successfully

patch /u01/stage/patch/psu_apr_2016/22738793/22502456  apply successful for home  /u01/app/11.2.4/grid
patch /u01/stage/patch/psu_apr_2016/22738793/22502549  apply successful for home  /u01/app/11.2.4/grid
patch /u01/stage/patch/psu_apr_2016/22738793/22502505  apply successful for home  /u01/app/11.2.4/grid

Starting CRS...
Installing Trace File Analyzer
CRS-4123: Oracle High Availability Services has been started.

opatch auto succeeded.
[root@dcdb02 grid]#
[root@dcdb02 ~]# cd /home/oracle/
[root@dcdb02 oracle]# . ./.bash_profile
[root@dcdb02 oracle]#  echo $ORACLE_HOME
/u01/app/oracle/product/11.2.4/db_1
[root@dcdb02 oracle]#
[root@dcdb02 oracle]# opatch auto /u01/stage/patch/psu_apr_2016/22738793 -oh /u01/app/oracle/product/11.2.4/db_1 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp
Executing /u01/app/11.2.4/grid/perl/bin/perl /u01/app/11.2.4/grid/OPatch/crs/patch11203.pl -patchdir /u01/stage/patch/psu_apr_2016 -patchn 22738793 -oh /u01/app/oracle/product/11.2.4/db_1 -ocmrf /u01/stage/patch/psu_apr_2016/ocm.rsp -paramfile /u01/app/11.2.4/grid/crs/install/crsconfig_params

This is the main log file: /u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_17-52-14.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.4/grid/cfgtoollogs/opatchauto2016-05-15_17-52-14.report.log

2016-05-15 17:52:14: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.4/grid/crs/install/crsconfig_params

Stopping RAC /u01/app/oracle/product/11.2.4/db_1 ...
Stopped RAC /u01/app/oracle/product/11.2.4/db_1 successfully

patch /u01/stage/patch/psu_apr_2016/22738793/22502456  apply successful for home  /u01/app/oracle/product/11.2.4/db_1
patch /u01/stage/patch/psu_apr_2016/22738793/22502549/custom/server/22502549  apply successful for home  /u01/app/oracle/product/11.2.4/db_1
patch /u01/stage/patch/psu_apr_2016/22738793/22674697  apply successful for home  /u01/app/oracle/product/11.2.4/db_1

Starting RAC /u01/app/oracle/product/11.2.4/db_1 ...
Started RAC /u01/app/oracle/product/11.2.4/db_1 successfully

opatch auto succeeded.
[root@dcdb02 oracle]# exit
[oracle@dcdb02 ~]$ opatch lsinventory
Oracle Interim Patch Installer version 11.2.0.3.12
Copyright (c) 2016, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/app/oracle/product/11.2.4/db_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/11.2.4/db_1/oraInst.loc
OPatch version    : 11.2.0.3.12
OUI version       : 11.2.0.4.0
Log file location : /u01/app/oracle/product/11.2.4/db_1/cfgtoollogs/opatch/opatch2016-05-16_12-25-55PM_1.log

Lsinventory Output file location : /u01/app/oracle/product/11.2.4/db_1/cfgtoollogs/opatch/lsinv/lsinventory2016-05-16_12-25-55PM.txt

--------------------------------------------------------------------------------
Local Machine Information::
Hostname: dcdb02.dc.com
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1):

Oracle Database 11g                                                  11.2.0.4.0
There are 1 products installed in this Oracle Home.


Interim patches (3) :

Patch  22674697     : applied on Sun May 15 17:56:06 BDT 2016
Unique Patch ID:  20038581
Patch description:  "OJVM PATCH SET UPDATE 11.2.0.4.160419"
   Created on 17 Mar 2016, 07:51:04 hrs PST8PDT
   Bugs fixed:
     19058059, 18933818, 19176885, 17201047, 19007266, 19554117, 14774730
     17285560, 19153980, 21911849, 18166577, 22670385, 19187988, 18458318
     19374518, 19006757, 17056813, 21811517, 19909862, 19223010, 22118835
     19895326, 22253904, 20408829, 19852360, 17804361, 21047766, 19231857
     17528315, 21566944

Patch  22502549     : applied on Sun May 15 17:55:38 BDT 2016
Unique Patch ID:  19880366
Patch description:  "OCW Patch Set Update : 11.2.0.4.160419 (22502549)"
   Created on 11 Mar 2016, 01:34:41 hrs PST8PDT
   Bugs fixed:
     19270660, 18328800, 18508710, 18691572, 20038451, 21251192, 21232394
     20365005, 17750548, 17387214, 17617807, 14497275, 20219458, 17733927
     18180541, 18962892, 17292250, 17378618, 16759171, 20110156, 17843489
     17065496, 13991403, 17273020, 17155238, 20012766, 21245437, 18261183
     18053580, 20218012, 17013634, 17886392, 20995001, 17039197, 17947785
     16317771, 10052729, 22353346, 20340620, 16237657, 20317221, 15917869
     18199185, 18399991, 20186278, 17374271, 18024089, 16849642, 20746251
     20246071, 14270845, 20552947, 18882642, 18414137, 17001914, 17927970
     14378120, 16346413, 15986647, 18068871, 21222147, 18143836, 16206997
     21982225, 19168690, 20235511, 18343490, 21875360, 16613232, 19276791
     17722664, 20440643, 12928658, 18952577, 18520351, 16249829, 18226143
     16076412, 18265482, 18229842, 17172091, 20676340, 17818075, 20091753
     18231837, 14373486, 20136892, 17483479, 20551654, 18120545, 18729166
     13843841, 21225209, 17405302, 18709496, 18330979, 18744838, 17087371
     20531190, 14525998, 18187697, 20598625, 14385860, 18348155, 19479503
     12928592, 17516024, 18370031, 17764053, 19272663, 17551223, 14671408
     18272135, 14207615, 21255373, 17500165, 18875012, 14769643, 18464784
     19558324, 18848125, 19241857, 14851828, 17955615, 20315294, 14693336
     16284825, 17352230, 20014326, 17238586, 17089344, 17405605, 21327402
     17531342, 19398098, 17159489, 17640316, 13823394, 16543190, 22024217
     17983675, 20795241, 17598201, 17481314, 16281493, 18346135, 15986311
     19601468, 17208793, 18700935, 18999857, 14076173, 18428146, 18352845
     17435488, 20408163, 17592037, 18352846, 19616601, 17391726, 17387779
     14777968, 15851860, 16206882, 20141091, 21113068, 20175174, 17305100
     15832129, 19885321, 16901346, 17985714, 18536826, 17780903, 18752378
     18946768, 16876500, 16875342, 17769597, 19955755, 16429265, 18336452
     17273003, 17209968, 16988311, 20094984, 19319357, 17059927, 17046460
     18053631, 16867761, 18774591, 21442094, 20235486, 19359787, 15869775
     19642566, 17447588, 16798862, 15920201

Patch  22502456     : applied on Sun May 15 17:54:42 BDT 2016
Unique Patch ID:  19856194
Patch description:  "Database Patch Set Update : 11.2.0.4.160419 (22502456)"
   Created on 21 Mar 2016, 11:49:22 hrs
Sub-patch  21948347; "Database Patch Set Update : 11.2.0.4.160119 (21948347)"
Sub-patch  21352635; "Database Patch Set Update : 11.2.0.4.8 (21352635)"
Sub-patch  20760982; "Database Patch Set Update : 11.2.0.4.7 (20760982)"
Sub-patch  20299013; "Database Patch Set Update : 11.2.0.4.6 (20299013)"
Sub-patch  19769489; "Database Patch Set Update : 11.2.0.4.5 (19769489)"
Sub-patch  19121551; "Database Patch Set Update : 11.2.0.4.4 (19121551)"
Sub-patch  18522509; "Database Patch Set Update : 11.2.0.4.3 (18522509)"
Sub-patch  18031668; "Database Patch Set Update : 11.2.0.4.2 (18031668)"
Sub-patch  17478514; "Database Patch Set Update : 11.2.0.4.1 (17478514)"
   Bugs fixed:
     17288409, 21051852, 17811429, 18607546, 17205719, 20506699, 17816865
     17922254, 17754782, 16934803, 13364795, 17311728, 17441661, 17284817
     16992075, 17446237, 14015842, 19972569, 21756677, 21538558, 20925795
     17449815, 17375354, 19463897, 13866822, 17982555, 17235750, 17478514
     18317531, 14338435, 18235390, 20803583, 13944971, 20142975, 17811789
     16929165, 18704244, 20506706, 17546973, 20334344, 14054676, 17088068
     17346091, 18264060, 17343514, 21538567, 19680952, 18471685, 19211724
     13951456, 21847223, 16315398, 18744139, 16850630, 19049453, 18673304
     17883081, 19915271, 18641419, 18262334, 17006183, 16065166, 18277454
     16833527, 10136473, 18051556, 17865671, 17852463, 18554871, 17853498
     18334586, 17551709, 17588480, 19827973, 17344412, 17842825, 18828868
     17025461, 11883252, 13609098, 17239687, 17602269, 19197175, 22195457
     18316692, 17313525, 12611721, 19544839, 18964939, 17600719, 18191164
     19393542, 17571306, 18482502, 20777150, 19466309, 17040527, 17165204
     18098207, 16785708, 17465741, 17174582, 16180763, 16777840, 12982566
     19463893, 22195465, 16875449, 12816846, 17237521, 19358317, 17811438
     17811447, 21983325, 17945983, 18762750, 16912439, 17184721, 18061914
     17282229, 18331850, 18202441, 17082359, 18723434, 21972320, 19554106
     14034426, 18339044, 19458377, 17752995, 20448824, 17891943, 17258090
     17767676, 16668584, 18384391, 17040764, 17381384, 15913355, 18356166
     14084247, 20596234, 20506715, 21756661, 13853126, 18203837, 14245531
     21756699, 16043574, 22195441, 17848897, 17877323, 21453153, 17468141
     20861693, 17786518, 17912217, 17037130, 18155762, 16956380, 17478145
     17394950, 18641461, 18189036, 18619917, 17027426, 21352646, 16268425
     22195492, 19584068, 18436307, 17265217, 17634921, 13498382, 21526048
     19258504, 20004087, 17443671, 22195485, 18000422, 20004021, 22321756
     17571039, 21067387, 16344544, 18009564, 14354737, 21286665, 18135678
     18614015, 20441797, 18362222, 17835048, 16472716, 17936109, 17050888
     17325413, 14010183, 18747196, 17761775, 16721594, 17082983, 20067212
     21179898, 17302277, 18084625, 15990359, 18203835, 17297939, 17811456
     16731148, 21168487, 13829543, 17215560, 14133975, 17694209, 17385178
     18091059, 8322815, 17586955, 17201159, 17655634, 18331812, 19730508
     18868646, 17648596, 16220077, 16069901, 17348614, 17393915, 17274537
     17957017, 18096714, 17308789, 18436647, 14285317, 19289642, 14764829
     18328509, 17622427, 22195477, 16943711, 22502493, 14368995, 17346671
     18996843, 17783588, 21343838, 16618694, 17672719, 18856999, 18783224
     17851160, 17546761, 17798953, 18273830, 22092979, 16596890, 19972566
     16384983, 17726838, 17360606, 22321741, 13645875, 18199537, 16542886
     21787056, 17889549, 14565184, 17071721, 17610798, 20299015, 21343897
     22893153, 20657441, 17397545, 18230522, 16360112, 19769489, 12905058
     18641451, 12747740, 18430495, 17016369, 17042658, 14602788, 17551063
     19972568, 21517440, 18508861, 19788842, 14657740, 17332800, 13837378
     19972564, 17186905, 18315328, 19699191, 17437634, 22353199, 18093615
     19006849, 19013183, 17296856, 18674024, 17232014, 16855292, 17762296
     14692762, 21051840, 17705023, 19121551, 21330264, 19854503, 21868720
     19309466, 18681862, 18554763, 20558005, 17390160, 18456514, 16306373
     13955826, 18139690, 17501491, 17752121, 21668627, 17299889, 17889583
     18673325, 19721304, 18293054, 17242746, 17951233, 17649265, 18094246
     19615136, 17011832, 16870214, 17477958, 18522509, 20631274, 16091637
     17323222, 16595641, 16524926, 18228645, 18282562, 17596908, 17156148
     18031668, 16494615, 22683225, 17545847, 17655240, 17614134, 13558557
     17341326, 17891946, 17716305, 16392068, 19271443, 21351877, 18092127
     18440047, 17614227, 14106803, 16903536, 18973907, 18673342, 19032867
     17389192, 17612828, 16194160, 17006570, 17721717, 17390431, 17570240
     16863422, 18325460, 19727057, 16422541, 19972570, 17267114, 18244962
     21538485, 18765602, 18203838, 16198143, 17246576, 14829250, 17835627
     18247991, 14458214, 21051862, 16692232, 17786278, 17227277, 16042673
     16314254, 16228604, 16837842, 17393683, 17787259, 20331945, 20074391
     15861775, 16399083, 18018515, 22683212, 18260550, 21051858, 17036973
     16613964, 17080436, 16579084, 18384537, 18280813, 20296213, 16901385
     15979965, 18441944, 16450169, 9756271, 17892268, 11733603, 16285691
     17587063, 21343775, 16538760, 18180390, 18193833, 21387964, 21051833
     17238511, 17824637, 16571443, 18306996, 14852021, 18674047, 17853456
     12364061, 22195448



Rac system comprising of multiple nodes
  Local node = dcdb02
  Remote node = dcdb01

--------------------------------------------------------------------------------

OPatch succeeded.
[oracle@dcdb02 ~]$
[oracle@dcdb02 ~]$ srvctl start instance -d dcdb -n dcdb02

[oracle@dcdb01 ~]$ cd $ORACLE_HOME/rdbms/admin
[oracle@dcdb01 admin]$ sqlplus "/as sysdba"

SQL*Plus: Release 11.2.0.4.0 Production on Sun May 15 18:03:45 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> SHUTDOWN
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> STARTUP;
ORACLE instance started.

Total System Global Area 6413680640 bytes
Fixed Size    2265224 bytes
Variable Size 3472887672 bytes
Database Buffers 2919235584 bytes
Redo Buffers   19292160 bytes
Database mounted.
Database opened.
SQL> @catbundle.sql psu apply

PL/SQL procedure successfully completed.


Function created.


Function created.

....

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
[oracle@dcdb01 admin]$
[oracle@dcdb01 admin]$ sqlplus "/as sysdba"

SQL*Plus: Release 11.2.0.4.0 Production on Sun May 15 18:03:45 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> @utlrp.sql

TIMESTAMP
--------------------------------------------------------------------------------
COMP_TIMESTAMP UTLRP_BGN  2016-05-15 18:18:38

DOC>   The following PL/SQL block invokes UTL_RECOMP to recompile invalid
DOC>   objects in the database. Recompilation time is proportional to the
DOC>   number of invalid objects in the database, so this command may take
DOC>   a long time to execute on a database with a large number of invalid
DOC>   objects.

......
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
[oracle@dcdb01 admin]$
[grid@dcdb01 ~]$ srvctl stop database -d dcdb
[oracle@dcdb01 catbundle]$ cd $ORACLE_HOME/sqlpatch/22674697
[oracle@dcdb01 22674697]$ sqlplus "/as sysdba"

SQL*Plus: Release 11.2.0.4.0 Production on Sun May 15 18:58:42 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup nomount;
ORACLE instance started.

Total System Global Area 6413680640 bytes
Fixed Size    2265224 bytes
Variable Size 3472887672 bytes
Database Buffers 2919235584 bytes
Redo Buffers   19292160 bytes
SQL> alter systemd  set cluster_database=false scope=spfile;

System altered.

SQL> shut
ORA-01507: database not mounted


ORACLE instance shut down.
SQL> startup upgrade
ORACLE instance started.

Total System Global Area 6413680640 bytes
Fixed Size    2265224 bytes
Variable Size 3472887672 bytes
Database Buffers 2919235584 bytes
Redo Buffers   19292160 bytes
Database mounted.
Database opened.
SQL> @postinstall.sql

PL/SQL procedure successfully completed.


Function created.


PL/SQL procedure successfully completed.

...

SQL> alter system set cluster_database=true scope=spfile;

System altered.

SQL> SHUTDOWN
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options
[oracle@dcdb01 22674697]$
[oracle@dcdb01 22674697]$ srvctl start database -d dcdb

Reference:

Patch Set Updates for Oracle Products (Doc ID 854428.1)

My Oracle Support Document 966023.1

How to Create an OCM Response file to Apply a Patch in Silent Mode - opatch silent (Doc ID 966023.1)

No comments:

Post a Comment