Wednesday, September 21, 2022

OCI VMDB System : Upgrade Database from 12c to 19c using dbcli


Environment : OCI VMDB System

OS          : Oracle Linux 7

Note        : dbcli commands should be run as ROOT user


=> Obtain Pre-Authenticated Request URL

You need a Pre-Authenticated Request URL (PAR URL), which is a unique URL generated for an Oracle Database 19c Oracle Home image that is stored as a zip file in the Oracle Cloud bucket.

In addition to the PAR URL, you will also need values for the following two parameters that will be supplied to the dbcli tool when you run the dbcli create-dbhome command in Step 2 of Section 2.

Version of the Oracle Database 19c Oracle Home image (--cloneVersion)

Checksum (--sha256sum) of the zip file

You can obtain the PAR URL and its two supporting parameters from Oracle Support by raising a service request as follows:

Log in to My Oracle Support and click the Contact Us button at the top right.

Click the Create Non-Technical SR button.

Enter Problem Summary as: Request PAR URL for Database Upgrade

Choose Problem Type: Software Download Issues > 02. Database Upgrade

Select your correct Support Identifier.

Click Next.

Bypass instructions to review displayed KM document by clicking Next.

Click Next.

Verify your contact information.

Click Submit.


=> Upgrade Grid Infrastructure to latest version 

Check below link on how to update dcs client and agent and upgrade GI 

https://www.apps-dba.com/2022/09/oci-vmdb-system-apply-gi-psu-using-dbcli.html


=> Install 19C Oracle Home using below command (No downtime needed)

Duration : 10 - 20 mins

# dbcli create-dbhome --cloneVersion 19.15.0.0.0 --parUrl https://objectstorage.us-phoenix-1.oraclecloud.com/p/##########/n/dev/b/DB-Images/o/CustomImage/db1915.220419.tar.gz --sha256sum ###################################


Job details
----------------------------------------------------------------
                     ID:  ba06af5a-4349-4e05-be63-f866ca4c7afc
            Description:  Database Home creation with Id : 03f77e36-98c1-4f0d-ac7f-923e9073a654
                 Status:  Created
                Created:  September 6, 2022 10:06:55 AM GST
               Progress:  0%
                Message:  Create Database Home
             Error Code:
Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------



[root@sitdb ~]# dbcli describe-job -i ba06af5a-4349-4e05-be63-f866ca4c7afc
Job details
----------------------------------------------------------------
                     ID:  ba06af5a-4349-4e05-be63-f866ca4c7afc
            Description:  Database Home creation with Id : 03f77e36-98c1-4f0d-ac7f-923e9073a654
                 Status:  Running
                Created:  September 6, 2022 10:06:55 AM GST
               Progress:  0%
                Message:  Create Database Home
             Error Code:
Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
Database home creation                                                   September 6, 2022 10:06:56 AM GST   September 6, 2022 10:06:56 AM GST   Running
[root@sitdb ~]#


[root@sitdb ~]# dbcli describe-job -i ba06af5a-4349-4e05-be63-f866ca4c7afc

Job details
----------------------------------------------------------------
                     ID:  ba06af5a-4349-4e05-be63-f866ca4c7afc
            Description:  Database Home creation with Id : 03f77e36-98c1-4f0d-ac7f-923e9073a654
                 Status:  Success
                Created:  September 6, 2022 10:06:55 AM GST
               Progress:  100%
                Message:  Create Database Home
             Error Code:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
Database home creation                                                   September 6, 2022 10:06:56 AM GST   September 6, 2022 10:16:18 AM GST   Success

[root@sitdb ~]#


=> Run the following commands to create an nls directory on each of the Oracle RAC nodes:


unset ORACLE_HOME
unset ORACLE_UNQNAME
unset LD_LIBRARY_PATH
unset ORACLE_SID
export PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/oracle/.local/bin:/home/oracle/bin

export ORACLE_HOME=/u01/app/oracle/product/19.0.0.0/dbhome_1
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH

[oracle@sitdb ~]$ $ORACLE_HOME/perl/bin/perl $ORACLE_HOME/nls/data/old/cr9idata.pl
Creating directory /u01/app/oracle/product/19.0.0.0/dbhome_1/nls/data/9idata ...
Copying files to /u01/app/oracle/product/19.0.0.0/dbhome_1/nls/data/9idata...
Copy finished.
Please reset environment variable ORA_NLS10 to /u01/app/oracle/product/19.0.0.0/dbhome_1/nls/data/9idata!
[oracle@sitdb ~]$


=> (Conditional) Revert group permissions on oracle executable.

It is recommended that you always start the database using srvctl, which automatically runs setasmgidwrap and resets the group to asmadmin.

Alternatively, you can manually run setasmgidwrap as the grid user to revert the group to asmadmin using the following commands:
$ sudo su grid
$ . oraenv
$ ORACLE_SID = [+ASM1] ? +ASM1
$ export PATH=$GRID_HOME/bin:$PATH
$ setasmgidwrap -o /u01/app/oracle/product/19.0.0.0/dbhome_1/bin/oracle


[oracle@sitdb 19cUpgd]$ ls -ltr /u01/app/oracle/product/19.0.0.0/dbhome_1/bin/oracle
-rwsr-s--x 1 oracle asmadmin 449406720 Sep  6 10:16 /u01/app/oracle/product/19.0.0.0/dbhome_1/bin/oracle
[oracle@sitdb 19cUpgd]$


=> Download and run hcheck.sql (in both CDB and PDB) from My Oracle Support Knowledge Document 136697.1, hcheck.sql - Script to Check for Known Problems in Oracle8i, Oracle9i, Oracle10g, Oracle 11g and Oracle 12c and Above.


The hcheck.sql script is a generic lightweight script that checks for this issue and other common data dictionary problems.

Note: If the hcheck.sql script reports any HCKE-nnnn errors, contact Oracle Support before proceeding further with the upgrade.


---------------------------------------
06-SEP-2022 11:44:29  Elapsed: 1 secs
---------------------------------------
Found 0 potential problem(s) and 0 warning(s)
Note that hcheck has been run on CDB$ROOT, consider to run it in each PDB

PL/SQL procedure successfully completed.

Statement processed.

Complete output is in trace file:
/u01/app/oracle/diag/rdbms/sitcdb_dxb1mz/SITCDB/trace/SITCDB_ora_13048_HCHECK.trc


=> Parameter Changes


SQL> create pfile='/home/oracle/init-orig.ora' from spfile;
File created.

=> unset local_listener parameter 

SQL> show parameter local_listener
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
local_listener                       string      LISTENER_SITCDB
SQL> alter system set local_listener='';
System altered.
SQL> show parameter local_listener
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
local_listener                       string
SQL>




=> Take count of invalid objects from CDB and PDB's 

select name,user from v$database;
select count(*) from dba_objects where status='INVALID';
create table invalids_ch as (select object_name,owner from dba_objects where status='INVALID');
select count(*) from invalids_ch ;


=> Get database id and oracle home id 



[root@sitdb ~]# dbcli list-databases

ID                                       DB Name    DB Type  DB Version           CDB        Class    Shape    Storage    Status        DbHomeID
---------------------------------------- ---------- -------- -------------------- ---------- -------- -------- ---------- ------------ ----------------------------------------
17982ac7-8753-4209-bed9-2d12b8a4f355     SITCDB     Si       12.1.0.2.220719      true       Oltp              ASM        Configured   6f64870a-8b63-4530-8a9d-132438e273a5
[root@sitdb ~]# dbcli list-dbhomes

ID                                       Name                 DB Version                               Home Location                                 Status
---------------------------------------- -------------------- ---------------------------------------- --------------------------------------------- ----------
6f64870a-8b63-4530-8a9d-132438e273a5     OraDB12102_home1     12.1.0.2.220719                          /u01/app/oracle/product/12.1.0.2/dbhome_1     Configured
03f77e36-98c1-4f0d-ac7f-923e9073a654     OraDB19000_home1     19.15.0.0.0                              /u01/app/oracle/product/19.0.0.0/dbhome_1     Configured

[root@sitdb ~]#


=> Run Pre-Check 


dbcli upgrade-database -i <source Oracle 12c database ID> -dh <target 19C Oracle Home ID> --upgradeOptions "-keepEvents" --precheck

[root@sitdb ~]# dbcli upgrade-database -i 17982ac7-8753-4209-bed9-2d12b8a4f355 -dh 03f77e36-98c1-4f0d-ac7f-923e9073a654 --upgradeOptions "-keepEvents" --precheck
{
  "jobId" : "2009ef7f-70a3-4043-9533-5e2e9392a10d",
  "status" : "Created",
  "message" : null,
  "errorCode" : "",
  "reports" : [ ],
  "createTimestamp" : "September 06, 2022 13:09:57 PM GST",
  "resourceList" : [ ],
  "description" : "Database upgrade precheck with dbResId : 17982ac7-8753-4209-bed9-2d12b8a4f355",
  "updatedTime" : "September 06, 2022 13:09:57 PM GST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}
[root@sitdb ~]#


[root@sitdb ~]# dbcli describe-job -i 2009ef7f-70a3-4043-9533-5e2e9392a10d -l Verbose

Job details
----------------------------------------------------------------
                     ID:  2009ef7f-70a3-4043-9533-5e2e9392a10d
            Description:  Database upgrade precheck with dbResId : 17982ac7-8753-4209-bed9-2d12b8a4f355
                 Status:  Success
                Created:  September 6, 2022 1:09:57 PM GST
               Progress:  100%
                Message:
             Error Code:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
Database Service upgrade precheck                                        September 6, 2022 1:09:57 PM GST    September 6, 2022 1:11:48 PM GST    Success
Database Service upgrade precheck                                        September 6, 2022 1:09:57 PM GST    September 6, 2022 1:11:47 PM GST    Success
Database Service upgrade precheck                                        September 6, 2022 1:09:57 PM GST    September 6, 2022 1:09:57 PM GST    Success
Database Service upgrade precheck                                        September 6, 2022 1:09:57 PM GST    September 6, 2022 1:09:57 PM GST    Success
Generating SSH key                                                       September 6, 2022 1:09:57 PM GST    September 6, 2022 1:09:57 PM GST    Success
Setting up ssh equivalance                                               September 6, 2022 1:09:57 PM GST    September 6, 2022 1:09:57 PM GST    Success
Database Service upgrade precheck                                        September 6, 2022 1:09:57 PM GST    September 6, 2022 1:09:57 PM GST    Success
Database Service upgrade precheck                                        September 6, 2022 1:09:58 PM GST    September 6, 2022 1:09:58 PM GST    Success
PreCheck executePreReqs                                                  September 6, 2022 1:09:58 PM GST    September 6, 2022 1:11:35 PM GST    Success
Removing ssh keys                                                        September 6, 2022 1:11:35 PM GST    September 6, 2022 1:11:47 PM GST    Success
Removing Znode                                                           September 6, 2022 1:11:35 PM GST    September 6, 2022 1:11:35 PM GST    Success
Removing keys from authorized_keys file                                  September 6, 2022 1:11:35 PM GST    September 6, 2022 1:11:35 PM GST    Success
Removing PrivatePublic key Files                                         September 6, 2022 1:11:35 PM GST    September 6, 2022 1:11:35 PM GST    Success
Running updatedb command                                                 September 6, 2022 1:11:35 PM GST    September 6, 2022 1:11:47 PM GST    Success
Running SSHKey Gen                                                       September 6, 2022 1:11:47 PM GST    September 6, 2022 1:11:47 PM GST    Success

[root@sitdb ~]#



=> Upgrade Database (Downtime)


Duration : 2 Hours 

[root@sitdb ~]# dbcli upgrade-database -i 17982ac7-8753-4209-bed9-2d12b8a4f355 -dh 03f77e36-98c1-4f0d-ac7f-923e9073a654 --upgradeOptions "-keepEvents"
{
  "jobId" : "cd96cef4-b030-4690-b38c-483a1748362f",
  "status" : "Created",
  "message" : null,
  "errorCode" : "",
  "reports" : [ ],
  "createTimestamp" : "September 06, 2022 13:13:54 PM GST",
  "resourceList" : [ ],
  "description" : "Database upgrade with resource Id : 17982ac7-8753-4209-bed9-2d12b8a4f355",
  "updatedTime" : "September 06, 2022 13:13:59 PM GST",
  "percentageProgress" : "0%",
  "cause" : null,
  "action" : null
}
[root@sitdb ~]


[root@sitdb ~]# dbcli describe-job -i cd96cef4-b030-4690-b38c-483a1748362f -l Verbose

Job details
----------------------------------------------------------------
                     ID:  cd96cef4-b030-4690-b38c-483a1748362f
            Description:  Database upgrade with resource Id : 17982ac7-8753-4209-bed9-2d12b8a4f355
                 Status:  Running
                Created:  September 6, 2022 1:13:54 PM GST
               Progress:  0%
                Message:
             Error Code:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
Database Service Upgrade                                                 September 6, 2022 1:13:59 PM GST    September 6, 2022 1:13:59 PM GST    Running
Database Service Upgrade                                                 September 6, 2022 1:13:59 PM GST    September 6, 2022 1:13:59 PM GST    Running
Database Service Upgrade                                                 September 6, 2022 1:13:59 PM GST    September 6, 2022 1:13:59 PM GST    Success
Database Service Upgrade                                                 September 6, 2022 1:13:59 PM GST    September 6, 2022 1:13:59 PM GST    Running
Database Service Upgrade                                                 September 6, 2022 1:13:59 PM GST    September 6, 2022 1:14:00 PM GST    Success
Generating SSH key                                                       September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:00 PM GST    Success
Setting up ssh equivalance                                               September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:00 PM GST    Success
Database Service Upgrade                                                 September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:00 PM GST    Success
Database Service Upgrade                                                 September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:00 PM GST    Success
Database Service Upgrade                                                 September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:00 PM GST    Success
Updating SqlNet for db upgrade.                                          September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:00 PM GST    Success
Update RECO disk Size                                                    September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:03 PM GST    Success
task:TaskZJsonRpcExt_1600                                                September 6, 2022 1:14:00 PM GST    September 6, 2022 1:14:03 PM GST    Success
PreCheck executePreReqs                                                  September 6, 2022 1:14:04 PM GST    September 6, 2022 1:14:04 PM GST    Running

[root@sitdb ~]#



[root@sitdb ~]# dbcli describe-job -i cd96cef4-b030-4690-b38c-483a1748362f

Job details
----------------------------------------------------------------
                     ID:  cd96cef4-b030-4690-b38c-483a1748362f
            Description:  Database upgrade with resource Id : 17982ac7-8753-4209-bed9-2d12b8a4f355
                 Status:  Success
                Created:  September 6, 2022 1:13:54 PM GST
               Progress:  100%
                Message:
             Error Code:

Task Name                                                                Start Time                          End Time                            Status
------------------------------------------------------------------------ ----------------------------------- ----------------------------------- ----------
PreCheck executePreReqs                                                  September 6, 2022 1:14:04 PM GST    September 6, 2022 1:15:21 PM GST    Success
Database Upgrade                                                         September 6, 2022 1:15:23 PM GST    September 6, 2022 2:54:10 PM GST    Success
drop GRP                                                                 September 6, 2022 2:56:34 PM GST    September 6, 2022 2:56:40 PM GST    Success

[root@sitdb ~]#



=> Check CDB and PDB Status 


SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 SITPDB                         READ WRITE YES


SQL> alter pluggable database SITPDB close immediate;

Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 SITPDB                         MOUNTED
SQL> alter pluggable database SITPDB open;

Pluggable database altered.

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 SITPDB                         READ WRITE NO
SQL>


SQL> alter pluggable database SITPDB save state; 

Pluggable database altered.


=> Check for any new invalid objects

SQL> select count(*) from dba_objects where status='INVALID' and object_name not in (select object_name from invalids_ch);

  COUNT(*)
----------
         0

SQL> alter session set container=SITPDB;

Session altered.

SQL> select count(*) from dba_objects where status='INVALID';

  COUNT(*)
----------
         0

SQL> exit


=> Check lspatches 


[oracle@sitdb ~]$ export PATH=$ORACLE_HOME/OPatch:$PATH
[oracle@sitdb ~]$ opatch lspatches
33308736;DATAPATCH HANGS AS CATCON TRIES TO START MANY PROCESSES IN EXA ENVIRONMENTS
31424070;APPSST19C  XTTS PDB - TABLE IMPORT/CREATION FAILED WITH ORA-39083 ORA-14334
34096213;Fix for Bug 34096213
33810360;CVE-2021-45943 REPORTED IN (OPEN SOURCE GEOSPATIAL FOUNDATION/GEOSPATIAL DATA ABSTRACTION LIBRARY/OPENGIS SIMPLE FEATURES REFERENCE IMPLEMENTATION (GDAL/OGR)/3.3.0)
34088989;ONE-OFF REQUEST FOR DELETE DIR FOR SUPTOOLS/TFA + SUPTOOLS/ORACHK + SUPTOOLS/ORACHK.ZIP FROM DB PSU/BP/RU/RUR
33809062;TRACKING BUG FOR REGRESSION RTI 24544369 CAUSED BY PKNAGGS_BUG-32472737 APPROVED/INCLUDED IN 21.0.0.0 ADBSBP
33613829;RDBMS - DSTV37 UPDATE - TZDATA2021E
32327201;RDBMS - DSTV36 UPDATE - TZDATA2020E
31335037;RDBMS - DSTV35 UPDATE - TZDATA2020A
30432118;MERGE REQUEST ON TOP OF 19.0.0.0.0 FOR BUGS 28852325 29997937
33808367;OJVM RELEASE UPDATE: 19.15.0.0.220419 (33808367)
31732095;UPDATE PERL IN 19C DATABASE ORACLE HOME TO V5.32
33810130;JDK BUNDLE PATCH 19.0.0.0.220419
33815596;OCW RELEASE UPDATE 19.15.0.0.0 (33815596)
33806152;Database Release Update : 19.15.0.0.220419 (33806152)

OPatch succeeded.
[oracle@sitdb ~]$



=> Database is successfully upgraded from 12C to 19C



No comments:

Post a Comment