Exadata X8M : Cell Disks and ASM Disks Overview

=> Run below command on any of the cell storage server as celladmin user to identify all cell disks attached to the storage server.

=> In Exadata, a LUN (Logical Unit) is a logical abstraction of a storage device. LUNs are based on hard disks, flash disks, and persistent memory devices. LUNs are automatically created when Exadata is initially configured.

[celladmin@cellstorage01 ~]$ cellcli
CellCLI: Release 20.1.4.0.0 - Production on Wed Jan 11 07:40:55 UTC 2023
Copyright (c) 2007, 2016, Oracle and/or its affiliates. All rights reserved.
CellCLI> list lun
/opt/oracle/cell/disks/raw/DISK00  /opt/oracle/cell/disks/raw/DISK00  normal
/opt/oracle/cell/disks/raw/DISK01  /opt/oracle/cell/disks/raw/DISK01  normal
/opt/oracle/cell/disks/raw/DISK02  /opt/oracle/cell/disks/raw/DISK02  normal
/opt/oracle/cell/disks/raw/DISK03  /opt/oracle/cell/disks/raw/DISK03  normal
/opt/oracle/cell/disks/raw/DISK04  /opt/oracle/cell/disks/raw/DISK04  normal
/opt/oracle/cell/disks/raw/DISK05  /opt/oracle/cell/disks/raw/DISK05  normal
/opt/oracle/cell/disks/raw/DISK06  /opt/oracle/cell/disks/raw/DISK06  normal
/opt/oracle/cell/disks/raw/DISK07  /opt/oracle/cell/disks/raw/DISK07  normal
/opt/oracle/cell/disks/raw/DISK08  /opt/oracle/cell/disks/raw/DISK08  normal
/opt/oracle/cell/disks/raw/DISK09  /opt/oracle/cell/disks/raw/DISK09  normal
/opt/oracle/cell/disks/raw/DISK10  /opt/oracle/cell/disks/raw/DISK10  normal
/opt/oracle/cell/disks/raw/DISK11  /opt/oracle/cell/disks/raw/DISK11  normal
/opt/oracle/cell/disks/raw/DXDDISK0 /opt/oracle/cell/disks/raw/DXDDISK0 normal
/opt/oracle/cell/disks/raw/DXDDISK1 /opt/oracle/cell/disks/raw/DXDDISK1 normal
/opt/oracle/cell/disks/raw/FLASH00 /opt/oracle/cell/disks/raw/FLASH00 normal
/opt/oracle/cell/disks/raw/FLASH01 /opt/oracle/cell/disks/raw/FLASH01 normal
/opt/oracle/cell/disks/raw/FLASH02 /opt/oracle/cell/disks/raw/FLASH02 normal
/opt/oracle/cell/disks/raw/FLASH03 /opt/oracle/cell/disks/raw/FLASH03 normal
CellCLI>

=> To get more details of a particular LUN run below command, note that each LUN is associated with one Physical Disk and one Cell Disk

CellCLI> list lun where name like '.*DISK09' detail
name:                   /opt/oracle/cell/disks/raw/DISK09
cellDisk:               CD_09_cellstorage01
deviceName:             /opt/oracle/cell/disks/raw/DISK09
diskType:               HardDisk
id:                     /opt/oracle/cell/disks/raw/DISK09
isSystemLun:            FALSE
lunSize:                11
physicalDrives:         /opt/oracle/cell/disks/raw/DISK09
raidLevel:              "RAID 0"
status:                 normal
CellCLI>

=> Examine the attributes of the hard disk associated with LUN. Exadata maintains the physical attributes of each hard disk in a physicaldisk object. A physicaldisk object is automatically created for each hard disk.

CellCLI> list physicaldisk where luns like '.*DISK09' detail
name:                   /opt/oracle/cell/disks/raw/DISK09
deviceName:             /opt/oracle/cell/disks/raw/DISK09
diskType:               HardDisk
luns:                   /opt/oracle/cell/disks/raw/DISK09
physicalInsertTime:     2021-03-13T00:41:36+00:00
physicalSerial:         /opt/oracle/cell/disks/raw/DISK09
physicalSize:           11
status:                 normal
CellCLI>

=> A cell disk is a higher-level storage abstraction. Each cell disk is based on a LUN and contains additional attributes and metadata. Examine the attributes of Cell Disk based on the previous output

CellCLI> list celldisk CD_09_cellstorage01 detail
name:                   CD_09_cellstorage01
comment:                
creationTime:           2021-03-13T00:41:37+00:00
deviceName:             /opt/oracle/cell/disks/raw/DISK09
devicePartition:        /opt/oracle/cell/disks/raw/DISK09
diskType:               HardDisk
errorCount:             0
freeSpace:              0
id:                     913f3c88-3729-4f4b-9913-6ced4f293fe3
physicalDisk:           /opt/oracle/cell/disks/raw/DISK09
size:                   2G
status:                 normal
CellCLI>

=> A grid disk defines an area of storage on a cell disk. Grid disks are consumed by ASM and are used as the storage for ASM disk groups. Each cell disk can contain a number of grid disks. Examine the grid disks associated with the cell disk you examined in the previous step. Note the names and sizes of the grid disks.

CellCLI> list griddisk where celldisk=CD_09_cellstorage01 detail
name:                   DATA_CD_09_cellstorage01
asmDiskGroupName:       DATA
asmDiskName:            DATA_CD_09_cellstorage01
asmFailGroupName:       cellstorage01
availableTo:            
cachingPolicy:          default
cellDisk:               CD_09_cellstorage01
comment:                
creationTime:           2021-03-13T00:41:37+00:00
diskType:               HardDisk
errorCount:             0
id:                     8c4fc63e-fa92-4942-af03-27c0167f3e5c
size:                   1.046875G
status:                 active
name:                   RECO_CD_09_cellstorage01
asmDiskGroupName:       RECO
asmDiskName:            RECO_CD_09_cellstorage01
asmFailGroupName:       cellstorage01
availableTo:            
cachingPolicy:          default
cellDisk:               CD_09_cellstorage01
comment:                
creationTime:           2021-03-13T00:41:37+00:00
diskType:               HardDisk
errorCount:             0
id:                     8ee1e13e-dc12-4787-85d0-a94bb8d76637
size:                   928M
status:                 active
CellCLI>

=> Establish a terminal connection to database server as the grid user and configure the session to access ASM using the +ASM1 instance.

Locate the grid disks inside ASM. Check that the sizes reported by ASM match the grid disk attributes reported in previous steps

[grid@exadb01~]$ . oraenv
ORACLE_SID = [grid] ? +ASM1
The Oracle base has been set to /u01/app/grid
[grid@exadb01 ~]$ sqlplus / as sysasm
SQL*Plus: Release 19.0.0.0.0 - Production on Wed Jan 11 10:10:23 2023
Version 19.9.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.9.0.0.0

SQL> select name, path, state, total_mb from v$asm_disk
  2  where name like '%_CD_09_cellstorage01'
  3  /

NAME                                  PATH                        STATE    TOTAL_MB
-------------------------------   --------------------------------------------------------------   ------------  --------------------- 
DATA_CD_09_cellstorage01   o/192.168.1.105;192.168.1.106/DATA_CD_09_cellstorage01 NORMAL        1072
RECO_CD_09_cellstorage01        o/192.168.1.105;192.168.1.106/RECO_CD_09_cellstorage01 NORMAL     928

SQL>

=> Determine which ASM disk group the grid disks from previous steps are assigned to.

SQL> select d.name disk, dg.name diskgroup
  2  from v$asm_disk d, v$asm_diskgroup dg
  3  where dg.group_number = d.group_number
  4  and d.name like '%_CD_09_cellstorage01'
  5  /
DISK        DISKGROUP
------------------------------ ------------------------------
DATA_CD_09_cellstorage01        DATA
RECO_CD_09_cellstorage01        RECO
SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.9.0.0.0
[grid@exadb01 ~]$