Wednesday, January 11, 2023

Exadata X8M : Cell Disks and ASM Disks Overview

Oracle Exadata Cell Disks Explained - From Physical Media to ASM Capacity
Oracle Exadata Series

Oracle Exadata Cell Disks Explained How physical media becomes CellCLI-managed capacity, then turns into grid disks and ASM space that databases can actually use.

Cell disks sit in the middle of the Exadata storage model. They are not the raw drives you pull from the chassis, and they are not the ASM disks your database team sees in SQL. A useful mental model is: physical disk or LUN first, cell disk second, grid disk third, ASM disk and disk group last. Once that layering is clear, Exadata storage tasks become much easier to reason about, especially when you are validating capacity, mapping failures, or planning changes.

4 layersPhysical to ASM path
1 objectCell disk per LUN
2 viewsCellCLI and ASM perspective
3 checksStatus, freespace, mapping

The mental model: cell disks are the storage-cell layer between hardware and ASM

Exadata deliberately separates storage objects into layers so each layer can be managed for a different purpose. The physical device or LUN is the hardware-facing end. A cell disk is the Exadata storage-software object created on top of that device or LUN. One or more grid disks are then carved from the cell disk, and those grid disks are what ASM discovers and treats as disks inside disk groups such as DATA or RECO.

The practical consequence is simple: when a DBA says a disk group is short on space, the answer is rarely visible at only one layer. You often need to inspect grid disks and cell disks together. When a storage admin says a disk has a problem, you need to know whether the fault is at the physical-disk level, the cell-disk level, or only in how the space is allocated above it.

1. Physical disk or LUN Drive, flash device, or presented LUN Hardware-facing identity 2. Cell disk Managed by CellCLI Exadata reserves the LUN here 3. Grid disks Logical slices from a cell disk Mapped to ASM usage 4. ASM disks and groups Discovered as Exadata paths What the database consumes Why this layering matters operationally Capacity is allocated in the cell-disk and grid-disk layers. Database visibility starts at ASM, so troubleshooting works best when you follow the full chain.
The layer that gets skipped most often in conversations is the cell disk. That omission is exactly why some storage discussions become confusing.

Hardware view

A physical disk or presented LUN describes the media. It does not yet tell you how Exadata storage software has reserved or divided it.

CellCLI view

Cell disks and grid disks are storage-cell objects. This is where you inspect mapping, free space, and allocation choices.

ASM view

ASM sees the Exadata-presented disks, not the internal storage-cell layering that produced them.

First principle

When you need to explain Exadata storage cleanly, avoid collapsing all layers into the word “disk”. The same word can mean a drive, a cell disk, a grid disk, or an ASM disk, and those are not interchangeable.

What a cell disk actually is, and what it is not

A cell disk is created from a physical disk or from a LUN presented to the storage cell. Exadata uses the cell-disk object to reserve that underlying storage for its own software stack. From there, the space can be divided into grid disks or pool disks depending on the storage design in use. That is why a cell disk is more than a simple label: it is the management boundary where the cell claims and organizes underlying capacity.

There is also an important sizing and identity rule: only one cell disk can be created on a given LUN. If the physical disk is not partitioned into LUNs, the cell disk can be created directly on the whole physical disk. That rule is one of the reasons mapping mistakes become easier to spot once you inspect the layer correctly.

One per LUNA LUN maps to at most one cell disk
CellCLI-ownedCreation and inspection happen at the storage cell
Allocation sourceGrid disks are carved from cell-disk capacity
Object What it represents Where you inspect it Why it matters
Physical disk / LUN The underlying storage device or externally presented logical unit. LIST PHYSICALDISK and hardware inventory. Tells you what media exists before Exadata carves it into managed objects.
Cell disk The Exadata storage-software object created on a physical disk or LUN. LIST CELLDISK, DESCRIBE CELLDISK. Defines the reserve-and-allocate layer from which higher objects are built.
Grid disk A logical allocation carved from a cell disk. LIST GRIDDISK. Connects cell capacity to a specific ASM use such as DATA or RECO.
ASM disk The database-facing disk discovered by ASM over Exadata storage paths. V$ASM_DISK, asmcmd lsdsk. Shows what the ASM instance can actually consume and rebalance.
ASM disk group A collection of ASM disks managed together. V$ASM_DISKGROUP, asmcmd lsdg. Represents the space the database teams usually think about first.
Operational caution

Most cell-disk work is infrastructural, not day-to-day SQL administration. Treat create, drop, or import operations as storage changes with database consequences, because the objects above the cell disk depend on it.

Commonly understood correctly

  • ASM disk groups are where databases consume capacity.
  • Grid disks are the unit most directly tied to those groups.
  • CellCLI is the right tool for the storage-cell layer.

Commonly blurred together

  • A physical drive is not the same thing as a cell disk.
  • A cell disk is not the same thing as an ASM disk.
  • Disk-group free space does not by itself explain the cell-side layout.

How a cell disk becomes database-visible capacity

The handoff from storage cell to database happens through grid disks. A grid disk records which cell disk it came from, its size, and the ASM disk group name it is intended for. On the ASM side, Exadata disks are discovered through Exadata-specific paths, which is why V$ASM_DISK and asmcmd show the database-facing picture rather than the internal cell allocation itself.

This layered split is useful during troubleshooting. If a grid disk looks fine in CellCLI but is missing or problematic in ASM, the conversation shifts toward discovery, disk state, or disk-group membership. If the grid disk is absent or undersized in CellCLI, you already know the issue is below ASM.

CellCLI: map cell disks to grid disks
-- On the storage cell as celladmin
CellCLI> LIST CELLDISK ATTRIBUTES name, physicalDisk, deviceName, size, freespace, status

-- Then inspect how those cell disks are allocated upward
CellCLI> LIST GRIDDISK ATTRIBUTES name, cellDisk, asmDiskgroupName, size, status

-- If you need the full object model, inspect the attribute list first
CellCLI> DESCRIBE CELLDISK
CellCLI> DESCRIBE GRIDDISK
ASM: inspect Exadata-visible disks
-- In the ASM instance
SELECT d.path,
       d.name,
       d.header_status,
       d.mode_status,
       g.name AS diskgroup_name
FROM   v$asm_disk d
       LEFT JOIN v$asm_diskgroup g
         ON d.group_number = g.group_number
WHERE  d.path LIKE 'o/%'
ORDER BY g.name, d.path;
Storage cell side Cell disk: CD_00_cell01 size, freespace, status Grid disk: DATA_CD_00_cell01 cellDisk, asmDiskgroupName, size Exadata path handoff ASM discovery uses paths under o/ Database sees the presented disk rather than the internal cell metadata ASM side Path like: o/cell01/DATA_CD_00_cell01 header_status, mode_status, group_number Disk group membership DATA, RECO, or other group design Troubleshooting is faster when you verify capacity from both sides of the handoff.
The object names above are illustrative, but the operational idea is exact: grid disks bridge cell-managed storage to ASM-visible disks.
Practical shortcut

If a disk-group conversation is vague, ask for two things immediately: the CellCLI mapping of grid disk to cell disk, and the ASM path and status for the corresponding Exadata disk. That usually narrows the problem faster than debating symptoms.

Capacity changes: freespace, resizing, and why geometry is more logical than physical

One of the most useful documented details in Exadata storage management is that grid disks can use space anywhere on their cell disks. They do not need to occupy one contiguous physical region. That means cell-disk free space is a logical allocation pool, not something you should picture as a single unbroken stripe waiting at the end of a device.

That design simplifies some capacity operations, but it does not make them casual. Before adjusting grid-disk sizes, you still want to inspect cell-disk freespace, understand which ASM disk group is affected, and account for the ASM consequences of the change. Storage layout and ASM rebalance behavior are linked operationally even when they are administered from different layers.

Change question What to inspect first Reasoning Typical safe posture
Can I grow a grid disk? Cell-disk freespace and current grid-disk mapping. Growth consumes cell-disk free capacity from the storage-cell side. Confirm room at the cell layer before thinking about ASM benefits.
Can I shrink a grid disk? ASM free space and disk-group pressure first, then current allocations. Storage can be reduced only if the database side can tolerate the lower capacity safely. Treat shrink work as a coordinated storage-plus-ASM change.
Is cell-disk free space “fragmented”? Look at the documented allocation model, not just a physical picture in your head. Grid disks do not need contiguous regions on the cell disk. Reason from reported free space and mappings, not from partition intuition.
Why does ASM still need attention? Disk group membership, rebalance expectations, and mode status. The database consumes the result of the cell-side change, not the cell disk directly. Validate on both layers before declaring a resize complete.

Good change-planning questions

  • Which cell disks supply the grid disks I am about to affect?
  • How much freespace is left per cell disk right now?
  • Which ASM disk group consumes those grid disks today?
  • What will verification look like after the change?

Questions that are too vague

  • “Does the cell have enough disk?” without naming the object layer.
  • “Can I resize storage?” without naming the grid disks or disk groups.
  • “ASM has room, so storage must be fine” without checking CellCLI.
  • “The drive is healthy, so the higher layers must be healthy” without mapping upward.
Useful nuance

Exadata exposes enough metadata to make storage reasoning concrete. A resize discussion becomes much safer when you anchor it in cellDisk, asmDiskgroupName, size, and freespace rather than in informal shorthand.

Diagnostics that matter: status, import signals, and mapping consistency

A quick health check usually starts with status. In a normal steady state, you expect healthy cell disks and grid disks to report clean status values and consistent mappings. If those mappings no longer line up, or if objects are present on one layer but not another, you have found a more precise troubleshooting path than a generic “storage issue”.

There is also a class of cases where disks are moved or reintroduced. Exadata supports exporting and importing grid disks and cell disks, and documented import-related statuses such as importRequired or importForceRequired are a sign to use those workflows deliberately rather than recreating objects blindly. That distinction matters because recreating storage objects unnecessarily can turn a recoverable metadata problem into a destructive rebuild.

Signals that point you to the cell layer

  • LIST CELLDISK or LIST GRIDDISK shows unexpected status.
  • The expected grid-disk-to-cell-disk mapping is missing.
  • Reported cell-disk free space does not fit the requested capacity change.

Signals that push you upward to ASM

  • Grid disks exist in CellCLI, but ASM visibility or membership is not what you expect.
  • V$ASM_DISK paths under o/ are missing or not in the expected group.
  • The storage-cell picture looks clean, but the database side still reports pressure or state issues.

Misconception: “Cell disk” and “ASM disk” are basically synonyms

They describe different layers. A cell disk is a storage-cell object; an ASM disk is the database-facing disk discovered after grid-disk presentation.

Misconception: a resize is just a physical partition problem

Exadata documents grid-disk allocation as non-contiguous if needed, so the right model is managed logical allocation, not simple partition geometry.

Misconception: healthy hardware guarantees healthy allocation

A good physical disk can still sit under confusing or mismatched higher-layer allocations if you never inspect CellCLI and ASM together.

Misconception: import-related states mean “drop and recreate”

Import workflows exist for a reason. If the disk metadata says import is required, treat that as a workflow clue, not a reason to improvise.

Avoid the expensive mistake

If the storage cell tells you an object needs import handling, stop and verify the intended workflow before issuing destructive commands. Exadata gives you object state precisely so you do not have to guess.

Validation lab: trace one path from physical storage to disk group membership

A strong Exadata validation pass walks the chain in order. Start at the physical-disk layer, confirm the cell-disk object, map the grid disks that sit on top of it, and then verify the corresponding disks in ASM. This sequence gives you a reproducible way to answer both capacity and troubleshooting questions from observed evidence at each layer.

1. Start with hardware identity

List the physical disks so you know the device or LUN you are actually talking about.

2. Confirm the cell disk

Check size, device mapping, status, and free space at the storage-software layer.

3. Map all grid disks

Verify which ASM disk groups consume space from that cell disk.

4. Verify in ASM

Confirm the Exadata-visible disks and their group membership from the database side.

CellCLI and SQL validation sequence
-- Storage cell: inspect physical disks first
CellCLI> LIST PHYSICALDISK ATTRIBUTES name, deviceName, diskType, status

-- Then confirm the cell-disk layer
CellCLI> LIST CELLDISK ATTRIBUTES name, physicalDisk, deviceName, size, freespace, status

-- Then map cell disks upward to grid disks
CellCLI> LIST GRIDDISK ATTRIBUTES name, cellDisk, asmDiskgroupName, size, status

-- ASM instance: verify the presented Exadata disks
SELECT d.path,
       d.name,
       d.header_status,
       d.mode_status,
       g.name AS diskgroup_name
FROM   v$asm_disk d
       LEFT JOIN v$asm_diskgroup g
         ON d.group_number = g.group_number
WHERE  d.path LIKE 'o/%'
ORDER BY g.name, d.path;

What a clean result looks like

  • The physical disk, cell disk, and grid disk mappings are internally consistent.
  • Status values are clean at the CellCLI layer.
  • ASM paths under o/ line up with the expected disk groups.
  • The capacity story agrees from both the cell side and the ASM side.

What should trigger a deeper review

  • Unexpected import-related status at the cell-disk layer.
  • A grid disk that has no clear upward match in ASM.
  • Conflicting capacity conclusions between CellCLI and ASM.
  • People discussing a “disk problem” without agreeing on which layer they mean.

Quick quiz

The questions below test whether the object boundaries are clear. Clear object boundaries make Exadata storage behavior much easier to reason about.

7 questions CellCLI + ASM Layer mapping
Q1. Which sequence best describes the storage path from hardware to database use on Exadata?
ASM disk group -> grid disk -> cell disk -> physical disk
Physical disk or LUN -> cell disk -> grid disk -> ASM disk group
Physical disk -> ASM disk -> cell disk -> disk group
Grid disk -> physical disk -> cell disk -> ASM
Correct answer: physical disk or LUN, then cell disk, then grid disk, then ASM consumption.
Q2. What is the most accurate description of a cell disk?
An ASM failure group entry
A database datafile stored on Smart Flash Cache
An Exadata storage-software object created on a physical disk or LUN
A synonym for an ASM disk discovered by asmcmd
Correct answer: a cell disk is the storage-cell object created on the underlying device or LUN.
Q3. Which statement about a LUN and cell disks is correct?
Only one cell disk can be created on a given LUN.
A LUN must be split into at least two cell disks.
Cell disks exist only for flash media, not for hard disks.
Cell disks are optional if ASM disk groups already exist.
Correct answer: a LUN can have at most one cell disk.
Q4. Which CellCLI command is the best first step for checking how grid disks map upward from cell disks?
LIST PHYSICALDISK only
DROP GRIDDISK
ALTER CELLDISK immediately
LIST GRIDDISK ATTRIBUTES name, cellDisk, asmDiskgroupName, size, status
Correct answer: inspect grid-disk attributes so the mapping is visible before you change anything.
Q5. Why is cell-disk freespace important during resize planning?
Because ASM cannot see disk groups without it
Because grid-disk growth consumes capacity from the cell-disk allocation layer
Because physical disks disappear when freespace reaches zero
Because it directly replaces ASM rebalance checks
Correct answer: cell-disk free space is the source pool for cell-side allocation changes.
Q6. What does it mean that grid disks need not be contiguous on a cell disk?
They are always mirrored automatically by the cell
They bypass ASM completely
The allocation model is logical and can draw from free space anywhere on the cell disk
There is no such thing as cell-disk free space
Correct answer: allocation can use free space anywhere on the cell disk rather than one continuous segment.
Q7. If a cell disk reports an import-related status such as importRequired, what is the best mindset?
Use the documented import workflow intentionally instead of improvising destructive recreation
Ignore it if ASM still sees disks
Assume the hardware has failed beyond recovery
Drop the disk group first and ask questions later
Correct answer: treat the status as guidance toward the right workflow, not as a reason to guess.

No comments:

Post a Comment