ASM files number 12 and 254

The staleness directory (ASM file number 12) contains metadata to map the slots in the staleness registry to particular disks and ASM clients. The staleness registry (ASM file number 254) tracks allocation units that become stale while the disks are offline. This applies to normal and high redundancy disk groups with the attribute COMPATIBLE.RDBMS set to 11.1 or higher. The staleness metadata is created when needed, and grows to accommodate additional offline disks.

When a disk goes offline, each RDBMS instance gets a slot in the staleness registry for that disk. This slot has a bit for each allocation unit in the offline disk. When an RDBMS instance I/O write is targeted for an offline disk, that instance sets the corresponding bit in the staleness registry.

When a disk is brought back online, ASM copies the allocation units, that have the staleness registry bit set, from the mirrored extents. Because only allocation units that should have changed while the disk was offline are updated, bringing a disk online is more efficient then adding a disk if was dropped instead of just offlined.

No stale disks

The staleness metadata structures are created as needed, which means the staleness directory and registry do not exist when all disks are online.

SQL> SELECT g.name "Disk group",
 g.group_number "Group#",
 d.disk_number "Disk#",
 d.name "Disk",
 d.mode_status "Disk status"
FROM v$asm_disk d, v$asm_diskgroup g
WHERE g.group_number=d.group_number and g.group_number<>0
ORDER BY 1, 2, 3;

Disk group       Group#      Disk# Disk         Disk status
------------ ---------- ---------- ------------ ------------
DATA                  1          0 ASMDISK1     ONLINE
                                 1 ASMDISK2     ONLINE
                                 2 ASMDISK3     ONLINE
RECO                  2          0 ASMDISK4     ONLINE
                                 1 ASMDISK5     ONLINE
                                 2 ASMDISK6     ONLINE

SQL> SELECT x.number_kffxp "File#",
 x.disk_kffxp "Disk#",
 x.xnum_kffxp "Extent",
 x.au_kffxp "AU",
 d.name "Disk name"
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
 and x.disk_kffxp=d.disk_number
 and x.number_kffxp in (12, 254)
ORDER BY 1, 2;

no rows selected

Stale disks

Staleness information will be created when a disk goes offline, but only when there are I/O writes intended for offline disks.

In the following example, I will offline the disk manually, with the ALTER DISKGROUP OFFLINE DISK command. But as far as stalenss metadata is concerned, it will be created irrespective of how and why a disk goes offline.

SQL> alter diskgroup RECO offline disk ASMDISK6;

Diskgroup altered.

SQL> SELECT g.name "Disk group",
 g.group_number "Group#",
 d.disk_number "Disk#",
 d.name "Disk",
 d.mode_status "Disk status"
FROM v$asm_disk d, v$asm_diskgroup g
WHERE g.group_number=d.group_number and g.group_number=2
ORDER BY 1, 2, 3;

Disk group       Group#      Disk# Disk         Disk status
------------ ---------- ---------- ------------ ------------
RECO                  2          0 ASMDISK4     ONLINE
                                 1 ASMDISK5     ONLINE
                                 2 ASMDISK6     OFFLINE

Database keeps writing to this disk group, and after a while we see the staleness directory and staleness registry created for this disk group

SQL> SELECT x.number_kffxp "File#",
 x.disk_kffxp "Disk#",
 x.xnum_kffxp "Extent",
 x.au_kffxp "AU",
 d.name "Disk name"
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
 and x.disk_kffxp=d.disk_number
 and d.group_number=2
 and x.number_kffxp in (12, 254)
ORDER BY 1, 2;

     File#      Disk#     Extent         AU Disk name
---------- ---------- ---------- ---------- ------------------------------
        12          0          0         86 ASMDISK4
                    1          0        101 ASMDISK5
                    2          0 4294967294 ASMDISK6
       254          0          0         85 ASMDISK4
                    1          0        100 ASMDISK5
                    2          0 4294967294 ASMDISK6

Look inside

There is not much to see in the actual metadata. Even kfed struggles to recognise these types of metadata blocks :)

$ kfed read /dev/oracleasm/disks/ASMDISK4 aun=86 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           21 ; 0x002: *** Unknown Enum ***
...
kffdnd.bnode.incarn:                  1 ; 0x000: A=1 NUMM=0x0
kffdnd.bnode.frlist.number:  4294967295 ; 0x004: 0xffffffff
kffdnd.bnode.frlist.incarn:           0 ; 0x008: A=0 NUMM=0x0
kffdnd.overfl.number:        4294967295 ; 0x00c: 0xffffffff
kffdnd.overfl.incarn:                 0 ; 0x010: A=0 NUMM=0x0
kffdnd.parent.number:                 0 ; 0x014: 0x00000000
kffdnd.parent.incarn:                 1 ; 0x018: A=1 NUMM=0x0
kffdnd.fstblk.number:                 0 ; 0x01c: 0x00000000
kffdnd.fstblk.incarn:                 1 ; 0x020: A=1 NUMM=0x0
kfdsde.entry.incarn:                  1 ; 0x024: A=1 NUMM=0x0
kfdsde.entry.hash:                    0 ; 0x028: 0x00000000
kfdsde.entry.refer.number:   4294967295 ; 0x02c: 0xffffffff
kfdsde.entry.refer.incarn:            0 ; 0x030: A=0 NUMM=0x0
kfdsde.cid:                       +ASMR ; 0x034: length=5
kfdsde.indlen:                        1 ; 0x074: 0x0001
kfdsde.flags:                         0 ; 0x076: 0x0000
kfdsde.spare1:                        0 ; 0x078: 0x00000000
kfdsde.spare2:                        0 ; 0x07c: 0x00000000
kfdsde.indices[0]:                    0 ; 0x080: 0x00000000
kfdsde.indices[1]:                    0 ; 0x084: 0x00000000
kfdsde.indices[2]:                    0 ; 0x088: 0x00000000
...

$ kfed read /dev/oracleasm/disks/ASMDISK4 aun=85 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           20 ; 0x002: *** Unknown Enum ***
...
kfdsHdrB.clientId:           1297301881 ; 0x000: 0x4d534179
kfdsHdrB.incarn:                      0 ; 0x004: 0x00000000
kfdsHdrB.dskNum:                      2 ; 0x008: 0x0002
kfdsHdrB.ub2spare:                    0 ; 0x00a: 0x0000
ub1[0]:                               0 ; 0x00c: 0x00
ub1[1]:                               0 ; 0x00d: 0x00
ub1[2]:                               0 ; 0x00e: 0x00
ub1[3]:                               0 ; 0x00f: 0x00
ub1[4]:                               0 ; 0x010: 0x00
ub1[5]:                               0 ; 0x011: 0x00
ub1[6]:                               0 ; 0x012: 0x00
ub1[7]:                              16 ; 0x013: 0x10
ub1[8]:                               0 ; 0x014: 0x00
...

Not much to see, as these are just bitmaps.

Conclusion

The staleness directory and staleness registry are supporting metadata structure for the disk offline and fast resync feature introduced in ASM version 11. The staleness directory contains metadata to map the slots in the staleness registry to particular disks and ASM clients. The staleness registry tracks allocation units that become stale while the disks are offline. This feature is relevant to normal and high redundancy disk groups only.