The views expressed on this blog are my own and do not necessarily reflect the views of Oracle

January 2, 2012

ASM file number 2

ASM file number 2 - the disk directory - keeps track of all disks in the disk group. As disk groups are independent storage units, each disk group will have its own disk directory.

As far as ASM is concerned, the disk directory is just another file. It has its own entry in the file directory, it is mirrored as per the disk group redundancy policy and it grows as needed.

Each disk directory entry maintains the following information:
  • Disk number
  • Disk state
  • ASM disk name (this can differ from the disk's OS name)
  • Fail group name
  • Create time-stamp
  • Failure time-stamp
  • Failure timer
  • Resize target
  • Relative location in the staleness registry
  • Disk repair time
  • Zone information
V$ASM_DISK view

Most of the information maintained in the disk directories can be accessed via V$ASM_DISK view. It displays one row for every disk discovered, including disks which are not part of any disk group. Note that ASM performs disk discovery every time this view is queried, so those queries are expensive.

The following example shows the query output from V$ASM_DISK in an ASM instance:

SQL> SELECT group_number, disk_number, state, name, mount_status
FROM v$asm_disk;

GROUP_NUMBER DISK_NUMBER STATE    NAME         MOUNT_S
------------ ----------- -------- ------------ -------
           0           0 NORMAL                CLOSED
           0           1 NORMAL                CLOSED
           0           2 NORMAL                CLOSED
           0           6 NORMAL                CLOSED
           1           0 NORMAL   ASMDISK1     CACHED
           1           1 NORMAL   ASMDISK2     CACHED
           1           2 NORMAL   ASMDISK3     CACHED
           1           3 NORMAL   ASMDISK4     CACHED
           2           0 NORMAL   ASMDISK5     CACHED
           2           1 NORMAL   ASMDISK6     CACHED

10 rows selected.

Note that we see all disks available to ASM, including those that do not belong to a mounted disk group (GROUP_NUMBER=0).

V$ASM_DISK_STAT view

V$ASM_DISK_STAT displays the same information V$ASM_DISK does, but without performing discovery of new disks. The information comes from the SGA of the ASM instance. This results in a less expensive operation, but since discovery is not performed, the output of this view does not include information about disks that are new to the system.

The following example shows the query output from V$ASM_DISK_STAT in an ASM instance:

SQL> SELECT group_number, disk_number, state, name, mount_status
FROM v$asm_disk_stat;

GROUP_NUMBER DISK_NUMBER STATE    NAME         MOUNT_S
------------ ----------- -------- ------------ -------
           1           0 NORMAL   ASMDISK1     CACHED
           1           1 NORMAL   ASMDISK2     CACHED
           1           2 NORMAL   ASMDISK3     CACHED
           1           3 NORMAL   ASMDISK4     CACHED
           2           0 NORMAL   ASMDISK5     CACHED
           2           1 NORMAL   ASMDISK6     CACHED

6 rows selected.

This time, we only see disks for the mounted disk groups.

Locating the disk directory

We can query fixed table X$KFFXP in ASM instance, to find out which allocation units belong to file 2. I will also join V$ASM_DISK_STAT to get the disk names. Let's do that for disk group 1:

SQL> SELECT x.xnum_kffxp "Extent",
x.au_kffxp "AU",
x.disk_kffxp "Disk #",
d.name "Disk name"
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
and x.disk_kffxp=d.disk_number
and x.group_kffxp=1
and x.number_kffxp=2
ORDER BY 1, 2;

    Extent         AU     Disk # Disk name
---------- ---------- ---------- -------------------
         0          2          3 ASMDISK4
         0          3          0 ASMDISK1
         0          3          1 ASMDISK2

The result shows two things - that the file directory is triple mirrored and that the current size of the file directory is 3 physical extents, which equals to 3 allocation units. It is worth pointing out that even in a normal redundancy disk group, the disk directory is tripple mirrored.

Let's now use kfed to look at the disk dirctory entries for disk group 1. As the data is the same in all three allocation units, we can look at the first one, i.e. allocation unit 2 on disk ASMDISK4:

$ kfed read /dev/oracleasm/disks/ASMDISK4 aun=2 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            6 ; 0x002: KFBTYP_DISKDIR
...
kfddde[0].entry.incarn:               1 ; 0x024: A=1 NUMM=0x0
kfddde[0].entry.hash:                 0 ; 0x028: 0x00000000
kfddde[0].entry.refer.number:4294967295 ; 0x02c: 0xffffffff
kfddde[0].entry.refer.incarn:         0 ; 0x030: A=0 NUMM=0x0
kfddde[0].dsknum:                     0 ; 0x034: 0x0000
kfddde[0].state:                      2 ; 0x036: KFDSTA_NORMAL
kfddde[0].ddchgfl:                  132 ; 0x037: 0x84
kfddde[0].dskname:             ASMDISK1 ; 0x038: length=8
kfddde[0].fgname:              ASMDISK1 ; 0x058: length=8
kfddde[0].crestmp.hi:          32960657 ; 0x078: HOUR=0x11 DAYS=0x4 MNTH=0xc YEAR=0x7db
kfddde[0].crestmp.lo:        2843202560 ; 0x07c: USEC=0x0 MSEC=0x1f5 SECS=0x17 MINS=0x2a
kfddde[0].failstmp.hi:                0 ; 0x080: HOUR=0x0 DAYS=0x0 MNTH=0x0 YEAR=0x0
kfddde[0].failstmp.lo:                0 ; 0x084: USEC=0x0 MSEC=0x0 SECS=0x0 MINS=0x0
kfddde[0].timer:                      0 ; 0x088: 0x00000000
kfddde[0].size:                    4094 ; 0x08c: 0x00000ffe
kfddde[0].srRloc.super.hiStart:       0 ; 0x090: 0x00000000
kfddde[0].srRloc.super.loStart:       0 ; 0x094: 0x00000000
kfddde[0].srRloc.super.length:        0 ; 0x098: 0x00000000
kfddde[0].srRloc.incarn:              0 ; 0x09c: 0x00000000
kfddde[0].dskrprtm:                   0 ; 0x0a0: 0x00000000
kfddde[0].zones[0].start:             0 ; 0x0a4: 0x00000000
kfddde[0].zones[0].size:           4094 ; 0x0a8: 0x00000ffe
kfddde[0].zones[0].used:             47 ; 0x0ac: 0x0000002f
...
kfddde[1].entry.incarn:               1 ; 0x1e4: A=1 NUMM=0x0
kfddde[1].entry.hash:                 1 ; 0x1e8: 0x00000001
kfddde[1].entry.refer.number:4294967295 ; 0x1ec: 0xffffffff
kfddde[1].entry.refer.incarn:         0 ; 0x1f0: A=0 NUMM=0x0
kfddde[1].dsknum:                     1 ; 0x1f4: 0x0001
kfddde[1].state:                      2 ; 0x1f6: KFDSTA_NORMAL
kfddde[1].ddchgfl:                  132 ; 0x1f7: 0x84
kfddde[1].dskname:             ASMDISK2 ; 0x1f8: length=8
...
kfddde[2].entry.incarn:               1 ; 0x3a4: A=1 NUMM=0x0
kfddde[2].entry.hash:                 2 ; 0x3a8: 0x00000002
kfddde[2].entry.refer.number:4294967295 ; 0x3ac: 0xffffffff
kfddde[2].entry.refer.incarn:         0 ; 0x3b0: A=0 NUMM=0x0
kfddde[2].dsknum:                     2 ; 0x3b4: 0x0002
kfddde[2].state:                      2 ; 0x3b6: KFDSTA_NORMAL
kfddde[2].ddchgfl:                  132 ; 0x3b7: 0x84
kfddde[2].dskname:             ASMDISK3 ; 0x3b8: length=8
...

Disk information is kept in kfddde fields, where kfddde[0] is about disk 0, kfddde[1] about disk 1 and so on. This way we can access all information about all disks in the disk directory for this disk group. As it can be seen, most of the information is externalized via V$ASM views.

Conclusion

ASM disk directories maintain the information about disks in ASM disk groups. That information is externalized via V$ASM_DISK* views and can also be accessed via the kfed utility, provided we know where to look.