The views expressed on this blog are my own and do not necessarily reflect the views of Oracle

January 3, 2012

ASM file number 3


When ASM needs to make an atomic change to multiple metadata blocks, a log record is written into the Active Change Directory (ACD), which is ASM metadata file number 3. These log records are written in a single I/O.

ACD is divided into chunks (threads) and each running ASM instance has its own 42 MB chunk. When a disk group is created, a single chunk is allocated for the ACD. As more instances mount that disk group, the ACD grows (by 42 MB) to accommodate every running instance with its own ACD chunk.

The ACD components are:
  • ACDC - ACD checkpoint
  • ABA - ACD block address
  • LGE - ACD redo log record
  • BCD - ACD block change descriptor
Locating the active change directory

We can query X$KFFXP to find the ACD allocation units. This is ASM file number 3 hence number_kffxp=3 in our query:

SQL> SELECT x.xnum_kffxp "Extent",
x.au_kffxp "AU",
x.disk_kffxp "Disk #",
d.name "Disk name"
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
and x.disk_kffxp=d.disk_number
and x.group_kffxp=1
and x.number_kffxp=3
ORDER BY 1, 2;

    Extent         AU     Disk # Disk name
---------- ---------- ---------- ------------------------------
         0          4          0 ASMDISK5
         1          2          1 ASMDISK6
         2          5          0 ASMDISK5
...
        39         21          1 ASMDISK6
        40         24          0 ASMDISK5
        41         22          1 ASMDISK6

42 rows selected.

The query returned 42 rows, i.e. 42 allocation units. As the allocation unit size for this disk group is 1MB, that means the total size of the ACD is 42 MB.

If I recreate the disk group with the larger allocation unit size, say 4 MB, we should still end up with a 42 MB ACD. Let's have a look:

SQL> create diskgroup RECO external redundancy
disk 'ORCL:ASMDISK5', 'ORCL:ASMDISK6'
attribute 'au_size'='4M';

Diskgroup created.

And now the query from x$kffxp and v$asm_disk_stat returns 11 rows (showing that the ACD size is still 42 MB):

    Extent         AU     Disk # Disk name
---------- ---------- ---------- ------------------------------
         0          3          1 ASMDISK6
         1          3          0 ASMDISK5
         2          4          1 ASMDISK6
...
        10          8          1 ASMDISK6

11 rows selected.

Closer look at the ACD

Let's look at the ACD using the kfed utility. The last query shows that the ACD start at AU 3 on disk ASMDISK6. Note that with the allocation unit size of 4 MB, I have to specify ausz=4m on the kfed command line:

$ kfed read /dev/oracleasm/disks/ASMDISK6 ausz=4m aun=3 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            7 ; 0x002: KFBTYP_ACDC
...
kfracdc.eyec[0]:                     65 ; 0x000: 0x41
kfracdc.eyec[1]:                     67 ; 0x001: 0x43
kfracdc.eyec[2]:                     68 ; 0x002: 0x44
kfracdc.eyec[3]:                     67 ; 0x003: 0x43
kfracdc.thread:                       1 ; 0x004: 0x00000001
kfracdc.lastAba.seq:         4294967295 ; 0x008: 0xffffffff
kfracdc.lastAba.blk:         4294967295 ; 0x00c: 0xffffffff
kfracdc.blk0:                         1 ; 0x010: 0x00000001
kfracdc.blks:                     11263 ; 0x014: 0x00002bff
kfracdc.ckpt.seq:                     2 ; 0x018: 0x00000002
kfracdc.ckpt.blk:                     2 ; 0x01c: 0x00000002
kfracdc.fcn.base:                    16 ; 0x020: 0x00000010
kfracdc.fcn.wrap:                     0 ; 0x024: 0x00000000
kfracdc.bufBlks:                    512 ; 0x028: 0x00000200
kfracdc.strt112.seq:                  0 ; 0x02c: 0x00000000
kfracdc.strt112.blk:                  0 ; 0x030: 0x00000000

The output shows that this is indeed an ACD block (kfbh.type=KFBTYP_ACDC). The only interesting piece of information here is kfracdc.thread=1, which meand that this ACD belong to ASM instance 1. In a cluster, this would match the ASM instance number.

That was block 0, the begining of the ACD. Let's now look at block 1 - the actual ACD data.

$ kfed read /dev/oracleasm/disks/ASMDISK6 ausz=4m aun=3 blkn=1 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            8 ; 0x002: KFBTYP_CHNGDIR
...
kfracdb.lge[0].valid:                 1 ; 0x00c: V=1 B=0 M=0
kfracdb.lge[0].chgCount:              1 ; 0x00d: 0x01
kfracdb.lge[0].len:                  52 ; 0x00e: 0x0034
kfracdb.lge[0].kfcn.base:            13 ; 0x010: 0x0000000d
kfracdb.lge[0].kfcn.wrap:             0 ; 0x014: 0x00000000
kfracdb.lge[0].bcd[0].kfbl.blk:       0 ; 0x018: blk=0
kfracdb.lge[0].bcd[0].kfbl.obj:       4 ; 0x01c: file=4
kfracdb.lge[0].bcd[0].kfcn.base:      0 ; 0x020: 0x00000000
kfracdb.lge[0].bcd[0].kfcn.wrap:      0 ; 0x024: 0x00000000
kfracdb.lge[0].bcd[0].oplen:          4 ; 0x028: 0x0004
kfracdb.lge[0].bcd[0].blkIndex:       0 ; 0x02a: 0x0000
kfracdb.lge[0].bcd[0].flags:         28 ; 0x02c: F=0 N=0 F=1 L=1 V=1 A=0 C=0
kfracdb.lge[0].bcd[0].opcode:       212 ; 0x02e: 0x00d4
kfracdb.lge[0].bcd[0].kfbtyp:         9 ; 0x030: KFBTYP_COD_BGO
kfracdb.lge[0].bcd[0].redund:        17 ; 0x031: SCHE=0x1 NUMB=0x1
kfracdb.lge[0].bcd[0].pad:        63903 ; 0x032: 0xf99f
kfracdb.lge[0].bcd[0].KFRCOD_CRASH:   1 ; 0x034: 0x00000001
kfracdb.lge[0].bcd[0].au[0]:          8 ; 0x038: 0x00000008
kfracdb.lge[0].bcd[0].disks[0]:       0 ; 0x03c: 0x0000
...

We see that the ACD block 1 is of type KFBTYP_CHNGDIR, and contains the elements of kfracdb.lge[i] structure - the ASM redo records. Some of the things of interest there are the operation being performed (opcode) and the operation type (kfbtyp). None of this is very useful outside of the ACD context, so we will leave it at that.

Conclusion

This is an informational post only, to complete the ASM metadata story, as there are no practical benefits of understanding the inner works of the ACD.