ASM file number 7

ASM metadata file number 7 - volume directory - keeps track of files associated with ASM Dynamic Volume Manager (ADVM) volumes.

An ADVM volume device is constructed from an ASM dynamic volume. One or more ADVM volume devices may be configured within each disk group. ASM Cluster File System (ACFS) is layered on ASM through the ADVM interface. ASM dynamic volume manager is another client of ASM - the same way the database is. When a volume is opened, the corresponding ASM file is opened and ASM extents are sent to the ADVM driver.

There are two file types associated with ADVM volumes

• ASMVOL – The volume file which is the container for the volume storage
• ASMVDRL – The file that contains the volume's Dirty Region Logging (DRL) information. This file is required for re-silvering mirrors

Turn up the ADVM volume

It is not necessary to create a dedicated disk group for ADVM, but it does make sense to do so. That way we keep the database files separate from the ACFS files. Let's have a look at an example.

SQL> create diskgroup ACFS
disk 'ORCL:ASMDISK5', 'ORCL:ASMDISK6'
attribute 'COMPATIBLE.ASM' = '11.2', 'COMPATIBLE.ADVM' = '11.2';

Diskgroup created.

To be able to add volumes to a disk group, attributes COMPATIBLE.ASM and COMPATIBLE.ADVM must be set to at least '11.2'. Also the ADVM/ACFS drivers have to loaded (this is always done in cluster environments, but it may have to be done manually in a single instance setup).

I can now create couple of volumes in this disk group.

$ asmcmd volcreate -G ACFS -s 2G ACFS_VOL1

$ asmcmd volcreate -G ACFS -s 2G ACFS_VOL2

$ asmcmd volinfo -a
Diskgroup Name: ACFS

Volume Name: ACFS_VOL1 Volume Device: /dev/asm/acfs_vol1-159 State: ENABLED Size (MB): 2048 Resize Unit (MB): 32 Redundancy: MIRROR Stripe Columns: 4 Stripe Width (K): 128 Usage: Mountpath: Volume Name: ACFS_VOL2 Volume Device: /dev/asm/acfs_vol2-159 State: ENABLED Size (MB): 2048 Resize Unit (MB): 32 Redundancy: MIRROR Stripe Columns: 4 Stripe Width (K): 128 Usage: Mountpath:

$

Note that there are no mount paths associated with the volumes as I haven't used them yet.

Let's now look at the ADVM volume metadata. First find the allocation units of the volume directory.

SQL> SELECT x.xnum_kffxp "Extent",
x.au_kffxp "AU",
x.disk_kffxp "Disk #",
d.name "Disk name"
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
and x.disk_kffxp=d.disk_number
and x.group_kffxp=2
and x.number_kffxp=7
ORDER BY 1, 2;

    Extent         AU     Disk # Disk name
---------- ---------- ---------- ------------------------------
         0         53          1 ASMDISK6
         0         53          0 ASMDISK5

Use kfed to have a look at the actual metadata.

$ kfed read /dev/oracleasm/disks/ASMDISK5 aun=53 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           22 ; 0x002: KFBTYP_VOLUMEDIR
...
kfvvde.entry.incarn:                  1 ; 0x024: A=1 NUMM=0x0
kfvvde.entry.hash:                    0 ; 0x028: 0x00000000
kfvvde.entry.refer.number:   4294967295 ; 0x02c: 0xffffffff
kfvvde.entry.refer.incarn:            0 ; 0x030: A=0 NUMM=0x0
kfvvde.volnm:           ++AVD_DG_NUMBER ; 0x034: length=15
kfvvde.usage:                           ; 0x054: length=0
kfvvde.dgname:                          ; 0x074: length=0
kfvvde.clname:                          ; 0x094: length=0
kfvvde.mountpath:                       ; 0x0b4: length=0
kfvvde.drlinit:                       0 ; 0x4b5: 0x00
kfvvde.pad1:                          0 ; 0x4b6: 0x0000
kfvvde.volfnum.number:                0 ; 0x4b8: 0x00000000
kfvvde.volfnum.incarn:                0 ; 0x4bc: 0x00000000
kfvvde.drlfnum.number:                0 ; 0x4c0: 0x00000000
kfvvde.drlfnum.incarn:                0 ; 0x4c4: 0x00000000
kfvvde.volnum:                        0 ; 0x4c8: 0x0000
kfvvde.avddgnum:                    159 ; 0x4ca: 0x009f
kfvvde.extentsz:                      0 ; 0x4cc: 0x00000000
kfvvde.volstate:                      4 ; 0x4d0: D=0 C=0 R=1
...

That was block 0 of the allocation unit 53. It only contains the marker for the ADVM volume (++AVD_DG_NUMBER). The actual volume info is in blocks 1 and up.

$ kfed read /dev/oracleasm/disks/ASMDISK5 aun=53 blkn=1 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           22 ; 0x002: KFBTYP_VOLUMEDIR
...
kfvvde.entry.incarn:                  1 ; 0x024: A=1 NUMM=0x0
kfvvde.entry.hash:                    0 ; 0x028: 0x00000000
kfvvde.entry.refer.number:   4294967295 ; 0x02c: 0xffffffff
kfvvde.entry.refer.incarn:            0 ; 0x030: A=0 NUMM=0x0
kfvvde.volnm:                 ACFS_VOL1 ; 0x034: length=9
kfvvde.usage:                           ; 0x054: length=0
kfvvde.dgname:                          ; 0x074: length=0
kfvvde.clname:                          ; 0x094: length=0
kfvvde.mountpath:                       ; 0x0b4: length=0
kfvvde.drlinit:                       0 ; 0x4b5: 0x00
kfvvde.pad1:                          0 ; 0x4b6: 0x0000
kfvvde.volfnum.number:              257 ; 0x4b8: 0x00000101
kfvvde.volfnum.incarn:        771971291 ; 0x4bc: 0x2e0358db
kfvvde.drlfnum.number:              256 ; 0x4c0: 0x00000100
kfvvde.drlfnum.incarn:        771971289 ; 0x4c4: 0x2e0358d9
kfvvde.volnum:                        1 ; 0x4c8: 0x0001
kfvvde.avddgnum:                    159 ; 0x4ca: 0x009f
kfvvde.extentsz:                      8 ; 0x4cc: 0x00000008
kfvvde.volstate:                      2 ; 0x4d0: D=0 C=1 R=0
...

$ kfed read /dev/oracleasm/disks/ASMDISK5 aun=53 blkn=2 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           22 ; 0x002: KFBTYP_VOLUMEDIR
...
kfvvde.entry.incarn:                  1 ; 0x024: A=1 NUMM=0x0
kfvvde.entry.hash:                    0 ; 0x028: 0x00000000
kfvvde.entry.refer.number:   4294967295 ; 0x02c: 0xffffffff
kfvvde.entry.refer.incarn:            0 ; 0x030: A=0 NUMM=0x0
kfvvde.volnm:                 ACFS_VOL2 ; 0x034: length=9
kfvvde.usage:                           ; 0x054: length=0
kfvvde.dgname:                          ; 0x074: length=0
kfvvde.clname:                          ; 0x094: length=0
kfvvde.mountpath:                       ; 0x0b4: length=0
kfvvde.drlinit:                       0 ; 0x4b5: 0x00
kfvvde.pad1:                          0 ; 0x4b6: 0x0000
kfvvde.volfnum.number:              259 ; 0x4b8: 0x00000103
kfvvde.volfnum.incarn:        771971303 ; 0x4bc: 0x2e0358e7
kfvvde.drlfnum.number:              258 ; 0x4c0: 0x00000102
kfvvde.drlfnum.incarn:        771971301 ; 0x4c4: 0x2e0358e5
kfvvde.volnum:                        2 ; 0x4c8: 0x0002
kfvvde.avddgnum:                    159 ; 0x4ca: 0x009f
kfvvde.extentsz:                      8 ; 0x4cc: 0x00000008
kfvvde.volstate:                      2 ; 0x4d0: D=0 C=1 R=0
...

Block 1 of ASM metadata file 7 has the information about the first volume (kfvvde.volnm: ACFS_VOL1). Note that there are two files associated with that volume:

• DRL file (kfvvde.drlfnum.number: 256)
• Volume file (kfvvde.volfnum.number: 257)

Block 2 has the information about the second volume (kfvvde.volnm: ACFS_VOL2). There are also two files associated with that volume:

• DRL file – kfvvde.drlfnum.number: 258
• Volume file – kfvvde.volfnum.number: 259

As these are special files, they are not shown in the output of 'asmcmd ls' command or when we query V$ASM_ALIAS. But they do show up in V$ASM_FILE view.

SQL> SELECT file_number "File #", bytes/1024/1024 "Size (MB)", type
FROM v$asm_file
WHERE group_number=2;

    File #  Size (MB) TYPE
---------- ---------- ----------
       256         17 ASMVDRL
       257       2048 ASMVOL
       258         17 ASMVDRL
       259       2048 ASMVOL

Create ASM cluster file system

I can now use the volume device to create an ASM cluster file system (ACFS).

# /sbin/mkfs -t acfs /dev/asm/acfs_vol1-159
mkfs.acfs: version                   = 11.2.0.3.0
mkfs.acfs: on-disk version           = 39.0
mkfs.acfs: volume                    = /dev/asm/acfs_vol1-159
mkfs.acfs: volume size               = 2147483648
mkfs.acfs: Format complete.

# mkdir /acfs1

# mount -t acfs /dev/asm/acfs_vol1-159 /acfs1

# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
...
oracleasmfs on /dev/oracleasm type oracleasmfs (rw)
/dev/asm/acfs_vol1-159 on /acfs1 type acfs (rw)

$ asmcmd volinfo -G ACFS ACFS_VOL1
Diskgroup Name: ACFS

 Volume Name: ACFS_VOL1
 Volume Device: /dev/asm/acfs_vol1-159
 State: ENABLED
 Size (MB): 2048
 Resize Unit (MB): 32
 Redundancy: MIRROR
 Stripe Columns: 4
 Stripe Width (K): 128
 Usage: ACFS
 Mountpath: /acfs1

$

Let's see if the mount path info now shows up in volume directory:

$ kfed read /dev/oracleasm/disks/ASMDISK6 aun=53 blkn=1 | grep mountpath
kfvvde.mountpath:                /acfs1 ; 0x0b4: length=6

It does as expected.

Conclusion

One or more ADVM volume devices may be configured within each disk group. ASM Cluster File System (ACFS) is layered on ASM through the ADVM interface. ASM dynamic volume manager is another client of ASM - the same way the database is.

There are two internal file types associated with ASM volumes:

• ASMVOL – The volume file which is the container for the volume storage
• ASMVDRL – The file that contains the volume's Dirty Region Logging (DRL) information