The views expressed on this blog are my own and do not necessarily reflect the views of Oracle

January 8, 2012

ASM file number 8


The disk Used Space Directory (USD) – ASM file number 8 - maintains the number of allocation units (AU) used per zone, per disk in a disk group. The USD is split into a set of Used Space Entries (USE). Each USE will maintain a counter for the number of used AUs per disk, per zone. A disk zone can be either HOT or COLD.

This structure is version 11.2 specific and is relevant to the Intelligent Data Placement feature. The USD will be present in a newly created disk group in version 11.2 or when the ASM compatibility is advanced to 11.2.

Locating the used space directory

Let's get the allocation units for the used space directory - for all disk groups.

SQL> break on Group#
SQL> SELECT d.group_number "Group#",
 x.disk_kffxp "Disk#",
 x.xnum_kffxp "Extent",
 x.au_kffxp "AU",
 d.name "Disk name"
FROM x$kffxp x, v$asm_disk_stat d
WHERE x.group_kffxp=d.group_number
 and x.disk_kffxp=d.disk_number
 and x.number_kffxp=8
ORDER BY 1, 2;

 Group#  Disk#  Extent     AU Disk name
------- ------ ------- ------ ------------
      1      0       0     51 ASMDISK5
             1       0     51 ASMDISK6
      2      0       0     41 ASMDISK1
             2       0     39 ASMDISK3
             3       0     38 ASMDISK4

Check the disk used space allocation for all disks in all disk groups.

SQL> SELECT group_number "Group#",
 name "Disk name",
 hot_used_mb "Hot (MB)",
 cold_used_mb "Cold (MB)"
FROM v$asm_disk_stat
ORDER BY 1;

 Group# Disk name      Hot (MB)  Cold (MB)
------- ------------ ---------- ----------
      1 ASMDISK5              0       4187
        ASMDISK6              0       4187
      2 ASMDISK4              0       1138
        ASMDISK2              0       1135
        ASMDISK1              0       1139
        ASMDISK3              0       1144

The result shows that all space in all disks is allocated in the cold disk zones. Let's have a closer look at the used space directory with kfed.

$ kfed read /dev/oracleasm/disks/ASMDISK5 aun=51 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           26 ; 0x002: KFBTYP_USEDSPC
...
kfdusde[0].used[0].spare:             0 ; 0x000: 0x00000000
kfdusde[0].used[0].hi:                0 ; 0x004: 0x00000000
kfdusde[0].used[0].lo:             4134 ; 0x008: 0x00001026
kfdusde[0].used[1].spare:             0 ; 0x00c: 0x00000000
kfdusde[0].used[1].hi:                0 ; 0x010: 0x00000000
kfdusde[0].used[1].lo:                0 ; 0x014: 0x00000000
kfdusde[1].used[0].spare:             0 ; 0x018: 0x00000000
kfdusde[1].used[0].hi:                0 ; 0x01c: 0x00000000
kfdusde[1].used[0].lo:             4134 ; 0x020: 0x00001026
kfdusde[1].used[1].spare:             0 ; 0x024: 0x00000000
kfdusde[1].used[1].hi:                0 ; 0x028: 0x00000000
kfdusde[1].used[1].lo:                0 ; 0x02c: 0x00000000
kfdusde[2].used[0].spare:             0 ; 0x030: 0x00000000
kfdusde[2].used[0].hi:                0 ; 0x034: 0x00000000
kfdusde[2].used[0].lo:                0 ; 0x038: 0x00000000
kfdusde[2].used[1].spare:             0 ; 0x03c: 0x00000000
kfdusde[2].used[1].hi:                0 ; 0x040: 0x00000000
kfdusde[2].used[1].lo:                0 ; 0x044: 0x00000000
...

There are two disks in disk group number 1, so only the first two kfdusde entries are populated. And both show that all the space is allocated in the cold zone.

Check the used space directory entries for disk group 2.

$ kfed read /dev/oracleasm/disks/ASMDISK1 aun=41 | more
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                           26 ; 0x002: KFBTYP_USEDSPC
...
kfdusde[0].used[0].spare:             0 ; 0x000: 0x00000000
kfdusde[0].used[0].hi:                0 ; 0x004: 0x00000000
kfdusde[0].used[0].lo:             1092 ; 0x008: 0x00000444
kfdusde[0].used[1].spare:             0 ; 0x00c: 0x00000000
kfdusde[0].used[1].hi:                0 ; 0x010: 0x00000000
kfdusde[0].used[1].lo:                0 ; 0x014: 0x00000000
kfdusde[1].used[0].spare:             0 ; 0x018: 0x00000000
kfdusde[1].used[0].hi:                0 ; 0x01c: 0x00000000
kfdusde[1].used[0].lo:             1093 ; 0x020: 0x00000445
kfdusde[1].used[1].spare:             0 ; 0x024: 0x00000000
kfdusde[1].used[1].hi:                0 ; 0x028: 0x00000000
kfdusde[1].used[1].lo:                0 ; 0x02c: 0x00000000
kfdusde[2].used[0].spare:             0 ; 0x030: 0x00000000
kfdusde[2].used[0].hi:                0 ; 0x034: 0x00000000
kfdusde[2].used[0].lo:             1098 ; 0x038: 0x0000044a
kfdusde[2].used[1].spare:             0 ; 0x03c: 0x00000000
kfdusde[2].used[1].hi:                0 ; 0x040: 0x00000000
kfdusde[2].used[1].lo:                0 ; 0x044: 0x00000000
kfdusde[3].used[0].spare:             0 ; 0x048: 0x00000000
kfdusde[3].used[0].hi:                0 ; 0x04c: 0x00000000
kfdusde[3].used[0].lo:             1094 ; 0x050: 0x00000446
kfdusde[3].used[1].spare:             0 ; 0x054: 0x00000000
kfdusde[3].used[1].hi:                0 ; 0x058: 0x00000000
kfdusde[3].used[1].lo:                0 ; 0x05c: 0x00000000
kfdusde[4].used[0].spare:             0 ; 0x060: 0x00000000
kfdusde[4].used[0].hi:                0 ; 0x064: 0x00000000
kfdusde[4].used[0].lo:                0 ; 0x068: 0x00000000
kfdusde[4].used[1].spare:             0 ; 0x06c: 0x00000000
kfdusde[4].used[1].hi:                0 ; 0x070: 0x00000000
kfdusde[4].used[1].lo:                0 ; 0x074: 0x00000000
...

Disk group 2 has four disks and again all space is allocated in the cold disk zones.

Hot files

Let's create a disk group template for hot files.

SQL> alter diskgroup DATA add template HOTFILE attributes (HOT);

Diskgroup altered.

Note that this feature requires the disk group attribute COMPATIBLE.RDBMS to be at least 11.2.

Now create a datafile that will be alocated in the disks' hot zones.

SQL> create tablespace T1_HOT datafile '+DATA(HOTFILE)' size 50M;

Tablespace created.

Let's check the space allocation now, by running the last query again.

SQL> SELECT group_number "Group#",
 name "Disk name",
 hot_used_mb "Hot (MB)",
 cold_used_mb "Cold (MB)"
FROM v$asm_disk_stat
ORDER BY 1;

    Group# Disk name                        Hot (MB)  Cold (MB)
---------- ------------------------------ ---------- ----------
         1 ASMDISK5                                0       4187
           ASMDISK6                                0       4187
         2 ASMDISK4                               13       1152
           ASMDISK2                               12       1153
           ASMDISK1                               13       1152
           ASMDISK3                               13       1153

The result shows that 51 MB (50 MB for the file and 1 MB for the file header) are now allocated in the hot zones across all disk in the disk group.

Warm up a file

I can also move an existing datafile into the hot zone. Let's find all datafiles in disk group DATA.

$ asmcmd find --type datafile +DATA "*"
+DATA/BR/DATAFILE/EXAMPLE.269.769030517
+DATA/BR/DATAFILE/NOT_IMPORTANT.273.771795255
+DATA/BR/DATAFILE/SYSAUX.257.769030245
+DATA/BR/DATAFILE/SYSTEM.256.769030243
+DATA/BR/DATAFILE/T1_HOT.274.772054033
+DATA/BR/DATAFILE/TRIPLE_C.272.771794469
+DATA/BR/DATAFILE/TRIPLE_M.271.771793293
+DATA/BR/DATAFILE/UNDOTBS1.258.769030245
+DATA/BR/DATAFILE/USERS.259.769030245

Let's move the undo tablespace datafile into the hot zone.

SQL> alter diskgroup DATA modify file '+DATA/BR/DATAFILE/UNDOTBS1.258.769030245' attributes (HOT);

Diskgroup altered.

This action triggers the rebalance for disk group DATA, as file extents have to be moved to disks' hot regions. Once the rebalance completes, the last query shows more data in hot region for disks in disk group number 2.

SQL> SELECT group_number "Group#",
 name "Disk name",
 hot_used_mb "Hot (MB)",
 cold_used_mb "Cold (MB)"
FROM v$asm_disk_stat
ORDER BY 1;

    Group# Disk name                        Hot (MB)  Cold (MB)
---------- ------------------------------ ---------- ----------
         1 ASMDISK5                                0       4187
           ASMDISK6                                0       4187
         2 ASMDISK4                               40       1125
           ASMDISK2                               39       1126
           ASMDISK1                               39       1126
           ASMDISK3                               39       1127

Conclusion

The disk Used Space Directory (USD) – ASM file number 8 - maintains the number of allocation units (AU) used per zone, per disk in a disk group. It is a supporting metadata structure for the Intelligent Data Placement feature in ASM version 11.2. One handy use of this feature is a control of datafile placement in disks' hot or cold zones.

Posted by

3 comments:

  1. Demon_A_Go-GoDecember 12, 2012 at 11:39 PM

    Hello Bane,

    Great post, it is actually not mentioned anywhere how ASM will judge which zone is hot and which is cold ? Do you have any thoughts about this.
    From docs it says:
    Because Intelligent Data Placement leverages the geometry of the disk, it is well suited to JBOD (just a bunch of disks).
    In contrast, a storage array with LUNs composed of concatenated volumes masks the geometry from Oracle ASM.

    Does this mean it can't decide which LUN part is hot which is cold with storage array LUNs ?

    BR
    Ognyan

    ReplyDelete
    Replies
    1. Bane RadulovicDecember 13, 2012 at 7:20 PM

      Thanks Ognyan,

      My understanding is that this hot/cold split is both automatic and dynamic. That means ASM will determine the percentage of the disk size to be the hot region and the rest will be cold. If the hot region becomes full and you want to move more files into the hot zone, ASM will extend the hot region and shrink the cold.

      You are right about the disks made of LUNs from the storage array. While ASM sees the disk 'tracks', it has no idea about the actual track placements. In short, it doesn't make any sense to implement this feature with such storage system.

      Cheers,
      Bane

      Delete
  2. Zoey RobertsJanuary 4, 2022 at 6:50 PM

    Thiis is a great post thanks

    ReplyDelete

Subscribe to: Post Comments (Atom)