The views expressed on this blog are my own and do not necessarily reflect the views of Oracle

July 27, 2014

How to reconfigure Oracle Restart

The other day I had to reconfigure an Oracle Restart 12c environment, and I couldn't find my blog post on that. It turns out that I never published it here, as my MOS document on this topic was created back in 2010 when this blog didn't exist!

The original document was written for Oracle Restart 11gR2, but it is still valid for 12c. Here it is.

Introduction

This document is about reconfiguring the Oracle Restart. One reason for such action might be the server rename. If the server was renamed and then rebooted, the ASM instance startup would fail with ORA-29701: unable to connect to Cluster Synchronization Service.

The solution is to reconfigure Oracle Restart as follows.

1. Remove Oracle Restart configuration

This step should be performed as the privileged user (root).

# $GRID_HOME/crs/install/roothas.pl -deconfig -force

The expected result is "Successfully deconfigured Oracle Restart stack".

2. Reconfigure Oracle Restart

This step should also be performed as the privileged user (root).

# $GRID_HOME/crs/install/roothas.pl

The expected result is "Successfully configured Oracle Grid Infrastructure for a Standalone Server"

3. Add ASM back to Oracle Restart configuration

This step should be performed as the Grid Infrastructure owner (grid user).

$ srvctl add asm

The expected result is no output, just a return to the operating system prompt.

4. Start ASM instance

This step should be performed as the Grid Infrastructure owner (grid user).

$ srvctl start asm

That should start the ASM instance.

Note that at this time there will be no ASM initialization or server parameter file.

5. Recreate ASM server parameter file (SPFILE)

This step should be performed as the Grid Infrastructure owner (grid user).

Create a temporary initialization parameter file (e.g. /tmp/init+ASM.ora) with the content similar to the this (of course, with your own disk group names):

asm_diskgroups='DATA','RECO'
large_pool_size=12M
remote_login_passwordfile='EXCLUSIVE'

Mount the disk group where the new server parameter file (SPFILE) will reside (e.g. DATA) and create the SPFILE:

$ sqlplus / as sysasm

SQL> alter diskgroup DATA mount;

Diskgroup altered.

SQL> create spfile='+DATA' from pfile='/tmp/init+ASM.ora';

File created.

SQL> show parameter spfile

NAME   TYPE   VALUE
------ ------ -------------------------------------------------
spfile string +DATA/asm/asmparameterfile/registry.253.707737977

6. Restart HAS stack

This step should be performed as the Grid Infrastructure owner (grid user).

$ crsctl stop has

$ crsctl start has

7. Add components back to Oracle Restart configuration

Add the database, the listener and other components, back into the Oracle Restart configuration.

7.1. Add database

This step should be performed as RDBMS owner (oracle user).

In 11gR2 the command is:

$ srvctl add database -d db_unique_name -o oracle_home

In 12c the command is:

$ srvctl add database -db db_unique_name -oraclehome oracle_home

7.2. Add listener

This step should be performed as the Grid Infrastructure owner (grid user).

$ srvctl add listener

7.3. Add other components

For information on how to add back additional components in 11gR2 see:
Oracle® Database Administrator's Guide 11g Release 2 (11.2)
Chapter 4 Configuring Automatic Restart of an Oracle Database

For 12c see:
Oracle® Database Administrator's Guide 12c Release 1 (12.1)
Chapter 4 Configuring Automatic Restart of an Oracle Database

Conclusion

As noted earlier, I have published a MOS document on this: How to Reconfigure Oracle Restart - MOS Doc ID 986740.1.

May 4, 2014

ASM Disk Group Attributes

Disk group attributes were introduced in ASM version 11.1. They are bound to a disk group, rather than the ASM instance. Some attributes can be set only at the time the disk group is created, some only after the disk group is created, and some attributes can be set at any time, by altering the disk group.

This is the follow up on the ASM Attributes Directory post.

ACCESS_CONTROL.ENABLED

This attribute determines whether ASM File Access Control is enabled for a disk group. The value can be TRUE or FALSE (default).

If the attribute is set to TRUE, accessing ASM files is subject to access control. If FALSE, any user can access every file in the disk group. All other operations behave independently of this attribute.

This attribute can only be set when altering a disk group.

ACCESS_CONTROL.UMASK

This attribute determines which permissions are masked out on the creation of an ASM file for the owner, group, and others not in the user group. This attribute applies to all files in a disk group.

The values can be combinations of three digits {0|2|6} {0|2|6} {0|2|6}. The default is 066.

Setting to '0' does not mask anything. Setting to '2' masks out write permission. Setting to '6' masks out both read and write permissions.

Before setting the ACCESS_CONTROL.UMASK disk group attribute, the ACCESS_CONTROL.ENABLED has to be set to TRUE.

This attribute can only be set when altering a disk group.

AU_SIZE

The AU_SIZE attribute controls the allocation unit size and can only be set when creating the disk group.

It is worth spelling out that each disk group can have a different allocation unit size.

CELL.SMART_SCAN_CAPABLE [Exadata]

This attribute is applicable to Exadata when the disk group is created from the grid disks on the storage cells. It enables the smart scan functionality for the objects stored in that disk group.

COMPATIBLE.ASM

The value for the disk group COMPATIBLE.ASM attribute determines the minimum software version for an ASM instance that can use the disk group. This setting also affects the format of the ASM metadata structures.

The default value for the COMPATIBLE.ASM is 10.1, when using the CREATE DISKGROUP statement, the ASMCMD mkdg command and the Enterprise Manager Create Disk Group page.

When creating a disk group with the ASMCA, the default value is 11.2 in ASM version 11gR2 and 12.1 in ASM version 12c.

COMPATIBLE.RDBMS

The value for the COMPATIBLE.RDBMS attribute determines the minimum COMPATIBLE database initialization parameter setting for any database instance that is allowed to use the disk group.

Before advancing the COMPATIBLE.RDBMS attribute, ensure that the values for the COMPATIBLE initialization parameter for all databases that access the disk group are set to at least the value of the new setting for the COMPATIBLE.RDBMS.

COMPATIBLE.ADVM

The value for the COMPATIBLE.ADVM attribute determines whether the disk group can contain the ASM volumes. The value must be set to 11.2 or higher. Before setting this attribute, the COMPATIBLE.ASM value must be 11.2 or higher. Also, the ADVM volume drivers must be loaded in the supported environment.

By default, the value of the COMPATIBLE.ADVM attribute is empty until set.

CONTENT.CHECK [12c]

The CONTENT.CHECK attributes enables or disables the content checking when performing the disk group rebalance. The attribute values can be TRUE or FALSE.

The content checking can include Hardware Assisted Resilient Data (HARD) checks on user data, validation of file types from the file directory against the block contents and the file directory information, and mirror side comparison.

When the attribute is set to TRUE, the logical content checking is enabled for all rebalance operations.

The content checking is also known as the disk scrubbing feature.

CONTENT.TYPE [11.2.0.3, Exadata]

The CONTENT.TYPE attribute identifies the disk group type that can be DATA, RECOVERY or SYSTEM. It determines the distance to the nearest partner disk/failgroup. The default value is DATA which specifies a distance of 1, the value of RECOVERY specifies a distance of 3 and the value of SYSTEM specifies a distance of 5.

The distance of 1 simply means that ASM considers all disks for partnership. The distance of 3 means that every 3rd disk will be considered for partnership and the distance of 5 means that every 5th disk will be considered for partnership.

The attribute can be specified when creating or altering a disk group. If the CONTENT.TYPE attribute is set or changed using the ALTER DISKGROUP, the new configuration does not take effect until a disk group rebalance is explicitly run.

The CONTENT.TYPE attribute is only valid for NORMAL and HIGH redundancy disk groups. The COMPATIBLE.ASM attribute must be set to 11.2.0.3 or higher to enable the CONTENT.TYPE attribute.

DISK_REPAIR_TIME

The value of the DISK_REPAIR_TIME attribute determines the amount of time ASM will keep the disk offline, before dropping it. This is relevant to the fast mirror resync feature for which the COMPATIBLE.ASM attribute must be set to 11.1 or higher.

This attribute can only be set when altering a disk group.

FAILGROUP_REPAIR_TIME [12c]

The FAILGROUP_REPAIR_TIME attribute specifies a default repair time for the failure groups in the disk group. The failure group repair time is used if ASM determines that an entire failure group has failed. The default value is 24 hours. If there is a repair time specified for a disk, such as with the DROP AFTER clause of the ALTER DISKGROUP OFFLINE DISK statement, that disk repair time overrides the failure group repair time.

This attribute can only be set when altering a disk group and is only applicable to NORMAL and HIGH redundancy disk groups.

IDP.BOUNDARY and IDP.TYPE [Exadata]

These attributes are used to configure Exadata storage, and are relevant for the Intelligent Data Placement feature.

PHYS_META_REPLICATED [12c]

The PHYS_META_REPLICATED attribute tracks the replication status of a disk group. When the ASM compatibility of a disk group is advanced to 12.1 or higher, the physical metadata of each disk is replicated. This metadata includes the disk header, free space table blocks and allocation table blocks. The replication is performed online asynchronously. This attribute value is set to true by ASM if the physical metadata of every disk in the disk group has been replicated.

This attribute is only defined in a disk group with the COMPATIBLE.ASM set to 12.1 and higher. The attribute is read-only and is intended for information only - a user cannot set or change its value. The values are either TRUE or FALSE.

SECTOR_SIZE

The SECTOR_SIZE attribute specifies the sector size for disks in a disk group and can only be set when creating a disk group.

The values for the SECTOR_SIZE can be 512, 4096 or 4K (provided the disks support those values). The default value is platform dependent. The COMPATIBLE.ASM and COMPATIBLE.RDBMS attributes must be set to 11.2 or higher to set the sector size to a value other than the default value.

NOTE: ASM Cluster File System (ACFS) does not support 4 KB sector drives.

STORAGE.TYPE

The STORAGE.TYPE attribute specifies the type of the disks in the disk group. The possible values are EXADATA, PILLAR, ZFSAS and OTHER. If the attribute is set to EXADATA|PILLAR|ZFSAS then all disks in the disk group must be of that type. If the attribute is set to OTHER, any types of disks can be in the disk group.

If the STORAGE.TYPE disk group attribute is set to PILLAR or ZFSAS, the Hybrid Columnar Compression (HCC) functionality can be enabled for the objects in the disk group. Exadata already supports HCC.

NOTE: The ZFS storage must be provisioned through Direct NFS (dNFS) and the Pillar Axiom storage must be provisioned via the SCSI or the Fiber Channel interface.

To set the STORAGE.TYPE attribute, the COMPATIBLE.ASM and COMPATIBLE.RDBMS disk group attributes must be set to 11.2.0.3 or higher. For maximum support with ZFS storage, set COMPATIBLE.ASM and COMPATIBLE.RDBMS disk group attributes to 11.2.0.4 or higher.

The STORAGE.TYPE attribute can be set when creating or altering a disk group. The attribute cannot be set when clients are connected to the disk group. For example, the attribute cannot be set when an ADVM volume is enabled on the disk group.

The attribute is not visible in the V$ASM_ATTRIBUTE view or with the ASMCMD lsattr command until the attribute has been set.

THIN_PROVISIONED [12c]

The THIN_PROVISIONED attribute enables or disables the functionality to discard unused storage space after a disk group rebalance is completed. The attribute value can be TRUE or FALSE (default).

Storage vendor products that support thin provisioning have the capability to reuse the discarded storage space for a more efficient overall physical storage utilization.

APPLIANCE.MODE [11.2.0.4, Exadata]

The APPLIANCE.MODE attribute improves the disk rebalance completion time when dropping one or more ASM disks. This means that redundancy is restored faster after a (disk) failure. The attribute is automatically enabled when creating a new disk group in Exadata. Existing disk groups must explicitly set the attribute using the ALTER DISKGROUP command. This feature is also known as fixed partnering.

The attribute can only be enabled on disk groups that meet the following requirements:

The Oracle ASM disk group attribute COMPATIBLE.ASM is set to release 11.2.0.4, or later.
The CELL.SMART_SCAN_CAPABLE attribute is set to TRUE.
All disks in the disk group are the same type of disk, such as all hard disks or all flash disks.
All disks in the disk group are the same size.
All failure groups in the disk group have an equal number of disks.
No disk in the disk group is offline.

Minimum software: Oracle Exadata Storage Server Software release 11.2.3.3 running Oracle Database 11g Release 2 (11.2) release 11.2.0.4

NOTE: This feature is not available in Oracle Database version 12.1.0.1.

Hidden disk group attributes

Not all disk group attributes are documented. Here are some of the more interesting ones.

_REBALANCE_COMPACT

The _REBALANCE_COMPACT attribute is related to the compacting phase of the rebalance. The attribute value can be TRUE (default) or FALSE. Setting the attribute to FALSE, disables the compacting phase of the disk group rebalance.

_EXTENT_COUNTS

The _EXTENT_COUNTS attribute, is related to the variable extents size feature, that determines the points at which the extent size will be incremented.

The value of the attribute is “20000 20000 214748367”, which means that the first 20000 extent sizes will be 1 AU, next 20000 extents will have the size determined by the second value of the _EXTENT_SIZES attribute, and the rest will have the size determined by the third value of the _EXTENT_SIZES attribute.

_EXTENT_SIZES

The _EXTENT_SIZES is the second attribute relevant to the variable extents size feature, and it determines the extent size increments - in the number of AUs.

In ASM version 11.1 the attribute value is "1 8 64". In ASM version 11.2 and later, the value of the _EXTENT_SIZES is "1 4 16".

V$ASM_ATTRIBUTE view and ASMCMD lsattr command

The disk group attributes can be queried via the V$ASM_ATTRIBUTE view and the ASMCMD lsattr command.

This is one way to list the attributes for disk group PLAY:

$ asmcmd lsattr -G PLAY –l

Name                    Value
access_control.enabled  FALSE
access_control.umask     066
au_size                  4194304
cell.smart_scan_capable FALSE
compatible.asm           11.2.0.0.0
compatible.rdbms         11.2.0.0.0
disk_repair_time         3.6h
sector_size              512
$

The disk group attributes can be modified via SQL ALTER DISKGROUP SET ATTRIBUTE, ASMCMD setattr command and the ASMCA. This is an example of using the ASMCMD setattr command to modify the DISK_REPAIR_TIME attribute for disk group PLAY:

$ asmcmd setattr -G PLAY disk_repair_time '4.5 H'

Check the new value:

$ asmcmd lsattr -G PLAY -l disk_repair_time
Name              Value
disk_repair_time  4.5 H
$

Conclusion

Disk group attributes, introduced in ASM version 11.1, are a great way to fine tune the disk group capabilities. Some of the attributes are Exadata specific (as marked) and some are available in certain versions only (as marked). Most disk group attributes are documented and accessible via the V$ASM_ATTRIBUTE view. Some of the undocumented attributes were also discussed and those should not be modified unless advised by Oracle Support.