The views expressed on this blog are my own and do not necessarily reflect the views of Oracle

April 18, 2010

Set up ASM with a single ASMCA command


ASM Configuration Assistant (ASMCA) was introduced in ASM version 11.2. It is used to configure ASM instances, and to create and manage disk groups, volumes and ASM cluster file systems (ACFS). ASMCA can be used in GUI or command-line mode.

In this post I will show how to use the ASMCA - in a non-cluster environment - to create and start an ASM  instance, create a disk group and start related/required services. I will use the ASMCA in a command line mode with the silent option.

Perform the Grid Infrastructure software only installation

You may want to do this with ASM job role separation option, in which case you should perform all steps as OS user grid. Otherwise, perform all steps as OS user oracle.

Set up the disks to be used by ASM

I used ASMLIB to create 4 disks for ASM, and I can see them as follows:

$ oracleasm listdisks
DISK1
DISK2
DISK3
DISK4

Configure ASM

Run the following command:

$ asmca -silent -configureASM -sysAsmPassword s3kr3t1 -asmsnmpPassword s3kr3t2 -diskString 'ORCL:*' -diskGroupName DATA -disk 'ORCL:*' -redundancy EXTERNAL

As I used ASMLIB disks, I specified 'ORCL:*' for ASM discovery string. Make sure you specify the correct value for your environment.

On a successful run, the above command should have returned:

ASM created and started successfully.
DiskGroup DATA created successfully.

And it should have performed the following:
  • Start the cluster synchronisation services daemon – ocssd.bin
  • Start three agents – cssdagent, oraagent.bin and orarootagent.bin
  • Start the disk monitor – diskmon.bin
  • Create and start ASM instance +ASM
  • Create the external redundancy disk group DATA
  • Create ASM spfile in disk group DATA
I have also published this in MOS Doc ID 1068788.1.

35 comments:

  1. Hi Sir ,

    I have configured 10gR2(Standard Ed),2 node RAC in RHEL5 with ASM ( SAN storage , mapped as raw device to both nodes but not mounted as filesystem).

    And there is a replication from this SAN to another SAN box.

    From this 2nd SAN all ASM raw devices are mapped to a new RHEL5 machine.

    Now i need to bring up the oracle instance ( with ASM) in this new RHEL5 machine using this replicated raw devices.

    Can you please give me your valuable suggestions on this.

    ReplyDelete
  2. Hi Ramesh,

    All you need to do on a new machine is specify the path to those raw devices via ASM init parameter asm_diskstring. In fact, I think asm_diskstring="/dev/raw/*" is default on Linux, so you might be able to start up ASM on the new system without specifying a value for asm_diskstring. Just check that is either not set or points to your raw devices.

    Let me know if you have any problems with this.

    Cheers,
    Bane

    ReplyDelete
  3. Hi Sir ,

    I have successfully brought up the single instance in 3rd RHEL5 machine.

    ****** Thank you very much for your prompt response **********

    Below are the things i have done as part the activity :

    1. Recreated CSS daemon in new RHEL5 machine.
    2. Taken init+ASM1.ora from RAC and modified as per the single instance ( Remove instance_number parameters).
    3. Mounted the ASM instance ( successfully mounted with asm_diskstring="/dev/raw/*" )
    4. Taken init file of RAC1's database and modified as per the single instance.
    ( have passed common DB name as db_name and instance_name)
    5. Mounted the database ( And delete the redo log groups relates to THREAD# 2)
    6. Open the database.
    7. Single instance is successfully opened in 3rd RHEL5 machine using replicated raw devices of RAC.

    Thanks ,
    Ramesh Togara.

    ReplyDelete
  4. I am glad to hear this Ramesh.
    Cheers,
    Bane

    ReplyDelete
  5. Hey Bane - love the blog.

    What about if you want to create a diskgroup with 4k sector size in 11gR2? As far as I can tell there is no way to get ASMCA to do this, i.e. to set the SECTOR_SIZE attribute as part of creating a diskgroup...

    Any idea if it's possible, or when it's coming in the future?

    ReplyDelete
  6. Hi FlashDBA,

    Yeah, it looks like ASMCA is a bit slow in adopting new features. I guess that will be fixed in the upcoming releases.

    You can still create disk groups with 4K sector size with sqlplus and asmcmd. Here is a command line example with asmcmd:

    $ cat /tmp/sector4k.xml
    [dg name="SECTOR4K" redundancy="high"]
    [dsk string="ORCL:ASMDISK1"/]
    [dsk string="ORCL:ASMDISK2"/]
    [dsk string="ORCL:ASMDISK3"/]
    [a name="compatible.asm" value="11.2"/]
    [a name="compatible.rdbms" value="11.2"/]
    [a name="sector_size" value="4096"/]
    [/dg]

    $ asmcmd mkdg /tmp/sector4k.xml

    Of course you need to replace all [ with < and all ] with > in sector4k.xml. This thing would not let me post the correct xml :(

    Cheers,
    Bane

    ReplyDelete
  7. Bane, that is genius! Do you know the syntax for running the installASM command using XML so that I can create the diskgroup with 4k sector size? I've been messing around with the XML but I can't seen to find any examples in the documentation.

    ReplyDelete
  8. Well, before you pronounce me a genius, note that the example I gave was for asmcmd command, not asmca :)
    I think it may be best that you create the ASM instance first (with asmca -silent) and add your 4k sector disk group later.
    Cheers,
    Bane

    ReplyDelete
  9. Ha I didn't notice that.

    I tried setting up the ASM instance first but of course you have to specify a diskgroup to do that and my ASM disks all reside on 4k sector disks. So I end up creating a 512 byte sector diskgroup that I don't want just so I can create ASM.

    I guess I need to wait for ASMCA to have sector_size added to its parameter list...

    ReplyDelete
  10. Is there a way to create 4k sector size diskgroup without using ASMLIB? i.e using rawdevices? I can create 4k SS disks with ASMLIB and having errors with raw devices : "ERROR at line 1:
    ORA-15018: diskgroup cannot be created
    ORA-15080: synchronous I/O operation to a disk failed"

    ReplyDelete
  11. No, ASMLIB is not a requirement for 4K sector size disk groups.
    Can you do I/O against the raw devices at the OS level, both read and write, say with the dd command?
    Do you have 512 byte sector raw disks, attached to the same server, that you can use to create a disk group? In other words, do you have this problem with 4k sector size disks only?
    What other diagnostics do you have?
    Cheers,
    Bane

    ReplyDelete
  12. Hi Bane,
    Thanks for the reply. Here is more info, and I used SSD.

    Test 1:
    Formated SSD with 512b
    Created Partitions (for 1MB lun alignment)
    Used ASMLIB, created ASM disks
    Successfully Created diskgroup with default 512b

    Test 2:
    Formated SSD with 4k
    Created Partitions (for 1MB lun alignment)
    Used ASMLIB, created ASM disks
    Successfully Created diskgroup with 4k sctor size

    Test 3:
    Formated SSD with 512b
    Created Partitions (for 1MB lun alignment)
    Configured in /etc/sysconfig/rawdevices
    Set permissions and restarted the service
    Successfully Created diskgroup with 512b sctor size

    Test 4:
    Formated SSD with 4k
    Created Partitions (for 1MB lun alignment)
    Configured in /etc/sysconfig/rawdevices
    Set permissions and restarted the service
    Got an error while creating diskgroup with 4k sctor size

    dd works fine; but Oracle throws an error. Thanks again.

    ReplyDelete
    Replies
    1. Interesting... How about using block devices instead of raw? You know that raw devices are not required and are being phased out anyway.

      Adjust ownership and permissions on your devices and add '/dev/sd*' to your asm_diskstring. Now create the disk group:

      create diskgroup D1
      disk '/dev/sdg', '/dev/sdh'
      attribute 'compatible.asm'='11.2', 'compatible.rdbms'='11.2', 'sector_size'='4096';

      Cheers,
      Bane

      Delete
  13. I've tried it as a block device and got the same errro. Just opened a SR with Oracle and awiting response. Thanks.

    ReplyDelete
    Replies
    1. Please let me know the outcome as I am curious to know what is going on. Unfortunately I don't have a 4k sector size disk to try this myself :(
      Cheers,
      Bane

      Delete
  14. Looks like its a New bug and the SR has been moved to the ASM development team.

    ReplyDelete
  15. bane,

    I am running the command as below
    Oracle@chirU>asmca -silent -configureASM -sysAsmPassword ora1n -asmsnmpPassword ora2n -diskString '/dev/dsk/c2*' -diskGroupName DATA -disk '/dev/dsk/c2*' -redundancy EXTERNAL

    Configuring ASM failed with the following message:
    Unable to copy the file "/tmp/oratab" to "/var/opt/oracle/oratab".

    It fails I have lot space. Also
    '/dev/dsk/c2* disk string is correct.

    ReplyDelete
    Replies
    1. This may not be the best forum to diagnose issues.
      You may start by looking at why oracle user cannot copy a file to /var/opt/oracle. You can then trace/strace asmca to see what is going on. If you cannot work it out please email me [bane.radulovic at gmail.com] your ASM version, OS version and the trace/strace file. Once we sort it out we can post the findings here.
      Cheers,
      Bane

      Delete
  16. hi Bane.

    Im getting "PRVF-5150 file name /dev/rdsk/xxxxx not a valid path" error during configuring 11gR2 grid.

    But when i checked , device is existed and permissions also set to "grid,asmadmin"

    Can you please help me on this.

    ReplyDelete
  17. Hi Ramesh,
    I haven't seen that error, but quick google search shows up few discussions that point to a bug and the advice that the error can be ignored. Please let me know you exact ASM version and I will check if that bug is still there in your version.
    Cheers,
    Bane

    ReplyDelete
  18. Hi Bane ,

    Thanks for your prompt response. Below are the configuration details.

    Grid Version : 11.2.0.3.0
    OS : Sun OS 5.10
    Storage : SAN slices given as raw devices.

    Error is showing in "prerequests cheking" in during grid installation.


    Thanks.

    ReplyDelete
    Replies
    1. Yeah, that issue is still there in 11.2.0.3. You can ignore it and proceed with the install.

      Delete
  19. Thanks bane..

    I will proceed further and let you know the status.

    ReplyDelete
  20. Hi Bane,

    We have ignored the "Warning" msg and proceeded with installation.
    at the end of the installation or while managing cluster we don't encountering any errors.

    Please clarify the below.

    How can we add the additional disks to existed "Diskgroup" in ASM which is having "NORMAL" redundancy ( 2 failgroups).

    Thanks in advance.

    ReplyDelete
    Replies
    1. If you didn't specify failgroup names yourself, you just add a new disk with:
      alter diskgroup add disk '';.

      If you did specify failgroups explicitly, and you had a good reason to do so, you need to use:
      alter diskgroup add failgroup disk '';.
      Then you need to add another disk to the other failgroup, as you want both failgroups to have the same number of disks and the same capacity.

      But I suspect, you did not manually name your failgroups, which is fine and how most people do it. In that case you just add new disks as needed, using the first command I gave you.

      Cheers

      Delete
    2. Ahhh. This site doesn't like angle brackets :(
      The first command should be:
      alter diskgroup dg_name add disk 'disk_name';
      The second, with failgroups should be:
      alter diskgroup dg_name add failgrup fg_name disk 'disk_name';

      Delete
  21. Hi Bane,
    I'm trying to install ASM by asmca. Either with graphical or command line mode I still receive the same error:
    Configuring ASM failed with the following message:
    Configuring HA resource failed. The following error occured:
    PRCR-1079 : Failed to start resource ora.asm
    ORA-00119: invalid specification for system parameter %s
    CRS-2674: Start of 'ora.asm' on 'my_server_name' failed
    Could you help me?
    Thanks,
    g10

    ReplyDelete
  22. Hi g10,
    It's not easy to tell what the problem is based on what you provided. Best I can suggest is google search for "PRCR-1079 ORA-00119 CRS-2674".
    If that doesn't help, please let me know:
    1. The exact command line (all arguments)
    2. ASM and OS version
    3. The result of 'ls -l '
    Cheers,
    Bane

    ReplyDelete
    Replies
    1. Hi Bane,
      I've tried before wrote to you to google about all the net and also in metalink, but I haven't found useful informations.
      ASM is 11gR2, os version is RHEL 5, exact command is:
      /orasw/app/11.2.0/grid/bin/asmca -silent -configureASM -sysAsmPassword s3kr3t1 -asmsnmpPassword s3kr3t2 -diskString 'ORCL:*' -diskGroupName DATA -disk 'ORCL:*' -redundancy EXTERNAL.
      Thanks in advance,
      g10

      Delete
  23. Hello Sir,
    I am not expert but a senior dba. I need your valuable advice if you would:
    I am trying to solve a db block corruption error on ASM disks and I am trying to solve the problem with some options.
    One of them is to create an ASM DG with sector size 4096 bytes instead of default 512 bytes.
    But when I tried to create DG with the following command, I encounter errors.
    Could you help me about this issue please?

    My environment :
    ---------------------------------------------------------
    Red Hat Enterprise Linux Server release 5.8 (Tikanga)
    Oracle RDBMS and ASM binary : 11.2.0.3.0, oracleasm lib
    Storage : HTS SAN disks (LUNs)

    My command and the error :
    ----------------------------------------------------------
    SQL> CREATE DISKGROUP TESTDG1
    2 EXTERNAL REDUNDANCY
    3 DISK 'ORCL:TESTDISK01'
    4 ATTRIBUTE
    5 'au_size'='64M',
    6 'sector_size'='4096',
    7 'compatible.asm' = '11.2',
    8 'compatible.rdbms' = '11.2';

    CREATE DISKGROUP TESTDG1
    *
    ERROR at line 1:
    ORA-15018: diskgroup cannot be created
    ORA-15038: disk '' mismatch on 'Sector Size' with target disk group [512][4096]

    Best Regards,
    Devrim



    ReplyDelete
    Replies
    1. Hi Devrim,

      That disk (TESTDISK01) has to be created with the 4KB sector size. Only then you can create the disk group with that command.

      You can check that with the following command:
      $ kfed read /dev/oracleasm/disks/TESTDISK01 | grep secsize

      That should return kfdhdb.secsize: 4096.
      If it returns kfdhdb.secsize: 512, you need to talk to your storage/sys admin and ask them to create the disk/LUN with 4KB sector size.

      Let me how that went.

      Cheers,
      Bane

      Delete
  24. Hi Bane,

    Thank your for quick response :)
    I just tried kfed command but it gave nothing :(
    Then I write first pat of command and I got the result as below :


    +ASM oracle@crixus:/home/oracle>kfed read /dev/oracleasm/disks/TESTDISK01
    kfbh.endian: 0 ; 0x000: 0x00
    kfbh.hard: 0 ; 0x001: 0x00
    kfbh.type: 0 ; 0x002: KFBTYP_INVALID
    kfbh.datfmt: 0 ; 0x003: 0x00
    kfbh.block.blk: 0 ; 0x004: blk=0
    kfbh.block.obj: 0 ; 0x008: file=0
    kfbh.check: 520109344 ; 0x00c: 0x1f003d20
    kfbh.fcn.base: 0 ; 0x010: 0x00000000
    kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
    kfbh.spare1: 0 ; 0x018: 0x00000000
    kfbh.spare2: 0 ; 0x01c: 0x00000000
    2AF5E2BBA400 00000000 00000000 00000000 1F003D20 [............ =..]
    2AF5E2BBA410 00000000 00000000 00000000 00000000 [................]
    2AF5E2BBA420 4C43524F 4B534944 54534554 4B534944 [ORCLDISKTESTDISK]
    2AF5E2BBA430 00003130 00000000 00000000 00000000 [01..............]
    2AF5E2BBA440 00000000 00000000 00000000 00000000 [................]
    Repeat 251 times
    KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]

    Then, I tried your command on working asm disk :

    +ASM oracle@crixus:/home/oracle>kfed read /dev/oracleasm/disks/DMMDATA01|grep secsize
    kfdhdb.secsize: 512 ; 0x0b8: 0x0200

    Actually TESTDISK01 may be created wrong but I thought its sector size 512 bytes possibly.

    Could you suggest me the method to create 4096 bytes ASM disk with oracleasm please?
    As I know there is no option to specify sector size with "oracleasm create disk" command.

    Regards,
    Devrim

    ReplyDelete
    Replies
    1. The output of 'kfed read /dev/oracleasm/disks/TESTDISK01' is normal for a disk that is not part of a disk group. Sorry, that command (with grep secsize) I gave you would show the sector size once the disk group is created.

      While you create ASMLIB disk (with oracleasm createdisk), you use a LUN that was given to you by your storage/sys admin. You need to have the LUN created with 4KB sector size and that is something your storage/sys admin has to do for you. The thing is that the device itself has to support 4KB sector size, and only then you can create the disk group with 4KB sector size.

      Hope this makes sense.

      Cheers,
      Bane

      Delete
  25. Hello Bane,

    I just shared thsi information with OS and SAN/Storage admins.
    They will try to find a solution to problem.

    Thank you and best regards,
    Devrim

    ReplyDelete