In this post I will show how to use the ASMCA - in a non-cluster environment - to create and start an ASM instance, create a disk group and start related/required services. I will use the ASMCA in a command line mode with the silent option.
Perform the Grid Infrastructure software only installation
You may want to do this with ASM job role separation option, in which case you should perform all steps as OS user grid. Otherwise, perform all steps as OS user oracle.
Set up the disks to be used by ASM
I used ASMLIB to create 4 disks for ASM, and I can see them as follows:
$ oracleasm listdisks
DISK1
DISK2
DISK3
DISK4
DISK1
DISK2
DISK3
DISK4
Configure ASM
Run the following command:
As I used ASMLIB disks, I specified 'ORCL:*' for ASM discovery string. Make sure you specify the correct value for your environment.
On a successful run, the above command should have returned:
And it should have performed the following:
$ asmca -silent -configureASM -sysAsmPassword s3kr3t1 -asmsnmpPassword s3kr3t2 -diskString 'ORCL:*' -diskGroupName DATA -disk 'ORCL:*' -redundancy EXTERNAL
As I used ASMLIB disks, I specified 'ORCL:*' for ASM discovery string. Make sure you specify the correct value for your environment.
On a successful run, the above command should have returned:
ASM created and started successfully.
DiskGroup DATA created successfully.
DiskGroup DATA created successfully.
And it should have performed the following:
- Start the cluster synchronisation services daemon – ocssd.bin
- Start three agents – cssdagent, oraagent.bin and orarootagent.bin
- Start the disk monitor – diskmon.bin
- Create and start ASM instance +ASM
- Create the external redundancy disk group DATA
- Create ASM spfile in disk group DATA
I have also published this in MOS Doc ID 1068788.1.
Hi Sir ,
ReplyDeleteI have configured 10gR2(Standard Ed),2 node RAC in RHEL5 with ASM ( SAN storage , mapped as raw device to both nodes but not mounted as filesystem).
And there is a replication from this SAN to another SAN box.
From this 2nd SAN all ASM raw devices are mapped to a new RHEL5 machine.
Now i need to bring up the oracle instance ( with ASM) in this new RHEL5 machine using this replicated raw devices.
Can you please give me your valuable suggestions on this.
Hi Ramesh,
ReplyDeleteAll you need to do on a new machine is specify the path to those raw devices via ASM init parameter asm_diskstring. In fact, I think asm_diskstring="/dev/raw/*" is default on Linux, so you might be able to start up ASM on the new system without specifying a value for asm_diskstring. Just check that is either not set or points to your raw devices.
Let me know if you have any problems with this.
Cheers,
Bane
Hi Sir ,
ReplyDeleteI have successfully brought up the single instance in 3rd RHEL5 machine.
****** Thank you very much for your prompt response **********
Below are the things i have done as part the activity :
1. Recreated CSS daemon in new RHEL5 machine.
2. Taken init+ASM1.ora from RAC and modified as per the single instance ( Remove instance_number parameters).
3. Mounted the ASM instance ( successfully mounted with asm_diskstring="/dev/raw/*" )
4. Taken init file of RAC1's database and modified as per the single instance.
( have passed common DB name as db_name and instance_name)
5. Mounted the database ( And delete the redo log groups relates to THREAD# 2)
6. Open the database.
7. Single instance is successfully opened in 3rd RHEL5 machine using replicated raw devices of RAC.
Thanks ,
Ramesh Togara.
I am glad to hear this Ramesh.
ReplyDeleteCheers,
Bane
Hey Bane - love the blog.
ReplyDeleteWhat about if you want to create a diskgroup with 4k sector size in 11gR2? As far as I can tell there is no way to get ASMCA to do this, i.e. to set the SECTOR_SIZE attribute as part of creating a diskgroup...
Any idea if it's possible, or when it's coming in the future?
Hi FlashDBA,
ReplyDeleteYeah, it looks like ASMCA is a bit slow in adopting new features. I guess that will be fixed in the upcoming releases.
You can still create disk groups with 4K sector size with sqlplus and asmcmd. Here is a command line example with asmcmd:
$ cat /tmp/sector4k.xml
[dg name="SECTOR4K" redundancy="high"]
[dsk string="ORCL:ASMDISK1"/]
[dsk string="ORCL:ASMDISK2"/]
[dsk string="ORCL:ASMDISK3"/]
[a name="compatible.asm" value="11.2"/]
[a name="compatible.rdbms" value="11.2"/]
[a name="sector_size" value="4096"/]
[/dg]
$ asmcmd mkdg /tmp/sector4k.xml
Of course you need to replace all [ with < and all ] with > in sector4k.xml. This thing would not let me post the correct xml :(
Cheers,
Bane
Bane, that is genius! Do you know the syntax for running the installASM command using XML so that I can create the diskgroup with 4k sector size? I've been messing around with the XML but I can't seen to find any examples in the documentation.
ReplyDeleteWell, before you pronounce me a genius, note that the example I gave was for asmcmd command, not asmca :)
ReplyDeleteI think it may be best that you create the ASM instance first (with asmca -silent) and add your 4k sector disk group later.
Cheers,
Bane
Ha I didn't notice that.
ReplyDeleteI tried setting up the ASM instance first but of course you have to specify a diskgroup to do that and my ASM disks all reside on 4k sector disks. So I end up creating a 512 byte sector diskgroup that I don't want just so I can create ASM.
I guess I need to wait for ASMCA to have sector_size added to its parameter list...
Is there a way to create 4k sector size diskgroup without using ASMLIB? i.e using rawdevices? I can create 4k SS disks with ASMLIB and having errors with raw devices : "ERROR at line 1:
ReplyDeleteORA-15018: diskgroup cannot be created
ORA-15080: synchronous I/O operation to a disk failed"
No, ASMLIB is not a requirement for 4K sector size disk groups.
ReplyDeleteCan you do I/O against the raw devices at the OS level, both read and write, say with the dd command?
Do you have 512 byte sector raw disks, attached to the same server, that you can use to create a disk group? In other words, do you have this problem with 4k sector size disks only?
What other diagnostics do you have?
Cheers,
Bane
Hi Bane,
ReplyDeleteThanks for the reply. Here is more info, and I used SSD.
Test 1:
Formated SSD with 512b
Created Partitions (for 1MB lun alignment)
Used ASMLIB, created ASM disks
Successfully Created diskgroup with default 512b
Test 2:
Formated SSD with 4k
Created Partitions (for 1MB lun alignment)
Used ASMLIB, created ASM disks
Successfully Created diskgroup with 4k sctor size
Test 3:
Formated SSD with 512b
Created Partitions (for 1MB lun alignment)
Configured in /etc/sysconfig/rawdevices
Set permissions and restarted the service
Successfully Created diskgroup with 512b sctor size
Test 4:
Formated SSD with 4k
Created Partitions (for 1MB lun alignment)
Configured in /etc/sysconfig/rawdevices
Set permissions and restarted the service
Got an error while creating diskgroup with 4k sctor size
dd works fine; but Oracle throws an error. Thanks again.
Interesting... How about using block devices instead of raw? You know that raw devices are not required and are being phased out anyway.
DeleteAdjust ownership and permissions on your devices and add '/dev/sd*' to your asm_diskstring. Now create the disk group:
create diskgroup D1
disk '/dev/sdg', '/dev/sdh'
attribute 'compatible.asm'='11.2', 'compatible.rdbms'='11.2', 'sector_size'='4096';
Cheers,
Bane
I've tried it as a block device and got the same errro. Just opened a SR with Oracle and awiting response. Thanks.
ReplyDeletePlease let me know the outcome as I am curious to know what is going on. Unfortunately I don't have a 4k sector size disk to try this myself :(
DeleteCheers,
Bane
Looks like its a New bug and the SR has been moved to the ASM development team.
ReplyDeleteLet's see what they say...
Deletebane,
ReplyDeleteI am running the command as below
Oracle@chirU>asmca -silent -configureASM -sysAsmPassword ora1n -asmsnmpPassword ora2n -diskString '/dev/dsk/c2*' -diskGroupName DATA -disk '/dev/dsk/c2*' -redundancy EXTERNAL
Configuring ASM failed with the following message:
Unable to copy the file "/tmp/oratab" to "/var/opt/oracle/oratab".
It fails I have lot space. Also
'/dev/dsk/c2* disk string is correct.
This may not be the best forum to diagnose issues.
DeleteYou may start by looking at why oracle user cannot copy a file to /var/opt/oracle. You can then trace/strace asmca to see what is going on. If you cannot work it out please email me [bane.radulovic at gmail.com] your ASM version, OS version and the trace/strace file. Once we sort it out we can post the findings here.
Cheers,
Bane
hi Bane.
ReplyDeleteIm getting "PRVF-5150 file name /dev/rdsk/xxxxx not a valid path" error during configuring 11gR2 grid.
But when i checked , device is existed and permissions also set to "grid,asmadmin"
Can you please help me on this.
Hi Ramesh,
ReplyDeleteI haven't seen that error, but quick google search shows up few discussions that point to a bug and the advice that the error can be ignored. Please let me know you exact ASM version and I will check if that bug is still there in your version.
Cheers,
Bane
Hi Bane ,
ReplyDeleteThanks for your prompt response. Below are the configuration details.
Grid Version : 11.2.0.3.0
OS : Sun OS 5.10
Storage : SAN slices given as raw devices.
Error is showing in "prerequests cheking" in during grid installation.
Thanks.
Yeah, that issue is still there in 11.2.0.3. You can ignore it and proceed with the install.
DeleteThanks bane..
ReplyDeleteI will proceed further and let you know the status.
Hi Bane,
ReplyDeleteWe have ignored the "Warning" msg and proceeded with installation.
at the end of the installation or while managing cluster we don't encountering any errors.
Please clarify the below.
How can we add the additional disks to existed "Diskgroup" in ASM which is having "NORMAL" redundancy ( 2 failgroups).
Thanks in advance.
If you didn't specify failgroup names yourself, you just add a new disk with:
Deletealter diskgroup add disk '';.
If you did specify failgroups explicitly, and you had a good reason to do so, you need to use:
alter diskgroup add failgroup disk '';.
Then you need to add another disk to the other failgroup, as you want both failgroups to have the same number of disks and the same capacity.
But I suspect, you did not manually name your failgroups, which is fine and how most people do it. In that case you just add new disks as needed, using the first command I gave you.
Cheers
Ahhh. This site doesn't like angle brackets :(
DeleteThe first command should be:
alter diskgroup dg_name add disk 'disk_name';
The second, with failgroups should be:
alter diskgroup dg_name add failgrup fg_name disk 'disk_name';
Hi Bane,
ReplyDeleteI'm trying to install ASM by asmca. Either with graphical or command line mode I still receive the same error:
Configuring ASM failed with the following message:
Configuring HA resource failed. The following error occured:
PRCR-1079 : Failed to start resource ora.asm
ORA-00119: invalid specification for system parameter %s
CRS-2674: Start of 'ora.asm' on 'my_server_name' failed
Could you help me?
Thanks,
g10
Hi g10,
ReplyDeleteIt's not easy to tell what the problem is based on what you provided. Best I can suggest is google search for "PRCR-1079 ORA-00119 CRS-2674".
If that doesn't help, please let me know:
1. The exact command line (all arguments)
2. ASM and OS version
3. The result of 'ls -l '
Cheers,
Bane
Hi Bane,
DeleteI've tried before wrote to you to google about all the net and also in metalink, but I haven't found useful informations.
ASM is 11gR2, os version is RHEL 5, exact command is:
/orasw/app/11.2.0/grid/bin/asmca -silent -configureASM -sysAsmPassword s3kr3t1 -asmsnmpPassword s3kr3t2 -diskString 'ORCL:*' -diskGroupName DATA -disk 'ORCL:*' -redundancy EXTERNAL.
Thanks in advance,
g10
Hello Sir,
ReplyDeleteI am not expert but a senior dba. I need your valuable advice if you would:
I am trying to solve a db block corruption error on ASM disks and I am trying to solve the problem with some options.
One of them is to create an ASM DG with sector size 4096 bytes instead of default 512 bytes.
But when I tried to create DG with the following command, I encounter errors.
Could you help me about this issue please?
My environment :
---------------------------------------------------------
Red Hat Enterprise Linux Server release 5.8 (Tikanga)
Oracle RDBMS and ASM binary : 11.2.0.3.0, oracleasm lib
Storage : HTS SAN disks (LUNs)
My command and the error :
----------------------------------------------------------
SQL> CREATE DISKGROUP TESTDG1
2 EXTERNAL REDUNDANCY
3 DISK 'ORCL:TESTDISK01'
4 ATTRIBUTE
5 'au_size'='64M',
6 'sector_size'='4096',
7 'compatible.asm' = '11.2',
8 'compatible.rdbms' = '11.2';
CREATE DISKGROUP TESTDG1
*
ERROR at line 1:
ORA-15018: diskgroup cannot be created
ORA-15038: disk '' mismatch on 'Sector Size' with target disk group [512][4096]
Best Regards,
Devrim
Hi Devrim,
DeleteThat disk (TESTDISK01) has to be created with the 4KB sector size. Only then you can create the disk group with that command.
You can check that with the following command:
$ kfed read /dev/oracleasm/disks/TESTDISK01 | grep secsize
That should return kfdhdb.secsize: 4096.
If it returns kfdhdb.secsize: 512, you need to talk to your storage/sys admin and ask them to create the disk/LUN with 4KB sector size.
Let me how that went.
Cheers,
Bane
Hi Bane,
ReplyDeleteThank your for quick response :)
I just tried kfed command but it gave nothing :(
Then I write first pat of command and I got the result as below :
+ASM oracle@crixus:/home/oracle>kfed read /dev/oracleasm/disks/TESTDISK01
kfbh.endian: 0 ; 0x000: 0x00
kfbh.hard: 0 ; 0x001: 0x00
kfbh.type: 0 ; 0x002: KFBTYP_INVALID
kfbh.datfmt: 0 ; 0x003: 0x00
kfbh.block.blk: 0 ; 0x004: blk=0
kfbh.block.obj: 0 ; 0x008: file=0
kfbh.check: 520109344 ; 0x00c: 0x1f003d20
kfbh.fcn.base: 0 ; 0x010: 0x00000000
kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
kfbh.spare1: 0 ; 0x018: 0x00000000
kfbh.spare2: 0 ; 0x01c: 0x00000000
2AF5E2BBA400 00000000 00000000 00000000 1F003D20 [............ =..]
2AF5E2BBA410 00000000 00000000 00000000 00000000 [................]
2AF5E2BBA420 4C43524F 4B534944 54534554 4B534944 [ORCLDISKTESTDISK]
2AF5E2BBA430 00003130 00000000 00000000 00000000 [01..............]
2AF5E2BBA440 00000000 00000000 00000000 00000000 [................]
Repeat 251 times
KFED-00322: Invalid content encountered during block traversal: [kfbtTraverseBlock][Invalid OSM block type][][0]
Then, I tried your command on working asm disk :
+ASM oracle@crixus:/home/oracle>kfed read /dev/oracleasm/disks/DMMDATA01|grep secsize
kfdhdb.secsize: 512 ; 0x0b8: 0x0200
Actually TESTDISK01 may be created wrong but I thought its sector size 512 bytes possibly.
Could you suggest me the method to create 4096 bytes ASM disk with oracleasm please?
As I know there is no option to specify sector size with "oracleasm create disk" command.
Regards,
Devrim
The output of 'kfed read /dev/oracleasm/disks/TESTDISK01' is normal for a disk that is not part of a disk group. Sorry, that command (with grep secsize) I gave you would show the sector size once the disk group is created.
DeleteWhile you create ASMLIB disk (with oracleasm createdisk), you use a LUN that was given to you by your storage/sys admin. You need to have the LUN created with 4KB sector size and that is something your storage/sys admin has to do for you. The thing is that the device itself has to support 4KB sector size, and only then you can create the disk group with 4KB sector size.
Hope this makes sense.
Cheers,
Bane
Hello Bane,
ReplyDeleteI just shared thsi information with OS and SAN/Storage admins.
They will try to find a solution to problem.
Thank you and best regards,
Devrim