Automatic Storage Management (ASM) lessons 2

ASM Processes
After you start your ASM instance, you will find that several of the Oracle processes you
are acquainted with will be running, such as PMON and DBWR. Additional ASM processes
will be started too. These processes include the following in 11g,

• The ARBn process, used to perform disk group rebalance operations. There may be one or more of these processes running.
• The ASMB process manages ASM storage and provides statistics. 
• The GMON process maintains disk membership in ASM disk groups. 
• The KATE process performs proxy I/O to ASM metadata files when a disk is offlined. 
• The MARK process is responsible for marking ASM allocation units as stale following a missed write to an offline disk.
• The RBAL process runs in both database and ASM instances. RBAL is responsible for performing a global open of ASM disks in normal databases. RBAL coordinates rebalance activity for disk groups in ASM instances.

ASM Disk Discovery

ASM disk discovery is the first step to setting up an ASM disk group. In this section, we

will cover configuring the ASM_DISKSTRING parameter, which helps with ASM disk discovery, and then we will discuss the topic of ASM disk discovery in general.

You may not need to set ASM_DISKSTRING. ASM_DISKSTRING has a number of different
default values depending on the platform you are using. Below table lists the platform-
specific default values (these will be set if ASM_DISKSTRING is set to a NULL value only).

#    Default asm_diskstring values for supported platforms:

#     Solaris (32/64 bit)   /dev/rdsk/*
#     Windows NT/XP         \\.\orcldisk* 
#     Linux (32/64 bit)     /dev/raw/*
#     HPUX                     /dev/rdsk/*
#     HPUX(Tru 64)          /dev/rdisk/*
#     AIX                   /dev/rhdisk/*

For example in my windows system i did'nt set any ASM_DISKSTRING, so it taken the default setting ,
SQL> show parameter asm

NAME                                 TYPE        VALUE
------------------------------------ ----------- ----------------------------
asm_diskgroups                       string      DATA, FLASH
asm_diskstring                       string
asm_power_limit                      integer     1
SQL>
SQL> select path from v$asm_disk;
PATH
-----------------------------------------------------------------------------
\\.\ORCLDISKDATA0
\\.\ORCLDISKFLASH0
SQL>
The ASM_DISKSTRING can be dynamically altered, which is nice if your friendly system administratoradds some storage to your system that you want Oracle to be able to use.

The asterisk is required when defining the ASM_DISKSTRING parameter. Here are
some examples of setting the ASM_DISKSTRING parameter. In this first example, ASM will
look for disks in devices when we create disk groups:
SQL> Alter system set ASM_DISKSTRING=’/devices/*‘;
In the next example, we are pointing ASM_DISKSTRING to ORACLE_HOME/disks:
SQL> Alter system set ASM_DISKSTRING=’?/disks/*‘;

ASM Disk Discovery on Instance Start:
When the ASM instance is started, it will use the paths listed in the ASM_DISKSTRING parameter and discover the disks that are available.Once discovery is complete and the ASM instance is open, you can review the disks discovered by looking at the V$ASM_DISK view, as

SQL> column path format a20

set lines 132

set pages 50

select path, group_number group_#, disk_number disk_#, mount_status,

header_status, state, total_mb, free_mb

from v$asm_disk

order by group_number;

PATH GROUP_# DISK_# MOUNT_S HEADER_STATU STATE TOTAL_MB FREE_MB

-------------- ------- ------ ------- ------------ ----- -------- ---------

/dev/raw/raw4 0 1 CLOSED FOREIGN NORMAL 39 0

/dev/raw/raw5 0 0 CLOSED FOREIGN NORMAL 39 0

/dev/raw/raw3 0 2 CLOSED FOREIGN NORMAL 39 0

/dev/raw/raw6 0 2 CLOSED CANIDATE NORMAL 2048 2048

ORCL:ASM01_004 1 3 CACHED MEMBER NORMAL 34212 30436

ORCL:ASM01_005 1 4 CACHED MEMBER NORMAL 34212 30408

ORCL:ASM01_006 1 5 CACHED MEMBER NORMAL 34212 30420

In this view, you see that there are three disks (/dev/raw/raw3,/dev/raw/raw4,/dev/raw/raw5) that are not assigned to any group (thosewith GROUP_# set to 0). These are unassigned disks that ASM has discovered but that have not been assigned to a disk group. Note the mount status of CLOSED on those three disks, which also indicates that the disk is not being accessed by ASM. The HEADER_STATUS of FOREIGN indicates that these disks contain data already and are owned by some process other than ASM (in this case, these are voting disks for a RAC). If the HEADER_STATUS says CANIDATE, as with /dev/raw/raw6, then we could add this disk to an ASM disk group.

Notice that most of the disks have a MOUNT_STATUS of CACHED and a HEADER_STATUS of

MEMBER. This means that the disk is currently part of an ASM disk group (which we will

discuss more in the next section).

Redundancy, Striping, and Other ASM Topic

Redundancy:-

When configuring an ASM disk group, you can use one of three different ASM redundancy

setting options to protect the data in your disk group:

Normal : Typically employs two-way mirroring by default and thus requires allocation

of two failure groups.

High  : Typically employs three-way mirroring by default and thus requires allocation of

three failure groups.

External  : Does not employ any mirroring. This setting is typically used when the disk

group is being assigned to an external disk that is attached to some device that already

employs some disk redundancy (rely on hardware mirroring) and have only one failure group.

Each failure group represents a logical allocation of one or more disks to the ASM disk group and provides for mirroring within that disk group. Thus, when you create an ASM disk group, you might have one disk assigned to failure group 1 and one disk assigned to failure group 2. This way your data is protected from failure.When you’re using ASM mirroring, ASM will allocate an extent on a disk that becomes the primary copy (one of the failure groups) and then allocate copies of that extent to the mirrored copies (the other failure groups). 

When you define the redundancy setting for a disk group, you are defining things such

as what kind of striping occurs and whether the data will be mirrored. These attributes are defined based on which template you have assigned to the ASM disk group. By default, when you create a disk group, Oracle will assign it the default template setting. You can optionally assign another ASM template to a given disk group. Each Oracle file type has its own default template.

Here is a list of the "default" (built-in) templates and their attributes:

Template name	Striping	Mirroring when using a normal redundancy disk group	Mirroring when using a high redundancy disk group	Mirroring when using an external redundancy disk group
Controlfile	Fine	3-Way Mirroring	3-Way Mirroring	No Mirroring
Datafile	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Onlinelog	Fine	2-Way Mirroring	3-Way Mirroring	No Mirroring
Archivelog	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Tempfile	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Backupset	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Parameterfile	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Dataguardconfig	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Flashback	Fine	2-Way Mirroring	3-Way Mirroring	No Mirroring
Changetracking	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Dumpset	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Xtransport	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring
Autobackup	Coarse	2-Way Mirroring	3-Way Mirroring	No Mirroring

In addition to these "named" templates you can also create user-defined templates.  These user-defined templates appear in the "name" column of the v$asm_template view.

Default ASM Template Redundancy Settings

So, if you create a disk group with normal redundancy using the default template and

you put datafiles on it, the datafile template would be used by default. In this case, a

datafile would use two-way mirroring and coarse striping (see the section “Striping”).

This means you would have to allocate at least two disks to an ASM disk group when

it was created, one assigned to a different failure group.The DBA can mention whether the files created via the template should be 2-way or 3-way mirrored and Coarse or fine striped.

Template attributes

Striping Attribute Value	Description
FINE	Striping in 128 KB chunks.
COARSE	Striping in 1 MB chunks.

Redundancy Attribute Value	Resulting Mirroring in Normal Redundancy Disk Group	Resulting Mirroring in High Redundancy Disk Group	Resulting Mirroring in External Redundancy Disk Group
MIRROR	Two-way mirroring	Three-way mirroring	(Not allowed)
HIGH	Three-way mirroring	Three-way mirroring	(Not allowed)
UNPROTECTED	No mirroring	(Not allowed)	No mirroring

Example for add template,

ALTER DISKGROUP disk_group_name ADD TEMPLATE template_name 
  ATTRIBUTES ([{MIRROR|HIGH|UNPROTECTED}] [{FINE|COARSE}]);

The following statement creates a new template named reliable for the normal redundancy disk group dgroup2:

ALTER DISKGROUP dgroup2 ADD TEMPLATE reliable ATTRIBUTES (HIGH FINE);

When you add a template to a disk group, the template cannot be retroactively

applied to files already in that disk group. As a result, you will need to use RMAN

to back up and then restore files that already exist in the disk group in order for them to

take on the attributes of the new template.

You can see the templates associated with a given disk group by querying the V$ASM_

TEMPLATE view, as shown in this example:

SQL> select * from v$asm_template where group_number=1;

We can alter diskgroup command. Notice in this example that we are actually changing one of the attributes of the default templates. You cannot drop the default templates, but you can modify them, as shown here:

SQL> ALTER DISKGROUP sp_dgroup2 ALTER TEMPLATE datafile ATTRIBUTES (coarse);

ASM Disk Group Attributes

Oracle Database 11g also allows you to define specific disk group attributes. Disk group attributes are set using the attribute clause of the create diskgroup and alter

diskgroup commands. The following attributes can be set on a specific ASM disk group:

Au_size-This is the disk group allocation unit (AU) size. The value defaults to 1MB and

can be set only when the disk group is created. You must modify the AU size of the disk

group if you want the disk group to be able to hold larger amounts of data. A disk group

with the default AU size will be able to grow to 35TB (normal redundancy). Increasing the

AU size will significantly increase the maximum size of the disk group. The maximum AU

size is 64MB.

Compatible.rdbms- Indicates the database version that the disk group is compatible with

at a minimum (default is 10.1). This value should be equal to or greater than the compatibility parameter of the database(s) accessing the ASM disk group. This value cannot be rolled back once set.

Compatible.asm- Indicates the ASM instance version that the disk group is compatible with ata minimum (default is 10.1). Compatible.asm must always be set to a value equal to or greater than compatible.rdbms. Once compatible.asm is set for a disk group, it can not be rolled back to an earlier value.

Disk_repair_time- Indicates the length of time that the disk resync process should maintain

change tracking before dropping an offline disk. The default for this parameter is 3.6 hours. Disk group attributes can be viewed using the V$ASM_ATTRIBUTE view. You can see some examples of setting compatibility here:

SQL>Create diskgroup robert01 external redundancy

Disk ’/oracle/asm/ASM_DISKGROUP_robert01.asm’

Attribute ’ccompatible.asm’=’11.1.0’;

SQL>Alter diskgroup robert01 set attribute ’DISK_REPAIR_TIME’=’1200M’;

SQL>Alter diskgroup robert01 set attribute ’compatible.asm’=’11.1.0’;

ASM Fast Disk Resync

Disk loss in ASM can result from a number of reasons, such as loss of controller cards, cable failures, or power-supply errors. In many cases, the disk itself is still intact. To allow for sufficient time to recover from disk failures that do not involve the actual failure of a disk, ASM provides the ASM fast disk resync feature.

By default, when a disk in an ASM disk group fails the disk will be taken offline automatically. The disk will be dropped some 3.6 hours later. As a result, you have only 3.6 hours by default to respond to a disk outage. If you correct the problem and the physical disk media is not corrupted, then ASM fast disk resync will quickly resynchronize the disk when it comes back online, correcting the problem very quickly.You can change the amount of time that Oracle will wait to automatically drop the disk by setting the disk_repair_time attribute for the individual disk groups using the alter diskgroup command, as shown in this example, where we set the disk_repair_time attribute to 18 hours:

SQL> Alter diskgroup dgroup1 set attribute ‘disk_repair_time’=’18h’;

ASM Preferred Mirror Read

Your ASM configuration may involve remote mirroring to disks that are a fair distance away. When some of your disk mirrors are far away then those disks may not be the best set of disks for a given instance to read from. For example, you might have a Real Application Cluster database with local and remote mirrored disks. In this case, you want to have the RAC instances primarily read from the local disks to ensure the best performance. ASM preferred mirror read is designed to indicate to Oracle which disk failgroup is the preferred read disk group.
ASM preferred mirror read is only available if you are using RAC. Also preferred mirror read is generally used only with clustered ASM instances, but this is not a requirement. To take advantage of ASM preferred mirror read, you should configure each disk failure group with specific geographically located set of disks. Use the Oracle 11g parameter, asm_preferred_read_failure_groups, to configure a databse instance with a list of preferred disk failure group names to use when that instance accesses ASM disks. The format of the values of the asm_preferred_read_failure_groups parameter is diskgroupname.failuregroupname where diskgroupname is the name of the disk group that the failure group belongs to and failuregroupname is the preferred failure groups name. Include multiple diskgroup/failgroup names by separating each preferred read group defined with a comma as seen in this example:

asm_preferred_read_failure_groups=dgroup1.fdisk2, dgroup2.fdisk2

In the event ASM cannot read from the preferred disk failure group, then the non-preferred failure groups will be read. To determine if a given disk file group is a preferred disk group you can use the PREFERRED_READ column of the V$ASM_DISK view.

Adding an ASM Disk Group

You use the create diskgroup command to create an ASM disk group,

SQL> CREATE DISKGROUP DATA_AREA NORMAL REDUNDANCY
failgroup diskcontrol1 DISK
‘/devices/diska1’,‘/devices/diska2’,‘/devices/diska3’,‘/devices/diska4’
failgroup diskcontrol2 DISK
‘/devices/diskb1’,‘/devices/diskb2’,‘/devices/diskb3’,‘/devices/diskb4’;

we created as diskgroup named DATA_AREA with normal redundancy having two failure group, each contains four disks.From the following figure you can see how data is mirrored in disk group.

If we used high redundancy, we would need to add a third disk to the command,

as shown here:

SQL> CREATE DISKGROUP DATA_AREA HIGH REDUNDANCY

failgroup diskcontrol1 DISK
‘/devices/diska1’,‘/devices/diska2’,‘/devices/diska3’,‘/devices/diska4’
failgroup diskcontrol2 DISK
‘/devices/diskb1’,‘/devices/diskb2’,‘/devices/diskb3’,‘/devices/diskb4’;

failgroup diskcontrol3 DISK
‘/devices/diskc1’,‘/devices/diskc2’,‘/devices/diskc3’,‘/devices/diskc4’;

 You can also name the disks being assigned to the ASM disk group using the name
clause of the create diskgroup command . For example we can use
‘/devices/diska1’ name diska1,‘/devices/diska2’ name diska2,‘/devices/diska3’ name diska3,‘/devices/diska4’ name diska4  instead of ‘/devices/diska1’,‘/devices/diska2’,‘/devices/diska3’,‘/devices/diska4’ in the above command. Failure to use the name clause will result in each disk receiving its own system-default assigned name.

Dropping an ASM Disk Group

To remove an ASM disk group, you use the drop diskgroup command. By default, if any

files exist in the disk group, ASM will not allow you to drop it unless you use the including

contents clause.

Here is an example of removing an ASM disk group with the drop diskgroup command:

SQL> Drop diskgroup sp_dgroup2;

If the ASM disk group has files in it, use this version:

SQL> Drop diskgroup sp_dgroup2 including contents;

Adding Disks to an ASM Disk Group

As databases grow, you need to add disk space. The alter diskgroup command allows

you to add disks to a given disk group to increase the amount of space available. Adding

a disk to an existing disk group is easy with the alter diskgroup command,

SQL>alter diskgroup cooked_dgroup1 add disk ‘c:\oracle\asm_disk\_file_disk3’

name new_disk;

In the preceding example we did not assign the disk to a specific failure group. As a result, each disk will be assigned to its own failure group when it’s created. For example, when we added the disk to the cooked_dgroup1 disk group, a new failure group called cooked_dgroup1_0002 was created, as shown in this output:

SQL> select disk_number, group_number, failgroup from v$asm_disk;

DISK_NUMBER GROUP_NUMBER failgroup

----------- ------------ ------------------------------

1 0

0 1 DISKCONTROL1

1 1 DISKCONTROL2

2 1 COOKED_DGROUP1_0002

We can add a disk to an existing failure group by using the failgroup parameter, as

shown in this example:

SQL> alter diskgroup cooked_dgroup1 add failgroup DISKCONTROL1

disk ‘c:\oracle\asm_disk\_file_disk4’ name new_disk;

Removing Disks from an ASM Disk Group

The alter diskgroup command allows you to remove disks from an ASM disk group using

the drop disk parameter. ASM will first rebalance the data on the disks to be dropped,

assuming enough space is available. If insufficient space is available to move the data from

the disk to be dropped to another disk, then an error will be raised. Here is an example

of dropping a disk from a disk group:

SQL>alter diskgroup cooked_dgroup1 drop disk ‘c:\oracle\asm_disk\_file_disk4’;

The alter diskgroup command also gives you the option to drop from a failure group

all disks that are assigned to the disk group. Use the in failgroup keyword and then indicatethe name of the failure group, as shown in this example:

SQL>alter diskgroup cooked_dgroup1 drop disks in failgroup diskcontrol1;

When you drop a disk from a disk group, the operation is asynchronous. Therefore,

when the SQL prompt returns, this does not indicate that the operation has completed. To

determine if the operation has completed, you will need to review the V$ASM_DISK view.

When the disk drop is complete the column HEADER_STATUS will take on the value of FORMER,as shown in this example:

SQL> select disk_number, header_status from v$asm_disk;

DISK_NUMBER HEADER_STATU

----------- ------------

0 FORMER

1 FORMER

1 MEMBER

2 MEMBER

If the drop is not complete (the V$ASM_DISK column STATE will read dropping), you can

check the V$ASM_OPERATION view and it will give you an idea of how long the operation is expected to take before it is complete. Here is an example query that will provide you with this information:

SQL> select group_number, operation, state, power, est_minutes from v$asm_operation;

Undropping Disks from an ASM Disk Group

If you have accidentally dropped a disk, simply use the alter diskgroup command with the undrop disks parameter, as shown here:

SQL> alter diskgroup sp_dgroup2 undrop disks;

To undrop a disk, the header_status from v$asm_disk view must be dropping. You cannot undrop a completely dropped disk.

Resizing Disks in an ASM Disk Group

ASM diskgroup is full and we want to add space to diskgroup. We have 2 options here,
1. Add new disks depending on type of diskgroup / number of failgroups
2. Increase the size of the existing LUN / LV / Disk partition

First approach is pretty straight forward For example, if we have a NORMAL redundancy ASM diskgroup DATADG with 2 failgroups then we can add two disk in each failure group as follows ,

SQL> alter diskgroup DATADG add 
FAILGROUP DATADG_FG1 disk '/dev/raw/raw3' 
FAILGROUP DATADG_FG2 disk '/dev/raw/raw4' ;

This approach does not need any down time i.e. disks can be added when ASM & DB are running (Rebal operation will have some overhead, so we should be doing this when system is not heavily used).
Now second approach , 

First we need to extend LUN or LV or disk partition, so that OS can see the revised size. But here is the twist, If we are using oracle 10g, we’ll have to bounce the ASM instance (and database), so that ASM can read the new disk size. This is due to Bug 4110313 : So if you try to increase the diskgroup size using following command, command will not have any effect (although OS can see revised disk size)

SQL> select name, total_mb, usable_file_mb from v$asm_diskgroup;
NAME TOTAL_MB USABLE_FILE_MB
------------------------------ ---------- --------------
DATADG 69688 3460

SQL> alter diskgroup DATADG resize all;
Diskgroup altered.

SQL> select name, total_mb, usable_file_mb from v$asm_diskgroup;
NAME TOTAL_MB USABLE_FILE_MB
------------------------------ ---------- --------------
DATADG 69688 3460

This problem is fixed in 11g and we don’t have to recycle ASM instance

SQL> select name, total_mb, usable_file_mb from v$asm_diskgroup;
NAME TOTAL_MB USABLE_FILE_MB
------------------------------ ---------- --------------
DATADG 69688 3460

SQL> alter diskgroup DATADG resize all;
Diskgroup altered.

SQL> select name, total_mb, usable_file_mb from v$asm_diskgroup;
NAME TOTAL_MB USABLE_FILE_MB
------------------------------ ---------- --------------
DATADG 139320 38276