O/S: RHEL 3
DB: 10.2.0.3


Hi all,

Our SAN storage failed with its zoning whilst the db was running. This has caused ASM to go into a panic. This is how the ASM storage looked before this problem occurred (the asm dg is setup as normal redundancy):
Code:
GROUP_NUMBER  DISK_NUMBER  COMPOUND_INDEX  INCARNATION  MOUNT_S  HEADER_STATU  MODE_ST  STATE  REDUNDA  LIBRARY  TOTAL_MB  FREE_MB  NAME  FAILGROUP  LABEL  PATH  UDID  PRODUCT  CREATE_DA  MOUNT_DAT  REPAIR_TIMER  READS  WRITES  READ_ERRS  WRITE_ERRS  READ_TIME  WRITE_TIME  BYTES_READ  BYTES_WRITTEN  
1  0  16777216  4042741803  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23255  DGGROUP_0000  PRIMARY     /dev/raw/raw1        30-NOV-07  28-OCT-08  0  36804  46018  0  0  786.1  582.06  6755644416  6763955200  
1  1  16777217  4042741804  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23370  DGGROUP_0001  PRIMARY     /dev/raw/raw2        30-NOV-07  28-OCT-08  0  44119  42268  0  0  867.22  603.79  7411873280  5409548288  
1  2  16777218  4042741805  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23271  DGGROUP_0002  PRIMARY     /dev/raw/raw3        30-NOV-07  28-OCT-08  0  32283  45450  0  0  869.36  649.66  7197868032  6651507712  
1  3  16777219  4042741809  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23398  DGGROUP_0003  PRIMARY     /dev/raw/raw7        25-FEB-08  28-OCT-08  0  34146  45752  0  0  711.88  687.74  9106022400  6570381824  
1  4  16777220  4042741810  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23272  DGGROUP_0004  FAILURE     /dev/raw/raw8        25-FEB-08  28-OCT-08  0  50672  42625  0  0  420.74  627.37  6999826432  5928142336  
1  5  16777221  4042741813  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1126399  21329  DGGROUP_0005  PRIMARY     /dev/raw/raw11        26-APR-08  28-OCT-08  0  43892  54946  0  0  815.75  880.13  8084622848  8770541056  
1  6  16777222  4042741806  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23342  DGGROUP_0006  FAILURE     /dev/raw/raw4        04-JAN-08  28-OCT-08  0  31169  45148  0  0  856.28  703.76  6300344320  5618552832  
1  7  16777223  4042741807  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23309  DGGROUP_0007  FAILURE     /dev/raw/raw5        04-JAN-08  28-OCT-08  0  37962  47718  0  0  1243.8  784.62  8271555072  9542723584  
1  8  16777224  4042741808  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  23362  DGGROUP_0008  FAILURE     /dev/raw/raw6        04-JAN-08  28-OCT-08  0  35696  47765  0  0  1063.55  810.58  7211213312  8064837120  
1  9  16777225  4042741814  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1126399  21366  DGGROUP_0009  PRIMARY     /dev/raw/raw12        26-APR-08  28-OCT-08  0  42290  52654  0  0  695.94  2057.64  8111098880  8949030912  
1  10  16777226  4042741811  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1126399  21371  DGGROUP_0010  FAILURE     /dev/raw/raw9        26-APR-08  28-OCT-08  0  57631  53387  0  0  973.89  947.48  8225914880  7936991232  
1  11  16777227  4042741812  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1126399  21308  DGGROUP_0011  FAILURE     /dev/raw/raw10        26-APR-08  28-OCT-08  0  45823  52156  0  0  926.72  2098.44  7714097152  7713911808  
1  12  16777228  4042741815  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  1034652  DGGROUP_0012  DGGROUP_0012     /dev/raw/raw13        28-OCT-08  28-OCT-08  0  2700  48486  0  0  159.19  469.68  918773760  1.0007E+10  
1  13  16777229  4042741816  CACHED  MEMBER  ONLINE  NORMAL  UNKNOWN  System  1048575  1034661  DGGROUP_0013  DGGROUP_0013     /dev/raw/raw14        28-OCT-08  28-OCT-08  0  2629  45224  0  0  72.31  1566.82  782823424  9782869504  


14 rows selected.

SQL>
The above was taken whilst a rebalance was going on as we had added disks.

This is how the ASM diskgroup looks now:
Code:
GROUP_NUMBER DISK_NUMBER HEADER_STATU MOUNT_S PATH           FAILGROUP
------------ ----------- ------------ ------- -------------- --------------------
           0           0 MEMBER       CLOSED  /dev/raw/raw3
           0           1 MEMBER       CLOSED  /dev/raw/raw7
           0           2 MEMBER       CLOSED  /dev/raw/raw11

           1           0 MEMBER       CACHED  /dev/raw/raw1  PRIMARY
           1           1 MEMBER       CACHED  /dev/raw/raw2  PRIMARY
           1           2 CANDIDATE    MISSING should be raw3
           1           6 MEMBER       CACHED  /dev/raw/raw4  FAILURE
           1           7 MEMBER       CACHED  /dev/raw/raw5  FAILURE
           1           8 MEMBER       CACHED  /dev/raw/raw6  FAILURE
           1           3 CANDIDATE    MISSING should be raw7
           1           4 MEMBER       CACHED  /dev/raw/raw8  FAILURE
           1          10 MEMBER       CACHED  /dev/raw/raw9  FAILURE
           1          11 MEMBER       CACHED  /dev/raw/raw10 FAILURE
           1           5 CANDIDATE    MISSING should be raw11
           1           9 MEMBER       CACHED  /dev/raw/raw12 PRIMARY
           1          12 MEMBER       CACHED  /dev/raw/raw13 DGGROUP_0012
           1          13 MEMBER       CACHED  /dev/raw/raw14 DGGROUP_0013

As can me seen raw3,7 and 11 have gone missing. Can anyone shed any light into how we get these disks back into the diskgroup. We tried:

1/ A repair, this succedded but didnt touch the offline disks.
2/ dropping the disks but this resulted in an error as well altough it started a rebalance which we stopped as we dont have enough space.

Please can anyone let us know how we can get these disks back?

Thanks in advance,
Chucks