Intermittent Media recovery on Dataguard

[Chucks_k]

O/S: RHEL 4 both LIVE and DRS
DB:2 NODE RAC live 10.2.0.4 and 2 NODE RAC DRS on 10.2.0.4 via Dataguard.

Hi all,

We have recently build a standby database however we are having problems with media recovery. It seems that every time the standby detects a gap it fails to initially fetch the log. This is the error:

Fetching gap sequence in thread 1, gap sequence 36193-36194
Error 12154 received logging on to the standby
FAL[client, MRP0]: Error 12154 connecting to bof211,bof212 for fetching gap sequence
Thu Apr  9 09:13:40 2009
Errors in file /home/oracle/product/10.2.0/db_1/admin/mmgdrs/bdump/mmgdrs1_mrp0_17444.trc:
ORA-12154: Message 12154 not found; No message file for product=RDBMS, facility=ORA
Thu Apr  9 09:14:32 2009
RFS[187]: Archived Log: '+LOG/mmgdrs/1_36193_665682703.dbf'
Thu Apr  9 09:14:35 2009
Media Recovery Log +LOG/mmgdrs/1_36193_665682703.dbf
Media Recovery Waiting for thread 1 sequence 36194 (in transit)
Thu Apr  9 09:15:09 2009
RFS[185]: Possible network disconnect with primary database

Now does the above mention there is a problem with the network? As the log is fetched later on successfully.


On live we see the below ( roughly same time as above)

hu Apr  9 09:17:11 2009
ARC0: Attempting destination LOG_ARCHIVE_DEST_2 network reconnect (3135)
ARC0: Destination LOG_ARCHIVE_DEST_2 network reconnect abandoned
Thu Apr  9 09:17:11 2009
Errors in file /opt/oracle/admin/bof2/bdump/bof211_arc0_12645.trc:
ORA-03135: Message 3135 not found; No message file for product=RDBMS, facility=ORA
FAL[server, ARC0]: FAL archive failed, see trace file.
Thu Apr  9 09:17:13 2009
Errors in file /opt/oracle/admin/bof2/bdump/bof211_arc0_12645.trc:
ORA-16055: Message 16055 not found; No message file for product=RDBMS, facility=ORA
ARCH: FAL archive failed. Archiver continuing
Thu Apr  9 09:17:13 2009
ORACLE Instance bof211 - Archival Error. Archiver continuing.
Thu Apr  9 09:17:13 2009
Errors in file /opt/oracle/admin/bof2/bdump/bof211_lns1_16918.trc:
ORA-03135: Message 3135 not found; No message file for product=RDBMS, facility=ORA
Thu Apr  9 09:17:13 2009

The log gets applied (after stopping and starting media recovery!)
:

Thu Apr  9 11:15:41 2009
RFS[187]: Archived Log: '+LOG/mmgdrs/1_36194_665682703.dbf'
Thu Apr  9 11:15:43 2009
Media Recovery Log +LOG/mmgdrs/1_36194_665682703.dbf

If it is a network issue, is there a way to make sure the FAL service keeps trying rather going to sleep? We have a very large lag and need to get it back down to zero.

Any advice is greatly appreciated.

Thanks in advance,
Chucks

[sunil_kandi]

Does this happen during RMAN backup on Primary? If so there is matching oracle bug# 7587686 which Oracle Support claim fixed in 10.2.0.4 but few customers reporting the same issue on 10.2.0.4. Looks like you should contact Oracle Support.

[Chucks_k]

Hi,

Thanks for that. We have no backup running and our databases are already at 10.2.0.4. We may have to raise this with Oracle as I cannot see anything wrong with the config.

Thanks again