Tablespace restore / recovery with RMAN

[Axr2]

Jeff / moderators,
May be I didn't phrase my question right. I still continue to get a "RMAN-06023: no backup or copy of datafile 3 found to restore" message while trying to restore the new tablespace.

I've simulated and pasted all the steps again:
1) I first performed a level 0 incremental backup.
2) THen created a tablespace.
3) Then renamed the underlying d/f.
4) Then tried restoring the tablespace.

I've "cut n paste" all the steps that I performed. By the way, one other issue cropped up while simulating the below restore..I was able to issue checkpoints (for a while) even after the underlying datafile was renamed at the OS level! Pretty bizzare..please see the below, you'll probably understand this a lil better.

I first kicked off the backup. It was successful. If you'd like I can paste the logfile later.

oracle[/export/home/oracle/rman] rman @testrmanincr0.rman log='/export/home/oracle/rman/testrmanincr0.log'


oracle[/export/home/oracle/rman] cat testrmanincr0.rman
connect target sys/blah@TESTRMAN
connect catalog rman/rmanblah@DWDEV
run { execute script testrman_incr0; }
===================
Here is the rman backup script being called by testrmanincr0.rman :

CREATE SCRIPT testrman_incr0
{
allocate channel t1 type 'sbt_tape' parms
'ENV=(TDPO_OPTFILE=/opt/tivoli/tsm/client/oracle/bin64/tdpo.opt)';
allocate channel t2 type 'sbt_tape' parms
'ENV=(TDPO_OPTFILE=/opt/tivoli/tsm/client/oracle/bin64/tdpo.opt)';
BACKUP INCREMENTAL LEVEL = 0 CUMULATIVE
format 'testrman_incr0_%t_%s_%p'
filesperset 5
DATABASE;
sql 'alter system archive log current';
backup
format 'testrman_incr0_archlog_%t_%s_%p'
archivelog all;
release channel t1;
release channel t2;
}
==================
Now to creating the new tablespace

SQL> create tablespace test_ts datafile '/u12/oradata/TESTRMAN/test_ts.dbf' size 1M;

Tablespace created.

SQL> !ls -l /u12/oradata/TESTRMAN/test_ts.dbf
-rw-r----- 1 oracle oinstall 1064960 Apr 13 13:39 /u12/oradata/TESTRMAN/test_ts.dbf

SQL> create table junk_on_test_ts (a varchar2(20)) tablespace test_ts;

Table created.

SQL> alter system switch logfile;

System altered.

SQL> alter system checkpoint;

System altered.

SQL> !mv /u12/oradata/TESTRMAN/test_ts.dbf /u12/oradata/TESTRMAN/test_ts.dbf.o

SQL> alter system checkpoint;

System altered.

SQL> !ls -l /u12/oradata/TESTRMAN/test_ts.dbf*
-rw-r----- 1 oracle oinstall 1064960 Apr 13 13:41 /u12/oradata/TESTRMAN/test_ts.dbf.o

SQL> alter system checkpoint;

System altered.

HOW IN THE WORLD DOES IT ISSUE CHECKPOINTS WHEN THE D/F HAS BEEN RENAMED?!! (Not yelling..just making this prominent)

No error messages in alertlog either!

oracle[/u01/app/oracle/admin/TESTRMAN/bdump] tail alert_TESTRMAN.log
Tue Apr 13 13:35:13 2004
ARC0: Evaluating archive log 1 thread 1 sequence 23
ARC0: Unable to archive log 1 thread 1 sequence 23
Log actively being archived by another process
Tue Apr 13 13:35:13 2004
ARCH: Completed archiving log 1 thread 1 sequence 23
Tue Apr 13 13:39:25 2004
create tablespace test_ts datafile '/u12/oradata/TESTRMAN/test_ts.dbf' size 1M
Tue Apr 13 13:39:25 2004
Completed: create tablespace test_ts datafile '/u12/oradata/T

After about 2-3 minutes of continuously issuing checkpoints, I eventually see the following errors in the alertlog. Why did it take so long for the smon to wake up and figure the checkpointing error?! So I could potentially have inserted rows into this table although the underlying datafile was gone! So would these rows have been written to?! This is bizzare.

oracle[/u01/app/oracle/admin/TESTRMAN/bdump] tail alert_TESTRMAN.log
Errors in file /u01/app/oracle/admin/TESTRMAN/bdump/testrman_smon_13050.trc:
ORA-01116: error in opening database file 3
ORA-01110: data file 3: '/u12/oradata/TESTRMAN/test_ts.dbf'
ORA-27041: unable to open file
SVR4 Error: 2: No such file or directory
Additional information: 3
Tue Apr 13 13:46:13 2004

Anyway, back to the restore issue at hand :

oracle[/u12/oradata/TESTRMAN] rman target / catalog rman/rmanblah@DWDEV

Recovery Manager: Release 9.2.0.4.0 - 64bit Production

Copyright (c) 1995, 2002, Oracle Corporation. All rights reserved.

connected to target database: TESTRMAN (DBID=2651072106)
connected to recovery catalog database

RMAN> run { allocate channel t1 type 'sbt_tape' parms
2> 'ENV=(TDPO_OPTFILE=/opt/tivoli/tsm/client/oracle/bin64/tdpo.opt)';
3> sql "alter tablespace TEST_TS offline immediate";
4> restore tablespace TEST_TS;
5> recover tablespace TEST_TS;
6> sql "alter tablespace TEST_TS online";
7> release channel t1;
8> }

starting full resync of recovery catalog
full resync complete
allocated channel: t1
channel t1: sid=11 devtype=SBT_TAPE
channel t1: Tivoli Data Protection for Oracle: version 5.2.0.0

sql statement: alter tablespace TEST_TS offline immediate

Starting restore at 13-APR-04

released channel: t1
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 04/13/2004 13:56:48
RMAN-06026: some targets not found - aborting restore
RMAN-06023: no backup or copy of datafile 3 found to restore