arch procs hang: shutdown

[rogerF]

Oracle People:

I've got a brand new oracle 8.1.7 server up with fiber, veritas and the stops and whistles.

I'm testing before I go production and I have problems with arch procs that hang during shutdown:

SYS@MYSHOP.SE> shutdown immediate
Database closed.
Database dismounted.

.... At this point the following arc procs hang forever and
the shutdown is NOT completed:


--------------------------------------------------------
oracle 23815 1 0 Nov 11 ? 0:00 ora_arc3_cqdb
oracle 4658 1 0 Nov 14 ? 0:00 ora_arc6_cqdb
oracle 17103 1 0 Nov 10 ? 0:00 ora_arc2_cqdb
oracle 21100 1 0 Nov 12 ? 0:00 ora_arc4_cqdb
oracle 11330 1 0 06:56:05 ? 0:00 ora_arc7_cqdb
oracle 27813 1 0 Nov 13 ? 0:00 ora_arc5_cqdb
--------------------------------------------------------

The archive log destination is writable by oracle.

Has anybody heard of this problem ? Does anybody have any tips ?

THANKS,

Roger

[dknight]

Have any logfiles successfully archived?

[rogerF]

Thanks for reply:

Yes, we had arch files a few days ago:


---------------------------------------------
-rw-r----- 1 oracle dba 506368 Nov 9 15:13 arch_1_3065.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3066.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3067.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3068.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3069.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3070.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3071.arc
-rw-r----- 1 oracle dba 509440 Nov 9 15:17 arch_1_3072.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3073.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3074.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3075.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3076.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3077.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3078.arc
-rw-r----- 1 oracle dba 512000 Nov 9 15:17 arch_1_3079.arc
-rw-r----- 1 oracle dba 511488 Nov 9 15:17 arch_1_3080.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3081.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:17 arch_1_3082.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3083.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3084.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3085.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3086.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3087.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3088.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3089.arc
-rw-r----- 1 oracle dba 512512 Nov 9 15:18 arch_1_3090.arc
------------------------------------------------

[dknight]

Have you run out of space?

> df -k

Can you delete some of the arch files?

[rogerF]

I've got tons off disk- and Veritas to expand if needed.

Space not a problem.

-Roger

[dknight]

I don't know what it means, but did you notice that files 3066 through 3082 all archived at the same time?

[gpsingh]

Are there any messages in alert file.
How many redo log groups do u have.
Why did u need so many arch processes, any waits.
Can u truss these processes (on Solaris, need to verify the utility to trace on other vendors.)

[rogerF]

Thanks again:

The alert logs show normal actions during the hotbackups:

Thu Nov 15 06:56:04 2001
Completed: alter tablespace USER03 end backup
Archive Log Stop
archiving is disabled
Thu Nov 15 06:56:05 2001
ARCH shutting down
Thu Nov 15 06:56:05 2001
ARCH shutting down
ARC1: Archival stopped
Thu Nov 15 06:56:05 2001
ARC0: Archival stopped
Archive Log Start
archiving is enabled
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=20
ARC0: Archival started
ARC1 started with pid=21
Thu Nov 15 06:56:05 2001
ARCH: STARTING ARCH PROCESSES COMPLETE
Thu Nov 15 06:56:05 2001
ARC1: Archival started
Thu Nov 15 06:56:05 2001
ARC6: STARTING ARCH PROCESSES
ARC7 started with pid=18
Thu Nov 15 06:56:06 2001
ARC6: STARTING ARCH PROCESSES COMPLETE
Thu Nov 15 06:56:06 2001
ARC7: Archival started
Thu Nov 15 06:56:06 2001
alter database backup controlfile to
'/u02/backup/cqdb/backup/control01_cqdb_011115.bck'
Thu Nov 15 06:56:07 2001
Completed: alter database backup controlfile to
'/u02/bac
Thu Nov 15 06:56:07 2001
alter database backup controlfile to trace
Completed: alter database backup controlfile to trace
Thu Nov 15 14:39:33 2001
Restarting dead background process EMN0
EMN0 started with pid=23
Thu Nov 15 14:39:35 2001
Shutting down instance (immediate)
License high water mark = 6
Thu Nov 15 14:39:39 2001
ALTER DATABASE CLOSE NORMAL
Thu Nov 15 14:39:39 2001
SMON: disabling tx recovery
SMON: disabling cache recovery
Thu Nov 15 14:39:39 2001
Thread 1 closed at log sequence 3091
Thu Nov 15 14:39:39 2001
Completed: ALTER DATABASE CLOSE NORMAL
Thu Nov 15 14:39:39 2001
ALTER DATABASE DISMOUNT
Completed: ALTER DATABASE DISMOUNT
archiving is disabled
Thu Nov 15 14:39:43 2001
ARCH shutting down
Thu Nov 15 14:39:43 2001
ARCH shutting down
Thu Nov 15 14:39:43 2001
ARC0: Archival stopped
Thu Nov 15 14:39:43 2001
ARC1: Archival stopped
Thu Nov 15 14:44:52 2001
SHUTDOWN: waiting for detached processes to terminate.
Thu Nov 15 15:14:36 2001
Starting ORACLE instance (normal)
Thu Nov 15 15:17:40 2001
Starting ORACLE instance (normal)

[dknight]

Maybe you can force a logfile switch:

SQL> alter system switch logfile;

and see what happens.

[rogerF]

I've killed the arc procs from the command line as this is not in production yet.

The strange thing is that SQLPLUS seems to be having problems to startup. While svrmgrl has no problem. This could mean that sqlplus also could of had problem with shutdown:
------------------------------------------------------
export ORACLE_SID=cqdb
bash-2.03$ sqlplus

SQL*Plus: Release 8.1.7.0.0 - Production on Thu Nov 15 16:46:52 2001

(c) Copyright 2000 Oracle Corporation. All rights reserved.

Enter user-name: sys
Enter password:
ERROR:
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
SVR4 Error: 2: No such file or directory
------------------------------------------

Yes, I have /etc/system set up for Solaris !
Yes, "svrmgrl" had no problem starting up the db.

-Roger