The production Oracle box a sunfire 280 is rebooting itself.
Oracle logs show nothing but a sudden restart so that tells me either a power cycle or solaris decided to restart.

Its compounded by another problem which may be totally unrelated.
I got on the console and saw that the year old scripts that predate me and have always worked - are in fact failing. So dbora call to dbstart is bombing as if the wrong shell is involved or the shell executables have become corrupt.

Here is the startup log I captured. I can however manually get Oracle up.

dbora and dbstart exist and have not change since 2002 - any ideas ?

sub-section of console :

> Aug 19 18:22:08 adrastea raid: Sense=700006000000009800000000A0000000000000000
> 000000000000000000000000000000000082C000000000000000000000000000B053154393438323
> 13832392020202020200301040000070000000000000000000000000000000000000000000000000
> 000000000000000000000000000000000000000000000000000000000000000000000000000044F3
> 038313930332F31373536343400000000000000
> Aug 19 18:22:08 adrastea raid: Sense=700006000000009800000000A0000000000000000
> 000000000000000000000000000000000082C000000000000000000000000000B053154393438323
> 13832392020202020200301040000070000000000000000000000000000000000000000000000000
> 000000000000000000000000000000000000000000000000000000000000000000000000000044F3
> 038313930332F31373536343400000000000000
> volume management starting.
> Aug 19 18:22:09 adrastea sendmail[420]: My unqualified host name (adrastea) unkn
> own; sleeping for retry
> Aug 19 18:22:09 adrastea sendmail[482]: My unqualified host name (adrastea) unkn
> own; sleeping for retry
> Aug 19 18:22:09 adrastea sendmail[461]: My unqualified host name (adrastea) unkn
> own; sleeping for retry
> Aug 19 18:22:09 adrastea sendmail[419]: My unqualified host name (adrastea) unkn
> own; sleeping for retry
> Aug 19 18:22:09 adrastea sendmail[417]: My unqualified host name (adrastea) unkn
> own; sleeping for retry
> /etc/rc3.d/S99dbora: [!: not found <--------corrupt shell processor ?
> Sun Microsystems Inc. SunOS 5.9 Generic May 2002
> Aug 19 18:22:12 adrastea sendmail[418]: My unqualified host name (adrastea) unkn
> own; sleeping for retry
> Aug 19 18:22:13 adrastea sendmail[513]: My unqualified host name (adrastea) unkn
> own; sleeping for retry
>


> I believe that is being generated in this Oracle startups script which has not changed so
> perhaps the shell script processor 'sh' has become corrupt ?
>
> if [! -f $ORA_HOME/bin/dbstart] <-- Cannot parse "[!" the file following it exists
> then
> echo "Oracle startup: cannot start"
> exit
> fi


Here is the full console:

> ===================FULL CONSOLE BELOW HERE=======================
>
> wn; sleeping for retry
> my own domain name (galileo) -- using short name
> Aug 19 16:00:01 galileo sendmail[3285]: My unqualified host name (galileo) unkno
> wn; sleeping for retry
> Aug 19 16:01:01 galileo sendmail[3285]: unable to qualify my own domain name (ga
> lileo) -- using short name
> Aug 19 16:01:01 galileo sendmail[3285]: [ID 702911 mail.alert] unable to qualify
> my own domain name (galileo) -- using short name

*****Above repeats and had to be removed

> /dev/rdsk/c2t0d0s3: 7109 files, 60706 used, 10019494 free
> /dev/rdsk/c2t0d0s3: (1158 frags, 1252292 blocks, 0.0% fragmentation)
> The system is coming up. Please wait.
> checking ufs filesystems
> /dev/rdsk/c1t5d5s0: is stable.
> /dev/rdsk/c1t5d7s0: is stable.
> /dev/rdsk/c1t5d6s0: is stable.
> /dev/rdsk/c1t5d3s0: is stable.
> /dev/rdsk/c1t5d4s0: is stable.
> /dev/rdsk/c1t5d2s1: is stable.
> /dev/rdsk/c2t0d0s7: is stable.
> /dev/rdsk/c1t5d0s0: is stable.
> /dev/rdsk/c2t1d0s7: is stable.
> /dev/rdsk/c1t5d1s0: is stable.
> /dev/rdsk/c1t5d2s0: is stable.
> starting rpc services: rpcbind done.
> Setting netmask of eri0 to 255.255.255.0
> Setting default IPv4 interface for multicast: add net 224.0/4: gateway adrastea
> syslog service starting.
> The NVSRAM settings of controller c1t5d0(1T94821829) are correct.
> nvutil command succeeded.
> Array Monitor initiated
> RDAC daemons initiated
> Aug 19 18:22:07 adrastea rdriver: ID[RAIDarray.rdaemon.1001] RDAC Resolution Dae
> mon locked in memory
> Aug 19 18:22:07 adrastea raid: AEN event Host=adrastea Ctrl=1T94821829 Dev=c1t5d
> 0
> Aug 19 18:22:07 adrastea raid: AEN event Host=adrastea Ctrl=1T94821829 Dev=c1t5d
> 0
> Aug 19 18:22:07 adrastea raid: ASC=3F ASCQ=C7 FRU=06 LUN=00 LUN Stat=00
> Aug 19 18:22:07 adrastea raid: ASC=3F ASCQ=C7 FRU=06 LUN=00 LUN Stat=00
> Aug 19 18:22:07 adrastea raid: Sense=7000060000000098000000003FC70600000000000
> 000000000000142000000000000000000080D000000000000000000000000000B053154393438323
> 13832392020202020200301040000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000004383
> 038313930332F31373536343300000000000000
> Aug 19 18:22:07 adrastea raid: Sense=7000060000000098000000003FC70600000000000
> 000000000000142000000000000000000080D000000000000000000000000000B053154393438323
> 13832392020202020200301040000000000000000000000000000000000000000000000000000000
> 00000000000000000000000000000000000000000000000000000000000000000000000000004383
Sense=700006000000009800000000A0000000000000000
> 000000000000000000000000000000000082C000000000000000000000000000B053154393438323
> 13832392020202020200301040000070000000000000000000000000000000000000000000000000
> 000000000000000000000000000000000000000000000000000000000000000000000000000044F3
> 038313930332F31373536343400000000000000
> Aug 19 18:22:08 adrastea raid: Sense=700006000000009800000000A0000000000000000
> 000000000000000000000000000000000082C000000000000000000000000000B053154393438323
> 13832392020202020200301040000070000000000000000000000000000000000000000000000000
> 000000000000000000000000000000000000000000000000000000000000000000000000000044F3
> 038313930332F31373536343400000000000000
> volume management starting.
> <$view"oracle@adrastea:~>$view racle@adrastea:~> /etc/hosts
> "/etc/hosts" [Read only] 16 lines, 573 characters


> *** ns.fsr.com can't find adrastea: Non-existent host/domain
> > exit
> <$view"oracle@adrastea:~>$view racle@adrastea:~> /etc/rc3.d/S99dbora
> "/etc/rc3.d/S99dbora" [Read only] 48 lines, 1300 characters
>
> "/etc/rc3.d/S99dbora" [Read only] 48 lines, 1300 characters
> #!/bin/sh
> # Place this file in /etc/init.d
> # Only root can write to /etc/init.d
> # Set ORA_HOME to be equivalent to the $ORACLE_HOME
> # from which you wish to execute dbstart and dbshut;
> #
> # Set ORA_OWNER to the user id of the owner of the
> # Oracle database in ORA_HOME.
> ORA_HOME=/ora/app/oracle/product/9.2
> ORA_OWNER=oracle
> if [! -f $ORA_HOME/bin/dbstart]
> then
> echo "Oracle startup: cannot start"
> exit
> fi
> case "$1" in
> 'start')
> :q
> <$ls"oracle@adrastea:~>$ls racle@adrastea:~>
> RMAN_README.TXT dbora rda.tar
> afiedt.buf dumps rman_blprod_bu2.log
> bldemo.html nsmail stdby_installed_prod.txt
> dba_scripts patches
> <$ls"oracle@adrastea:~>$ls racle@adrastea:~> -l $ORA_HOME/bin/dbstart
> /bin/dbstart: No such file or directory
> <$ls"oracle@adrastea:~>$ls racle@adrastea:~> -l /ora/app/oracle/product/9.2/bin/dbs
> dbshut dbsnmp dbsnmp0 dbsnmpj dbsnmpj0 dbsnmpwd dbstart
> <$ls"oracle@adrastea:~>$ls racle@adrastea:~>> -l /ora/app/oracle/product/9.2/bin/dbstart
> -rwxr-xr-x 1 oracle dba 4468 Apr 26 2002 /ora/app/oracle/product/9.
> 2/bin/dbstart
> <$ls"oracle@adrastea:~>$ls racle@adrastea:~> -l /bin/sh
> -r-xr-xr-x 4 root root 95488 Apr 6 2002 /bin/sh
> <$"oracle@adrastea:~>$ racle@adrastea:~> /etc/rc3.d/S99dbora
> bash: /etc/rc3.d/S99dbora: bad interpreter: Permission denied
> <$"oracle@adrastea:~>$ racle@adrastea:~>
> úU seeprom format: 0000.0000.0000.0002
> Fatal Error reported by: (PCI-PBM)