how many databases one RAC can have? any limit?

[xyz2000]

Hi,

We have one RAC/ASM two node system.

There are total 36 databases on this RAC now.

we can only start 31 databases with srvctl, rest can only be started with sqlplus.

errors we have is:

SQL> ORA-27102: out of memory
SVR4 Error: 28: No space left on device
SQL> Dis'

file system is not full. asm have free space also.

change /etc/system or /etc/project will not help anything here.
RAC is 10.2.0.4

OS is Sun 10.

srvctl can start any 31 databases out of total 36 databases. Not related with any databases itself.

Thanks.

xyz2000

[davey23uk]

31 databases on a server? are you crazy?

you have run out of resources

but 31 databases on a server? utter madness

[xyz2000]

this is not my design, some one designed. We told them, it is too much, no one listen to us.

[xyz2000]

When more and more database added to RAC, we see more and more problems.

First, when we try to add more databases to rac, we see OCR get locked, we have to wait for while to try again.

Second, when we add more listeners to RACusing netca, we can not add listener on two node at same time, each time, we have to add listener on one node ....

Third, now we can not start database with srvctl, we can start databases using sqlplus.


Not sure what next problems will be. So far, each time, when we have problems, we can find a work around. So no one cares.

[xyz2000]

so far there are total 43 databases on that one server now.

36 databases out of 43 databases are on RAC. 7 databases are 9i stand alone databases.

[xyz2000]

this server have 256G memory and 32 CPU.

[xyz2000]

sorry, some number not right

there are total 43 databases on that server.
40 are on RAC,
they have 2 node cluster and 40 databases.

After reboot , only 33 databases are able to come up online by themselves.
Rest of them fail to come up.

When trying to manually start instance, still getting error.

From ora.inst file in racg folder found
===
SQL> ORA-27102: out of memory
SVR4 Error: 28: No space left on device
===

So change project and kernel settings as below

Modified shmmax in /etc/system file as below
set shmsys:shminfo_shmmax=6442450944

% projmod -s -K "project.max-shm-memory=(privileged,2.00TB,deny)" system
% projmod -s -K "process.max-sem-nsems=(privileged,2048,deny)" system

Rebooted node after these changes, still only 33 came up online, rest failed to come up with same error

The workaround as of now is to start remaining databases manually using sqlplus.

[PAVB]

Please tell us you are kidding.

[hrishy]

Hi

If 43 databases were single instance database that would have been okay .But you still have the problem of sharing of resources.

If out of 3 you have 40 RAC instances then you are in a big soup mate.
RAC was designed to elimnate a single point of failure and dsitribute workload your design works exactly against this a single bad program/query would bring down the entire node because of RAC now .

I sympathise with you as your option is very attractive from a management perspective but technically useless.

Better option would be to use VM and virtualize the environment and have one database per guest OS and then cluster with another host sharing resources in RAC is a very bad idea without a VM.

[davey23uk]

the workaround is to sort out your mess, not to make it worse