how many databases one RAC can have? any limit?

Hi,

We have one RAC/ASM two node system.

There are total 36 databases on this RAC now.

we can only start 31 databases with srvctl, rest can only be started with sqlplus.

errors we have is:

SQL> ORA-27102: out of memory
SVR4 Error: 28: No space left on device
SQL> Dis'

file system is not full. asm have free space also.

change /etc/system or /etc/project will not help anything here.
RAC is 10.2.0.4

OS is Sun 10.

srvctl can start any 31 databases out of total 36 databases. Not related with any databases itself.

Thanks.

xyz2000

31 databases on a server? are you crazy?

you have run out of resources

but 31 databases on a server? utter madness

this is not my design, some one designed. We told them, it is too much, no one listen to us.

When more and more database added to RAC, we see more and more problems.

First, when we try to add more databases to rac, we see OCR get locked, we have to wait for while to try again.

Second, when we add more listeners to RACusing netca, we can not add listener on two node at same time, each time, we have to add listener on one node ....

Third, now we can not start database with srvctl, we can start databases using sqlplus.


Not sure what next problems will be. So far, each time, when we have problems, we can find a work around. So no one cares.

so far there are total 43 databases on that one server now.

36 databases out of 43 databases are on RAC. 7 databases are 9i stand alone databases.

this server have 256G memory and 32 CPU.

sorry, some number not right

there are total 43 databases on that server.
40 are on RAC,
they have 2 node cluster and 40 databases.

After reboot , only 33 databases are able to come up online by themselves.
Rest of them fail to come up.

When trying to manually start instance, still getting error.

From ora.inst file in racg folder found
===
SQL> ORA-27102: out of memory
SVR4 Error: 28: No space left on device
===

So change project and kernel settings as below

Modified shmmax in /etc/system file as below
set shmsys:shminfo_shmmax=6442450944

% projmod -s -K "project.max-shm-memory=(privileged,2.00TB,deny)" system
% projmod -s -K "process.max-sem-nsems=(privileged,2048,deny)" system

Rebooted node after these changes, still only 33 came up online, rest failed to come up with same error

The workaround as of now is to start remaining databases manually using sqlplus.

Please tell us you are kidding.

Hi

If 43 databases were single instance database that would have been okay .But you still have the problem of sharing of resources.

If out of 3 you have 40 RAC instances then you are in a big soup mate.
RAC was designed to elimnate a single point of failure and dsitribute workload your design works exactly against this a single bad program/query would bring down the entire node because of RAC now .

I sympathise with you as your option is very attractive from a management perspective but technically useless.

Better option would be to use VM and virtualize the environment and have one database per guest OS and then cluster with another host sharing resources in RAC is a very bad idea without a VM.

the workaround is to sort out your mess, not to make it worse

Quote:

---

Originally Posted by hrishy

If 43 databases were single instance database that would have been okay

---

Really?

Hi Pavb

If o/p can consolidate 43 databases into a single instance and make 1 database and then have RAC and use services he should be okay.RAC in my opinion is great when you can consolidate databases and use services to isolate applications

If you think otherwise please elaborate here .

currently i am in a enviroment where management for cost saving reasons has decided to run 3-4 instances on a 3 node RAC and when things go wrong its difficult to troubleshoot.

It would be great to know your thoughts and i look forward to it

regards
Hrishy

P.S:PAVB offtopic i was curious if you would come out with book on Data Warehousing one day as i have carefully read your posts on those topics and they are quite helpful

Hi hrishy --

I do like the strategy of consolidating several "databases" into a single instance providing performance and backup/recovery issues are worked out in advance.

Answering the second question... yes, I've being working for a while in putting together a book on data warehousing - still have to work out the storytelling format people seems to like on "Understanding Database Administration". Thank you for your kind words - I do like your postings too.

Apologies to the O/P since i am diverting from the topic.

Hi Pablo

Its great to know that you have been working towards putting together a book on datawarehousing.

It would be great if you can share the topic of contents of the Data Warehousing book.

I managed to steal a glance at the contents of your book Database Administration on amazon and the book did sound interesting in the way the topic of Database Administration is approached.

It would be great to have a datawarehousing book also be technology agnostic covering DW the way its done across all RDBMSes and specalised databases like netezza.