I have some questions to following scenario:

0) Instances A and B configured as RAC
1) Update transaction on Instance A changes row
2) commit;
3) LGWR writes changes to online log
(fast commit)
4) commit complete.
5) SELECT on Instance B references row
6) Instance A crashes

Question is, if DBWR has not written the "dirty block" to
disk yet then what does RAC present to the Instance B user
whose query began after committed change but before change
was persisted to datafiles from Instance A buffer cache? I
surmised that the query may just get an ORA-nnn error but
then perhaps Instance B should roll-forward rebuild
the block from Instance A's log files and give the correct
answer.

I got anwer for this question form
James Morle
Scale Abilities, Ltd
Author of
"Scaling Oracle8i - Building Highly Scalable OLTP System Architectures"

> When the first instance crashes, the second instance performs instance
> recovery on its behalf. While recovery is in progress, the resources
> that are in flux are effectively frozen out by the DLM (GCS,GES in new
> parlance): The second instance cannot get any lock mode on any of the
> resources, and will effectively hang. The instance recovery determines
> which resources were a) mastered on the dead node and need remastering
> on a survivor, and b) which resources were in a non-null mode. Then
> cache recovery takes place for the crashed instance, just as a single
> node performs its own cache recovery on startup after a crash. The redo
> logs are replayed to rollforward the transactions. Once instance
> recovery is complete, the query continues unabated!

and I'm going to test this answer.

I have:
-- 2 sun Solaris boxes with Sun Cluster 3.0.
-- EMC/Concorde Disk array
-- Sun Solaris and Linux boxes as clients.
-- all db files use RAW devices
I have 2 sets of redo log files for each instance
first - place on RAW devices
second - place on local disks of each Instance

Most hard question is:
How I can decrise probabitity of write operations of "dirty blocks"
from db cache Instance A to disk between point 4 and 6 scenario.

----------------------------------------------------------------
I'll very glad for any suggestions that help me build test enveronment.