This is not a recurring problem. It occurred only on Jun 23, 2008. This RAC has been in commission for 9 months now and so far we have not done any OS upgrades; We did apply CPU patch, but that was in Jan, 2008: not since then.
I do not think it is a clusterware issue. My thinking is, somehow, there was a network gap between cluster interconnect that forced node1 to reset node2.
Any other ideas?
As I said your servers lost communication between them.
o2net is ocfs2 hearbeat