check your hit ratio and try to reduce the db buffer cache.

How did you find the memory usage of each oracle process? Using top? Top mis guides..