memory scan rate on host box

[Harry77]

hp-ux 11.31
4 databases 10.2.0.3

vmstat every so often shows the memory page scan rate going through the roof which occasionally brings the box to a standstill.

the problem we are having is marrying up the scan rate usage to a process or indeed a series of processes.

There does not appear to be any unusual activity in the databases. Nor can see how rogue processes or again something unusual.

We have tracked when the issue first started (begining of the month) using grid, and are currently trying to ascertain what has changed on the system. But fundamentally if we can see what process(es) are casuing the scan rate to ramp up we'd have a better idea.

Does anyone know how to do this?

[Harry77]

Ok update

32Gb of memory.

Glance shows 20Gb used under Sys Mem

I have a lot of oracle processes, genuine connections it seems, but they have a parent process id of 1...which struck me as odd.

Anyway, I'm still struggling to pinpoint the memory leak.

[JChiappa]

Well, I don´t have the answers, dunno about OS methods for pinpoint paging, but some points can be done here :

a) the symptom "glance shows 20Gb used under Sys Mem" out of 32 Gb IS very suspect : check the docs/links in http://unix.derkeiler.com/Mailing-Li...4-06/0010.html and http://forums11.itrc.hp.com/service/...threadId=68965

b) "I have a lot of oracle processes" : http://www.dba-oracle.com/t_tuning_cpu_usage_vmstat.htm speak about it, but the fact is that in outage situations (ie, low CPU power remaining, low memory available, etc) the system begins to steal pages for the processes - the assumption here is, you installed something new, this thing (maybe due to bugs) is consuming a lot of resources, this is the root cause... OR maybe not, simply your applications begun to do more work (bigger volumes, new programs doing more work, etc) and the timing is just coincidence , the new installs are not interfering...

My initial advice : check the links, see about configs, AND search for intensive processes (using top, iostat, glance in the OS, and the Oracle tools/dictionary views inside the db), tune them/stop them....


best regards,


Chiappa

[Harry77]

Thanks for the post.

We ended up rebooting the machine, very, very annoying thing to have to do. But now sys mem is down to about 9Gb (in line with the DR machine) so definitely some kind of memory leak. Also the page scan rate is minimal (close to 0), i.e. what it should it be.

I don't know enough of how glance calculates 'system memory' so it is something I need to look into.

We've been through everything that has been recently installed and nothing appears to be the culprit. Saying that, we have noticed an issue with one application that uses dbms_pipes...and doesn't always close them. So that is something to consider, although the amount of memory a pipe uses is tiny.