Process information

today we have a system which kicked all users out of the system due to memory being flushed or so it seems.  Whilst yes we have monitoring and can see memory & cpu usage, we dont actually have post event visibility of the processes that caused this.

Both top and ps provide this information, but only if we capture it. Thus the following will grab the process information  covering both cpu, overall memory consumption as total % followed by actual values for  RSS & VSZ.

ps -A -o comm,%cpu,%mem,rss,vsz |sed 's/\(\w*\)\/\(\w*\)/\1/g'|awk 'NR==1 { print} NR!=1 {cpu[$1]+=$2;mem[$1]=$3;rss[$1]=$4;vsz[$1]=$5} END { for (i in cpu) {print i,cpu[i],mem[i],rss[i],vsz[i]}}'| sort -r +1 -2 -| head -6 | tail -5

example output

mysqld 8.6 83.8 27557052 29039096
puppetd 0 0.1 64416 174908
zabbix_agentd 0 0.0 412 52292
watchdog 0 0.0 0 0
vmw_pvscsi_wq_2 0 0.0 0 0

note, it reports back both the RSS (Resident Set Size) and VSZ (virtual memory size).  There is a flaw here, but in essence, VSZ will show the entire process size as available – but not necessarily loaded into physical ram, whilst RSS will show the approximate amount of physical memory currently loaded.

I mention RSS as being approximate memory size – be aware that it doesn’t include the page size for any shared libraries.