Problem
You’re working on a Solaris system, and irate users are reporting the database application is slow. You have multiple databases running on this box and want to identify which processes are consuming the most CPU resources. Once the resource-consuming processes are identified at the OS, then you want to map them (if possible) to a database process.
CHAPTER 6 ■ ANALYZING OPERATING SYSTEM PERFORMANCE
■ Note If you are not running Solaris, then see the solution in Recipe 6-2.
Solution
On most Solaris systems, the prstat utility is used to identify which processes are consuming the most CPU resources. For example, you can instruct the prstat to report system statistics every five seconds:
$ prstat 5
Here is some sample output:
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP 16609 oracle 2364M 1443M cpu2 60 0 3:14:45 20% oracle/11 27565 oracle 2367M 1590M cpu3 21 0 0:11:28 16% oracle/14 23632 oracle 2284M 1506M run 46 2 0:16:18 6.1% oracle/11 4066 oracle 2270M 1492M sleep 59 0 0:02:52 1.7% oracle/35 15630 oracle 2274M 1482M sleep 48 0 19:40:41 1.2% oracle/11
Type q or press Ctrl+C to exit prstat. In the prior output, process 16609 is consistently showing up as a top CPU-consuming process.
After identifying a top resource-consuming process, you can determine which database the process is associated with by using the ps command. This example reports on process information associated with the PID of 16609:
$ ps -ef | grep 16609
oracle 16609 3021 18 Mar 09 ? 196:29 ora_dw00_ENGDEV
In this example, the name of the process is ora_dw00_ENGDEV and the associated database is ENGDEV.
How It Works
If you’re working on a Solaris server, the top utility is oftentimes not installed. In these environments, the prstat command can be used to determine top resource-consuming processes on the system. Table 6-2 describes several of the columns displayed in the default output of prstat.
Table 6-2. Column Descriptions of the top Output Column Description
PID Unique process identifier
USERNAME OS username running the process SIZE Virtual memory size of the process RSS Resident set size of process
STATE State of process (running, stopped, and so on) PRI Priority of process
NICE Nice value used to compute priority TIME Cumulative execution time
CPU Percent of CPU consumption PROCESS Name of the executed file NLWP Number of LWPs in the process
6-4. Identifying Top Server-Consuming Resources (top)
Problem
You have a Linux server that hosts multiple databases. Users are reporting sluggishness with an application that uses one of the databases. You want to identify which processes are consuming the most resources on the server and then determine if the top consuming process is associated with a database.
Solution
The top command shows a real-time display of the highest resource-consuming processes on a server.
Here’s the simplest way to run top:
$ top
CHAPTER 6 ■ ANALYZING OPERATING SYSTEM PERFORMANCE
Listed next is a fragment of the output:
top - 04:40:05 up 353 days, 15:16, 3 users, load average: 2.84, 2.34, 2.45 Tasks: 454 total, 4 running, 450 sleeping, 0 stopped, 0 zombie
Cpu(s): 64.3%us, 3.4%sy, 0.0%ni, 20.6%id, 11.8%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 7645184k total, 6382956k used, 1262228k free, 176480k buffers Swap: 4128760k total, 184k used, 4128576k free, 3953512k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 19888 oracle 25 0 148m 13m 11m R 100.1 0.2 313371:45 oracle 19853 oracle 25 0 148m 13m 11m R 99.8 0.2 313375:41 oracle 9722 oracle 18 0 1095m 287m 150m R 58.6 3.8 0:41.89 oracle 445 root 11 -5 0 0 0 S 0.3 0.0 8:32.67 kjournald 9667 oracle 15 0 954m 55m 50m S 0.3 0.7 0:01.03 oracle 2 root RT -5 0 0 0 S 0.0 0.0 2:17.99 migration/0
Type q or press Ctrl+C to exit top. In the prior output, the first section of the output displays general system information such as how long the server has been running, number of users, CPU information, and so on. The second section shows which processes are consuming the most CPU resources (listed top to bottom). In the prior output, the process ID of 19888 is consuming a large amount of CPU. To
determine which database this process is associated with, use the ps command:
$ ps 19888
Here is the associated output:
PID TTY STAT TIME COMMAND
19888 ? Rs 313393:32 oracleO11R2 (DESCRIPTION=(LOCAL=YES)
In the prior output, the fourth column displays the value of oracleO11R2. This indicates that this is an Oracle process associated with the O11R2 database. If the process continues to consume resources, you can next determine if there is a SQL statement associated with the process (see Recipe 6-9) or terminate the process (see Recipe 6-10).
■ Tip
If you work in a Solaris operating system environment, use the
prstatcommand to view the top CPU-consuming processes (see Recipe 6-3 for details).
How It Works
If installed, the top utility is often the first investigative tool employed by DBAs and system administrators to identify resource-intensive processes on a server. If a process is continuously consuming excessive system resources, then you should further determine if the process is associated with a database and a specific SQL statement.
By default, top will repetitively refresh (every few seconds) information regarding the most CPU-intensive processes. While top is running, you can interactively change its output. For example, if you type >, this will move the column that top is sorting one position to the right. Table 6-3 lists the most useful hot key features to alter the top display to the desired format.
Table 6-3. Commands to Interactively Change the top Output Command Function
Spacebar Immediately refreshes the output
< or > Moves the sort column one to the left or to the right; by default, top sorts on the CPU column.
d Changes the refresh time R Reverses the sort order z Toggles the color output
h Displays help menu
F or O Chooses a sort column
Table 6-4 describes several of the columns displayed by top. Use these descriptions to help interpret the output.
Table 6-4. Column Descriptions of the top Output Column Description
PID Unique process identifier
USER OS username running the process PR Priority of the process
NI Nice value or process; negative value means high priority; positive value means low priority.
VIRT Total virtual memory used by process RES Non-swapped physical memory used SHR Shared memory used by process S Process status
%CPU Processes percent of CPU consumption since last screen refresh
%MEM Percent of physical memory the process is consuming
CHAPTER 6 ■ ANALYZING OPERATING SYSTEM PERFORMANCE
Column Description
TIME Total CPU time used by process
TIME+ Total CPU time, showing hundredths of seconds COMMAND Command line used to start a process