查看进程资源

参考《Linux® Debugging and Performance Tuning: Tips and Techniques》 chapter 6

1. The wchan option is the most interesting option for finding the location of the hang in this case. If this is a kernel address, ps uses /proc/kallsyms to find the nearest symbolic location.

2. The next ps command shows every process with the PID number, % of CPU, memory size, name, and what syscall the process is currently executing. The output is similar to this:

/home/a/j/nomad2:ps -eo pid,%cpu,vsz,args,wchan
  PID %CPU    VSZ COMMAND                     WCHAN
    1  0.0   4020 /sbin/init                  -
    2  0.0      0 [kthreadd]                  kthreadd
    3  0.0      0 [migration/0]               migration_thread
    4  0.0      0 [ksoftirqd/0]               ksoftirqd
    5  0.0      0 [watchdog/0]                watchdog

3. pgrep Lists the Process IDs That Match the Input Criteria

/home/a/j/nomad2:pgrep bash
1305
3494
3931
4704
5652

4. pstree Displays a Tree of Processes

/home/a/j/nomad2:pstree
init-+-atd
     |-console-kit-dae---61*[{console-kit-dae}]
     |-cron-+-2*[cron---sh---python]
     |      `-cron---sh
     |-5*[dbus-daemon]
     |-dd
     |-gdm---gdm---kill
     |-6*[getty]
     |-4*[gvfsd]
     |-4*[gvfsd-burn]
     |-4*[gvfsd-trash]
     |-hald---hald-runner-+-hald-addon-acpi
     |                    |-hald-addon-inpu
     |                    `-hald-addon-stor
     |-klogd
     |-loop.sh---sleep
     |-nscd---8*[{nscd}]
     |-portmap
     |-python
     |-rpc.statd
     |-sh---sh---wget
     |-sshd-+-13*[sshd---sshd---bash]
     |      |-sshd---sshd---bash---pstree
     |      |-14*[sshd---sshd]
     |      |-sshd---sshd---bash-+-man---pager
     |      |                    `-vim
     |      |-2*[sshd---sshd---sftp-server]
     |      `-sshd---sshd---bash---vi
     |-svnserve
     |-syslogd
     |-system-tools-ba
     |-tinyproxy---10*[tinyproxy]
     |-udevd
     |-vsftpd
     `-xinetd

5. strace can be a very effective way to debug an application if you think a system call is failing.

/home/a/j/nomad2:strace -o /tmp/1 ./a.out

6. back trace
A back trace can be an effective way to identify which process is hung on a system.

If your Linux system is hanging but your keyboard is still functioning, use the following method to help resolve the source of the hang. These steps perform a back trace of the current running process and all processes using the magic key sequence:

1) The kernel that is running on the system must be built with CONFIG_MAGIC_SYS-REQ enabled. The system must also be in text mode. Pressing Ctrl-Alt-F1 places the system in text mode. Pressing Ctrl-Alt-F7 places the system back in X Window.
2) While in text mode, press Alt-ScrollLock followed by Ctrl-ScrollLock. These magic keystrokes give you a stack trace of the currently running processes and all processes, respectively.
3) Look in the system's /var/log/messages file for the back trace. If everything is set up correctly, the system should have converted the symbolic kernel addresses.

7. lsof Lists Open Files

/home/a/j/nomad2:lsof -p 1305
COMMAND  PID   USER   FD   TYPE DEVICE    SIZE     NODE NAME
bash    1305 nomad2  cwd    DIR   0,21    4096 29609555 /home/a/j/nomad2/linux/ch6 (192.168.2.5:/store/home)
bash    1305 nomad2  rtd    DIR    8,1    4096        2 /
bash    1305 nomad2  txt    REG    8,1  813912 17694722 /bin/bash
bash    1305 nomad2  mem    REG    8,1 1436976 17825953 /lib/libc-2.7.so
bash    1305 nomad2  mem    REG    8,1   14624 17825970 /lib/libdl-2.7.so
bash    1305 nomad2  mem    REG    8,1  241408 17825802 /lib/libncurses.so.5.6
bash    1305 nomad2  mem    REG    8,1  127480 17825944 /lib/ld-2.7.so
bash    1305 nomad2  mem    REG    8,1  702576  8323249 /var/cache/nscd/passwd
bash    1305 nomad2    0u   CHR 136,19               21 /dev/pts/19
bash    1305 nomad2    1u   CHR 136,19               21 /dev/pts/19
bash    1305 nomad2    2u   CHR 136,19               21 /dev/pts/19
bash    1305 nomad2  255u   CHR 136,19               21 /dev/pts/19

8. the network debugging tools ifconfig, arp, tcpdump, ethereal, and netstat.

The option for netstat -tap, which is a good way to determine what programs are serving from your system. It can be used to look for rogue connections to your server.