Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 86487 - "ps" hangs in proc_info_read->alloc_inode
Summary: "ps" hangs in proc_info_read->alloc_inode
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: kernel
Version: 8.0
Hardware: athlon
OS: Linux
medium
high
Target Milestone: ---
Assignee: Arjan van de Ven
QA Contact: Brian Brock
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2003-03-24 04:05 UTC by Konstantin Olchanski
Modified: 2007-04-18 16:52 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2004-09-30 15:40:41 UTC


Attachments (Terms of Use)

Description Konstantin Olchanski 2003-03-24 04:05:36 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030301

Description of problem:
"ps", "top" and others hang on reads from /proc/PID/stat. The hung processes
cannot be killed and the system eventually becomes unusable and has to be reset.

This hung happens rarely, less than 1 per month per 20 systems. I have seen it
with kernels older than 2.4.18-19.8.0, never with vanilla 2.4.20 and (never?)
with redhat-7.x systems. I will be updating the currently hung system to
kernel-smp-2.4.18-27.8.0.athlon.rpm and will report if the hang reappears.

I am attaching the output of "strace ps -ef" and "alt-sysrq-t". Note that "ps"
is hung on a read() from "/proc/30731/stat" and the kernel stack trace shows it
waiting on a lock inside sys_read()->proc_info_read()->alloc_inide(). Please let
me know if you need any additional information from the hung systems.

K.O. olchansk@triumf.ca


Version-Release number of selected component (if applicable):
kernel-smp-2.4.18-19.8.0smp

How reproducible:
Sometimes

Steps to Reproduce:
1. blah...
2.
3.
    

Additional info:

Comment 1 Konstantin Olchanski 2003-03-24 04:06:36 UTC
Output from "strace ps -ef":

stat64("/proc/29709", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/29709/stat", O_RDONLY)      = 7
read(7, "29709 (onx.exe) S 13281 29709 13"..., 511) = 206
close(7)                                = 0
open("/proc/29709/statm", O_RDONLY)     = 7
read(7, "5414 133 69 0 0 133 64\n", 511) = 23
close(7)                                = 0
open("/proc/29709/status", O_RDONLY)    = 7
read(7, "Name:\tonx.exe\nState:\tS (sleeping"..., 511) = 466
close(7)                                = 0
open("/proc/29709/cmdline", O_RDONLY)   = 7
read(7, "/OSC/OnX/v12r0/Linux-i686/onx.ex"..., 2047) = 34
close(7)                                = 0
open("/proc/29709/environ", O_RDONLY)   = -1 EACCES (Permission denied)
stat64("/proc/29769", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/29769/stat", O_RDONLY)      = 7
read(7, "29769 (kdesktop_lock) S 13237 13"..., 511) = 202
close(7)                                = 0
open("/proc/29769/statm", O_RDONLY)     = 7
read(7, "660 240 240 0 0 240 0\n", 511) = 22
close(7)                                = 0
open("/proc/29769/status", O_RDONLY)    = 7
read(7, "Name:\tkdesktop_lock\nState:\tS (sl"..., 511) = 472
close(7)                                = 0
open("/proc/29769/cmdline", O_RDONLY)   = 7
read(7, "/usr/bin/kdesktop_lock\0--forcelo"..., 2047) = 35
close(7)                                = 0
open("/proc/29769/environ", O_RDONLY)   = -1 EACCES (Permission denied)
stat64("/proc/29770", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/29770/stat", O_RDONLY)      = 7
read(7, "29770 (kbanner.kss) S 29769 1318"..., 511) = 208
close(7)                                = 0
open("/proc/29770/statm", O_RDONLY)     = 7
read(7, "709 322 249 4 0 318 73\n", 511) = 23
close(7)                                = 0
open("/proc/29770/status", O_RDONLY)    = 7
read(7, "Name:\tkbanner.kss\nState:\tS (slee"..., 511) = 470
close(7)                                = 0
open("/proc/29770/cmdline", O_RDONLY)   = 7
read(7, "kbanner.kss\0-window-id\0004194308\0", 2047) = 31
close(7)                                = 0
open("/proc/29770/environ", O_RDONLY)   = -1 EACCES (Permission denied)
stat64("/proc/20402", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/20402/stat", O_RDONLY)      = 7
read(7, "20402 (mozilla-bin) S 10267 1318"..., 511) = 197
close(7)                                = 0
open("/proc/20402/statm", O_RDONLY)     = 7
read(7, "5790 4135 3120 6 0 4129 1935\n", 511) = 29
close(7)                                = 0
open("/proc/20402/status", O_RDONLY)    = 7
read(7, "Name:\tmozilla-bin\nState:\tS (slee"..., 511) = 471
close(7)                                = 0
open("/proc/20402/cmdline", O_RDONLY)   = 7
read(7, "/usr/lib/mozilla-1.0.1/mozilla-b"..., 2047) = 35
close(7)                                = 0
open("/proc/20402/environ", O_RDONLY)   = -1 EACCES (Permission denied)
stat64("/proc/30731", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
open("/proc/30731/stat", O_RDONLY)      = 7
read(7, 
 <unfinished ...>


Comment 2 Konstantin Olchanski 2003-03-24 04:07:20 UTC
Output from alt-sysrq-t:

Mar 23 19:22:35 dork kernel: SysRq : Show State
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel:                          free                     
  sibling
Mar 23 19:22:35 dork kernel:   task             PC    stack   pid father child
younger older
Mar 23 19:22:35 dork kernel: init          S F7FEAE00     0     1      0 18266 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xc36b7eec))
Mar 23 19:22:35 dork kernel: t [kernel] 0xad (0xf17afedc))
Mar 23 19:22:35 dork kernel: [<c01339cb>] handle_mm_fault [kernel] 0x10b
(0xf17afee0))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xf17aff14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xf17aff74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf17aff94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf17affc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: tcsh          S C68E1500     0 13443  13260       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xcccb1edc))
Mar 23 19:22:35 dork kernel: [<c01339cb>] handle_mm_fault [kernel] 0x10b
(0xcccb1ee0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xcccb1efc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xcccb1f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xcccb1f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xcccb1f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xcccb1fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: tcsh          S C705FD80     4 13444  13257       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xcb865edc))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xcb865ee0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xcb865efc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xcb865f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xcb865f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xcb865f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xcb865fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: tcsh          S C7D8F0C0     0 13445  13262       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xe9d71edc))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xe9d71ee0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xe9d71efc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xe9d71f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xe9d71f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xe9d71f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe9d71fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: tcsh          S F5B6E500     4 13453  13248       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xf0335edc))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xf0335ee0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xf0335efc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xf0335f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xf0335f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf0335f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf0335fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: tcsh          S F7764780     4 13457  13250       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xd9c39edc))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xd9c39ee0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xd9c39efc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xd9c39f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xd9c39f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd9c39f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd9c39fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: kdeinit       S E46913C0     4 14308  13186 14310 
 10258 13267 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xe9dcbeec))
Mar 23 19:22:35 dork kernel: [<c018bf55>] tty_poll [kernel] 0x85 (0xe9dcbf0c))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xe9dcbf24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xe9dcbf60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe9dcbfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: tcsh          S C03AC880     0 14310  14308       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c01905c9>] change_termios [kernel]
0x1a9 (0xda8b9ec8))
Mar 23 19:22:35 dork kernel: [<c012b0cd>] schedule_timeout [kernel] 0xad
(0xda8b9edc))
Mar 23 19:22:35 dork kernel: [<c01339cb>] handle_mm_fault [kernel] 0x10b
(0xda8b9ee0))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xda8b9f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xda8b9f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xda8b9f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xda8b9fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mozilla-bin   S C03AC880     0 10258  13186 10267 
       14308 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xe4b9bf0c))
Mar 23 19:22:35 dork kernel: [<c0155917>] pipe_poll [kernel] 0x37 (0xe4b9bf10))
Mar 23 19:22:35 dork kernel: [<c015d21b>] do_pollfd [kernel] 0x5b (0xe4b9bf28))
Mar 23 19:22:35 dork kernel: [<c015d326>] do_pollfd [kernel] 0x166 (0xe4b9bf44))
Mar 23 19:22:35 dork kernel: [<c015d4e3>] sys_poll [kernel] 0x163 (0xe4b9bf78))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe4b9bfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mozilla-bin   S C03AD240     0 10267  10258 15990 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xc51a9f0c))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xc51a9f2c))
Mar 23 19:22:35 dork kernel: [<c015d326>] do_pollfd [kernel] 0x166 (0xc51a9f44))
Mar 23 19:22:35 dork kernel: [<c015d4e3>] sys_poll [kernel] 0x163 (0xc51a9f78))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xc51a9fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mozilla-bin   S C03AD240  2384 10268  10267       
 10269       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xee2bdf0c))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xee2bdf2c))
Mar 23 19:22:35 dork kernel: [<c015d326>] do_pollfd [kernel] 0x166 (0xee2bdf44))
Mar 23 19:22:35 dork kernel: [<c015d4e3>] sys_poll [kernel] 0x163 (0xee2bdf78))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xee2bdfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mozilla-bin   S F5AD5C00  2384 10269  10267       
 10271 10268 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c01082db>] sys_rt_sigsuspend [kernel]
0xeb (0xeec67f98))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xeec67fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mozilla-bin   S F5AD5C00  2384 10271  10267       
 20402 10269 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xd8d83f64))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xd8d83f84))
Mar 23 19:22:35 dork kernel: [<c012b1b1>] sys_nanosleep [kernel] 0xd1 (0xd8d83f9c))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd8d83fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: onx.exe       S C03AD240     0 29709  13281       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xd4b07eec))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xd4b07f0c))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xd4b07f24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xd4b07f60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd4b07fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: kdesktop_lock S C03AC880     0 29769  13237 29770 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xccd4beec))
Mar 23 19:22:35 dork kernel: [<c024fa6e>] unix_poll [kernel] 0x2e (0xccd4bf00))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xccd4bf24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xccd4bf60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xccd4bfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: kbanner.kss   S C03AC880     0 29770  29769       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xd9b73eec))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xd9b73f0c))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xd9b73f24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xd9b73f60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd9b73fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mozilla-bin   S C03AC880    48 20402  10267       
 15990 10271 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c01082db>] sys_rt_sigsuspend [kernel]
0xeb (0xe7cb7f98))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe7cb7fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mofia         D F5DF1D80  3276 30731      1       
 31123 13275 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0136d99>] __lock_page [kernel] 0x99
(0xe5263958))
Mar 23 19:22:35 dork kernel: [<c0136dec>] lock_page [kernel] 0x1c (0xe5263980))
Mar 23 19:22:35 dork kernel: [<c013864e>] filemap_nopage [kernel] 0x11e
(0xe5263988))
Mar 23 19:22:35 dork kernel: [<c01336b0>] do_no_page [kernel] 0x80 (0xe52639b8))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xe52639f8))
Mar 23 19:22:35 dork kernel: [<c01324bd>] get_user_pages [kernel] 0xdd (0xe5263a24))
Mar 23 19:22:35 dork kernel: [<c016e7e0>] elf_core_dump [kernel] 0x950 (0xe5263a54))
Mar 23 19:22:35 dork kernel: [<c0154fb0>] do_coredump [kernel] 0x180 (0xe5263ed4))
Mar 23 19:22:35 dork kernel: [<c012bc14>] dequeue_signal [kernel] 0x64 (0xe5263efc))
Mar 23 19:22:35 dork kernel: [<c01091d1>] do_signal [kernel] 0x251 (0xe5263f14))
Mar 23 19:22:35 dork kernel: [<c0126b54>] bh_action [kernel] 0x54 (0xe5263f64))
Mar 23 19:22:35 dork kernel: [<c01269f7>] tasklet_hi_action [kernel] 0x67
(0xe5263f6c))
Mar 23 19:22:35 dork kernel: [<c012adbe>] update_process_times [kernel] 0x3e
(0xe5263f8c))
Mar 23 19:22:35 dork kernel: [<c011b6f0>] do_page_fault [kernel] 0x0 (0xe5263fb8))
Mar 23 19:22:35 dork kernel: [<c0109498>] signal_return [kernel] 0x14 (0xe5263fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mofia         D C8791680  3508 31123      1       
  5555 30731 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0136d99>] __lock_page [kernel] 0x99
(0xf033d958))
Mar 23 19:22:35 dork kernel: [<c0136dec>] lock_page [kernel] 0x1c (0xf033d980))
Mar 23 19:22:35 dork kernel: [<c013864e>] filemap_nopage [kernel] 0x11e
(0xf033d988))
Mar 23 19:22:35 dork kernel: [<c01336b0>] do_no_page [kernel] 0x80 (0xf033d9b8))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xf033d9f8))
Mar 23 19:22:35 dork kernel: [<c01324bd>] get_user_pages [kernel] 0xdd (0xf033da24))
Mar 23 19:22:35 dork kernel: [<c016e7e0>] elf_core_dump [kernel] 0x950 (0xf033da54))
Mar 23 19:22:35 dork kernel: [<c0154fb0>] do_coredump [kernel] 0x180 (0xf033ded4))
Mar 23 19:22:35 dork kernel: [<c012bc14>] dequeue_signal [kernel] 0x64 (0xf033defc))
Mar 23 19:22:35 dork kernel: [<c01091d1>] do_signal [kernel] 0x251 (0xf033df14))
Mar 23 19:22:35 dork kernel: [<c018ae1d>] tty_write [kernel] 0xfd (0xf033df74))
Mar 23 19:22:35 dork kernel: [<c018f420>] write_chan [kernel] 0x0 (0xf033df78))
Mar 23 19:22:35 dork kernel: [<c014a429>] sys_write [kernel] 0x109 (0xf033df94))
Mar 23 19:22:35 dork kernel: [<c011b6f0>] do_page_fault [kernel] 0x0 (0xf033dfb8))
Mar 23 19:22:35 dork kernel: [<c0109498>] signal_return [kernel] 0x14 (0xf033dfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AD240    16 31206   1011 31210 
 17566       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xd73c9e54))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xd73c9e60))
Mar 23 19:22:35 dork kernel: [<c024f35f>] unix_stream_data_wait [kernel] 0xcf
(0xd73c9e8c))
Mar 23 19:22:35 dork kernel: [<c024f7a1>] unix_stream_recvmsg [kernel] 0x3f1
(0xd73c9ec4))
Mar 23 19:22:35 dork kernel: [<c01f8548>] sock_recvmsg [kernel] 0x58 (0xd73c9f08))
Mar 23 19:22:35 dork kernel: [<c01490da>] dentry_open [kernel] 0x15a (0xd73c9f3c))
Mar 23 19:22:35 dork kernel: [<c01f8687>] sock_read [kernel] 0xa7 (0xd73c9f50))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd73c9f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd73c9fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S F5B6E200    16 31210  31206 31213 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xc67e7eec))
Mar 23 19:22:35 dork kernel: [<c021f9c6>] tcp_poll [kernel] 0x36 (0xc67e7ef4))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xc67e7f24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xc67e7f60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xc67e7fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: tcsh          S F5B6EBC0     0 31213  31210  5117 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xd9f0bedc))
Mar 23 19:22:35 dork kernel: [<c01339cb>] handle_mm_fault [kernel] 0x10b
(0xd9f0bee0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xd9f0befc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xd9f0bf14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xd9f0bf74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd9f0bf94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd9f0bfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: emacs         S C03AC880  2384   814  31213       
  5071       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xca221eec))
Mar 23 19:22:35 dork kernel: [<c021f9c6>] tcp_poll [kernel] 0x36 (0xca221ef4))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xca221f0c))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xca221f24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xca221f60))
Mar 23 19:22:35 dork kernel: [<c0108588>] restore_sigcontext [kernel] 0x128
(0xca221f7c))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xca221fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: emacs         S C03AC880  2384  5071  31213       
  5117   814 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xc8027eec))
Mar 23 19:22:35 dork kernel: [<c021f9c6>] tcp_poll [kernel] 0x36 (0xc8027ef4))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xc8027f0c))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xc8027f24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xc8027f60))
Mar 23 19:22:35 dork kernel: [<c0108588>] restore_sigcontext [kernel] 0x128
(0xc8027f7c))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xc8027fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: emacs         S C03AD240  2388  5117  31213       
        5071 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xdeb8feec))
Mar 23 19:22:35 dork kernel: [<c021f9c6>] tcp_poll [kernel] 0x36 (0xdeb8fef4))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xdeb8ff0c))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xdeb8ff24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xdeb8ff60))
Mar 23 19:22:35 dork kernel: [<c0108588>] restore_sigcontext [kernel] 0x128
(0xdeb8ff7c))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xdeb8ffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: top           D C03AC880     0  5504      1       
  5519  5655 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xc8f5de44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xc8f5de58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xc8f5de7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xc8f5de88))
Mar 23 19:22:35 dork kernel: [<c0163257>] new_inode [kernel] 0x17 (0xc8f5dea4))
Mar 23 19:22:35 dork kernel: [<c01702d8>] proc_pid_make_inode [kernel] 0x88
(0xc8f5deac))
Mar 23 19:22:35 dork kernel: [<c0170621>] proc_base_lookup [kernel] 0x111
(0xc8f5dec0))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xc8f5def8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xc8f5df0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xc8f5df3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xc8f5df6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xc8f5df94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xc8f5dfa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xc8f5dfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AC880     4  5519      1       
  9965  5504 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xe883de44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xe883de58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xe883de7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xe883de88))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xe883deb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xe883decc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xe883def8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xe883df0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xe883df3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xe883df6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xe883df94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xe883dfa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe883dfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: top           D C03AC880     0  5555      1       
  5598 31123 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xce17fe44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xce17fe58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xce17fe7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xce17fe88))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xce17feb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xce17fecc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xce17fef8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xce17ff0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xce17ff3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xce17ff6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xce17ff94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xce17ffa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xce17ffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AD240     4  5598      1       
  5655  5555 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xc656de44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xc656de58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xc656de7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xc656de88))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xc656deb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xc656decc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xc656def8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xc656df0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xc656df3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xc656df6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xc656df94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xc656dfa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xc656dfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AC880     0  5655      1       
  5504  5598 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xf7bafe44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xf7bafe58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xf7bafe7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xf7bafe88))
Mar 23 19:22:35 dork kernel: [<c012c705>] wake_up_parent [kernel] 0x25 (0xf7bafefc))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xf7baff3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xf7baff6c))
Mar 23 19:22:35 dork kernel: [<c011d0c6>] schedule [kernel] 0x186 (0xf7baff7c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf7baff94))
Mar 23 19:22:35 dork kernel: [<c010dc31>] syscall_trace [kernel] 0x51 (0xf7baffac))
Mar 23 19:22:35 dork kernel: [<c01094cb>] tracesys [kernel] 0x1f (0xf7baffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AC880  2384  9965      1       
 17515  5519 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xcc323e44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xcc323e58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xcc323e7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xcc323e88))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xcc323eb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xcc323ecc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xcc323ef8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xcc323f0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xcc323f3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xcc323f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xcc323f94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xcc323fa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xcc323fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: crond         S F5DF1480    16 13845   1129 13849 
  9899       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xf0fcdf38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xf0fcdf6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf0fcdf94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf0fcdfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: triumf-up2dat S F5DF1780     0 13849  13845 13864 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0124f37>] sys_wait4 [kernel] 0x117
(0xd63fbf80))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd63fbfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash2         S C36CC500     0 13863  13849 13869 
 13864       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xf75bdf38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xf75bdf6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf75bdf94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf75bdfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: triumf-up2dat S F5DF1180     0 13864  13849 13867 
       13863 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xf0423f38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xf0423f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf0423f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf0423fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: cat           S F5B6E080     0 13867  13864       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xda7f9f38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xda7f9f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xda7f9f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xda7f9fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash2         S F5B6E2C0     0 13869  13863 13873 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0124f37>] sys_wait4 [kernel] 0x117
(0xd6a87f80))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd6a87fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AD240     0 13870  13869       
 13871       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xd47c1e44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xd47c1e58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xd47c1e7c))
Mar 23 19:22:35 dork kernel: [<c0141273>] __alloc_pages [kernel] 0x93 (0xd47c1e98))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xd47c1eb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xd47c1ecc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xd47c1ef8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xd47c1f0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xd47c1f3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xd47c1f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd47c1f94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xd47c1fa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd47c1fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: grep          S CA1E33C0     0 13871  13869       
 13872 13870 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xcbebff38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xcbebff6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xcbebff94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xcbebffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: wc            S C03AD240    64 13872  13869       
 13873 13871 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xd250df38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xd250df6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd250df94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd250dfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: cut           S C03AD240     0 13873  13869       
       13872 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xdd15df38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xdd15df6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xdd15df94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xdd15dfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: crond         S D8B8B200     0  9899   1129  9903 
       13845 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xdbd3ff38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xdbd3ff6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xdbd3ff94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xdbd3ffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: triumf-up2dat S D8B8BB00     0  9903   9899  9917 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0124f37>] sys_wait4 [kernel] 0x117
(0xd020ff80))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd020ffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash2         S D8B8B980     4  9915   9903  9922 
  9917       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xd0595f38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xd0595f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd0595f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd0595fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: triumf-up2dat S DB6E5480     0  9917   9903  9920 
        9915 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xe463df38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xe463df6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xe463df94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe463dfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: cat           S CA1E39C0     0  9920   9917       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xd6f33f38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xd6f33f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd6f33f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd6f33fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash2         S D8B8BC80    64  9922   9915  9926 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0124f37>] sys_wait4 [kernel] 0x117
(0xd5ebff80))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd5ebffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AC880    24  9923   9922       
  9924       (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xdc973e44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xdc973e58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xdc973e7c))
Mar 23 19:22:35 dork kernel: [<c0141273>] __alloc_pages [kernel] 0x93 (0xdc973e98))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xdc973eb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xdc973ecc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xdc973ef8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xdc973f0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xdc973f3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xdc973f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xdc973f94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xdc973fa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xdc973fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: grep          S C36CCC80    64  9924   9922       
  9925  9923 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xd8ab3f38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xd8ab3f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd8ab3f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd8ab3fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: wc            S C36CC200    64  9925   9922       
  9926  9924 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xe6703f38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xe6703f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xe6703f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe6703fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: cut           S C03AD240     4  9926   9922       
        9925 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0155358>] pipe_wait [kernel] 0x88
(0xe6121f38))
Mar 23 19:22:35 dork kernel: [<c0155450>] pipe_read [kernel] 0xc0 (0xe6121f6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xe6121f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe6121fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mozilla-bin   S C03AD240     0 15990  10267       
       20402 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xdf155f64))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xdf155f84))
Mar 23 19:22:35 dork kernel: [<c012b1b1>] sys_nanosleep [kernel] 0xd1 (0xdf155f9c))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xdf155fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: top           D C03AC880     4 17515      1       
 17565  9965 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xcd3dfe44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xcd3dfe58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xcd3dfe7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xcd3dfe88))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xcd3dfeb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xcd3dfecc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xcd3dfef8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xcd3dff0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xcd3dff3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xcd3dff6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xcd3dff94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xcd3dffa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xcd3dffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AC880     0 17565      1       
 18266 17515 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xdb22fe44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xdb22fe58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xdb22fe7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xdb22fe88))
Mar 23 19:22:35 dork kernel: [<c012c705>] wake_up_parent [kernel] 0x25 (0xdb22fefc))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xdb22ff3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xdb22ff6c))
Mar 23 19:22:35 dork kernel: [<c011d0c6>] schedule [kernel] 0x186 (0xdb22ff7c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xdb22ff94))
Mar 23 19:22:35 dork kernel: [<c010dc31>] syscall_trace [kernel] 0x51 (0xdb22ffac))
Mar 23 19:22:35 dork kernel: [<c01094cb>] tracesys [kernel] 0x1f (0xdb22ffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AC880    28 17566   1011 17568 
 18007 31206 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xd845fe54))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xd845fe60))
Mar 23 19:22:35 dork kernel: [<c024f35f>] unix_stream_data_wait [kernel] 0xcf
(0xd845fe8c))
Mar 23 19:22:35 dork kernel: [<c024f7a1>] unix_stream_recvmsg [kernel] 0x3f1
(0xd845fec4))
Mar 23 19:22:35 dork kernel: [<c01f8548>] sock_recvmsg [kernel] 0x58 (0xd845ff08))
Mar 23 19:22:35 dork kernel: [<c01490da>] dentry_open [kernel] 0x15a (0xd845ff3c))
Mar 23 19:22:35 dork kernel: [<c01f8687>] sock_read [kernel] 0xa7 (0xd845ff50))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd845ff94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd845ffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AC880     0 17568  17566 17570 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xdd1cdeec))
Mar 23 19:22:35 dork kernel: [<c024fa6e>] unix_poll [kernel] 0x2e (0xdd1cdf00))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xdd1cdf24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xdd1cdf60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xdd1cdfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash          S C03AD240    24 17570  17568       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c01905c9>] change_termios [kernel]
0x1a9 (0xe0ecbec8))
Mar 23 19:22:35 dork kernel: [<c012b0cd>] schedule_timeout [kernel] 0xad
(0xe0ecbedc))
Mar 23 19:22:35 dork kernel: [<c01339cb>] handle_mm_fault [kernel] 0x10b
(0xe0ecbee0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xe0ecbefc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xe0ecbf14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xe0ecbf74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xe0ecbf94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe0ecbfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AD240    64 18007   1011 18011 
 18109 17566 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xed583e54))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xed583e60))
Mar 23 19:22:35 dork kernel: [<c024f35f>] unix_stream_data_wait [kernel] 0xcf
(0xed583e8c))
Mar 23 19:22:35 dork kernel: [<c024f7a1>] unix_stream_recvmsg [kernel] 0x3f1
(0xed583ec4))
Mar 23 19:22:35 dork kernel: [<c01f8548>] sock_recvmsg [kernel] 0x58 (0xed583f08))
Mar 23 19:22:35 dork kernel: [<c01490da>] dentry_open [kernel] 0x15a (0xed583f3c))
Mar 23 19:22:35 dork kernel: [<c01f8687>] sock_read [kernel] 0xa7 (0xed583f50))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xed583f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xed583fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AC880     0 18011  18007 18015 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xf32ffeec))
Mar 23 19:22:35 dork kernel: [<c021f9c6>] tcp_poll [kernel] 0x36 (0xf32ffef4))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xf32fff24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xf32fff60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf32fffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash          S C03AD240     0 18015  18011 18067 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c01269f7>] tasklet_hi_action [kernel]
0x67 (0xe89a1f6c))
Mar 23 19:22:35 dork kernel: [<c0124f37>] sys_wait4 [kernel] 0x117 (0xe89a1f80))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xe89a1fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D C03AC880     0 18067  18015       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xf5aefe44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xf5aefe58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xf5aefe7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xf5aefe88))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xf5aefeb8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xf5aefecc))
Mar 23 19:22:35 dork kernel: [<c0160af0>] dput [kernel] 0x30 (0xf5aefef8))
Mar 23 19:22:35 dork kernel: [<c0156de6>] link_path_walk [kernel] 0x656
(0xf5aeff0c))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xf5aeff3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xf5aeff6c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf5aeff94))
Mar 23 19:22:35 dork kernel: [<c01495c2>] sys_open [kernel] 0xa2 (0xf5aeffa8))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf5aeffc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AD240    64 18109   1011 18114 
 18177 18007 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xd0109e54))
Mar 23 19:22:35 dork kernel: [<c0133947>] handle_mm_fault [kernel] 0x87
(0xd0109e60))
Mar 23 19:22:35 dork kernel: [<c024f35f>] unix_stream_data_wait [kernel] 0xcf
(0xd0109e8c))
Mar 23 19:22:35 dork kernel: [<c024f7a1>] unix_stream_recvmsg [kernel] 0x3f1
(0xd0109ec4))
Mar 23 19:22:35 dork kernel: [<c01f8548>] sock_recvmsg [kernel] 0x58 (0xd0109f08))
Mar 23 19:22:35 dork kernel: [<c01490da>] dentry_open [kernel] 0x15a (0xd0109f3c))
Mar 23 19:22:35 dork kernel: [<c01f8687>] sock_read [kernel] 0xa7 (0xd0109f50))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd0109f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd0109fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AD240     0 18114  18109 18119 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xcc321eec))
Mar 23 19:22:35 dork kernel: [<c021f9c6>] tcp_poll [kernel] 0x36 (0xcc321ef4))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xcc321f24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xcc321f60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xcc321fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash          S F7764B40    64 18119  18114 18171 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0124f37>] sys_wait4 [kernel] 0x117
(0xf1f31f80))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf1f31fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: strace        S F7764480    64 18171  18119 18172 
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c0124f37>] sys_wait4 [kernel] 0x117
(0xf17cbf80))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf17cbfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: ps            D 66656463     0 18172  18171       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c025ecdc>] rwsem_down_failed_common
[kernel] 0x5c (0xd8d0be44))
Mar 23 19:22:35 dork kernel: [<c025eb39>] rwsem_down_read_failed [kernel] 0x29
(0xd8d0be58))
Mar 23 19:22:35 dork kernel: [<c0172deb>] .text.lock.array [kernel] 0x51
(0xd8d0be7c))
Mar 23 19:22:35 dork kernel: [<c01623bc>] alloc_inode [kernel] 0xbc (0xd8d0be88))
Mar 23 19:22:35 dork kernel: [<c012c705>] wake_up_parent [kernel] 0x25 (0xd8d0befc))
Mar 23 19:22:35 dork kernel: [<c014901e>] dentry_open [kernel] 0x9e (0xd8d0bf3c))
Mar 23 19:22:35 dork kernel: [<c016f947>] proc_info_read [kernel] 0x77 (0xd8d0bf6c))
Mar 23 19:22:35 dork kernel: [<c011d0c6>] schedule [kernel] 0x186 (0xd8d0bf7c))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xd8d0bf94))
Mar 23 19:22:35 dork kernel: [<c010dc31>] syscall_trace [kernel] 0x51 (0xd8d0bfac))
Mar 23 19:22:35 dork kernel: [<c01094cb>] tracesys [kernel] 0x1f (0xd8d0bfc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: sshd          S C03AC880    64 18177   1011 18189 
       18109 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xd8d01eec))
Mar 23 19:22:35 dork kernel: [<c021f9c6>] tcp_poll [kernel] 0x36 (0xd8d01ef4))
Mar 23 19:22:35 dork kernel: [<c015cb9d>] do_select [kernel] 0x12d (0xd8d01f24))
Mar 23 19:22:35 dork kernel: [<c015d03e>] sys_select [kernel] 0x34e (0xd8d01f60))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xd8d01fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: bash          S D44450C0    28 18189  18177       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b0cd>] schedule_timeout [kernel]
0xad (0xf03d5edc))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xf03d5efc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xf03d5f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xf03d5f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf03d5f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf03d5fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: mingetty      S 00004C0F     0 18266      1       
       17565 (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c01e6a99>] vgacon_cursor [kernel]
0xf9 (0xf40f7ec4))
Mar 23 19:22:35 dork kernel: [<c012b0cd>] schedule_timeout [kernel] 0xad
(0xf40f7edc))
Mar 23 19:22:35 dork kernel: [<c019905b>] set_cursor [kernel] 0x7b (0xf40f7ee0))
Mar 23 19:22:35 dork kernel: [<c019cdb0>] con_flush_chars [kernel] 0x60
(0xf40f7ef0))
Mar 23 19:22:35 dork kernel: [<c018f582>] write_chan [kernel] 0x162 (0xf40f7efc))
Mar 23 19:22:35 dork kernel: [<c018f048>] read_chan [kernel] 0x298 (0xf40f7f14))
Mar 23 19:22:35 dork kernel: [<c018acf3>] tty_read [kernel] 0x113 (0xf40f7f74))
Mar 23 19:22:35 dork kernel: [<c014a263>] sys_read [kernel] 0xa3 (0xf40f7f94))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xf40f7fc0))
Mar 23 19:22:35 dork kernel: 
Mar 23 19:22:35 dork kernel: portmap       S C03AD240     0 18288    729       
             (NOTLB)
Mar 23 19:22:35 dork kernel: Call Trace: [<c012b07d>] schedule_timeout [kernel]
0x5d (0xdf641f0c))
Mar 23 19:22:35 dork kernel: [<c012b010>] process_timeout [kernel] 0x0 (0xdf641f2c))
Mar 23 19:22:35 dork kernel: [<c015d326>] do_pollfd [kernel] 0x166 (0xdf641f44))
Mar 23 19:22:35 dork kernel: [<c015d4e3>] sys_poll [kernel] 0x163 (0xdf641f78))
Mar 23 19:22:35 dork kernel: [<c010945f>] system_call [kernel] 0x33 (0xdf641fc0))


Comment 3 Konstantin Olchanski 2003-04-11 04:19:00 UTC
Okey, I see the same problem on a redhat 7.3 system with the 2.4.18-24.7.xsmp
kernel. The symptoms are slightly different- userland still hangs on reads from
/proc/PID/stat, but the alt-sysqr-t output now show them hanging in
sys_read->proc_info_read->__alloc_pages->.text.lock.array: (full trace available
on request)

Apr 10 21:12:20 lin06 kernel: top           D CAB10680     4 13219  13218      
  13220       (NOTLB)
Apr 10 21:12:20 lin06 kernel: Call Trace: [<c014dbe0>] cached_lookup [kernel]
0x10 (0xdfd43ec8))
Apr 10 21:12:20 lin06 kernel: [<c023b4c5>] rwsem_down_read_failed [kernel] 0x135
(0xdfd43ee4))
Apr 10 21:12:20 lin06 kernel: [<c0167197>] .text.lock.array [kernel] 0x73
(0xdfd43f10))
Apr 10 21:12:20 lin06 kernel: [<c013b39b>] __alloc_pages [kernel] 0x6b (0xdfd43f3c))
Apr 10 21:12:20 lin06 kernel: [<c016443c>] proc_info_read [kernel] 0x4c
(0xdfd43f58))
Apr 10 21:12:20 lin06 kernel: [<c01434a6>] sys_read [kernel] 0x96 (0xdfd43f7c))
Apr 10 21:12:20 lin06 kernel: [<c0142e37>] sys_open [kernel] 0x57 (0xdfd43fac))
Apr 10 21:12:20 lin06 kernel: [<c0108ca3>] system_call [kernel] 0x33 (0xdfd43fc0))

K.O.


Comment 4 Bugzilla owner 2004-09-30 15:40:41 UTC
Thanks for the bug report. However, Red Hat no longer maintains this version of
the product. Please upgrade to the latest version and open a new bug if the problem
persists.

The Fedora Legacy project (http://fedoralegacy.org/) maintains some older releases, 
and if you believe this bug is interesting to them, please report the problem in
the bug tracker at: http://bugzilla.fedora.us/



Note You need to log in before you can comment on or make changes to this bug.