You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Doron Fediuck <do...@redhat.com> on 2010/09/01 18:01:55 UTC
High CPU utilization of ClientInputStreamPump
Hi,
I have an implementation of SSH client using Mina SSHD.
I've noticed that working with it on several hosts will take the CPU to 100% for every SSH
client session I have. For me this is a severe state, so I ran some debug / profiling stuff.
First I used jstack to see what's running.
I saw-
"ClientInputStreamPump" daemon prio=6 tid=0x34db8400 nid=0xec8 runnable [0x478df000]
java.lang.Thread.State: RUNNABLE
at org.apache.sshd.client.channel.ChannelSession.securedRead(ChannelSession.java:107)
at org.apache.sshd.client.channel.ChannelSession.pumpInputStream(ChannelSession.java:81)
at org.apache.sshd.client.channel.ChannelSession$1.run(ChannelSession.java:53)
Locked ownable synchronizers:
- None
The, I wrote an application that connects to the other VM and list the thread utilization...
Here's what I got-
T ID: 2637 Name: JMX server connection timeout 2637
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 2636 Name: RMI TCP Connection(310)-10.35.17.171
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 2611 Name: RMI TCP Accept-0
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 2528 Name: WorkerThread#0[127.0.0.1:2566]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 2519 Name: ClientInputStreamPump
endCpuTime : 17624 sec's
endUserTime : 17619 sec's
T ID: 1101 Name: http-0.0.0.0-8080-27
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1100 Name: http-0.0.0.0-8080-26
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1099 Name: http-0.0.0.0-8080-25
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1095 Name: http-0.0.0.0-8080-24
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1093 Name: http-0.0.0.0-8080-23
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1092 Name: http-0.0.0.0-8080-22
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1091 Name: http-0.0.0.0-8080-21
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1090 Name: http-0.0.0.0-8080-20
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1089 Name: http-0.0.0.0-8080-19
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1088 Name: http-0.0.0.0-8080-18
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1087 Name: http-0.0.0.0-8080-17
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1086 Name: http-0.0.0.0-8080-16
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1085 Name: http-0.0.0.0-8080-15
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1084 Name: http-0.0.0.0-8080-14
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1083 Name: http-0.0.0.0-8080-13
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1082 Name: http-0.0.0.0-8080-12
endCpuTime : 114 sec's
endUserTime : 103 sec's
T ID: 1081 Name: http-0.0.0.0-8080-11
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1080 Name: http-0.0.0.0-8080-10
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1079 Name: http-0.0.0.0-8080-9
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1078 Name: http-0.0.0.0-8080-8
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1077 Name: http-0.0.0.0-8080-7
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1076 Name: http-0.0.0.0-8080-6
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1075 Name: http-0.0.0.0-8080-5
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1074 Name: http-0.0.0.0-8080-4
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 1073 Name: pool-22-thread-6
endCpuTime : 544 sec's
endUserTime : 534 sec's
T ID: 274 Name: http-0.0.0.0-8080-3
endCpuTime : 13 sec's
endUserTime : 13 sec's
T ID: 201 Name: MultiThreadedHttpConnectionManager cleanup
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 181 Name: http-0.0.0.0-8080-2
endCpuTime : 497 sec's
endUserTime : 444 sec's
T ID: 173 Name: http-0.0.0.0-8080-1
endCpuTime : 67 sec's
endUserTime : 60 sec's
T ID: 172 Name: ajp-0.0.0.0-8009-Acceptor-0
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 171 Name: http-0.0.0.0-8080-Acceptor-0
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 170 Name: JBossLifeThread
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 161 Name: IdleRemover
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 154 Name: Thread-17
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 153 Name: JCA PoolFiller
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 152 Name: HSQLDB Timer @1dc8b22
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 151 Name: QuartzScheduler_QuartzSchedulerThread
endCpuTime : 23 sec's
endUserTime : 22 sec's
T ID: 150 Name: QuartzScheduler_Worker-99
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 149 Name: QuartzScheduler_Worker-98
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 148 Name: QuartzScheduler_Worker-97
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 147 Name: QuartzScheduler_Worker-96
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 146 Name: QuartzScheduler_Worker-95
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 145 Name: QuartzScheduler_Worker-94
endCpuTime : 11 sec's
endUserTime : 10 sec's
T ID: 144 Name: QuartzScheduler_Worker-93
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 143 Name: QuartzScheduler_Worker-92
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 142 Name: QuartzScheduler_Worker-91
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 141 Name: QuartzScheduler_Worker-90
endCpuTime : 13 sec's
endUserTime : 10 sec's
T ID: 140 Name: QuartzScheduler_Worker-89
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 139 Name: QuartzScheduler_Worker-88
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 138 Name: QuartzScheduler_Worker-87
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 137 Name: QuartzScheduler_Worker-86
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 136 Name: QuartzScheduler_Worker-85
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 135 Name: QuartzScheduler_Worker-84
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 134 Name: QuartzScheduler_Worker-83
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 133 Name: QuartzScheduler_Worker-82
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 132 Name: QuartzScheduler_Worker-81
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 131 Name: QuartzScheduler_Worker-80
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 130 Name: QuartzScheduler_Worker-79
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 129 Name: QuartzScheduler_Worker-78
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 128 Name: QuartzScheduler_Worker-77
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 127 Name: QuartzScheduler_Worker-76
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 126 Name: QuartzScheduler_Worker-75
endCpuTime : 11 sec's
endUserTime : 10 sec's
T ID: 125 Name: QuartzScheduler_Worker-74
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 124 Name: QuartzScheduler_Worker-73
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 123 Name: QuartzScheduler_Worker-72
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 122 Name: QuartzScheduler_Worker-71
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 121 Name: QuartzScheduler_Worker-70
endCpuTime : 11 sec's
endUserTime : 9 sec's
T ID: 120 Name: QuartzScheduler_Worker-69
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 119 Name: QuartzScheduler_Worker-68
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 118 Name: QuartzScheduler_Worker-67
endCpuTime : 11 sec's
endUserTime : 9 sec's
T ID: 117 Name: QuartzScheduler_Worker-66
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 116 Name: QuartzScheduler_Worker-65
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 115 Name: QuartzScheduler_Worker-64
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 114 Name: QuartzScheduler_Worker-63
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 113 Name: QuartzScheduler_Worker-62
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 112 Name: QuartzScheduler_Worker-61
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 111 Name: QuartzScheduler_Worker-60
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 110 Name: QuartzScheduler_Worker-59
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 109 Name: QuartzScheduler_Worker-58
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 108 Name: QuartzScheduler_Worker-57
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 107 Name: QuartzScheduler_Worker-56
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 106 Name: QuartzScheduler_Worker-55
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 105 Name: QuartzScheduler_Worker-54
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 104 Name: QuartzScheduler_Worker-53
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 103 Name: QuartzScheduler_Worker-52
endCpuTime : 11 sec's
endUserTime : 10 sec's
T ID: 102 Name: QuartzScheduler_Worker-51
endCpuTime : 11 sec's
endUserTime : 10 sec's
T ID: 101 Name: QuartzScheduler_Worker-50
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 100 Name: QuartzScheduler_Worker-49
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 99 Name: QuartzScheduler_Worker-48
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 98 Name: QuartzScheduler_Worker-47
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 97 Name: QuartzScheduler_Worker-46
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 96 Name: QuartzScheduler_Worker-45
endCpuTime : 13 sec's
endUserTime : 10 sec's
T ID: 95 Name: QuartzScheduler_Worker-44
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 94 Name: QuartzScheduler_Worker-43
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 93 Name: QuartzScheduler_Worker-42
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 92 Name: QuartzScheduler_Worker-41
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 91 Name: QuartzScheduler_Worker-40
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 90 Name: QuartzScheduler_Worker-39
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 89 Name: QuartzScheduler_Worker-38
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 88 Name: QuartzScheduler_Worker-37
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 87 Name: QuartzScheduler_Worker-36
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 86 Name: QuartzScheduler_Worker-35
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 85 Name: QuartzScheduler_Worker-34
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 84 Name: QuartzScheduler_Worker-33
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 83 Name: QuartzScheduler_Worker-32
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 82 Name: QuartzScheduler_Worker-31
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 81 Name: QuartzScheduler_Worker-30
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 80 Name: QuartzScheduler_Worker-29
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 79 Name: QuartzScheduler_Worker-28
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 78 Name: QuartzScheduler_Worker-27
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 77 Name: QuartzScheduler_Worker-26
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 76 Name: QuartzScheduler_Worker-25
endCpuTime : 11 sec's
endUserTime : 9 sec's
T ID: 75 Name: QuartzScheduler_Worker-24
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 74 Name: QuartzScheduler_Worker-23
endCpuTime : 12 sec's
endUserTime : 11 sec's
T ID: 73 Name: QuartzScheduler_Worker-22
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 72 Name: QuartzScheduler_Worker-21
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 71 Name: QuartzScheduler_Worker-20
endCpuTime : 11 sec's
endUserTime : 9 sec's
T ID: 70 Name: QuartzScheduler_Worker-19
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 69 Name: QuartzScheduler_Worker-18
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 68 Name: QuartzScheduler_Worker-17
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 67 Name: QuartzScheduler_Worker-16
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 66 Name: QuartzScheduler_Worker-15
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 65 Name: QuartzScheduler_Worker-14
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 64 Name: QuartzScheduler_Worker-13
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 63 Name: QuartzScheduler_Worker-12
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 62 Name: QuartzScheduler_Worker-11
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 61 Name: QuartzScheduler_Worker-10
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 60 Name: QuartzScheduler_Worker-9
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 59 Name: QuartzScheduler_Worker-8
endCpuTime : 11 sec's
endUserTime : 10 sec's
T ID: 58 Name: QuartzScheduler_Worker-7
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 57 Name: QuartzScheduler_Worker-6
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 56 Name: QuartzScheduler_Worker-5
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 55 Name: QuartzScheduler_Worker-4
endCpuTime : 12 sec's
endUserTime : 10 sec's
T ID: 54 Name: QuartzScheduler_Worker-3
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 53 Name: QuartzScheduler_Worker-2
endCpuTime : 13 sec's
endUserTime : 11 sec's
T ID: 52 Name: QuartzScheduler_Worker-1
endCpuTime : 11 sec's
endUserTime : 9 sec's
T ID: 51 Name: QuartzScheduler_Worker-0
endCpuTime : 14 sec's
endUserTime : 11 sec's
T ID: 50 Name: WorkManager(2)-1
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 49 Name: AOPListner
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 48 Name: ContainerBackgroundProcessor[StandardEngine[jboss.web]]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 47 Name: RMI TCP Accept-4444
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 46 Name: PooledInvokerAcceptor#0-4445
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 43 Name: Thread-14
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 44 Name: Listener:4712
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 42 Name: Thread-13
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 41 Name: Thread-12
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 40 Name: Thread-11
endCpuTime : 5 sec's
endUserTime : 5 sec's
T ID: 38 Name: Listener:4713
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 37 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=4446]]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 36 Name: ServerSocketRefresh
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 35 Name: HDScanner
endCpuTime : 319 sec's
endUserTime : 136 sec's
T ID: 34 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=3873]]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 33 Name: ServerSocketRefresh
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 32 Name: secondaryServerSocketThread[0]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 31 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=4457]]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 30 Name: ServerSocketRefresh
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 28 Name: RMI RenewClean-[qumranetyk:1175]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 27 Name: RMI Scheduler(0)
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 25 Name: RMI TCP Accept-0
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 24 Name: RMI TCP Accept-1090
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 23 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=5400]]
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 22 Name: ServerSocketRefresh
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 21 Name: JBoss System Threads(1)-2
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 20 Name: GC Daemon
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 19 Name: RMI Reaper
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 18 Name: RMI TCP Accept-1098
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 17 Name: JBoss System Threads(1)-1
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 16 Name: Timer-Log4jService
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 15 Name: ZipFile Lock Reaper
endCpuTime : 1 sec's
endUserTime : 1 sec's
T ID: 14 Name: Timer-0
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 11 Name: DestroyJavaVM
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 5 Name: Attach Listener
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 4 Name: Signal Dispatcher
endCpuTime : 0 sec's
endUserTime : 0 sec's
T ID: 3 Name: Finalizer
endCpuTime : 8 sec's
endUserTime : 7 sec's
T ID: 2 Name: Reference Handler
endCpuTime : 2 sec's
endUserTime : 1 sec's
Just in case you missed it-
T ID: 2519 Name: ClientInputStreamPump
endCpuTime : 17624 sec's
endUserTime : 17619 sec's
It looks like ClientInputStreamPump is pumping my CPU :(
Any idea how to resolve it ?
--
/d
Re: High CPU utilization of ClientInputStreamPump
Posted by Emmanuel Lécharny <el...@apache.org>.
On 9/14/10 9:36 AM, Doron Fediuck wrote:
> On Monday 13 September 2010 09:50:27 Emmanuel Lecharny wrote:
>>> Anyone ?
>>> Can someone please take a look on this issue ?
>> It just seems that you are heavily using the server, sending a hell lot
>> of data to it. Do you have any releavnt informations about the load on
>> your server ?
>>
>> If you look at the code pumping out CPU, you'll see that there is no
>> black magic in it :
>>
>> //
>> // On some platforms, a call to System.in.read(new byte[65536],
>> 0,32768) always throws an IOException.
>> // So we need to protect against that and chunk the call into
>> smaller calls.
>> // This problem was found on Windows, JDK 1.6.0_03-b05.
>> //
>> protected int securedRead(InputStream in, byte[] buf, int off, int
>> len) throws IOException {
>> int n = 0;
>> for (;;) {
>> int nread = in.read(buf, off + n, Math.min(1024, len - n));
>> if (nread<= 0) {
>> return (n == 0) ? nread : n;
>> }
>> n += nread;
>> if (n>= len) {
>> return n;
>> }
>> // if not closed but no bytes available, return
>> if (in != null&& in.available()<= 0) {
>> return n;
>> }
>> }
>> }
>>
>>
>>
>>
> Just to clarify, I'm not using the SSHD server side, just client side,
> so I'm not sure your remark about heavy server usage it relevant.
>
> As for the code, it looks harmless, but I do not think
> networking or I/O should take CPU to 100%. We're missing
> something and that's why I posted it here. It could be something
> in the JVM or similar, but the issue is severe.
There is another potential issue due to the underlying MINA layer used
by sshd : there is a bug in the way the select() method is processed by
the JVM in some corner cases, which drive the JVM to eat 100% CPU. What
would help here is a thread dump when your CPU hits 100%
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: High CPU utilization of ClientInputStreamPump
Posted by Emmanuel Lecharny <el...@gmail.com>.
> Anyone ?
> Can someone please take a look on this issue ?
It just seems that you are heavily using the server, sending a hell lot
of data to it. Do you have any releavnt informations about the load on
your server ?
If you look at the code pumping out CPU, you'll see that there is no
black magic in it :
//
// On some platforms, a call to System.in.read(new byte[65536],
0,32768) always throws an IOException.
// So we need to protect against that and chunk the call into
smaller calls.
// This problem was found on Windows, JDK 1.6.0_03-b05.
//
protected int securedRead(InputStream in, byte[] buf, int off, int
len) throws IOException {
int n = 0;
for (;;) {
int nread = in.read(buf, off + n, Math.min(1024, len - n));
if (nread <= 0) {
return (n == 0) ? nread : n;
}
n += nread;
if (n >= len) {
return n;
}
// if not closed but no bytes available, return
if (in != null && in.available() <= 0) {
return n;
}
}
}
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: High CPU utilization of ClientInputStreamPump
Posted by Doron Fediuck <do...@redhat.com>.
Anyone ?
Can someone please take a look on this issue ?
This is a very serious problem.
On Wednesday 01 September 2010 19:01:55 Doron Fediuck wrote:
> Hi,
> I have an implementation of SSH client using Mina SSHD.
> I've noticed that working with it on several hosts will take the CPU to 100% for every SSH
> client session I have. For me this is a severe state, so I ran some debug / profiling stuff.
>
> First I used jstack to see what's running.
> I saw-
> "ClientInputStreamPump" daemon prio=6 tid=0x34db8400 nid=0xec8 runnable [0x478df000]
> java.lang.Thread.State: RUNNABLE
> at org.apache.sshd.client.channel.ChannelSession.securedRead(ChannelSession.java:107)
> at org.apache.sshd.client.channel.ChannelSession.pumpInputStream(ChannelSession.java:81)
> at org.apache.sshd.client.channel.ChannelSession$1.run(ChannelSession.java:53)
>
> Locked ownable synchronizers:
> - None
>
> The, I wrote an application that connects to the other VM and list the thread utilization...
> Here's what I got-
>
> T ID: 2637 Name: JMX server connection timeout 2637
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 2636 Name: RMI TCP Connection(310)-10.35.17.171
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 2611 Name: RMI TCP Accept-0
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 2528 Name: WorkerThread#0[127.0.0.1:2566]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 2519 Name: ClientInputStreamPump
> endCpuTime : 17624 sec's
> endUserTime : 17619 sec's
> T ID: 1101 Name: http-0.0.0.0-8080-27
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1100 Name: http-0.0.0.0-8080-26
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1099 Name: http-0.0.0.0-8080-25
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1095 Name: http-0.0.0.0-8080-24
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1093 Name: http-0.0.0.0-8080-23
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1092 Name: http-0.0.0.0-8080-22
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1091 Name: http-0.0.0.0-8080-21
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1090 Name: http-0.0.0.0-8080-20
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1089 Name: http-0.0.0.0-8080-19
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1088 Name: http-0.0.0.0-8080-18
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1087 Name: http-0.0.0.0-8080-17
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1086 Name: http-0.0.0.0-8080-16
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1085 Name: http-0.0.0.0-8080-15
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1084 Name: http-0.0.0.0-8080-14
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1083 Name: http-0.0.0.0-8080-13
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1082 Name: http-0.0.0.0-8080-12
> endCpuTime : 114 sec's
> endUserTime : 103 sec's
> T ID: 1081 Name: http-0.0.0.0-8080-11
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1080 Name: http-0.0.0.0-8080-10
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1079 Name: http-0.0.0.0-8080-9
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1078 Name: http-0.0.0.0-8080-8
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1077 Name: http-0.0.0.0-8080-7
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1076 Name: http-0.0.0.0-8080-6
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1075 Name: http-0.0.0.0-8080-5
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1074 Name: http-0.0.0.0-8080-4
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 1073 Name: pool-22-thread-6
> endCpuTime : 544 sec's
> endUserTime : 534 sec's
> T ID: 274 Name: http-0.0.0.0-8080-3
> endCpuTime : 13 sec's
> endUserTime : 13 sec's
> T ID: 201 Name: MultiThreadedHttpConnectionManager cleanup
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 181 Name: http-0.0.0.0-8080-2
> endCpuTime : 497 sec's
> endUserTime : 444 sec's
> T ID: 173 Name: http-0.0.0.0-8080-1
> endCpuTime : 67 sec's
> endUserTime : 60 sec's
> T ID: 172 Name: ajp-0.0.0.0-8009-Acceptor-0
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 171 Name: http-0.0.0.0-8080-Acceptor-0
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 170 Name: JBossLifeThread
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 161 Name: IdleRemover
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 154 Name: Thread-17
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 153 Name: JCA PoolFiller
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 152 Name: HSQLDB Timer @1dc8b22
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 151 Name: QuartzScheduler_QuartzSchedulerThread
> endCpuTime : 23 sec's
> endUserTime : 22 sec's
> T ID: 150 Name: QuartzScheduler_Worker-99
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 149 Name: QuartzScheduler_Worker-98
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 148 Name: QuartzScheduler_Worker-97
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 147 Name: QuartzScheduler_Worker-96
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 146 Name: QuartzScheduler_Worker-95
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 145 Name: QuartzScheduler_Worker-94
> endCpuTime : 11 sec's
> endUserTime : 10 sec's
> T ID: 144 Name: QuartzScheduler_Worker-93
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 143 Name: QuartzScheduler_Worker-92
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 142 Name: QuartzScheduler_Worker-91
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 141 Name: QuartzScheduler_Worker-90
> endCpuTime : 13 sec's
> endUserTime : 10 sec's
> T ID: 140 Name: QuartzScheduler_Worker-89
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 139 Name: QuartzScheduler_Worker-88
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 138 Name: QuartzScheduler_Worker-87
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 137 Name: QuartzScheduler_Worker-86
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 136 Name: QuartzScheduler_Worker-85
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 135 Name: QuartzScheduler_Worker-84
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 134 Name: QuartzScheduler_Worker-83
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 133 Name: QuartzScheduler_Worker-82
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 132 Name: QuartzScheduler_Worker-81
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 131 Name: QuartzScheduler_Worker-80
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 130 Name: QuartzScheduler_Worker-79
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 129 Name: QuartzScheduler_Worker-78
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 128 Name: QuartzScheduler_Worker-77
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 127 Name: QuartzScheduler_Worker-76
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 126 Name: QuartzScheduler_Worker-75
> endCpuTime : 11 sec's
> endUserTime : 10 sec's
> T ID: 125 Name: QuartzScheduler_Worker-74
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 124 Name: QuartzScheduler_Worker-73
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 123 Name: QuartzScheduler_Worker-72
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 122 Name: QuartzScheduler_Worker-71
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 121 Name: QuartzScheduler_Worker-70
> endCpuTime : 11 sec's
> endUserTime : 9 sec's
> T ID: 120 Name: QuartzScheduler_Worker-69
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 119 Name: QuartzScheduler_Worker-68
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 118 Name: QuartzScheduler_Worker-67
> endCpuTime : 11 sec's
> endUserTime : 9 sec's
> T ID: 117 Name: QuartzScheduler_Worker-66
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 116 Name: QuartzScheduler_Worker-65
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 115 Name: QuartzScheduler_Worker-64
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 114 Name: QuartzScheduler_Worker-63
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 113 Name: QuartzScheduler_Worker-62
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 112 Name: QuartzScheduler_Worker-61
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 111 Name: QuartzScheduler_Worker-60
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 110 Name: QuartzScheduler_Worker-59
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 109 Name: QuartzScheduler_Worker-58
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 108 Name: QuartzScheduler_Worker-57
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 107 Name: QuartzScheduler_Worker-56
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 106 Name: QuartzScheduler_Worker-55
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 105 Name: QuartzScheduler_Worker-54
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 104 Name: QuartzScheduler_Worker-53
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 103 Name: QuartzScheduler_Worker-52
> endCpuTime : 11 sec's
> endUserTime : 10 sec's
> T ID: 102 Name: QuartzScheduler_Worker-51
> endCpuTime : 11 sec's
> endUserTime : 10 sec's
> T ID: 101 Name: QuartzScheduler_Worker-50
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 100 Name: QuartzScheduler_Worker-49
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 99 Name: QuartzScheduler_Worker-48
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 98 Name: QuartzScheduler_Worker-47
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 97 Name: QuartzScheduler_Worker-46
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 96 Name: QuartzScheduler_Worker-45
> endCpuTime : 13 sec's
> endUserTime : 10 sec's
> T ID: 95 Name: QuartzScheduler_Worker-44
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 94 Name: QuartzScheduler_Worker-43
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 93 Name: QuartzScheduler_Worker-42
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 92 Name: QuartzScheduler_Worker-41
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 91 Name: QuartzScheduler_Worker-40
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 90 Name: QuartzScheduler_Worker-39
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 89 Name: QuartzScheduler_Worker-38
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 88 Name: QuartzScheduler_Worker-37
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 87 Name: QuartzScheduler_Worker-36
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 86 Name: QuartzScheduler_Worker-35
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 85 Name: QuartzScheduler_Worker-34
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 84 Name: QuartzScheduler_Worker-33
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 83 Name: QuartzScheduler_Worker-32
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 82 Name: QuartzScheduler_Worker-31
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 81 Name: QuartzScheduler_Worker-30
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 80 Name: QuartzScheduler_Worker-29
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 79 Name: QuartzScheduler_Worker-28
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 78 Name: QuartzScheduler_Worker-27
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 77 Name: QuartzScheduler_Worker-26
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 76 Name: QuartzScheduler_Worker-25
> endCpuTime : 11 sec's
> endUserTime : 9 sec's
> T ID: 75 Name: QuartzScheduler_Worker-24
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 74 Name: QuartzScheduler_Worker-23
> endCpuTime : 12 sec's
> endUserTime : 11 sec's
> T ID: 73 Name: QuartzScheduler_Worker-22
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 72 Name: QuartzScheduler_Worker-21
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 71 Name: QuartzScheduler_Worker-20
> endCpuTime : 11 sec's
> endUserTime : 9 sec's
> T ID: 70 Name: QuartzScheduler_Worker-19
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 69 Name: QuartzScheduler_Worker-18
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 68 Name: QuartzScheduler_Worker-17
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 67 Name: QuartzScheduler_Worker-16
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 66 Name: QuartzScheduler_Worker-15
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 65 Name: QuartzScheduler_Worker-14
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 64 Name: QuartzScheduler_Worker-13
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 63 Name: QuartzScheduler_Worker-12
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 62 Name: QuartzScheduler_Worker-11
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 61 Name: QuartzScheduler_Worker-10
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 60 Name: QuartzScheduler_Worker-9
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 59 Name: QuartzScheduler_Worker-8
> endCpuTime : 11 sec's
> endUserTime : 10 sec's
> T ID: 58 Name: QuartzScheduler_Worker-7
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 57 Name: QuartzScheduler_Worker-6
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 56 Name: QuartzScheduler_Worker-5
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 55 Name: QuartzScheduler_Worker-4
> endCpuTime : 12 sec's
> endUserTime : 10 sec's
> T ID: 54 Name: QuartzScheduler_Worker-3
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 53 Name: QuartzScheduler_Worker-2
> endCpuTime : 13 sec's
> endUserTime : 11 sec's
> T ID: 52 Name: QuartzScheduler_Worker-1
> endCpuTime : 11 sec's
> endUserTime : 9 sec's
> T ID: 51 Name: QuartzScheduler_Worker-0
> endCpuTime : 14 sec's
> endUserTime : 11 sec's
> T ID: 50 Name: WorkManager(2)-1
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 49 Name: AOPListner
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 48 Name: ContainerBackgroundProcessor[StandardEngine[jboss.web]]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 47 Name: RMI TCP Accept-4444
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 46 Name: PooledInvokerAcceptor#0-4445
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 43 Name: Thread-14
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 44 Name: Listener:4712
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 42 Name: Thread-13
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 41 Name: Thread-12
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 40 Name: Thread-11
> endCpuTime : 5 sec's
> endUserTime : 5 sec's
> T ID: 38 Name: Listener:4713
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 37 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=4446]]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 36 Name: ServerSocketRefresh
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 35 Name: HDScanner
> endCpuTime : 319 sec's
> endUserTime : 136 sec's
> T ID: 34 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=3873]]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 33 Name: ServerSocketRefresh
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 32 Name: secondaryServerSocketThread[0]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 31 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=4457]]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 30 Name: ServerSocketRefresh
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 28 Name: RMI RenewClean-[qumranetyk:1175]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 27 Name: RMI Scheduler(0)
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 25 Name: RMI TCP Accept-0
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 24 Name: RMI TCP Accept-1090
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 23 Name: AcceptorThread[ServerSocket[addr=/0.0.0.0,port=0,localport=5400]]
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 22 Name: ServerSocketRefresh
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 21 Name: JBoss System Threads(1)-2
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 20 Name: GC Daemon
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 19 Name: RMI Reaper
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 18 Name: RMI TCP Accept-1098
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 17 Name: JBoss System Threads(1)-1
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 16 Name: Timer-Log4jService
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 15 Name: ZipFile Lock Reaper
> endCpuTime : 1 sec's
> endUserTime : 1 sec's
> T ID: 14 Name: Timer-0
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 11 Name: DestroyJavaVM
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 5 Name: Attach Listener
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 4 Name: Signal Dispatcher
> endCpuTime : 0 sec's
> endUserTime : 0 sec's
> T ID: 3 Name: Finalizer
> endCpuTime : 8 sec's
> endUserTime : 7 sec's
> T ID: 2 Name: Reference Handler
> endCpuTime : 2 sec's
> endUserTime : 1 sec's
>
> Just in case you missed it-
> T ID: 2519 Name: ClientInputStreamPump
> endCpuTime : 17624 sec's
> endUserTime : 17619 sec's
>
> It looks like ClientInputStreamPump is pumping my CPU :(
> Any idea how to resolve it ?
>
>
--
/d