You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Francesco Malvezzi <fr...@unimore.it> on 2012/02/06 12:41:48 UTC

thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

I'm a bit confused about the jmx management of camel. According to
http://camel.apache.org/camel-jmx.html, jmx should be enabled on every camel
project.

Actually from jconsole I can see a org.codehaus.classwords:launcher item
which resembles camel very closely. And indeed it should be, because if I
connect the camel log gets populated with a long thread strace:

"Camel (camel-1) thread #4 - JmsConsumer[username.to]" daemon prio=10
tid=0x8ecb8800 nid=0x1977 in Object.wait() [0x8d2ae000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xadb6bc48> (a java.lang.Object)
	at
org.apache.activemq.SimplePriorityMessageDispatchChannel.dequeue(SimplePriorityMessageDispatchChannel.java:89)
	- locked <0xadb6bc48> (a java.lang.Object)
	at
org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:452)
	at
org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:577)
	at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:429)
	at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
	at
org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
	at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
	at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
	at
org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
[...]

"VM Thread" prio=10 tid=0x080d3c00 nid=0x1903 runnable 

"GC task thread#0 (ParallelGC)" prio=10 tid=0x0805f000 nid=0x1901 runnable 

"GC task thread#1 (ParallelGC)" prio=10 tid=0x08060800 nid=0x1902 runnable 

"VM Periodic Task Thread" prio=10 tid=0x080fb800 nid=0x190a waiting on
condition 

JNI global references: 1341

Heap
 PSYoungGen      total 160832K, used 103451K [0xa9580000, 0xb3e80000,
0xb4020000)
  eden space 150528K, 65% used [0xa9580000,0xaf668ce0,0xb2880000)
  from space 10304K, 40% used [0xb3470000,0xb388e070,0xb3e80000)
  to   space 11264K, 0% used [0xb2880000,0xb2880000,0xb3380000)
 PSOldGen        total 82432K, used 51728K [0x94020000, 0x990a0000,
0xa9580000)
  object space 82432K, 62% used [0x94020000,0x972a4378,0x990a0000)
 PSPermGen       total 65536K, used 41593K [0x90020000, 0x94020000,
0x94020000)
  object space 65536K, 63% used [0x90020000,0x928be6f0,0x94020000)

What am I missing? My camel version in 2.8.4

thank you,

Francesco

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5459905.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Babak Vahdat <ba...@swissonline.ch>.
Hi

maybe the reported bugs below could be also of your interest:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=539288
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=523368

Babak

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5465878.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Francesco Malvezzi <fr...@unimore.it>.
Babak Vahdat wrote
> 
> Sorry, I messed it up!
> 
> I mixed the the JDK build version versus the Hotspot build version you
> pasted at http://pastebin.com/WS6y9DEG. So simply forget my advice...
> 
> I've still got no idea why you can not connect through the JConsole.
> 
> Babak
> 

Actually your hint about the old java version looks sensible, because I'm
yet using a Debian/GNU Lenny (old stable). I'm going to upgrade to check
further,

thank you,

Francesco

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5465799.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Babak Vahdat <ba...@swissonline.ch>.
Sorry, I messed it up!

I mixed the the JDK build version versus the Hotspot build version you
pasted at http://pastebin.com/WS6y9DEG. So simply forget my advice...

I've still got no idea why you can not connect through the JConsole.

Babak 

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5462977.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Babak Vahdat <ba...@swissonline.ch>.
Hi,

You do launch your Camel application under a pretty old JDK (20.1-b02)
installed on your box as well and *not* the (26-b03) build!

- Remove the bin directory of this old JDK installation from your unix PATH
and put the bin directory of the one you already mentioned by your post
- Set JAVA_HOME *explicitly* to the the correct JDK (build 1.6.0_26-b03)

Now do the same again (mvn camel:run) to see if this would resolve the
problem, also make sure you make use of the *same* JConsole
comming along with the JDK distribution already mentioned.

Babak

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5462917.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Babak Vahdat <ba...@swissonline.ch>.
And maybe do a "mvn clean install" beforehand as well to make sure your class
files are generated using the "correct" JDK compiler.

Babak

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5462936.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Francesco Malvezzi <fr...@unimore.it>.
Babak Vahdat wrote
> 
> Where does that stack trace blows up on the JConsole or on your App side?
> Can you attach that huge stack trace as an Attachment to your Post? 
> 
First of all, thank you for helping me to post a sensible message.

Stack trace is too long, but you can glance it at:

http://pastebin.com/WS6y9DEG

The JConsole is quite silent: it just says in the dialog box: 'Connection
Failed. Retry?'



> And how do you deploy your Camel App? Standalone, JEE-Server (if yes which
> one and which version), OSGi (the same information as well).
> 
> The more information you provide, the more likely the people on the Forum
> can help you.
> 
> Babak
> 

It is a stand-alone Camel app: it was launched with mvn camel:run. Platform
is Debian GNU/Linux Lenny, Java is 
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) Server VM (build 20.1-b02, mixed mode)
camel is camel-2.8.4.

The stack trace bounces up from the console from which the mvn camel:run
command is issued.

ciao,

Francesco




--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5462495.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Babak Vahdat <ba...@swissonline.ch>.
Where does that stack trace blows up on the JConsole or on your App side? Can
you attach that huge stack trace as an Attachment to your Post? And how do
you deploy your Camel App? Standalone, JEE-Server (if yes which one and
which version), OSGi (the same information as well).

The more information you provide, the more likely the people on the Forum
can help you.

Babak

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5460410.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Francesco Malvezzi <fr...@unimore.it>.
Babak Vahdat wrote
> 
> Hi
> 
> Can you please precisely describe the problem you're facing? The thread
> state TIMED_WAITING is a legitimate state:
> 
> http://docs.oracle.com/javase/6/docs/api/java/lang/Thread.State.html#TIMED_WAITING
> 
Just connecting with jconsole to camel apps throws a huge thread stack
trace. Pardon, TIMED_WAITING is legitimate. Sure I can't read stack trace
(which is very long), but at the very beginning:

Full thread dump Java HotSpot(TM) Server VM (20.1-b02 mixed mode):

"InactivityMonitor WriteCheck" daemon prio=10 tid=0x087ad400 nid=0x5c1f in
Object.wait() [0x8ca6e000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xaea3fbd0> (a java.util.TaskQueue)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0xaea3fbd0> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

"InactivityMonitor ReadCheck" daemon prio=10 tid=0x09442400 nid=0x5c1e in
Object.wait() [0x8cabe000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0xaea3f700> (a java.util.TaskQueue)
	at java.util.TimerThread.mainLoop(Timer.java:509)
	- locked <0xaea3f700> (a java.util.TaskQueue)
	at java.util.TimerThread.run(Timer.java:462)

could it be a concurrency issue?


Babak Vahdat wrote
> 
> And check the link you've already provided regarding the note about the
> required jars for the JMX support on Camel 2.8 or older which says:
> 
> If these jars is not on the classpath Camel will fallback to non JMX mode!
> 
> Babak
> 

Thank you for the advise: actually it looks I already included the jars
thanks to maven. Also adding these jars to pom.xml doesn't change anything.

Thank you again,

Francesco


--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5460320.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Babak Vahdat <ba...@swissonline.ch>.
Hi

Can you please precisely describe the problem you're facing? The thread
state TIMED_WAITING is a legitimate state:

http://docs.oracle.com/javase/6/docs/api/java/lang/Thread.State.html#TIMED_WAITING

And check the link you've already provided regarding the note about the
required jars for the JMX support on Camel 2.8 or older which says:

If these jars is not on the classpath Camel will fallback to non JMX mode!

Babak

--
View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5460210.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: thread error with java.lang.Thread.State: TIMED_WAITING from jconsole

Posted by Claus Ibsen <cl...@gmail.com>.
Hi

This is expected. The TIMED_WAITING thread is a JmsConsumer thread
which pickup new messages from the ActiveMQ message queue.
Currently there queue is empty so there is no messages to pickup and
therefore the thread is waiting.

If you put messages on that queue, then the thread will pickup and
process the messages.

In the Camel distribution you can also try the
camel-example-management, and then stop route 1.
Then eventually the message queue will dry out and route 3 has no more
messages to pickup and will do a TIMED_WAITING as in your case

"Camel (camel) thread #3 - JmsConsumer[stock]" daemon prio=5
tid=7ff28e7b0000 nid=0x119608000 in Object.wait() [119607000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <7a3b43880> (a java.lang.Object)
	at org.apache.activemq.SimplePriorityMessageDispatchChannel.dequeue(SimplePriorityMessageDispatchChannel.java:89)
	- locked <7a3b43880> (a java.lang.Object)
	at org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:452)
	at org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:577)
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:429)



On Mon, Feb 6, 2012 at 12:41 PM, Francesco Malvezzi
<fr...@unimore.it> wrote:
> I'm a bit confused about the jmx management of camel. According to
> http://camel.apache.org/camel-jmx.html, jmx should be enabled on every camel
> project.
>
> Actually from jconsole I can see a org.codehaus.classwords:launcher item
> which resembles camel very closely. And indeed it should be, because if I
> connect the camel log gets populated with a long thread strace:
>
> "Camel (camel-1) thread #4 - JmsConsumer[username.to]" daemon prio=10
> tid=0x8ecb8800 nid=0x1977 in Object.wait() [0x8d2ae000]
>   java.lang.Thread.State: TIMED_WAITING (on object monitor)
>        at java.lang.Object.wait(Native Method)
>        - waiting on <0xadb6bc48> (a java.lang.Object)
>        at
> org.apache.activemq.SimplePriorityMessageDispatchChannel.dequeue(SimplePriorityMessageDispatchChannel.java:89)
>        - locked <0xadb6bc48> (a java.lang.Object)
>        at
> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:452)
>        at
> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:577)
>        at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:429)
>        at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:310)
>        at
> org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)
>        at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1058)
>        at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1050)
>        at
> org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:947)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:662)
> [...]
>
> "VM Thread" prio=10 tid=0x080d3c00 nid=0x1903 runnable
>
> "GC task thread#0 (ParallelGC)" prio=10 tid=0x0805f000 nid=0x1901 runnable
>
> "GC task thread#1 (ParallelGC)" prio=10 tid=0x08060800 nid=0x1902 runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x080fb800 nid=0x190a waiting on
> condition
>
> JNI global references: 1341
>
> Heap
>  PSYoungGen      total 160832K, used 103451K [0xa9580000, 0xb3e80000,
> 0xb4020000)
>  eden space 150528K, 65% used [0xa9580000,0xaf668ce0,0xb2880000)
>  from space 10304K, 40% used [0xb3470000,0xb388e070,0xb3e80000)
>  to   space 11264K, 0% used [0xb2880000,0xb2880000,0xb3380000)
>  PSOldGen        total 82432K, used 51728K [0x94020000, 0x990a0000,
> 0xa9580000)
>  object space 82432K, 62% used [0x94020000,0x972a4378,0x990a0000)
>  PSPermGen       total 65536K, used 41593K [0x90020000, 0x94020000,
> 0x94020000)
>  object space 65536K, 63% used [0x90020000,0x928be6f0,0x94020000)
>
> What am I missing? My camel version in 2.8.4
>
> thank you,
>
> Francesco
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/thread-error-with-java-lang-Thread-State-TIMED-WAITING-from-jconsole-tp5459905p5459905.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/