You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Eugeniu <eu...@hotmail.com> on 2012/05/22 18:00:37 UTC

Redelivery failure

/Hi

I am doing load testing of a camel route. The route has a netty endpoint
which sends data over tcp. I am getting the following errors:/

18:50:44,771 | WARN  | nt worker #109-7 | DefaultChannelPipeline           | 
-  -  | An exception was thrown by a user handler while handling an
exception event ([id: 0x7e6421f8, /192.168.2.3:37594 :> /192.168.2.3:9091]
EXCEPTION: java.lang.IllegalStateException: IDLE,initial)
java.lang.IllegalStateException: Queue full
	at java.util.AbstractQueue.add(AbstractQueue.java:71)[:1.6.0_31]
	at
java.util.concurrent.ArrayBlockingQueue.add(ArrayBlockingQueue.java:207)[:1.6.0_31]
	at
org.apache.camel.impl.DefaultServicePool.release(DefaultServicePool.java:94)
	at org.apache.camel.impl.ProducerCache$1.done(ProducerCache.java:296)
	at
org.apache.camel.processor.SendProcessor$2$1.done(SendProcessor.java:119)
	at
org.apache.camel.component.netty.handlers.ClientChannelHandler.exceptionCaught(ClientChannelHandler.java:84)
	at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238)
	at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)
	at
org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
	at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
	at
org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_31]
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_31]
	at java.lang.Thread.run(Thread.java:662)[:1.6.0_31]
18:50:44,760 | WARN  | t worker #162-11 | DefaultErrorHandler              |
91 - org.apache.camel.camel-core - 2.8.4 | Failed delivery for exchangeId:
ID-PC-11083-1337692449682-0-1750741. On delivery attempt: 4 caught:
java.lang.IllegalStateException: IDLE,initial
18:50:44,755 | WARN  | t worker #162-10 | DefaultErrorHandler              |
91 - org.apache.camel.camel-core - 2.8.4 | Failed delivery for exchangeId:
ID-PC-11083-1337692449682-0-1751118. On delivery attempt: 4 caught:
java.lang.IllegalStateException: IDLE,initial

/Also this:/

18:50:53,067 | WARN  | nt worker #185-4 | DefaultChannelPipeline           | 
-  -  | An exception was thrown by a user handler while handling an
exception event ([id: 0x60ef0168, /192.168.2.3:38847 :> /192.168.2.3:9091]
EXCEPTION: java.lang.IllegalStateException: IDLE,initial)
java.lang.NullPointerException
	at
org.apache.camel.processor.RedeliveryErrorHandler.processAsyncErrorHandler(RedeliveryErrorHandler.java:413)
	at
org.apache.camel.processor.RedeliveryErrorHandler$AsyncRedeliveryTask$2.done(RedeliveryErrorHandler.java:167)
	at
org.apache.camel.processor.interceptor.TraceInterceptor$1.done(TraceInterceptor.java:179)[91:org.apache.camel.camel-core:2.8.4]
	at
org.apache.camel.management.InstrumentationProcessor$1.done(InstrumentationProcessor.java:78)
	at org.apache.camel.impl.ProducerCache$1.done(ProducerCache.java:307)
	at
org.apache.camel.processor.SendProcessor$2$1.done(SendProcessor.java:119)
	at
org.apache.camel.component.netty.handlers.ClientChannelHandler.exceptionCaught(ClientChannelHandler.java:84)
	at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238)
	at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)
	at
org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
	at
org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
	at
org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
	at
org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_31]
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_31]
	at java.lang.Thread.run(Thread.java:662)[:1.6.0_31]

*/What configuration parameters can I put to avoid the "Queue full"
exception?/*

--
View this message in context: http://camel.465427.n5.nabble.com/Redelivery-failure-tp5713347.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Redelivery failure

Posted by Claus Ibsen <cl...@gmail.com>.
When you post stacktraces its important to also say which versions of
the projects you us.
Otherwise the line numbers may not match up.


On Tue, May 22, 2012 at 6:00 PM, Eugeniu <eu...@hotmail.com> wrote:
> /Hi
>
> I am doing load testing of a camel route. The route has a netty endpoint
> which sends data over tcp. I am getting the following errors:/
>
> 18:50:44,771 | WARN  | nt worker #109-7 | DefaultChannelPipeline           |
> -  -  | An exception was thrown by a user handler while handling an
> exception event ([id: 0x7e6421f8, /192.168.2.3:37594 :> /192.168.2.3:9091]
> EXCEPTION: java.lang.IllegalStateException: IDLE,initial)
> java.lang.IllegalStateException: Queue full
>        at java.util.AbstractQueue.add(AbstractQueue.java:71)[:1.6.0_31]
>        at
> java.util.concurrent.ArrayBlockingQueue.add(ArrayBlockingQueue.java:207)[:1.6.0_31]
>        at
> org.apache.camel.impl.DefaultServicePool.release(DefaultServicePool.java:94)
>        at org.apache.camel.impl.ProducerCache$1.done(ProducerCache.java:296)
>        at
> org.apache.camel.processor.SendProcessor$2$1.done(SendProcessor.java:119)
>        at
> org.apache.camel.component.netty.handlers.ClientChannelHandler.exceptionCaught(ClientChannelHandler.java:84)
>        at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238)
>        at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)
>        at
> org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>        at
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_31]
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_31]
>        at java.lang.Thread.run(Thread.java:662)[:1.6.0_31]
> 18:50:44,760 | WARN  | t worker #162-11 | DefaultErrorHandler              |
> 91 - org.apache.camel.camel-core - 2.8.4 | Failed delivery for exchangeId:
> ID-PC-11083-1337692449682-0-1750741. On delivery attempt: 4 caught:
> java.lang.IllegalStateException: IDLE,initial
> 18:50:44,755 | WARN  | t worker #162-10 | DefaultErrorHandler              |
> 91 - org.apache.camel.camel-core - 2.8.4 | Failed delivery for exchangeId:
> ID-PC-11083-1337692449682-0-1751118. On delivery attempt: 4 caught:
> java.lang.IllegalStateException: IDLE,initial
>
> /Also this:/
>
> 18:50:53,067 | WARN  | nt worker #185-4 | DefaultChannelPipeline           |
> -  -  | An exception was thrown by a user handler while handling an
> exception event ([id: 0x60ef0168, /192.168.2.3:38847 :> /192.168.2.3:9091]
> EXCEPTION: java.lang.IllegalStateException: IDLE,initial)
> java.lang.NullPointerException
>        at
> org.apache.camel.processor.RedeliveryErrorHandler.processAsyncErrorHandler(RedeliveryErrorHandler.java:413)
>        at
> org.apache.camel.processor.RedeliveryErrorHandler$AsyncRedeliveryTask$2.done(RedeliveryErrorHandler.java:167)
>        at
> org.apache.camel.processor.interceptor.TraceInterceptor$1.done(TraceInterceptor.java:179)[91:org.apache.camel.camel-core:2.8.4]
>        at
> org.apache.camel.management.InstrumentationProcessor$1.done(InstrumentationProcessor.java:78)
>        at org.apache.camel.impl.ProducerCache$1.done(ProducerCache.java:307)
>        at
> org.apache.camel.processor.SendProcessor$2$1.done(SendProcessor.java:119)
>        at
> org.apache.camel.component.netty.handlers.ClientChannelHandler.exceptionCaught(ClientChannelHandler.java:84)
>        at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238)
>        at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)
>        at
> org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
>        at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>        at
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_31]
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_31]
>        at java.lang.Thread.run(Thread.java:662)[:1.6.0_31]
>
> */What configuration parameters can I put to avoid the "Queue full"
> exception?/*
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Redelivery-failure-tp5713347.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: cibsen@fusesource.com
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/

Re: Redelivery failure

Posted by Christian Müller <ch...@gmail.com>.
The code which shows your route would be also helpful...

Best,
Christian

Sent from a mobile device
Am 23.05.2012 09:56 schrieb "Eugeniu" <eu...@hotmail.com>:

> /Hi
>
> I am doing load testing of a camel route. The route has a netty endpoint
> which sends data over tcp. I am getting the following errors:/
>
> 18:50:44,771 | WARN  | nt worker #109-7 | DefaultChannelPipeline
> |
> -  -  | An exception was thrown by a user handler while handling an
> exception event ([id: 0x7e6421f8, /192.168.2.3:37594 :> /192.168.2.3:9091]
> EXCEPTION: java.lang.IllegalStateException: IDLE,initial)
> java.lang.IllegalStateException: Queue full
>        at java.util.AbstractQueue.add(AbstractQueue.java:71)[:1.6.0_31]
>        at
>
> java.util.concurrent.ArrayBlockingQueue.add(ArrayBlockingQueue.java:207)[:1.6.0_31]
>        at
>
> org.apache.camel.impl.DefaultServicePool.release(DefaultServicePool.java:94)
>        at
> org.apache.camel.impl.ProducerCache$1.done(ProducerCache.java:296)
>        at
> org.apache.camel.processor.SendProcessor$2$1.done(SendProcessor.java:119)
>        at
>
> org.apache.camel.component.netty.handlers.ClientChannelHandler.exceptionCaught(ClientChannelHandler.java:84)
>        at
>
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238)
>        at
> org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)
>        at
>
> org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
>
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>        at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>        at
>
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>        at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>        at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_31]
>        at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_31]
>        at java.lang.Thread.run(Thread.java:662)[:1.6.0_31]
> 18:50:44,760 | WARN  | t worker #162-11 | DefaultErrorHandler
>  |
> 91 - org.apache.camel.camel-core - 2.8.4 | Failed delivery for exchangeId:
> ID-PC-11083-1337692449682-0-1750741. On delivery attempt: 4 caught:
> java.lang.IllegalStateException: IDLE,initial
> 18:50:44,755 | WARN  | t worker #162-10 | DefaultErrorHandler
>  |
> 91 - org.apache.camel.camel-core - 2.8.4 | Failed delivery for exchangeId:
> ID-PC-11083-1337692449682-0-1751118. On delivery attempt: 4 caught:
> java.lang.IllegalStateException: IDLE,initial
>
> /Also this:/
>
> 18:50:53,067 | WARN  | nt worker #185-4 | DefaultChannelPipeline
> |
> -  -  | An exception was thrown by a user handler while handling an
> exception event ([id: 0x60ef0168, /192.168.2.3:38847 :> /192.168.2.3:9091]
> EXCEPTION: java.lang.IllegalStateException: IDLE,initial)
> java.lang.NullPointerException
>        at
>
> org.apache.camel.processor.RedeliveryErrorHandler.processAsyncErrorHandler(RedeliveryErrorHandler.java:413)
>        at
>
> org.apache.camel.processor.RedeliveryErrorHandler$AsyncRedeliveryTask$2.done(RedeliveryErrorHandler.java:167)
>        at
>
> org.apache.camel.processor.interceptor.TraceInterceptor$1.done(TraceInterceptor.java:179)[91:org.apache.camel.camel-core:2.8.4]
>        at
>
> org.apache.camel.management.InstrumentationProcessor$1.done(InstrumentationProcessor.java:78)
>        at
> org.apache.camel.impl.ProducerCache$1.done(ProducerCache.java:307)
>        at
> org.apache.camel.processor.SendProcessor$2$1.done(SendProcessor.java:119)
>        at
>
> org.apache.camel.component.netty.handlers.ClientChannelHandler.exceptionCaught(ClientChannelHandler.java:84)
>        at
>
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238)
>        at
> org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)
>        at
>
> org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
>
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:76)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299)
>        at
>
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)
>        at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)
>        at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:351)
>        at
>
> org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:282)
>        at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:202)
>        at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_31]
>        at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_31]
>        at java.lang.Thread.run(Thread.java:662)[:1.6.0_31]
>
> */What configuration parameters can I put to avoid the "Queue full"
> exception?/*
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Redelivery-failure-tp5713347.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>

Re: Redelivery failure

Posted by Ashwin Karpe <ak...@fusesource.com>.
Hi,

As Claus and Christian suggested, the route (scrubbed of course) as well as
camel version information would help.

Based on a superficial inspection of the error, it seems like the capacity
of the java.util.concurrent.ArrayBlockingQueue is being exceeded. This seems
to suggest that exchanges being pushed into the netty endpoint are not being
sent to the destination and hanging around for some reason (due be comm
exceptions, retries, etc).

I would look at whether the port/endpoints are up and the rate at which the
consumer on the server is picking up its messages.  It is quite clear the
load testing is pushing a lot off messages to the endpoint, but the quality
of connection and ability to pick up messages at the consumer is not clear.

Disclaimer: This is however my superficial read of the situation. My
response is only as good as the data provided.

Cheers,

Ashwin...
 

-----
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
FUSESource (a Progress Software Corporation subsidiary)
http://fusesource.com 

Blog: http://opensourceknowledge.blogspot.com 
---------------------------------------------------------
--
View this message in context: http://camel.465427.n5.nabble.com/Redelivery-failure-tp5713347p5713384.html
Sent from the Camel - Users mailing list archive at Nabble.com.