Re: NPE in RouteContextProcessor

[Steve Drach <St...@glam.com>]

It is a custom component built several years ago by a prior engineer in my company, when the system was on Camel 1.4.  I'm not sure if DefaultExchange was thread safe back then or not, but it's certainly not thread safe today.  When I realized the issue I changed the routes from seda and sedax to direct since we do input spooling before sticking the message into the Camel pipeline and I stopped getting NPE's and more importantly stopped losing messages.  This is a high volume email system and people, for years, have been wondering why some mail doesn't get through ;-)

So what is Camel's multi thread story/philosophy?  Are there guide lines to follow?

From: Willem jiang <wi...@gmail.com>>
Date: Thursday, August 2, 2012 1:15 AM
To: Steve Drach <st...@glam.com>>
Subject: Re: NPE in RouteContextProcessor


It looks like you are using sedax component.
I didn't see this component before, is it a custom component which you build by yourself?

--
Willem jiang



On Wednesday, August 1, 2012 at 12:21 AM, Steve Drach wrote:


On 7/31/12 2:40 AM, "Willem jiang" <wi...@gmail.com>> wrote:

What's your Camel route look like ?

Here's the subset that shows the path taken, starting with "spool:core".
The section marked with
<==== is the last one I see in the trace.

/* From Core */

from("spool:core:store")
.process(new IllegalSMTPAddressFixer())
.process(new CopyFromAndToIntact())
.process(new AddressValidationFilter())
.choice()
.when(header(DO_NOT_SEND).isNotNull())
.to("seda:drop")
.otherwise()
.to("sedax:delivery");

Š

/* Shared */
from("sedax:delivery?threads=" + numOutgoingThreads) <=====
.process(new MessageTypeProcessor())
.process(logger)
.to("smtp:out",
"spool:*:delete");

from("seda:drop").to("direct:drop");

from("direct:drop")
.process(new ConvertDeliveryToDeadDelivery())
.process(new LogDeadDelivery())
.to("spool:*:delete");





--
Willem Jiang



On Tuesday, July 31, 2012 at 7:52 AM, Steve Drach wrote:

I'm using Camel 2.4.10. I'm get a NPE in RouteContextProcessor at line
42:

29 public class RouteContextProcessor extends DelegateAsyncProcessor {
...
38 @Override
39 protected boolean processNext(final Exchange exchange, final
AsyncCallback callback) {
40 // push the current route context
41 if (exchange.getUnitOfWork() != null) {
42 exchange.getUnitOfWork().pushRouteContext(routeContext);
43 }

I am able to demonstrate that even though the test in line 41 is true,
getUnitOfWork() in null in line 43. I do this by instrumenting the code
as
follows:

@Override
protected boolean processNext(final Exchange exchange, final
AsyncCallback callback) {
// push the current route context
try {
if (exchange.getUnitOfWork() != null) {
exchange.getUnitOfWork().pushRouteContext(routeContext);
}
} catch (Exception e) {
System.out.println("********************\n"+exchange);
System.out.println(exchange.getUnitOfWork());
e.printStackTrace();
System.exit(1);

}

And I can clearly see the null value. This implies to me somebody else
has
a reference to the same exchange object and is changing it between
lines 42
and line 43. This only happens occasionally (1 in 1000 times).

My code has 10 threads, so I wouldn't be surprised if it was my
problem, but
I just can't see it. I am not doing anything with the UnitOfWork in an
exchange. At most, I change the body of the Message and perhaps the
headers.

This is very difficult to debug because it happens so infrequently. Does
anyone have a suggestion that could help me?




--
View this message in context:
http://camel.465427.n5.nabble.com/NPE-in-RouteContextProcessor-tp5716620.
html
Sent from the Camel - Users mailing list archive at Nabble.com<http://Nabble.com>
(http://Nabble.com).

Re: NPE in RouteContextProcessor

[Christian Müller <ch...@gmail.com>]

All your Processor's, Bean's, ... are shared among all exchanges and are
used in multiple threads concurrent. They MUST be thread safe and that's
the responsibility of the author.
If you have to be stateful, safe the state as exchange property, message
header, ... Quite simple...

Did you discovered another behavior?

Best,
Christian

On Thu, Aug 2, 2012 at 6:18 PM, Steve Drach <St...@glam.com> wrote:

> It is a custom component built several years ago by a prior engineer in my
> company, when the system was on Camel 1.4.  I'm not sure if DefaultExchange
> was thread safe back then or not, but it's certainly not thread safe today.
>  When I realized the issue I changed the routes from seda and sedax to
> direct since we do input spooling before sticking the message into the
> Camel pipeline and I stopped getting NPE's and more importantly stopped
> losing messages.  This is a high volume email system and people, for years,
> have been wondering why some mail doesn't get through ;-)
>
> So what is Camel's multi thread story/philosophy?  Are there guide lines
> to follow?
>
> From: Willem jiang <wi...@gmail.com>>
> Date: Thursday, August 2, 2012 1:15 AM
> To: Steve Drach <st...@glam.com>>
> Subject: Re: NPE in RouteContextProcessor
>
>
> It looks like you are using sedax component.
> I didn't see this component before, is it a custom component which you
> build by yourself?
>
> --
> Willem jiang
>
>
>
> On Wednesday, August 1, 2012 at 12:21 AM, Steve Drach wrote:
>
>
> On 7/31/12 2:40 AM, "Willem jiang" <willem.jiang@gmail.com<mailto:
> willem.jiang@gmail.com>> wrote:
>
> What's your Camel route look like ?
>
> Here's the subset that shows the path taken, starting with "spool:core".
> The section marked with
> <==== is the last one I see in the trace.
>
> /* From Core */
>
> from("spool:core:store")
> .process(new IllegalSMTPAddressFixer())
> .process(new CopyFromAndToIntact())
> .process(new AddressValidationFilter())
> .choice()
> .when(header(DO_NOT_SEND).isNotNull())
> .to("seda:drop")
> .otherwise()
> .to("sedax:delivery");
>
> Š
>
> /* Shared */
> from("sedax:delivery?threads=" + numOutgoingThreads) <=====
> .process(new MessageTypeProcessor())
> .process(logger)
> .to("smtp:out",
> "spool:*:delete");
>
> from("seda:drop").to("direct:drop");
>
> from("direct:drop")
> .process(new ConvertDeliveryToDeadDelivery())
> .process(new LogDeadDelivery())
> .to("spool:*:delete");
>
>
>
>
>
> --
> Willem Jiang
>
>
>
> On Tuesday, July 31, 2012 at 7:52 AM, Steve Drach wrote:
>
> I'm using Camel 2.4.10. I'm get a NPE in RouteContextProcessor at line
> 42:
>
> 29 public class RouteContextProcessor extends DelegateAsyncProcessor {
> ...
> 38 @Override
> 39 protected boolean processNext(final Exchange exchange, final
> AsyncCallback callback) {
> 40 // push the current route context
> 41 if (exchange.getUnitOfWork() != null) {
> 42 exchange.getUnitOfWork().pushRouteContext(routeContext);
> 43 }
>
> I am able to demonstrate that even though the test in line 41 is true,
> getUnitOfWork() in null in line 43. I do this by instrumenting the code
> as
> follows:
>
> @Override
> protected boolean processNext(final Exchange exchange, final
> AsyncCallback callback) {
> // push the current route context
> try {
> if (exchange.getUnitOfWork() != null) {
> exchange.getUnitOfWork().pushRouteContext(routeContext);
> }
> } catch (Exception e) {
> System.out.println("********************\n"+exchange);
> System.out.println(exchange.getUnitOfWork());
> e.printStackTrace();
> System.exit(1);
>
> }
>
> And I can clearly see the null value. This implies to me somebody else
> has
> a reference to the same exchange object and is changing it between
> lines 42
> and line 43. This only happens occasionally (1 in 1000 times).
>
> My code has 10 threads, so I wouldn't be surprised if it was my
> problem, but
> I just can't see it. I am not doing anything with the UnitOfWork in an
> exchange. At most, I change the body of the Message and perhaps the
> headers.
>
> This is very difficult to debug because it happens so infrequently. Does
> anyone have a suggestion that could help me?
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/NPE-in-RouteContextProcessor-tp5716620.
> html
> Sent from the Camel - Users mailing list archive at Nabble.com<
> http://Nabble.com>
> (http://Nabble.com).
>
>