Threading issue with processors

[Char <ch...@gmx.net>]

Hello camel-breeders!

We are trying to poll a huge amount of files by the jbi:file component. All
these files are processed by a custom processor during the next step:

    
from("jbi:endpoint:http://myCompany.net/InboundRoutingService/EDIFileInEndpoint")
    	.intercept(new StreamCachingInterceptor())
   	.process(new DataProcessor(new EdiPreProcessor(),
DataProcessor.ATTACHMENT_TO_BODY))

There is a problem with the custom EdiPreProcessor. It's job is quite
simple. It has to wrap the incoming data in xml and to extract some
header-data to xml-attributes. Most times, it runs well and there is no
problem. But if you put a lot of files into the polling-directory, there are
exceptions thrown.
We debugged and saw that there are multiple threads of this
processor-instance running in case of multiple files to process... and if
one of these threads changes a global boolean attribute of the processor
object, this attribute is changed in every other thread as well.
I already tried some experiments with "volatile" or "synchronized" keywords,
but all of them failed. I just want that every single thread has it's own
attributes.

Is there a way to do this from camel's point of view? Or how would you solve
this problem?

thx in advance for your help
-- 
View this message in context: http://www.nabble.com/Threading-issue-with-processors-tp20661633s22882p20661633.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Threading issue with processors

[Char <ch...@gmx.net>]

We are working on it to make the processors thread-safe. It was just an
understanding-question of whether we have to do it or not, but now it is
obvious that it will just work with thread-safe processors.

thanks.
-- 
View this message in context: http://www.nabble.com/Threading-issue-with-processors-tp20661633s22882p20680417.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Threading issue with processors

[Gert Vanthienen <ge...@skynet.be>]

L.S.,

First of all, do you really need the global attributes?  If all you do 
in the process() method is transform a single exchange and there's no 
need to keep track of data over exchanges, you can probably avoid the 
the global attributes alltogether.

If you do need to maintain state over several exchanges, you can use 
some helper classes in the JDK (e.g. a ThreadLocal or the 
AtomicBoolean/Integer/...) to build a more thread-safe implementation of 
your code.  If you post us a snippet or some pseudo-code of what you're 
trying to accomplish, we can try to give you some points on how to make 
that processor thread-safe.

Regards,

Gert

Char wrote:
> Hello camel-breeders!
>
> We are trying to poll a huge amount of files by the jbi:file component. All
> these files are processed by a custom processor during the next step:
>
>     
> from("jbi:endpoint:http://myCompany.net/InboundRoutingService/EDIFileInEndpoint")
>     	.intercept(new StreamCachingInterceptor())
>    	.process(new DataProcessor(new EdiPreProcessor(),
> DataProcessor.ATTACHMENT_TO_BODY))
>
> There is a problem with the custom EdiPreProcessor. It's job is quite
> simple. It has to wrap the incoming data in xml and to extract some
> header-data to xml-attributes. Most times, it runs well and there is no
> problem. But if you put a lot of files into the polling-directory, there are
> exceptions thrown.
> We debugged and saw that there are multiple threads of this
> processor-instance running in case of multiple files to process... and if
> one of these threads changes a global boolean attribute of the processor
> object, this attribute is changed in every other thread as well.
> I already tried some experiments with "volatile" or "synchronized" keywords,
> but all of them failed. I just want that every single thread has it's own
> attributes.
>
> Is there a way to do this from camel's point of view? Or how would you solve
> this problem?
>
> thx in advance for your help
>