Re: [ConsumerTemplate] Concurrency problems in Camel 2.11.2 (attached test)

[Aida <ai...@gmail.com>]

Hi,

About reading files concurrently with the consumerTemplate, if this can help
somebody in the future:

I have found out that without using an implementation of GenericFileFilter
in order to get the right file (but using the "fileName" endpoint parameter
instead) there are no problems reading concurrently (all files are read and
there are no mixtures of data).

So the problem seems to happen when the same endpoint is read concurrently.


I had this (see the link to the test of the previous message for more info):

	    	GenericFileFilterByName filter = new
GenericFileFilterByName(fileNameEncoded);	    	
	    	GenericFileEndpoint endpoint2Consume = (GenericFileEndpoint)
CamelContextHelper.getMandatoryEndpoint(camelContext, ENDPOINT_TO_CONSUME);
	    	endpoint2Consume.setFilter(filter);
	    	
	    	Exchange dataExchange = consumer.receive(endpoint2Consume, 10000);


And now if I just add the fileName option (and without using the filter), it
works as expected:

	    	Exchange dataExchange = consumer.receive(ENDPOINT_TO_CONSUME +
"?fileName=" + fileName, 10000);


Conclusion: apparently, if you want to read files with the consumerTemplate
in parallel you must use different endpoints for each file


BR,

   Aida



Aida wrote
> Hi,
> 
> I have upgraded to Camel 2.11.2 . I'm trying to use the consumerTemplate
> for read files in parallel but I have two kind of errors:
> 
> - Files that are not read (the receive method returns null when the file
> exists)
> - The content of the files is mixed (If I have the file "1.txt" with
> content "1111" and "2.txt" with content "2222" the consumerTemplate gives
> me the "2.txt" with content "11111")
> 
> 
> I have written a test class that tests this (sometimes fails because of
> the first error and others because of the second).
> 
> I attach it in case it can be of any help. The test works if I use the
> consumerTemplate synchronizing the method that uses it, but it doesn´t
> when I remove it.
> 
> What the test does:
> 
> 	- Write a specified number of files (by default 5) in the target dir. All
> files are different with different content. The file "N.txt" has
> "NNNNNNNN......" as content
> 	- Start a route that reads in parallel (using a bean that uses the
> consumerTemplate) all these files (one by one, in parallel)
> 	- The test retrieves the read exchanges and makes assertions to check
> that all files have been read and that these files have the right content.
> 	
> All what is needed is in the same class (to execute it the package
> declaration must be updated for the appropiate one).
> 
> 
> I supposed that in Camel 2.11.2, as the consumerTemplate is threadSafe I
> could do this (read different files in parallel). Maybe It's not possible
> ...
> 
> 
> Thanks in advance.
> 
>   Aida.
> 
> The test class is this: 
> ConsumerTemplateUsageTest.java
> <http://camel.465427.n5.nabble.com/file/n5740295/ConsumerTemplateUsageTest.java>  




--
View this message in context: http://camel.465427.n5.nabble.com/ConsumerTemplate-Concurrency-problems-in-Camel-2-11-2-attached-test-tp5740295p5740698.html
Sent from the Camel - Users mailing list archive at Nabble.com.