Re: [ConsumerTemplate] CamelLocks deleted before file is completely processed

[Aida <ai...@gmail.com>]

Hi,

I have been investigating about this issue, because using the same
ConsumerTemplate doesn´t seem to fix it.

Just in case it can help somebody, I have seen that the
MarkerFileExclusiveReadLockStrategy#deleteLockFiles method is called once
per endpoint. So if two endpoints differ only in the "fileName" parameter
(but both files are in the same dir) this method is called and the
.camelLocks are removed.

If somebody has this problem he or she could try using a GenericFileFilter
implementation to filter by fileName. I haven´t tried it because this
approach has problems consuming files in parallel as I will comment in [1]
in some minutes. But if there is no need of concurrency, give it a try.

BR.

[1]
http://camel.465427.n5.nabble.com/ConsumerTemplate-Concurrency-problems-in-Camel-2-11-2-attached-test-td5740295.html


Claus Ibsen-2 wrote
> Hi
> 
> Yes create the template once and reuse it. See this page it also
> applies to consumer template
> http://camel.apache.org/why-does-camel-use-too-many-threads-with-producertemplate.html
> 
> On Fri, Sep 20, 2013 at 1:55 PM, Aida <

> ai.desu@

> > wrote:
>> Hi,
>>
>> I'm using the consumer template for reading files, and I'm experiencing
>> something that makes me think that I'm not using it correctly. What
>> happens
>> to me is:
>>
>> 1. I have a dir with multiple files
>> 2. The consumer template reads one of those files, creating a camelLock
>> file
>> for the file that is going to be processed
>> 3. The consumer template reads another file (same route, different
>> thread)
>> and, since the camelLock of the previous file still exists, deletes it
>> and
>> creates the camelLock for the file that is going to be processed now (the
>> previous file haven´t ended its processing and haven´t been "finished"
>> using
>> the doneUnitOfWork)
>> 4. The same for each file that is read (the camelLocks are removed and a
>> WARNING appears in the log " Deleting orphaned lock file: .../..
>> .camelLock")
>>
>> I think that maybe this is because I'm creating a new consumerTemplate
>> each
>> time I have to read something (could it be?), because I have seen that it
>> has a ConsumerCache.
>>
>> The problem is that I use some "rules" to use it (one of them creating
>> the
>> CT each time), because of some problems I have had in the past:
>>
>> - Don´t use the disconnect option for FTPs (Nullpointer, at least in
>> Camel
>> 2.9.0 that leaves threads alive, Camel post in [1])
>> - Create a new Consumer template each time I read a file (same reasons as
>> previous sentence) and stop it after use
>> - I synchronize the method that consumes an endpoint because otherwise,
>> when
>> reading files in parallel instead of reading two files, the consumer
>> reads
>> one twice
>>
>>
>> In case it can helps, I put my code here:
>>
>>         ConsumerTemplate cTemplate =
>> camelContext.createConsumerTemplate();
>>         Exchange dataExchange = null;
>>         if(timeoutInMilis == 0){
>>                 dataExchange = cTemplate.receiveNoWait(endpoint2Consume);
>>         }
>>         else{
>>                 dataExchange = cTemplate.receive(endpoint2Consume,
>> timeoutInMilis);
>>         }
>>         try {
>>                         cTemplate.stop();
>>                 } catch (Exception e) {
>>                         log ...
>>                 }
>>
>>
>> Thanks in advance
>>
>> Camel Version: 2.10.4
>> [1]
>> http://camel.465427.n5.nabble.com/FTP-ConsumerTemplate-Threads-remaining-alive-td5548338.html#a5723551
>>
>>
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/ConsumerTemplate-CamelLocks-deleted-before-file-is-completely-processed-tp5739870.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 
> -- 
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: 

> cibsen@

> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen





--
View this message in context: http://camel.465427.n5.nabble.com/ConsumerTemplate-CamelLocks-deleted-before-file-is-completely-processed-tp5739870p5740681.html
Sent from the Camel - Users mailing list archive at Nabble.com.