You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by sian <si...@gmail.com> on 2016/10/07 14:19:32 UTC

Skip duplicate for idempotent file2 component

I am using camel over a clustered environment and want to use a readlock on
my file consumer endpoint. The only cluster safe readlock is the idempotent
readlock however this requires an idempotentRepository to be set on the file
uri.

I use an idempotent consumer within the route which moves any duplicate
files to an error folder and logs the error to a specific file.

Is there a way to use this idempotent consumer with the idempotent readlock?
Or a way to set the idempotentRepository in the file component to not skip
the duplicates so they are picked up by the idempotent consumer in the route
instead?



--
View this message in context: http://camel.465427.n5.nabble.com/Skip-duplicate-for-idempotent-file2-component-tp5788485.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Skip duplicate for idempotent file2 component

Posted by sian <si...@gmail.com>.
Problem has been solved by using two different keys for the
idempotentConsumer and idempotentRepository then setting
readLockRemoveOnCommit=true so once the readlock has been released by the
node consuming the file, it is removed from the repository. This leaves the
idempotentConsumer to record and handle any duplicates.



--
View this message in context: http://camel.465427.n5.nabble.com/Skip-duplicate-for-idempotent-file2-component-tp5788485p5788562.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Skip duplicate for idempotent file2 component

Posted by sian <si...@gmail.com>.
Hi Joe,

Thanks for the response.

I am using a database table as the repository for the idempotent consumer as
you described. My issue is that if I use the idempotentRepository file
component (as required to use the idempotent readlock) then I cannot find a
way to move the file out of the directory or log the error.



--
View this message in context: http://camel.465427.n5.nabble.com/Skip-duplicate-for-idempotent-file2-component-tp5788485p5788552.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Re: Skip duplicate for idempotent file2 component

Posted by "Joseph E. Kampf Jr." <jo...@gmail.com>.
The idempotent repository can be a database table. You define a key for the message and that prevents the file from being consumed on multiple nodes. You should still move the file out of the directory when you are done processing. 

Thanks,

Joe

> On Oct 7, 2016, at 10:19 AM, sian <si...@gmail.com> wrote:
> 
> I am using camel over a clustered environment and want to use a readlock on
> my file consumer endpoint. The only cluster safe readlock is the idempotent
> readlock however this requires an idempotentRepository to be set on the file
> uri.
> 
> I use an idempotent consumer within the route which moves any duplicate
> files to an error folder and logs the error to a specific file.
> 
> Is there a way to use this idempotent consumer with the idempotent readlock?
> Or a way to set the idempotentRepository in the file component to not skip
> the duplicates so they are picked up by the idempotent consumer in the route
> instead?
> 
> 
> 
> --
> View this message in context: http://camel.465427.n5.nabble.com/Skip-duplicate-for-idempotent-file2-component-tp5788485.html
> Sent from the Camel - Users mailing list archive at Nabble.com.