You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by shirazi <me...@farhad.eu> on 2010/04/27 09:39:53 UTC
Removing entries from FileIdempotentRepository don't get persisted
Hi,
It seems that removing an entry (file name) programatically from a file
idempotent repository has only effect on the cache and does not get
persisted on the file repository.
Do I need to do more beside calling "remove" on the repository object to get
it persisted? Or is this a bug?
Bests,
-Farhad S.
--
View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Removing entries from FileIdempotentRepository don't get
persisted
Posted by shirazi <me...@farhad.eu>.
Thanks Claus,
I just created a minor bug-report
(https://issues.apache.org/activemq/browse/CAMEL-2678).
I'll have a look on extending the FileIdempotentRepository and make a
"flush" on remove.
Bests,
Farhad S.
Claus Ibsen-2 wrote:
>
> On Tue, Apr 27, 2010 at 10:29 AM, shirazi <me...@farhad.eu> wrote:
>>
>> Thanks Claus,
>> Acutally there are some scenarios in which one may need to rerun a file
>> through a workflow and it makes neccessary to remove the file from the
>> idempotent repository. My specific problem is that upon a server restart,
>> the file is not picked up by the workflow because my remove was not
>> persisted.
>> I'm not sure if I can remove the entry from the repository-file before
>> the
>> camel context loads it, or preferably just after removing it from the
>> cache.
>>
>
> You can implement your own repository.
> And/or extend the code from Camel to force a flush on remove.
> Or submit a patch to camel-core to have it flush on remove, which
> makes to do as a remove is not being executed that many times compared
> to add.
>
>
>
>
>> Bests,
>> -Farhad S.
>>
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> Hi
>>>
>>> On Tue, Apr 27, 2010 at 9:39 AM, shirazi <me...@farhad.eu> wrote:
>>>>
>>>> Hi,
>>>> It seems that removing an entry (file name) programatically from a file
>>>> idempotent repository has only effect on the cache and does not get
>>>> persisted on the file repository.
>>>> Do I need to do more beside calling "remove" on the repository object
>>>> to
>>>> get
>>>> it persisted? Or is this a bug?
>>>>
>>>
>>> The file based store is just a dump of the LRUCache, which means that
>>> in essence removing is not spooled to disk, only adds.
>>> This is much easier as add is just appending to an existing file.
>>>
>>> The only situation I can see is that a file should be removed due it
>>> could not be successfully processed, and hence it ought to be retried
>>> again.
>>> And that your server crashes before it re processes the file. Then the
>>> file based repo will contain that file name. You can easily fix this
>>> manually by removing
>>> the entry from the file before starting the server.
>>>
>>>
>>>
>>>> Bests,
>>>> -Farhad S.
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374328.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
--
View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374937.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Removing entries from FileIdempotentRepository don't get
persisted
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Apr 27, 2010 at 10:29 AM, shirazi <me...@farhad.eu> wrote:
>
> Thanks Claus,
> Acutally there are some scenarios in which one may need to rerun a file
> through a workflow and it makes neccessary to remove the file from the
> idempotent repository. My specific problem is that upon a server restart,
> the file is not picked up by the workflow because my remove was not
> persisted.
> I'm not sure if I can remove the entry from the repository-file before the
> camel context loads it, or preferably just after removing it from the cache.
>
You can implement your own repository.
And/or extend the code from Camel to force a flush on remove.
Or submit a patch to camel-core to have it flush on remove, which
makes to do as a remove is not being executed that many times compared
to add.
> Bests,
> -Farhad S.
>
>
>
> Claus Ibsen-2 wrote:
>>
>> Hi
>>
>> On Tue, Apr 27, 2010 at 9:39 AM, shirazi <me...@farhad.eu> wrote:
>>>
>>> Hi,
>>> It seems that removing an entry (file name) programatically from a file
>>> idempotent repository has only effect on the cache and does not get
>>> persisted on the file repository.
>>> Do I need to do more beside calling "remove" on the repository object to
>>> get
>>> it persisted? Or is this a bug?
>>>
>>
>> The file based store is just a dump of the LRUCache, which means that
>> in essence removing is not spooled to disk, only adds.
>> This is much easier as add is just appending to an existing file.
>>
>> The only situation I can see is that a file should be removed due it
>> could not be successfully processed, and hence it ought to be retried
>> again.
>> And that your server crashes before it re processes the file. Then the
>> file based repo will contain that file name. You can easily fix this
>> manually by removing
>> the entry from the file before starting the server.
>>
>>
>>
>>> Bests,
>>> -Farhad S.
>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374328.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Re: Removing entries from FileIdempotentRepository don't get
persisted
Posted by shirazi <me...@farhad.eu>.
Thanks Claus,
Acutally there are some scenarios in which one may need to rerun a file
through a workflow and it makes neccessary to remove the file from the
idempotent repository. My specific problem is that upon a server restart,
the file is not picked up by the workflow because my remove was not
persisted.
I'm not sure if I can remove the entry from the repository-file before the
camel context loads it, or preferably just after removing it from the cache.
Bests,
-Farhad S.
Claus Ibsen-2 wrote:
>
> Hi
>
> On Tue, Apr 27, 2010 at 9:39 AM, shirazi <me...@farhad.eu> wrote:
>>
>> Hi,
>> It seems that removing an entry (file name) programatically from a file
>> idempotent repository has only effect on the cache and does not get
>> persisted on the file repository.
>> Do I need to do more beside calling "remove" on the repository object to
>> get
>> it persisted? Or is this a bug?
>>
>
> The file based store is just a dump of the LRUCache, which means that
> in essence removing is not spooled to disk, only adds.
> This is much easier as add is just appending to an existing file.
>
> The only situation I can see is that a file should be removed due it
> could not be successfully processed, and hence it ought to be retried
> again.
> And that your server crashes before it re processes the file. Then the
> file based repo will contain that file name. You can easily fix this
> manually by removing
> the entry from the file before starting the server.
>
>
>
>> Bests,
>> -Farhad S.
>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
--
View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374328.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Removing entries from FileIdempotentRepository don't get
persisted
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
On Tue, Apr 27, 2010 at 9:39 AM, shirazi <me...@farhad.eu> wrote:
>
> Hi,
> It seems that removing an entry (file name) programatically from a file
> idempotent repository has only effect on the cache and does not get
> persisted on the file repository.
> Do I need to do more beside calling "remove" on the repository object to get
> it persisted? Or is this a bug?
>
The file based store is just a dump of the LRUCache, which means that
in essence removing is not spooled to disk, only adds.
This is much easier as add is just appending to an existing file.
The only situation I can see is that a file should be removed due it
could not be successfully processed, and hence it ought to be retried
again.
And that your server crashes before it re processes the file. Then the
file based repo will contain that file name. You can easily fix this
manually by removing
the entry from the file before starting the server.
> Bests,
> -Farhad S.
>
>
> --
> View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus