You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by shirazi <me...@farhad.eu> on 2010/04/27 10:29:18 UTC
Re: Removing entries from FileIdempotentRepository don't get
persisted
Thanks Claus,
Acutally there are some scenarios in which one may need to rerun a file
through a workflow and it makes neccessary to remove the file from the
idempotent repository. My specific problem is that upon a server restart,
the file is not picked up by the workflow because my remove was not
persisted.
I'm not sure if I can remove the entry from the repository-file before the
camel context loads it, or preferably just after removing it from the cache.
Bests,
-Farhad S.
Claus Ibsen-2 wrote:
>
> Hi
>
> On Tue, Apr 27, 2010 at 9:39 AM, shirazi <me...@farhad.eu> wrote:
>>
>> Hi,
>> It seems that removing an entry (file name) programatically from a file
>> idempotent repository has only effect on the cache and does not get
>> persisted on the file repository.
>> Do I need to do more beside calling "remove" on the repository object to
>> get
>> it persisted? Or is this a bug?
>>
>
> The file based store is just a dump of the LRUCache, which means that
> in essence removing is not spooled to disk, only adds.
> This is much easier as add is just appending to an existing file.
>
> The only situation I can see is that a file should be removed due it
> could not be successfully processed, and hence it ought to be retried
> again.
> And that your server crashes before it re processes the file. Then the
> file based repo will contain that file name. You can easily fix this
> manually by removing
> the entry from the file before starting the server.
>
>
>
>> Bests,
>> -Farhad S.
>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
--
View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374328.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Removing entries from FileIdempotentRepository don't get
persisted
Posted by shirazi <me...@farhad.eu>.
Thanks Claus,
I just created a minor bug-report
(https://issues.apache.org/activemq/browse/CAMEL-2678).
I'll have a look on extending the FileIdempotentRepository and make a
"flush" on remove.
Bests,
Farhad S.
Claus Ibsen-2 wrote:
>
> On Tue, Apr 27, 2010 at 10:29 AM, shirazi <me...@farhad.eu> wrote:
>>
>> Thanks Claus,
>> Acutally there are some scenarios in which one may need to rerun a file
>> through a workflow and it makes neccessary to remove the file from the
>> idempotent repository. My specific problem is that upon a server restart,
>> the file is not picked up by the workflow because my remove was not
>> persisted.
>> I'm not sure if I can remove the entry from the repository-file before
>> the
>> camel context loads it, or preferably just after removing it from the
>> cache.
>>
>
> You can implement your own repository.
> And/or extend the code from Camel to force a flush on remove.
> Or submit a patch to camel-core to have it flush on remove, which
> makes to do as a remove is not being executed that many times compared
> to add.
>
>
>
>
>> Bests,
>> -Farhad S.
>>
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> Hi
>>>
>>> On Tue, Apr 27, 2010 at 9:39 AM, shirazi <me...@farhad.eu> wrote:
>>>>
>>>> Hi,
>>>> It seems that removing an entry (file name) programatically from a file
>>>> idempotent repository has only effect on the cache and does not get
>>>> persisted on the file repository.
>>>> Do I need to do more beside calling "remove" on the repository object
>>>> to
>>>> get
>>>> it persisted? Or is this a bug?
>>>>
>>>
>>> The file based store is just a dump of the LRUCache, which means that
>>> in essence removing is not spooled to disk, only adds.
>>> This is much easier as add is just appending to an existing file.
>>>
>>> The only situation I can see is that a file should be removed due it
>>> could not be successfully processed, and hence it ought to be retried
>>> again.
>>> And that your server crashes before it re processes the file. Then the
>>> file based repo will contain that file name. You can easily fix this
>>> manually by removing
>>> the entry from the file before starting the server.
>>>
>>>
>>>
>>>> Bests,
>>>> -Farhad S.
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374328.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
>
--
View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374937.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Removing entries from FileIdempotentRepository don't get
persisted
Posted by Claus Ibsen <cl...@gmail.com>.
On Tue, Apr 27, 2010 at 10:29 AM, shirazi <me...@farhad.eu> wrote:
>
> Thanks Claus,
> Acutally there are some scenarios in which one may need to rerun a file
> through a workflow and it makes neccessary to remove the file from the
> idempotent repository. My specific problem is that upon a server restart,
> the file is not picked up by the workflow because my remove was not
> persisted.
> I'm not sure if I can remove the entry from the repository-file before the
> camel context loads it, or preferably just after removing it from the cache.
>
You can implement your own repository.
And/or extend the code from Camel to force a flush on remove.
Or submit a patch to camel-core to have it flush on remove, which
makes to do as a remove is not being executed that many times compared
to add.
> Bests,
> -Farhad S.
>
>
>
> Claus Ibsen-2 wrote:
>>
>> Hi
>>
>> On Tue, Apr 27, 2010 at 9:39 AM, shirazi <me...@farhad.eu> wrote:
>>>
>>> Hi,
>>> It seems that removing an entry (file name) programatically from a file
>>> idempotent repository has only effect on the cache and does not get
>>> persisted on the file repository.
>>> Do I need to do more beside calling "remove" on the repository object to
>>> get
>>> it persisted? Or is this a bug?
>>>
>>
>> The file based store is just a dump of the LRUCache, which means that
>> in essence removing is not spooled to disk, only adds.
>> This is much easier as add is just appending to an existing file.
>>
>> The only situation I can see is that a file should be removed due it
>> could not be successfully processed, and hence it ought to be retried
>> again.
>> And that your server crashes before it re processes the file. Then the
>> file based repo will contain that file name. You can easily fix this
>> manually by removing
>> the entry from the file before starting the server.
>>
>>
>>
>>> Bests,
>>> -Farhad S.
>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28373878.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://old.nabble.com/Removing-entries-from-FileIdempotentRepository-don%27t-get-persisted-tp28373878p28374328.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
--
Claus Ibsen
Apache Camel Committer
Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus