You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Krzysztof Hołdanowicz <ho...@gmail.com> on 2017/11/28 07:55:13 UTC
File idempotent store problem
Hi all,
I recently noticed that there is wrong entry order in file using
FileIdempotentRepository implementation.
The effect is that instead of having order like:
file1.txt.20171123
file2.txt.20171123
file1.txt.20171124
file3.txt.20171125
file2.txt.20171126
we have:
file1.txt.20171123
file1.txt.20171124
file2.txt.20171123
file2.txt.20171126
file3.txt.20171125
where date extension represents order in which particular file was consumed
by the idempotent file consumer.
As a consequence instead of initializing memory cache with newest values,
it is initialized (probably) based on hash function from truncStore method
and we consume same file more than once:
protected void trunkStore() {
LOG.info("Trunking idempotent filestore: {}", fileStore);
FileOutputStream fos = null;
try {
fos = new FileOutputStream(fileStore);
for (String key : *cache.keySet()*) {
fos.write(key.getBytes());
fos.write(STORE_DELIMITER.getBytes());
}
} catch (IOException e) {
throw ObjectHelper.wrapRuntimeCamelException(e);
} finally {
IOHelper.close(fos, "Trunking file idempotent repository", LOG);
}
}
LRUCache:
@Override
public Set<K> keySet() {
return map.keySet();
}
where previously it was:
@Override
public Set<K> keySet() {
return map.*ascendingKeySet*();
}
Regards
Kris
--
Pozdrawiam
Krzysztof Hołdanowicz
Re: File idempotent store problem
Posted by Onder SEZGIN <on...@gmail.com>.
There are points i agree/disagree.
This can stay as the limitation imho. Because using cache as fifo sounds a
bit advance care of idempotent repository feature and it would require
taking all the burdens of such implementation for a single minor feature by
diving into caffeine. This would be too much to implement in camel-core
ootb. in case you require such an advanced feature, idempotent repository
is a pluggable feature and it can be implemented with fifo based custom
cache as my first sight and imho.
On Wed, 3 Jan 2018 at 22:51, Krzysztof Hołdanowicz <ho...@gmail.com>
wrote:
> Hi,
>
> regarding CAMEL-12058 I don't know if you are aware of all consequences of
> wrong order in the idempotent file store.
> The wrong order in the file is not the problem itself as long as elemens
> are added and eviceted on runtime, because caffeine provides an api for
> ordering like:
>
> - @Override public Map<K, V> coldest(int limit)
> - @Override public Map<K, V> hottest(int limit)
> - @Override public Map<K, V> oldest(int limit),
> - @Override public Map<K, V> youngest(int limit) )
>
> however the consequences of this appears after RESTART. The memory cache
> does not contain the proper entries (in case of reaching the max limit
> size) because it does not load elements from hottest to coldest but with
> the file entries order hence some of the files are consumed multiple times.
> It means that current implementation of file idempotent store is not usable
> at all anymore. Ignoring the issue (CAMEL-12058) means that Camel library
> does not provide any implementation of idempotent file store as the current
> behaviour is completely wrong and causes consuming multiple times the same
> file after reaching max size limit and after restarting application.
>
>
> Regards
> Kris
>
> sob., 2 gru 2017 o 15:14 użytkownik Krzysztof Hołdanowicz <
> holdanowicz@gmail.com> napisał:
>
> > I don't know if I understood you correctly.
> > Instead of looping via cache.keySet() you mean looping via:
> > Map.Entry<String, Object> entry : cache.entrySet() or cache.foreach((k,
> v)
> > -> {...})?
> >
> > If yes what is the difference? Isn't it the same unordered collection?
> > If Caffeince returns unordered collection how we can get ordered entries?
> > Isn't it related with:
> > https://github.com/ben-manes/caffeine/issues/86
> >
> > Shouldn't we use a kind of LinkedHashMap implementation?
> >
> > Regards
> > Kris
> >
> > wt., 28 lis 2017 o 18:40 użytkownik Claus Ibsen-2 [via Camel] <
> > ml+s465427n5815878h97@n5.nabble.com> napisał:
> >
> >> Hi
> >>
> >> Ah well spotted.
> >> I think we should for loop via Map.Entry (or (k,v) via lambda) which I
> >> think will be in the correct order.
> >>
> >> You are welcome to log a JIRA. And also work on unit test and patch.
> >> http://camel.apache.org/contributing
> >>
> >> On Tue, Nov 28, 2017 at 8:55 AM, Krzysztof Hołdanowicz
> >> <[hidden email] <http://
> /user/SendEmail.jtp?type=node&node=5815878&i=0>>
> >> wrote:
> >>
> >> > Hi all,
> >> >
> >> > I recently noticed that there is wrong entry order in file using
> >> > FileIdempotentRepository implementation.
> >> > The effect is that instead of having order like:
> >> >
> >> > file1.txt.20171123
> >> > file2.txt.20171123
> >> > file1.txt.20171124
> >> > file3.txt.20171125
> >> > file2.txt.20171126
> >> >
> >> > we have:
> >> >
> >> > file1.txt.20171123
> >> > file1.txt.20171124
> >> > file2.txt.20171123
> >> > file2.txt.20171126
> >> > file3.txt.20171125
> >> >
> >> > where date extension represents order in which particular file was
> >> consumed
> >> > by the idempotent file consumer.
> >> > As a consequence instead of initializing memory cache with newest
> >> values,
> >> > it is initialized (probably) based on hash function from truncStore
> >> method
> >> > and we consume same file more than once:
> >> >
> >> > protected void trunkStore() {
> >> > LOG.info("Trunking idempotent filestore: {}", fileStore);
> >> > FileOutputStream fos = null;
> >> > try {
> >> > fos = new FileOutputStream(fileStore);
> >> > for (String key : *cache.keySet()*) {
> >> > fos.write(key.getBytes());
> >> > fos.write(STORE_DELIMITER.getBytes());
> >> > }
> >> > } catch (IOException e) {
> >> > throw ObjectHelper.wrapRuntimeCamelException(e);
> >> > } finally {
> >> > IOHelper.close(fos, "Trunking file idempotent repository",
> >> LOG);
> >> > }
> >> > }
> >> >
> >> > LRUCache:
> >> >
> >> > @Override
> >> > public Set<K> keySet() {
> >> > return map.keySet();
> >> > }
> >> >
> >> > where previously it was:
> >> >
> >> > @Override
> >> > public Set<K> keySet() {
> >> > return map.*ascendingKeySet*();
> >> > }
> >> >
> >> > Regards
> >> > Kris
> >> > --
> >> >
> >> > Pozdrawiam
> >> >
> >> > Krzysztof Hołdanowicz
> >>
> >>
> >>
> >> --
> >> Claus Ibsen
> >> -----------------
> >> http://davsclaus.com @davsclaus
> >> Camel in Action 2: https://www.manning.com/ibsen2
> >>
> >>
> >> ------------------------------
> >> If you reply to this email, your message will be added to the discussion
> >> below:
> >>
> >>
> http://camel.465427.n5.nabble.com/File-idempotent-store-problem-since-2-20-tp5815868p5815878.html
> >> To unsubscribe from Camel - Users, click here
> >> <
> http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=aG9sZGFub3dpY3pAZ21haWwuY29tfDQ2NTQyOHwtMTk2MTQxMTc0NQ==
> >
> >> .
> >> NAML
> >> <
> http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >>
> > --
> >
> > Pozdrawiam
> >
> > Krzysztof Hołdanowicz
> >
>
>
> --
>
> Pozdrawiam
>
> Krzysztof Hołdanowicz
>
Re: File idempotent store problem
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Yeah I can see the point that its better to have the default
implementation work better. I have reworked the implementation so the
file store and cache are independent and the cache is only used for
quick lookup. Also the default sizes of the file store has been
increased to allow storing more entries, and it now "free space" by
deleting the oldest 1000 entries (all configurable).
On Wed, Jan 3, 2018 at 8:51 PM, Krzysztof Hołdanowicz
<ho...@gmail.com> wrote:
> Hi,
>
> regarding CAMEL-12058 I don't know if you are aware of all consequences of
> wrong order in the idempotent file store.
> The wrong order in the file is not the problem itself as long as elemens
> are added and eviceted on runtime, because caffeine provides an api for
> ordering like:
>
> - @Override public Map<K, V> coldest(int limit)
> - @Override public Map<K, V> hottest(int limit)
> - @Override public Map<K, V> oldest(int limit),
> - @Override public Map<K, V> youngest(int limit) )
>
> however the consequences of this appears after RESTART. The memory cache
> does not contain the proper entries (in case of reaching the max limit
> size) because it does not load elements from hottest to coldest but with
> the file entries order hence some of the files are consumed multiple times.
> It means that current implementation of file idempotent store is not usable
> at all anymore. Ignoring the issue (CAMEL-12058) means that Camel library
> does not provide any implementation of idempotent file store as the current
> behaviour is completely wrong and causes consuming multiple times the same
> file after reaching max size limit and after restarting application.
>
>
> Regards
> Kris
>
> sob., 2 gru 2017 o 15:14 użytkownik Krzysztof Hołdanowicz <
> holdanowicz@gmail.com> napisał:
>
>> I don't know if I understood you correctly.
>> Instead of looping via cache.keySet() you mean looping via:
>> Map.Entry<String, Object> entry : cache.entrySet() or cache.foreach((k, v)
>> -> {...})?
>>
>> If yes what is the difference? Isn't it the same unordered collection?
>> If Caffeince returns unordered collection how we can get ordered entries?
>> Isn't it related with:
>> https://github.com/ben-manes/caffeine/issues/86
>>
>> Shouldn't we use a kind of LinkedHashMap implementation?
>>
>> Regards
>> Kris
>>
>> wt., 28 lis 2017 o 18:40 użytkownik Claus Ibsen-2 [via Camel] <
>> ml+s465427n5815878h97@n5.nabble.com> napisał:
>>
>>> Hi
>>>
>>> Ah well spotted.
>>> I think we should for loop via Map.Entry (or (k,v) via lambda) which I
>>> think will be in the correct order.
>>>
>>> You are welcome to log a JIRA. And also work on unit test and patch.
>>> http://camel.apache.org/contributing
>>>
>>> On Tue, Nov 28, 2017 at 8:55 AM, Krzysztof Hołdanowicz
>>> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=5815878&i=0>>
>>> wrote:
>>>
>>> > Hi all,
>>> >
>>> > I recently noticed that there is wrong entry order in file using
>>> > FileIdempotentRepository implementation.
>>> > The effect is that instead of having order like:
>>> >
>>> > file1.txt.20171123
>>> > file2.txt.20171123
>>> > file1.txt.20171124
>>> > file3.txt.20171125
>>> > file2.txt.20171126
>>> >
>>> > we have:
>>> >
>>> > file1.txt.20171123
>>> > file1.txt.20171124
>>> > file2.txt.20171123
>>> > file2.txt.20171126
>>> > file3.txt.20171125
>>> >
>>> > where date extension represents order in which particular file was
>>> consumed
>>> > by the idempotent file consumer.
>>> > As a consequence instead of initializing memory cache with newest
>>> values,
>>> > it is initialized (probably) based on hash function from truncStore
>>> method
>>> > and we consume same file more than once:
>>> >
>>> > protected void trunkStore() {
>>> > LOG.info("Trunking idempotent filestore: {}", fileStore);
>>> > FileOutputStream fos = null;
>>> > try {
>>> > fos = new FileOutputStream(fileStore);
>>> > for (String key : *cache.keySet()*) {
>>> > fos.write(key.getBytes());
>>> > fos.write(STORE_DELIMITER.getBytes());
>>> > }
>>> > } catch (IOException e) {
>>> > throw ObjectHelper.wrapRuntimeCamelException(e);
>>> > } finally {
>>> > IOHelper.close(fos, "Trunking file idempotent repository",
>>> LOG);
>>> > }
>>> > }
>>> >
>>> > LRUCache:
>>> >
>>> > @Override
>>> > public Set<K> keySet() {
>>> > return map.keySet();
>>> > }
>>> >
>>> > where previously it was:
>>> >
>>> > @Override
>>> > public Set<K> keySet() {
>>> > return map.*ascendingKeySet*();
>>> > }
>>> >
>>> > Regards
>>> > Kris
>>> > --
>>> >
>>> > Pozdrawiam
>>> >
>>> > Krzysztof Hołdanowicz
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> -----------------
>>> http://davsclaus.com @davsclaus
>>> Camel in Action 2: https://www.manning.com/ibsen2
>>>
>>>
>>> ------------------------------
>>> If you reply to this email, your message will be added to the discussion
>>> below:
>>>
>>> http://camel.465427.n5.nabble.com/File-idempotent-store-problem-since-2-20-tp5815868p5815878.html
>>> To unsubscribe from Camel - Users, click here
>>> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=aG9sZGFub3dpY3pAZ21haWwuY29tfDQ2NTQyOHwtMTk2MTQxMTc0NQ==>
>>> .
>>> NAML
>>> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>> --
>>
>> Pozdrawiam
>>
>> Krzysztof Hołdanowicz
>>
>
>
> --
>
> Pozdrawiam
>
> Krzysztof Hołdanowicz
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: File idempotent store problem
Posted by Krzysztof Hołdanowicz <ho...@gmail.com>.
Hi,
regarding CAMEL-12058 I don't know if you are aware of all consequences of
wrong order in the idempotent file store.
The wrong order in the file is not the problem itself as long as elemens
are added and eviceted on runtime, because caffeine provides an api for
ordering like:
- @Override public Map<K, V> coldest(int limit)
- @Override public Map<K, V> hottest(int limit)
- @Override public Map<K, V> oldest(int limit),
- @Override public Map<K, V> youngest(int limit) )
however the consequences of this appears after RESTART. The memory cache
does not contain the proper entries (in case of reaching the max limit
size) because it does not load elements from hottest to coldest but with
the file entries order hence some of the files are consumed multiple times.
It means that current implementation of file idempotent store is not usable
at all anymore. Ignoring the issue (CAMEL-12058) means that Camel library
does not provide any implementation of idempotent file store as the current
behaviour is completely wrong and causes consuming multiple times the same
file after reaching max size limit and after restarting application.
Regards
Kris
sob., 2 gru 2017 o 15:14 użytkownik Krzysztof Hołdanowicz <
holdanowicz@gmail.com> napisał:
> I don't know if I understood you correctly.
> Instead of looping via cache.keySet() you mean looping via:
> Map.Entry<String, Object> entry : cache.entrySet() or cache.foreach((k, v)
> -> {...})?
>
> If yes what is the difference? Isn't it the same unordered collection?
> If Caffeince returns unordered collection how we can get ordered entries?
> Isn't it related with:
> https://github.com/ben-manes/caffeine/issues/86
>
> Shouldn't we use a kind of LinkedHashMap implementation?
>
> Regards
> Kris
>
> wt., 28 lis 2017 o 18:40 użytkownik Claus Ibsen-2 [via Camel] <
> ml+s465427n5815878h97@n5.nabble.com> napisał:
>
>> Hi
>>
>> Ah well spotted.
>> I think we should for loop via Map.Entry (or (k,v) via lambda) which I
>> think will be in the correct order.
>>
>> You are welcome to log a JIRA. And also work on unit test and patch.
>> http://camel.apache.org/contributing
>>
>> On Tue, Nov 28, 2017 at 8:55 AM, Krzysztof Hołdanowicz
>> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=5815878&i=0>>
>> wrote:
>>
>> > Hi all,
>> >
>> > I recently noticed that there is wrong entry order in file using
>> > FileIdempotentRepository implementation.
>> > The effect is that instead of having order like:
>> >
>> > file1.txt.20171123
>> > file2.txt.20171123
>> > file1.txt.20171124
>> > file3.txt.20171125
>> > file2.txt.20171126
>> >
>> > we have:
>> >
>> > file1.txt.20171123
>> > file1.txt.20171124
>> > file2.txt.20171123
>> > file2.txt.20171126
>> > file3.txt.20171125
>> >
>> > where date extension represents order in which particular file was
>> consumed
>> > by the idempotent file consumer.
>> > As a consequence instead of initializing memory cache with newest
>> values,
>> > it is initialized (probably) based on hash function from truncStore
>> method
>> > and we consume same file more than once:
>> >
>> > protected void trunkStore() {
>> > LOG.info("Trunking idempotent filestore: {}", fileStore);
>> > FileOutputStream fos = null;
>> > try {
>> > fos = new FileOutputStream(fileStore);
>> > for (String key : *cache.keySet()*) {
>> > fos.write(key.getBytes());
>> > fos.write(STORE_DELIMITER.getBytes());
>> > }
>> > } catch (IOException e) {
>> > throw ObjectHelper.wrapRuntimeCamelException(e);
>> > } finally {
>> > IOHelper.close(fos, "Trunking file idempotent repository",
>> LOG);
>> > }
>> > }
>> >
>> > LRUCache:
>> >
>> > @Override
>> > public Set<K> keySet() {
>> > return map.keySet();
>> > }
>> >
>> > where previously it was:
>> >
>> > @Override
>> > public Set<K> keySet() {
>> > return map.*ascendingKeySet*();
>> > }
>> >
>> > Regards
>> > Kris
>> > --
>> >
>> > Pozdrawiam
>> >
>> > Krzysztof Hołdanowicz
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> http://davsclaus.com @davsclaus
>> Camel in Action 2: https://www.manning.com/ibsen2
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://camel.465427.n5.nabble.com/File-idempotent-store-problem-since-2-20-tp5815868p5815878.html
>> To unsubscribe from Camel - Users, click here
>> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=aG9sZGFub3dpY3pAZ21haWwuY29tfDQ2NTQyOHwtMTk2MTQxMTc0NQ==>
>> .
>> NAML
>> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
> --
>
> Pozdrawiam
>
> Krzysztof Hołdanowicz
>
--
Pozdrawiam
Krzysztof Hołdanowicz
Re: File idempotent store problem
Posted by Krzysztof Hołdanowicz <ho...@gmail.com>.
I don't know if I understood you correctly.
Instead of looping via cache.keySet() you mean looping via:
Map.Entry<String, Object> entry : cache.entrySet() or cache.foreach((k, v)
-> {...})?
If yes what is the difference? Isn't it the same unordered collection?
If Caffeince returns unordered collection how we can get ordered entries?
Isn't it related with:
https://github.com/ben-manes/caffeine/issues/86
Shouldn't we use a kind of LinkedHashMap implementation?
Regards
Kris
wt., 28 lis 2017 o 18:40 użytkownik Claus Ibsen-2 [via Camel] <
ml+s465427n5815878h97@n5.nabble.com> napisał:
> Hi
>
> Ah well spotted.
> I think we should for loop via Map.Entry (or (k,v) via lambda) which I
> think will be in the correct order.
>
> You are welcome to log a JIRA. And also work on unit test and patch.
> http://camel.apache.org/contributing
>
> On Tue, Nov 28, 2017 at 8:55 AM, Krzysztof Hołdanowicz
> <[hidden email] <http:///user/SendEmail.jtp?type=node&node=5815878&i=0>>
> wrote:
>
> > Hi all,
> >
> > I recently noticed that there is wrong entry order in file using
> > FileIdempotentRepository implementation.
> > The effect is that instead of having order like:
> >
> > file1.txt.20171123
> > file2.txt.20171123
> > file1.txt.20171124
> > file3.txt.20171125
> > file2.txt.20171126
> >
> > we have:
> >
> > file1.txt.20171123
> > file1.txt.20171124
> > file2.txt.20171123
> > file2.txt.20171126
> > file3.txt.20171125
> >
> > where date extension represents order in which particular file was
> consumed
> > by the idempotent file consumer.
> > As a consequence instead of initializing memory cache with newest
> values,
> > it is initialized (probably) based on hash function from truncStore
> method
> > and we consume same file more than once:
> >
> > protected void trunkStore() {
> > LOG.info("Trunking idempotent filestore: {}", fileStore);
> > FileOutputStream fos = null;
> > try {
> > fos = new FileOutputStream(fileStore);
> > for (String key : *cache.keySet()*) {
> > fos.write(key.getBytes());
> > fos.write(STORE_DELIMITER.getBytes());
> > }
> > } catch (IOException e) {
> > throw ObjectHelper.wrapRuntimeCamelException(e);
> > } finally {
> > IOHelper.close(fos, "Trunking file idempotent repository",
> LOG);
> > }
> > }
> >
> > LRUCache:
> >
> > @Override
> > public Set<K> keySet() {
> > return map.keySet();
> > }
> >
> > where previously it was:
> >
> > @Override
> > public Set<K> keySet() {
> > return map.*ascendingKeySet*();
> > }
> >
> > Regards
> > Kris
> > --
> >
> > Pozdrawiam
> >
> > Krzysztof Hołdanowicz
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://camel.465427.n5.nabble.com/File-idempotent-store-problem-since-2-20-tp5815868p5815878.html
> To unsubscribe from Camel - Users, click here
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=aG9sZGFub3dpY3pAZ21haWwuY29tfDQ2NTQyOHwtMTk2MTQxMTc0NQ==>
> .
> NAML
> <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
Pozdrawiam
Krzysztof Hołdanowicz
Re: File idempotent store problem
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Ah well spotted.
I think we should for loop via Map.Entry (or (k,v) via lambda) which I
think will be in the correct order.
You are welcome to log a JIRA. And also work on unit test and patch.
http://camel.apache.org/contributing
On Tue, Nov 28, 2017 at 8:55 AM, Krzysztof Hołdanowicz
<ho...@gmail.com> wrote:
> Hi all,
>
> I recently noticed that there is wrong entry order in file using
> FileIdempotentRepository implementation.
> The effect is that instead of having order like:
>
> file1.txt.20171123
> file2.txt.20171123
> file1.txt.20171124
> file3.txt.20171125
> file2.txt.20171126
>
> we have:
>
> file1.txt.20171123
> file1.txt.20171124
> file2.txt.20171123
> file2.txt.20171126
> file3.txt.20171125
>
> where date extension represents order in which particular file was consumed
> by the idempotent file consumer.
> As a consequence instead of initializing memory cache with newest values,
> it is initialized (probably) based on hash function from truncStore method
> and we consume same file more than once:
>
> protected void trunkStore() {
> LOG.info("Trunking idempotent filestore: {}", fileStore);
> FileOutputStream fos = null;
> try {
> fos = new FileOutputStream(fileStore);
> for (String key : *cache.keySet()*) {
> fos.write(key.getBytes());
> fos.write(STORE_DELIMITER.getBytes());
> }
> } catch (IOException e) {
> throw ObjectHelper.wrapRuntimeCamelException(e);
> } finally {
> IOHelper.close(fos, "Trunking file idempotent repository", LOG);
> }
> }
>
> LRUCache:
>
> @Override
> public Set<K> keySet() {
> return map.keySet();
> }
>
> where previously it was:
>
> @Override
> public Set<K> keySet() {
> return map.*ascendingKeySet*();
> }
>
> Regards
> Kris
> --
>
> Pozdrawiam
>
> Krzysztof Hołdanowicz
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2