[jira] [Assigned] (CAMEL-20850) LRUCache evicts entries unexpectedly

["Nicolas Filotto (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nicolas Filotto reassigned CAMEL-20850:
---------------------------------------

    Assignee: Nicolas Filotto

> LRUCache evicts entries unexpectedly
> ------------------------------------
>
>                 Key: CAMEL-20850
>                 URL: https://issues.apache.org/jira/browse/CAMEL-20850
>             Project: Camel
>          Issue Type: Bug
>            Reporter: Timur
>            Assignee: Nicolas Filotto
>            Priority: Minor
>
> h1. Summary
> We encountered an infinite loop while downloading files using the SFTP endpoint.
> During our investigation, we discovered the {{InProgressRepository}} utilizing the {{SimpleLRUCache}}. This cache incorporates an unconventional eviction implementation that involves a changes tracking queue.
> Any attempt to modify the cache, even when putting the same element, results in an increase in the cache size, potentially causing eviction to occur even for a single element.
> The code is as follows:
> {code:java}
> Map<Object, Object> lruCache = LRUCacheFactory.newLRUCache(1, 1);
> int numberOfInserts = 0;
> lruCache.put("key", "value");
> while  (lruCache.size() > 0) {
>       lruCache.put("key", "value");
>       numberOfInserts++;
> }
> throw new IllegalStateException("We should never reach this point, we inserted the same element " + numberOfInserts + " times");
> {code}
> In the provided code, the failure occurs when the number of inserts reaches 2.
> h1. Expected Result
> Utilize the LRU Cache limited only by the maximum size passed into it, rather than by some internal implementation limits.
> h1. Proposed Solution
> Implement a standard LRUCache, say, by leveraging a {{LRUMap}} from Apache Commons.
> We may provide the PR later on.
> h1. The Background
> We need to use a dynamic SFTP route (the download parameters depend on some configuration passed from the outside). By default, this is impossible, so we will have to use {{pollEnrich}}:
> {code:java}
> .pollEnrich().simple(SFTP_DYNAMIC_URI)
>              .timeout(RETURN_CURRENT_RESULT_WITHOUT_WAIT)
> {code}
> The pollEnrich takes only one element (see {{GenericFilePollingConsumer}}). The file consumer requests from the SFTP endpoint the list of all files, then takes the first, then gets the list again and takes the second file, and so on.
> The combination of {{InProgressRepository}} and idempotent consumer is used to avoid handling the same files again.
> However, every time the SFTP endpoint lists files, the endpoints and polling consumers adjust the list in the {{InProgressRepository}}, thus affecting the {{SimpleLRUCache}}.
> The number of elements in the cache doesn't grow, however, the number of changes grows with almost n^2 speed.
> With the limit of 50k files, the SimpleLRUCache is evicted already on ~600(!) files. This leads to an infinite loop while processing the files.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)