You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Joseph Gresock (JIRA)" <ji...@apache.org> on 2017/03/13 15:29:41 UTC
[jira] [Assigned] (NIFI-3582) DistributedMapCacheServer does not
evict persistent records
[ https://issues.apache.org/jira/browse/NIFI-3582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joseph Gresock reassigned NIFI-3582:
------------------------------------
Assignee: Joseph Gresock (was: Mark Payne)
> DistributedMapCacheServer does not evict persistent records
> -----------------------------------------------------------
>
> Key: NIFI-3582
> URL: https://issues.apache.org/jira/browse/NIFI-3582
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.1.1
> Reporter: Joseph Gresock
> Assignee: Joseph Gresock
>
> The PersistentMapCache does not appear to be evicting records when used by the DistributedMapCacheServer, at least in the FIFO eviction mode.
> To replicate this behavior in NiFi 1.1.1, add the following to your flow:
> *Services*
> * DistributedMapCacheServer, port 4557, maximum cache entries = 10,000, FIFO eviction, persistence directory specified
> * DistributedMapCacheClientService, point to the same host and port
> *Flow*
> GenerateFlowFile (randomize 1K binary files in batches of 10, schedule 10 threads) -> HashContent (md5) into hash.value -> DetectDuplicate with identifier = $\{hash.value\}, description = ., no age off, select your cache client, cache identifier = true
> This should cause the snapshot file in your cache's persistence directory to exceed 100,000 keys pretty quickly, and as far as I can tell, it never goes back down.
> On our production system, we have the cache server configured for 100,000 max entries with FIFO eviction, and we recently saw this log statement showing the writeahead log with over 4 million entries:
> {code}
> nifi-app.log:2017-03-09 15:03:00,670 INFO [Distributed Cache Server Communications Thread: ac907dec-49a4-439e-99f5-1558f2358d87] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@40569408
> checkpointed with 4262902 Records and 0 Swap Files in 256302 milliseconds (Stop-the-world time = 1378 milliseconds, Clear Edit Logs time = 19 millis), max Transaction ID 4263237
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)