[jira] [Resolved] (MINIFICPP-1127) Provenance repo performance should be improved

["Arpad Boda (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MINIFICPP-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arpad Boda resolved MINIFICPP-1127.
-----------------------------------
    Resolution: Fixed

> Provenance repo performance should be improved
> ----------------------------------------------
>
>                 Key: MINIFICPP-1127
>                 URL: https://issues.apache.org/jira/browse/MINIFICPP-1127
>             Project: Apache NiFi MiNiFi C++
>          Issue Type: Bug
>    Affects Versions: 0.7.0
>            Reporter: Arpad Boda
>            Assignee: Arpad Boda
>            Priority: Major
>             Fix For: 0.8.0
>
>          Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> There are several major issues in the current implementation of Provenance repo:
> * Repo size is not maintained consistently, repo never gets full, new elements are inserted without any restriction. Filling the underlying disc is just a question of time. 
> * There is a logic to remove older records, but that's never triggered due to improper size handling. (The counter never reaches the threshold limit)
> * Even if it would be triggered it is very inefficient, resulting in cpu load spikes. (The logic queries each key one by one just to maintain the size of data deleted based on the values received)
> * Write buffers are not configured properly, the result is sawtooth appearing in memory consumption (up to 64MB). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)