[jira] [Comment Edited] (NIFI-9056) Content Repository Filling Up

["Steven Youtsey (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419990#comment-17419990 ] 

Steven Youtsey edited comment on NIFI-9056 at 9/24/21, 9:04 PM:
----------------------------------------------------------------

We're seeing the same thing running with a 1.12 release. We are not enabling archiving and fill up the content repo with both RocksDBFlowFileRepo and WriteAheadFlowFileRepo. 
There was a major change to the flow file repo infrastructure in this release...see NIFI-6849.


was (Author: slyouts):
We're seeing the same thing running with a 1.12 release. There was a major change to the flow file repo infrastructure in this release...see NIFI-6849.

> Content Repository Filling Up
> -----------------------------
>
>                 Key: NIFI-9056
>                 URL: https://issues.apache.org/jira/browse/NIFI-9056
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.13.2
>            Reporter: Andrew Heys
>            Priority: Major
>
> We have a clustered nifi setup that has recently been upgraded to 1.13.2 from 1.11.4. After upgrading, one of the issues we have run into is that the Content Repository will fill up to the nifi.content.repository.archive.backpressure.percentage mark and lock the processing & canvas. The only solution is to restart nifi at this point. We have the following properties set:
> nifi.content.repository.archive.backpressure.percentage=95%
> nifi.content.repository.archive.max.usage.percentage=25%
> nifi.content.repository.archive.max.retention.period=2 hours
> The max usage property seems to be completed ignored. Monitoring the nifi cluster disk % for content repository shows that it slowly fills up over time and never decreasing. If we pause the input to entire nifi flow and let all the processing clear out with 0 flowfiles remaining on the canvas for 15+ minutes, the content repository disk usage does not decrease. Currently, our only solution is to restart nifi on a daily cron schedule. After restarting the nifi, it will clear out the 80+ GB of the content repository and usage falls down to 0%. 
>  
> There seems to be an issue removing the older content claims in 1.13.2.
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)