[jira] [Commented] (NIFI-9993) Processors cannot write to Content Repo: Unable to write flowfile content to content repository container default due to archive file size constraints; waiting for archive cleanup

["Mark Payne (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-9993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17532514#comment-17532514 ] 

Mark Payne commented on NIFI-9993:
----------------------------------

After analyzing, I have been able to draw the following conclusions:
 * Upon startup, the File System Repository scans for any files that are in 'archive' directories and increments an internal counter that is used for keeping track of how many files are in the archive directory.
 * When checking if backpressure should be applied, we have the following check to 'short-circuit' the rest of the logic: 
{code:java}
if (archivedFileCount.get() == 0) {
    LOG.debug("Waiting to write to container {} is not required because archivedFileCount is 0", containerName);
    return false;
}{code}
 

 * It appears that the logic we use in the initialization is faulty, as when I restart NiFi, the value is set to 0. As a result, the archivedFileCount counter is not incremented for each file in the 'archive' on startup. So when those files get cleaned up, the counter is decremented too much and can become negative. As a result, the check for (archivedFileCount.get() == 0) is false, and we apply backpressure.
 * This will only happen if there are no files in the archive and the storage space that's used up in the content repository is greater than the threshold configured in nifi.properties via the following properties: 
{code:java}
nifi.content.repository.archive.max.usage.percentage
nifi.content.repository.archive.backpressure.percentage{code}

> Processors cannot write to Content Repo: Unable to write flowfile content to content repository container default due to archive file size constraints; waiting for archive cleanup
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-9993
>                 URL: https://issues.apache.org/jira/browse/NIFI-9993
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>
> Sometimes, we can see NiFi log the following log message over and over:
> {code:java}
> 2022-05-05 10:47:03,267 WARN [Timer-Driven Process Thread-142] o.a.n.c.repository.FileSystemRepository Unable to write to container default due to archive file size constraints; waiting for archive cleanup {code}
> However, we can also see that there's nothing in the archive to cleanup, based on the fact that we also continually see this log message:
> {code:java}
> 2022-05-05 10:01:02,515 INFO [Cleanup Archive for default] o.a.n.c.repository.FileSystemRepository Successfully deleted 0 files (0 bytes) from archive {code}
> If there are any files in the archive directory, we should see them getting cleaned up. The fact that we constantly see "Successfully deleted 0 files (0 bytes) from archive" implies there is nothing in the archive.
> But if there is nothing in the archive, then we should not be seeing any notices about "Unable to write to container default due to archive file size constraints."
> This results in processors getting blocked for indefinite periods of time.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)