[jira] [Comment Edited] (NIFI-1527) Resource Claim counts not incremented on restart for FlowFiles that are swapped out

["Joseph Witt (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-1527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15156449#comment-15156449 ] 

Joseph Witt edited comment on NIFI-1527 at 2/22/16 4:25 AM:
------------------------------------------------------------

Have tested numerous scenarios and all are giving solid results.
- Sustained running tests
- Tested with content repo w/archive where nothing in it and already beyond target archive retention
- Tested with content repo w/archive where has some space to keep data
- Tested where I changed from it having some archive space to not having any while data was in repo
- Tested with backlogs including expiration on the queue
- Tested with archiving turned off
- Tested restarting NiFi halfway through a few of these other tests.

+1  having tested the current support/nifi-0.5.x branch with the latest 2KB applied.

However, in looking at the latest 37KB patch (as is on master & support/050), latest 39KB patch, latest 2KB patch I cannot quite tell that it is sufficient to push the 2KB patch now.  There appears to be some Javadoc changes and some structural/code changes this misses between the 37KB+2KB patch and the previous 39KB patch.  So [~markap14] please verify and clearly articulate the patch you mean to provide rebased against master (which presumably means it will apply to support/nifi-0.5.x)


was (Author: joewitt):
Have tested numerous scenarios and all are giving solid results.
- Sustained running tests
- Tested with content repo w/archive where nothing in it and already beyond target archive retention
- Tested with content repo w/archive where has some space to keep data
- Tested where I changed from it having some archive space to not having any while data was in repo
- Tested with backlogs including expiration on the queue
- Tested with archiving turned off
- Tested restarting NiFi halfway through a few of these other tests.

+1  Will merge to master and support/0.5.x branches.

> Resource Claim counts not incremented on restart for FlowFiles that are swapped out
> -----------------------------------------------------------------------------------
>
>                 Key: NIFI-1527
>                 URL: https://issues.apache.org/jira/browse/NIFI-1527
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Blocker
>             Fix For: 0.6.0, 0.5.1
>
>         Attachments: 0001-NIFI-1527-Ensure-that-we-increment-Claimant-Counts-f.patch, 0001-NIFI-1527-Ensure-that-we-increment-Claimant-Counts-f.patch, 0001-NIFI-1527-Ensure-that-we-increment-Claimant-Counts-f.patch, 0001-NIFI-1527-Fixed-issue-that-resulted-in-resource-clai.patch
>
>
> When NiFi starts, it tallies the count of how many FlowFiles reference each Resource Claim and then removes or archives any file in the Content Repository for which there is no claim.
> However, the claim counts are not incremented for data that is swapped out. As a result, the content repository could delete or archive data on restart that it should not. This could potentially result in data loss, if NiFi is restarted while both of the following conditions are met:
> * All FlowFiles that reference a Resource Claim are swapped out
> * Content Repository's archive is disabled, or the archival threshold is already exceeded from data outside of the Content Repository



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)