[jira] [Updated] (NIFI-32) Swap File format should include summary at end for faster NiFi system startup

["Mark Payne (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Payne updated NIFI-32:
---------------------------
    Fix Version/s:     (was: 1.2.0)
                   1.1.0

> Swap File format should include summary at end for faster NiFi system startup
> -----------------------------------------------------------------------------
>
>                 Key: NIFI-32
>                 URL: https://issues.apache.org/jira/browse/NIFI-32
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Minor
>             Fix For: 1.1.0
>
>
> When NiFi is restarted with many hundreds or thousands of Swap Files, it can take a very long time to startup.
> If when we write out the Swap Files, we append a summary of the Swap File (queue identifier, number of FlowFiles, size of FlowFiles, and ContentClaim info/count map), we can then read far less data for restart. This means we would serialize the summary to a ByteArrayOutputStream, then get the length, and then write out the summary, followed by length. On restore, we read just the last 8 bytes, and that allows us to generate the offset into the file to start reading the summary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)