[jira] [Assigned] (NIFI-10273) MergeContent tar compression fails on FlowFiles > 8.5GB

["David Handermann (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-10273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Handermann reassigned NIFI-10273:
---------------------------------------

    Assignee: Michael Hogue

> MergeContent tar compression fails on FlowFiles > 8.5GB
> -------------------------------------------------------
>
>                 Key: NIFI-10273
>                 URL: https://issues.apache.org/jira/browse/NIFI-10273
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.16.3
>            Reporter: Michael Hogue
>            Assignee: Michael Hogue
>            Priority: Minor
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> When attempting to tar compress FlowFiles > 8.5GB, MergeContent fails with the following error:
> {noformat}
>  java.lang.RuntimeException: entry size '<FlowFileSize>' is too big (> 8589934591){noformat}
> This is because apache commons-compress library has a hard maximum :
> [https://commons.apache.org/proper/commons-compress/apidocs/src-html/org/apache/commons/compress/archivers/tar/TarConstants.html#line.94]
> 077777777777 octal bytes is 8589934591 decimal bytes (~8.5GB).
> This can be addressed by configuring a BigNumberMode on the {{{}TarArchiveOutputStream{}}}:
> [https://commons.apache.org/proper/commons-compress/javadocs/api-1.10/org/apache/commons/compress/archivers/tar/TarArchiveOutputStream.html#setBigNumberMode(int)]
> {{Here's an example of another project doing this:}}
> {{[https://github.com/DataConservancy/dcs-packaging-tool/pull/36/files]}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)