[jira] [Commented] (NIFI-6964) Use compression level for xz-lzma2 format of the CompressContent processor

["Joe Witt (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001747#comment-17001747 ] 

Joe Witt commented on NIFI-6964:
--------------------------------

Nice contrib!  It is a bit easier/better for us to handle things that come in via github pull requests.  If you're up for giving that a go please do.  It makes tracking a little easier, gives us better chain of contrib/review, and it kicks off automated testing.  Without those things it slows down/adds to review effort and thus can end up sitting. This is ok too but let me know if you can try the PR route.

One thing that isn't great about the code as-is would be that it technically changes the behavior from the default you noted of level 6 to the specified default of 1.  That said, we did not specify the level the user would get so arguably it being unspecified means we can change it.  I think I'm ok with this as-is but we should probably flag this for migration docs in the release notes for user to be aware of.





> Use compression level for xz-lzma2 format of the CompressContent processor
> --------------------------------------------------------------------------
>
>                 Key: NIFI-6964
>                 URL: https://issues.apache.org/jira/browse/NIFI-6964
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>    Affects Versions: 1.10.0
>            Reporter: John Pierce
>            Priority: Minor
>              Labels: xz-lzma2
>         Attachments: CompressContent-enable-xz-compression-levels.patch
>
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> The CompressContent processor does not use the Compression Level property of the processor except for when using the GZIP compression format. On the contrary, the xz-lzma2 compression format defaults to using XZ compression level 6 for that specific format (I read the CompressContent.java source code to verify this) – disregarding whatever compression level you set on the processor itself.
> I have a use case where I must use the xz-lzma2 format (don't ask why) and I have to send (using the XZ format) already highly-compressed content that is +*NOT*+ XZ format to begin with. I have in excess of 500 GB of this sort of already highly compressed content to further compress into the XZ format on a daily basis.
> The attached patch will enhance the CompressContent.java source code enabling the compression level property to be used in both the GZIP and the XZ-LZMA2 formats.
> Please consider adding this patch to the baseline for this processor. I've tested it and the results are fantastic because I can crank down the compression level to 0 for XZ-LZMA2 now and use a lot less CPU. I'm generally seeing a 66% improvement in elapsed time to process highly compressed content using XZ format with compression level of 0 versus the hard-coded level 6 of the baseline code.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)