You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/04/30 02:08:00 UTC
[jira] [Commented] (NIFI-11466) Add a ModifyCompression processor
[ https://issues.apache.org/jira/browse/NIFI-11466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718001#comment-17718001 ]
ASF subversion and git services commented on NIFI-11466:
--------------------------------------------------------
Commit 0e93dfae832e9f77d510f5e4e69399436d72c076 in nifi's branch refs/heads/main from Matt Burgess
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0e93dfae83 ]
NIFI-11466 Added ModifyCompression Processor
- Added nifi-compress-bundle with nifi-compress-nar
This closes #7180
Co-authored-by: David Handermann <ex...@apache.org>
Signed-off-by: David Handermann <ex...@apache.org>
> Add a ModifyCompression processor
> ---------------------------------
>
> Key: NIFI-11466
> URL: https://issues.apache.org/jira/browse/NIFI-11466
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Major
> Fix For: 2.latest
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> If a user would like to convert from one compression format to another, they currently have to use CompressContent to decompress, then another CompressContent to compress into a different format. Two processors plus disk I/O for the FlowFiles and their underlying content claims can be I/O intensive in that case.
> Instead, a new ModifyCompression processor is proposed, to allow for both decompression of the incoming FlowFile and compression for the outgoing FlowFile, using appropriate memory buffers for the decompression/recompression. Adding "no decompression" and "no compression" options for the respective properties could allow this property to function like CompressContent does now, plus the ability to convert from one compression format (gzip, e.g.) to another (snappy-hadoop, e.g.). One example of a use case where this would be helpful is an I/O bound flow to get compressed data from a legacy source system into HDFS for faster (and larger-volume / distributed) processing of the data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)