You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Etienne Chauchot (Jira)" <ji...@apache.org> on 2023/09/07 10:10:00 UTC

[jira] [Created] (FLINK-33059) Support transparent compression for file-connector for all file input formats

Etienne Chauchot created FLINK-33059:
----------------------------------------

             Summary: Support transparent compression for file-connector for all file input formats
                 Key: FLINK-33059
                 URL: https://issues.apache.org/jira/browse/FLINK-33059
             Project: Flink
          Issue Type: Technical Debt
          Components: Connectors / FileSystem
            Reporter: Etienne Chauchot
            Assignee: Etienne Chauchot


Delimited file input formats (contrary to binary input format etc...) do not support compression via the existing decorator because split length is determined by the compressed file length lead to [this|https://issues.apache.org/jira/browse/FLINK-30314] bug .  We should force reading the whole file split (like it is done for binary input formats) on compressed files. Parallelism is still done at the file level (as now)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)