You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "ABC (Jira)" <ji...@apache.org> on 2020/09/04 02:26:00 UTC

[jira] [Created] (FLINK-19137) Bump Apache Parquet to 1.11.1

ABC created FLINK-19137:
---------------------------

             Summary: Bump Apache Parquet to 1.11.1
                 Key: FLINK-19137
                 URL: https://issues.apache.org/jira/browse/FLINK-19137
             Project: Flink
          Issue Type: Improvement
          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
            Reporter: ABC
         Attachments: image-2020-09-04-10-21-00-688.png, image-2020-09-04-10-24-42-480.png

Apache Parquet 1.11.1 fixed some important issues:
 * https://issues.apache.org/jira/browse/PARQUET-1309
 * https://issues.apache.org/jira/browse/PARQUET-1510
 * https://issues.apache.org/jira/browse/PARQUET-1485

Now Flink master branch relies parquet 1.10.0, and flink-sql-parquet artifact shaded parquet class files into flink-sql-parquet.jar. So this may lead to direct memory leak in PARQUET-1485 or parquet properties bug in PARQUET-1309 or repeat values with dictionary encoding error in PARQUET-1510.

 

For example in PARQUET-1309:

!image-2020-09-04-10-21-00-688.png!

then in Flink:

[https://github.com/C08061/flink/blob/master/flink-formats/flink-parquet/src/main/java/org/apache/flink/formats/parquet/ParquetInputFormat.java#L166]

!image-2020-09-04-10-24-42-480.png!

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)