You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "tartarus (Jira)" <ji...@apache.org> on 2021/08/11 04:10:00 UTC

[jira] [Created] (FLINK-23715) Support for reading fields that do not exist in Parquet files

tartarus created FLINK-23715:
--------------------------------

             Summary: Support for reading fields that do not exist in Parquet files
                 Key: FLINK-23715
                 URL: https://issues.apache.org/jira/browse/FLINK-23715
             Project: Flink
          Issue Type: Improvement
          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
            Reporter: tartarus


In a production environment, it is often encountered that users add fields to the hive table, but do not refresh the data of the historical partition. Therefore, if the new field is not in the historical partition file, an error will be reported when reading the historical partition.

General users would expect that if there is no such field, then fill in null and return.

The current flink Parquet format does not support this function.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)