You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/11/06 16:44:00 UTC

[jira] [Updated] (SPARK-54220) NullType/VOID/UNKNOWN Type Support in Parquet

     [ https://issues.apache.org/jira/browse/SPARK-54220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated SPARK-54220:
-----------------------------------
    Labels: parquet parquetReader parquetWriter pull-request-available  (was: parquet parquetReader parquetWriter)

> NullType/VOID/UNKNOWN Type Support in Parquet
> ---------------------------------------------
>
>                 Key: SPARK-54220
>                 URL: https://issues.apache.org/jira/browse/SPARK-54220
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 4.0.1
>            Reporter: Ziya Mukhtarov
>            Priority: Major
>              Labels: parquet, parquetReader, parquetWriter, pull-request-available
>
> Currently, Spark does not support NullType columns in Parquet files. This is blocked at the ParquetFileFormat level, which throws an exception if there's an attempt to read or write NullType column. Parquet has [UNKNOWN logical type annotation|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#unknown-always-null], which can be used for columns whose values are always null. This can be used to write NullType columns into Parquet files, and when reading from Parquet files that already has this annotation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org