You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yuming Wang (Jira)" <ji...@apache.org> on 2023/06/08 04:29:00 UTC
[jira] [Updated] (SPARK-43273) Support writing Parquet files with lz4raw compression
[ https://issues.apache.org/jira/browse/SPARK-43273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuming Wang updated SPARK-43273:
--------------------------------
Summary: Support writing Parquet files with lz4raw compression (was: Spark can't read parquet files with a newer LZ4_RAW compression)
> Support writing Parquet files with lz4raw compression
> -----------------------------------------------------
>
> Key: SPARK-43273
> URL: https://issues.apache.org/jira/browse/SPARK-43273
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.2.4, 3.3.3, 3.3.2, 3.4.0
> Reporter: Andrew Grigorev
> Priority: Trivial
>
> hadoop-parquet version should be updated to 1.3.0 (together with other parquet-mr libs)
> {code:java}
> java.util.concurrent.ExecutionException: org.apache.spark.SparkException: Job aborted due to stage failure: Task 2 in stage 1.0 failed 1 times, most recent failure: Lost task 2.0 in stage 1.0 (TID 3) (f2b63fdfa0a6 executor driver): java.lang.IllegalArgumentException: No enum constant org.apache.parquet.hadoop.metadata.CompressionCodecName.LZ4_RAW
> at java.base/java.lang.Enum.valueOf(Enum.java:273)
> at org.apache.parquet.hadoop.metadata.CompressionCodecName.valueOf(CompressionCodecName.java:26)
> at org.apache.parquet.format.converter.ParquetMetadataConverter.fromFormatCodec(ParquetMetadataConverter.java:636)
> ... {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org