You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2018/07/31 01:14:00 UTC
[jira] [Resolved] (SPARK-24952) Support LZMA2 compression by Avro
datasource
[ https://issues.apache.org/jira/browse/SPARK-24952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-24952.
----------------------------------
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21902
[https://github.com/apache/spark/pull/21902]
> Support LZMA2 compression by Avro datasource
> --------------------------------------------
>
> Key: SPARK-24952
> URL: https://issues.apache.org/jira/browse/SPARK-24952
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: Maxim Gekk
> Assignee: Maxim Gekk
> Priority: Minor
> Fix For: 2.4.0
>
>
> LZMA2 (XZ) has much more better compression ratio comparing to currently supported snappy and deflate. Underlying Avro library supports the compression codec already. Need to set parameters for the codec and allow users to specify "xz" compression via AvroOptions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org