You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Charles Pritchard (JIRA)" <ji...@apache.org> on 2016/09/13 23:49:20 UTC
[jira] [Commented] (SPARK-14482) Change default compression codec
for Parquet from gzip to snappy
[ https://issues.apache.org/jira/browse/SPARK-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488848#comment-15488848 ]
Charles Pritchard commented on SPARK-14482:
-------------------------------------------
I don't think this fully made it into the manual; this impacts default compression for Parquet, but I think the manual still shows the default as gzip.
> Change default compression codec for Parquet from gzip to snappy
> ----------------------------------------------------------------
>
> Key: SPARK-14482
> URL: https://issues.apache.org/jira/browse/SPARK-14482
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Reynold Xin
> Assignee: Reynold Xin
> Fix For: 2.0.0
>
>
> Based on our tests, gzip decompression is very slow (< 100MB/s), making queries decompression bound. Snappy can decompress at ~ 500MB/s on a single core.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org