You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Reynold Xin (JIRA)" <ji...@apache.org> on 2014/07/15 08:10:08 UTC

[jira] [Commented] (SPARK-2399) Add support for LZ4 compression

    [ https://issues.apache.org/jira/browse/SPARK-2399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061736#comment-14061736 ] 

Reynold Xin commented on SPARK-2399:
------------------------------------

Do you mind submitting a pull request for this? I was actually looking into lz4 and wanted to add one.

> Add support for LZ4 compression
> -------------------------------
>
>                 Key: SPARK-2399
>                 URL: https://issues.apache.org/jira/browse/SPARK-2399
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>            Reporter: Greg Bowyer
>              Labels: compression, lz4
>         Attachments: SPARK-2399-Make-spark-compression-able-to-use-LZ4.patch
>
>
> LZ4 is a compression codec of the same ideas as googles snappy, but has some advantages:
> * It is faster than snappy with a similar compression ratio
> * The implementation is Apache licensed and not GPL
> It has shown promise in both the lucene and hadoop communities, and it looks like its a really easy add to spark io compression.
> Attached is a patch that does this



--
This message was sent by Atlassian JIRA
(v6.2#6252)