Event Log Compression

[Bryan Jeffrey <br...@gmail.com>]

All,

I am running Spark 1.6.1. I enabled 'spark.eventLog.compress', and the data
is now being compressed using lz4.  I would like to move that back to
Snappy as I have some third party tools that require using Snappy.

Is there a variable used to control Spark eventLog compression algorithm?

Thank you,

Bryan Jeffrey

Re: Event Log Compression

[Jacek Laskowski <ja...@japila.pl>]

Hi,

Guess it's spark.io.compression.codec with lz4 being default.
Supported are lzf or snappy.

Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski


On Tue, Jul 26, 2016 at 3:33 PM, Bryan Jeffrey <br...@gmail.com> wrote:
> All,
>
> I am running Spark 1.6.1. I enabled 'spark.eventLog.compress', and the data
> is now being compressed using lz4.  I would like to move that back to Snappy
> as I have some third party tools that require using Snappy.
>
> Is there a variable used to control Spark eventLog compression algorithm?
>
> Thank you,
>
> Bryan Jeffrey

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org