You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Brandon White <bw...@gmail.com> on 2015/08/05 04:00:23 UTC

Turn Off Compression for Textfiles

How do you turn off gz compression for saving as textfiles? Right now, I am
reading ,gz files and it is saving them as .gz. I would love to not
compress them when I save.

1) DStream.saveAsTextFiles() //no compression

2) RDD.saveAsTextFile() //no compression

Any ideas?

Re: Turn Off Compression for Textfiles

Posted by Philip Weaver <ph...@gmail.com>.

The .gz extension indicates that the file is compressed with gzip. Choose a
different extension (e.g. .txt) when you save them.

On Tue, Aug 4, 2015 at 7:00 PM, Brandon White <bw...@gmail.com>
wrote:

> How do you turn off gz compression for saving as textfiles? Right now, I
> am reading ,gz files and it is saving them as .gz. I would love to not
> compress them when I save.
>
> 1) DStream.saveAsTextFiles() //no compression
>
> 2) RDD.saveAsTextFile() //no compression
>
> Any ideas?
>