You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Serge Blazhiyevskyy <Se...@nice.com> on 2012/09/14 20:52:15 UTC
Best splittable compression for Hadoop 1.0.0
Hi all,
I am trying to investigate the best splittable format for hadoop 1.0.0
Anybody have any recommendation?
Thanks
Serge
Re: Best splittable compression for Hadoop 1.0.0
Posted by Harsh J <ha...@cloudera.com>.
Serge,
I'd look at Avro DataFiles, with snappy or deflate compression
(tunable levels). It is documented at
http://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileWriter.html.
Read more on Avro and why its good at http://avro.apache.org.
On Sat, Sep 15, 2012 at 12:22 AM, Serge Blazhiyevskyy
<Se...@nice.com> wrote:
> Hi all,
>
>
> I am trying to investigate the best splittable format for hadoop 1.0.0
>
> Anybody have any recommendation?
>
>
> Thanks
> Serge
--
Harsh J
Re: Best splittable compression for Hadoop 1.0.0
Posted by Harsh J <ha...@cloudera.com>.
Serge,
I'd look at Avro DataFiles, with snappy or deflate compression
(tunable levels). It is documented at
http://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileWriter.html.
Read more on Avro and why its good at http://avro.apache.org.
On Sat, Sep 15, 2012 at 12:22 AM, Serge Blazhiyevskyy
<Se...@nice.com> wrote:
> Hi all,
>
>
> I am trying to investigate the best splittable format for hadoop 1.0.0
>
> Anybody have any recommendation?
>
>
> Thanks
> Serge
--
Harsh J
Re: Best splittable compression for Hadoop 1.0.0
Posted by Harsh J <ha...@cloudera.com>.
Serge,
I'd look at Avro DataFiles, with snappy or deflate compression
(tunable levels). It is documented at
http://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileWriter.html.
Read more on Avro and why its good at http://avro.apache.org.
On Sat, Sep 15, 2012 at 12:22 AM, Serge Blazhiyevskyy
<Se...@nice.com> wrote:
> Hi all,
>
>
> I am trying to investigate the best splittable format for hadoop 1.0.0
>
> Anybody have any recommendation?
>
>
> Thanks
> Serge
--
Harsh J
Re: Best splittable compression for Hadoop 1.0.0
Posted by Harsh J <ha...@cloudera.com>.
Serge,
I'd look at Avro DataFiles, with snappy or deflate compression
(tunable levels). It is documented at
http://avro.apache.org/docs/current/api/java/org/apache/avro/file/DataFileWriter.html.
Read more on Avro and why its good at http://avro.apache.org.
On Sat, Sep 15, 2012 at 12:22 AM, Serge Blazhiyevskyy
<Se...@nice.com> wrote:
> Hi all,
>
>
> I am trying to investigate the best splittable format for hadoop 1.0.0
>
> Anybody have any recommendation?
>
>
> Thanks
> Serge
--
Harsh J