You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Abhishek Pratap Singh <ma...@gmail.com> on 2011/12/14 20:32:52 UTC

Regarding pointers for LZO compression in Hive and Hadoop

Hi,

I m looking for some useful docs on enabling LZO on hadoop cluster. I tried
few of the blogs, but somehow its not working.
Here is my requirement.

I have a hadoop 0.20.2 and Hive 0.6. I have some tables with 1.5 TB of
data, i want to compress them using LZO and enable LZO in hive as well as
in hadoop.
Let me know if you have any useful docs or pointers for the same.


Regards,
Abhishek

Re: Regarding pointers for LZO compression in Hive and Hadoop

Posted by James Warren <ja...@stanfordalumni.org>.

Hi Abhishek -

(Redirecting to user@hive, bcc'ing common-user)

I found this blog to be particularly useful when incorporating Hive and LZO:

http://www.mrbalky.com/2011/02/24/hive-tables-partitions-and-lzo-compression/

And if you're having issues setting up LZO with Hadoop in general, check out

https://github.com/toddlipcon/hadoop-lzo

cheers,
-James



On Wed, Dec 14, 2011 at 11:32 AM, Abhishek Pratap Singh <manu.infy@gmail.com
> wrote:

> Hi,
>
> I m looking for some useful docs on enabling LZO on hadoop cluster. I tried
> few of the blogs, but somehow its not working.
> Here is my requirement.
>
> I have a hadoop 0.20.2 and Hive 0.6. I have some tables with 1.5 TB of
> data, i want to compress them using LZO and enable LZO in hive as well as
> in hadoop.
> Let me know if you have any useful docs or pointers for the same.
>
>
> Regards,
> Abhishek
>

Re: Regarding pointers for LZO compression in Hive and Hadoop

Posted by Prashant Kommireddi <pr...@gmail.com>.

http://code.google.com/p/hadoop-gpl-packing/

Thanks,
Prashant

On Wed, Dec 14, 2011 at 11:32 AM, Abhishek Pratap Singh <manu.infy@gmail.com
> wrote:

> Hi,
>
> I m looking for some useful docs on enabling LZO on hadoop cluster. I tried
> few of the blogs, but somehow its not working.
> Here is my requirement.
>
> I have a hadoop 0.20.2 and Hive 0.6. I have some tables with 1.5 TB of
> data, i want to compress them using LZO and enable LZO in hive as well as
> in hadoop.
> Let me know if you have any useful docs or pointers for the same.
>
>
> Regards,
> Abhishek
>

Re: Regarding pointers for LZO compression in Hive and Hadoop

Posted by James Warren <ja...@stanfordalumni.org>.

Hi Abhishek -

(Redirecting to user@hive, bcc'ing common-user)

I found this blog to be particularly useful when incorporating Hive and LZO:

http://www.mrbalky.com/2011/02/24/hive-tables-partitions-and-lzo-compression/

And if you're having issues setting up LZO with Hadoop in general, check out

https://github.com/toddlipcon/hadoop-lzo

cheers,
-James



On Wed, Dec 14, 2011 at 11:32 AM, Abhishek Pratap Singh <manu.infy@gmail.com
> wrote:

> Hi,
>
> I m looking for some useful docs on enabling LZO on hadoop cluster. I tried
> few of the blogs, but somehow its not working.
> Here is my requirement.
>
> I have a hadoop 0.20.2 and Hive 0.6. I have some tables with 1.5 TB of
> data, i want to compress them using LZO and enable LZO in hive as well as
> in hadoop.
> Let me know if you have any useful docs or pointers for the same.
>
>
> Regards,
> Abhishek
>