You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Manoj Babu <ma...@gmail.com> on 2012/10/21 09:40:55 UTC
extracting lzo compressed files
Hi,
Is there any option to extract the lzo compressed file in HDFS from
command line and any option to find the original size of the compressed
file.
Thanks in Advance!
Cheers!
Manoj.
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi,
By using lzop -l can able to get the compressed uncompr size in local fs.
Is there any way to do the same for files in HDFS?
$:~> lzop -l input/part-00008.lzo
method compressed uncompr. ratio uncompressed_name
LZO1X-1 40955533 497723469 8.2% input/part-00008
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 6:59 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> I am sorry. I can able to see the file size of compressed one but i am
> trying to find what will be size of the file if it is not compressed and by
> without extracting all set of files.
>
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi Bejoy,
>>
>> 'hadoop fs -ls' is not displaying the file size. is there any other way
>> to find the original file size.
>>
>> Thanks in advance.
>>
>> Cheers!
>> Manoj.
>>
>>
>>
>> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>>> **
>>> Hi Manoj
>>>
>>> You can get the file in a readable format using
>>> hadoop fs -text <fileName>
>>>
>>> Provided you have lzo codec within the property 'io.compression.codecs'
>>> in core-site.xml
>>>
>>> A 'hadoop fs -ls' command would itself display the file size.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Manoj Babu <ma...@gmail.com>
>>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>>> *To: *<us...@hadoop.apache.org>
>>> *ReplyTo: * user@hadoop.apache.org
>>> *Subject: *extracting lzo compressed files
>>>
>>> Hi,
>>>
>>> Is there any option to extract the lzo compressed file in HDFS from
>>> command line and any option to find the original size of the compressed
>>> file.
>>>
>>> Thanks in Advance!
>>>
>>> Cheers!
>>> Manoj.
>>>
>>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi,
By using lzop -l can able to get the compressed uncompr size in local fs.
Is there any way to do the same for files in HDFS?
$:~> lzop -l input/part-00008.lzo
method compressed uncompr. ratio uncompressed_name
LZO1X-1 40955533 497723469 8.2% input/part-00008
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 6:59 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> I am sorry. I can able to see the file size of compressed one but i am
> trying to find what will be size of the file if it is not compressed and by
> without extracting all set of files.
>
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi Bejoy,
>>
>> 'hadoop fs -ls' is not displaying the file size. is there any other way
>> to find the original file size.
>>
>> Thanks in advance.
>>
>> Cheers!
>> Manoj.
>>
>>
>>
>> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>>> **
>>> Hi Manoj
>>>
>>> You can get the file in a readable format using
>>> hadoop fs -text <fileName>
>>>
>>> Provided you have lzo codec within the property 'io.compression.codecs'
>>> in core-site.xml
>>>
>>> A 'hadoop fs -ls' command would itself display the file size.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Manoj Babu <ma...@gmail.com>
>>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>>> *To: *<us...@hadoop.apache.org>
>>> *ReplyTo: * user@hadoop.apache.org
>>> *Subject: *extracting lzo compressed files
>>>
>>> Hi,
>>>
>>> Is there any option to extract the lzo compressed file in HDFS from
>>> command line and any option to find the original size of the compressed
>>> file.
>>>
>>> Thanks in Advance!
>>>
>>> Cheers!
>>> Manoj.
>>>
>>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi,
By using lzop -l can able to get the compressed uncompr size in local fs.
Is there any way to do the same for files in HDFS?
$:~> lzop -l input/part-00008.lzo
method compressed uncompr. ratio uncompressed_name
LZO1X-1 40955533 497723469 8.2% input/part-00008
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 6:59 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> I am sorry. I can able to see the file size of compressed one but i am
> trying to find what will be size of the file if it is not compressed and by
> without extracting all set of files.
>
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi Bejoy,
>>
>> 'hadoop fs -ls' is not displaying the file size. is there any other way
>> to find the original file size.
>>
>> Thanks in advance.
>>
>> Cheers!
>> Manoj.
>>
>>
>>
>> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>>> **
>>> Hi Manoj
>>>
>>> You can get the file in a readable format using
>>> hadoop fs -text <fileName>
>>>
>>> Provided you have lzo codec within the property 'io.compression.codecs'
>>> in core-site.xml
>>>
>>> A 'hadoop fs -ls' command would itself display the file size.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Manoj Babu <ma...@gmail.com>
>>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>>> *To: *<us...@hadoop.apache.org>
>>> *ReplyTo: * user@hadoop.apache.org
>>> *Subject: *extracting lzo compressed files
>>>
>>> Hi,
>>>
>>> Is there any option to extract the lzo compressed file in HDFS from
>>> command line and any option to find the original size of the compressed
>>> file.
>>>
>>> Thanks in Advance!
>>>
>>> Cheers!
>>> Manoj.
>>>
>>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi,
By using lzop -l can able to get the compressed uncompr size in local fs.
Is there any way to do the same for files in HDFS?
$:~> lzop -l input/part-00008.lzo
method compressed uncompr. ratio uncompressed_name
LZO1X-1 40955533 497723469 8.2% input/part-00008
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 6:59 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> I am sorry. I can able to see the file size of compressed one but i am
> trying to find what will be size of the file if it is not compressed and by
> without extracting all set of files.
>
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
>
>> Hi Bejoy,
>>
>> 'hadoop fs -ls' is not displaying the file size. is there any other way
>> to find the original file size.
>>
>> Thanks in advance.
>>
>> Cheers!
>> Manoj.
>>
>>
>>
>> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>>> **
>>> Hi Manoj
>>>
>>> You can get the file in a readable format using
>>> hadoop fs -text <fileName>
>>>
>>> Provided you have lzo codec within the property 'io.compression.codecs'
>>> in core-site.xml
>>>
>>> A 'hadoop fs -ls' command would itself display the file size.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Manoj Babu <ma...@gmail.com>
>>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>>> *To: *<us...@hadoop.apache.org>
>>> *ReplyTo: * user@hadoop.apache.org
>>> *Subject: *extracting lzo compressed files
>>>
>>> Hi,
>>>
>>> Is there any option to extract the lzo compressed file in HDFS from
>>> command line and any option to find the original size of the compressed
>>> file.
>>>
>>> Thanks in Advance!
>>>
>>> Cheers!
>>> Manoj.
>>>
>>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
I am sorry. I can able to see the file size of compressed one but i am
trying to find what will be size of the file if it is not compressed and by
without extracting all set of files.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> 'hadoop fs -ls' is not displaying the file size. is there any other way to
> find the original file size.
>
> Thanks in advance.
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>
>> **
>> Hi Manoj
>>
>> You can get the file in a readable format using
>> hadoop fs -text <fileName>
>>
>> Provided you have lzo codec within the property 'io.compression.codecs'
>> in core-site.xml
>>
>> A 'hadoop fs -ls' command would itself display the file size.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Manoj Babu <ma...@gmail.com>
>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>> *To: *<us...@hadoop.apache.org>
>> *ReplyTo: * user@hadoop.apache.org
>> *Subject: *extracting lzo compressed files
>>
>> Hi,
>>
>> Is there any option to extract the lzo compressed file in HDFS from
>> command line and any option to find the original size of the compressed
>> file.
>>
>> Thanks in Advance!
>>
>> Cheers!
>> Manoj.
>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
I am sorry. I can able to see the file size of compressed one but i am
trying to find what will be size of the file if it is not compressed and by
without extracting all set of files.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> 'hadoop fs -ls' is not displaying the file size. is there any other way to
> find the original file size.
>
> Thanks in advance.
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>
>> **
>> Hi Manoj
>>
>> You can get the file in a readable format using
>> hadoop fs -text <fileName>
>>
>> Provided you have lzo codec within the property 'io.compression.codecs'
>> in core-site.xml
>>
>> A 'hadoop fs -ls' command would itself display the file size.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Manoj Babu <ma...@gmail.com>
>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>> *To: *<us...@hadoop.apache.org>
>> *ReplyTo: * user@hadoop.apache.org
>> *Subject: *extracting lzo compressed files
>>
>> Hi,
>>
>> Is there any option to extract the lzo compressed file in HDFS from
>> command line and any option to find the original size of the compressed
>> file.
>>
>> Thanks in Advance!
>>
>> Cheers!
>> Manoj.
>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
I am sorry. I can able to see the file size of compressed one but i am
trying to find what will be size of the file if it is not compressed and by
without extracting all set of files.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> 'hadoop fs -ls' is not displaying the file size. is there any other way to
> find the original file size.
>
> Thanks in advance.
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>
>> **
>> Hi Manoj
>>
>> You can get the file in a readable format using
>> hadoop fs -text <fileName>
>>
>> Provided you have lzo codec within the property 'io.compression.codecs'
>> in core-site.xml
>>
>> A 'hadoop fs -ls' command would itself display the file size.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Manoj Babu <ma...@gmail.com>
>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>> *To: *<us...@hadoop.apache.org>
>> *ReplyTo: * user@hadoop.apache.org
>> *Subject: *extracting lzo compressed files
>>
>> Hi,
>>
>> Is there any option to extract the lzo compressed file in HDFS from
>> command line and any option to find the original size of the compressed
>> file.
>>
>> Thanks in Advance!
>>
>> Cheers!
>> Manoj.
>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
I am sorry. I can able to see the file size of compressed one but i am
trying to find what will be size of the file if it is not compressed and by
without extracting all set of files.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 3:28 PM, Manoj Babu <ma...@gmail.com> wrote:
> Hi Bejoy,
>
> 'hadoop fs -ls' is not displaying the file size. is there any other way to
> find the original file size.
>
> Thanks in advance.
>
> Cheers!
> Manoj.
>
>
>
> On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
>
>> **
>> Hi Manoj
>>
>> You can get the file in a readable format using
>> hadoop fs -text <fileName>
>>
>> Provided you have lzo codec within the property 'io.compression.codecs'
>> in core-site.xml
>>
>> A 'hadoop fs -ls' command would itself display the file size.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Manoj Babu <ma...@gmail.com>
>> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
>> *To: *<us...@hadoop.apache.org>
>> *ReplyTo: * user@hadoop.apache.org
>> *Subject: *extracting lzo compressed files
>>
>> Hi,
>>
>> Is there any option to extract the lzo compressed file in HDFS from
>> command line and any option to find the original size of the compressed
>> file.
>>
>> Thanks in Advance!
>>
>> Cheers!
>> Manoj.
>>
>>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
'hadoop fs -ls' is not displaying the file size. is there any other way to
find the original file size.
Thanks in advance.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Manoj
>
> You can get the file in a readable format using
> hadoop fs -text <fileName>
>
> Provided you have lzo codec within the property 'io.compression.codecs' in
> core-site.xml
>
> A 'hadoop fs -ls' command would itself display the file size.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Manoj Babu <ma...@gmail.com>
> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *extracting lzo compressed files
>
> Hi,
>
> Is there any option to extract the lzo compressed file in HDFS from
> command line and any option to find the original size of the compressed
> file.
>
> Thanks in Advance!
>
> Cheers!
> Manoj.
>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
'hadoop fs -ls' is not displaying the file size. is there any other way to
find the original file size.
Thanks in advance.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Manoj
>
> You can get the file in a readable format using
> hadoop fs -text <fileName>
>
> Provided you have lzo codec within the property 'io.compression.codecs' in
> core-site.xml
>
> A 'hadoop fs -ls' command would itself display the file size.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Manoj Babu <ma...@gmail.com>
> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *extracting lzo compressed files
>
> Hi,
>
> Is there any option to extract the lzo compressed file in HDFS from
> command line and any option to find the original size of the compressed
> file.
>
> Thanks in Advance!
>
> Cheers!
> Manoj.
>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
'hadoop fs -ls' is not displaying the file size. is there any other way to
find the original file size.
Thanks in advance.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Manoj
>
> You can get the file in a readable format using
> hadoop fs -text <fileName>
>
> Provided you have lzo codec within the property 'io.compression.codecs' in
> core-site.xml
>
> A 'hadoop fs -ls' command would itself display the file size.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Manoj Babu <ma...@gmail.com>
> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *extracting lzo compressed files
>
> Hi,
>
> Is there any option to extract the lzo compressed file in HDFS from
> command line and any option to find the original size of the compressed
> file.
>
> Thanks in Advance!
>
> Cheers!
> Manoj.
>
>
Re: extracting lzo compressed files
Posted by Manoj Babu <ma...@gmail.com>.
Hi Bejoy,
'hadoop fs -ls' is not displaying the file size. is there any other way to
find the original file size.
Thanks in advance.
Cheers!
Manoj.
On Sun, Oct 21, 2012 at 1:47 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Manoj
>
> You can get the file in a readable format using
> hadoop fs -text <fileName>
>
> Provided you have lzo codec within the property 'io.compression.codecs' in
> core-site.xml
>
> A 'hadoop fs -ls' command would itself display the file size.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Manoj Babu <ma...@gmail.com>
> *Date: *Sun, 21 Oct 2012 13:10:55 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *extracting lzo compressed files
>
> Hi,
>
> Is there any option to extract the lzo compressed file in HDFS from
> command line and any option to find the original size of the compressed
> file.
>
> Thanks in Advance!
>
> Cheers!
> Manoj.
>
>
Re: extracting lzo compressed files
Posted by Bejoy KS <be...@gmail.com>.
Hi Manoj
You can get the file in a readable format using
hadoop fs -text <fileName>
Provided you have lzo codec within the property 'io.compression.codecs' in core-site.xml
A 'hadoop fs -ls' command would itself display the file size.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Manoj Babu <ma...@gmail.com>
Date: Sun, 21 Oct 2012 13:10:55
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: extracting lzo compressed files
Hi,
Is there any option to extract the lzo compressed file in HDFS from
command line and any option to find the original size of the compressed
file.
Thanks in Advance!
Cheers!
Manoj.
Re: extracting lzo compressed files
Posted by Bejoy KS <be...@gmail.com>.
Hi Manoj
You can get the file in a readable format using
hadoop fs -text <fileName>
Provided you have lzo codec within the property 'io.compression.codecs' in core-site.xml
A 'hadoop fs -ls' command would itself display the file size.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Manoj Babu <ma...@gmail.com>
Date: Sun, 21 Oct 2012 13:10:55
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: extracting lzo compressed files
Hi,
Is there any option to extract the lzo compressed file in HDFS from
command line and any option to find the original size of the compressed
file.
Thanks in Advance!
Cheers!
Manoj.
Re: extracting lzo compressed files
Posted by Bejoy KS <be...@gmail.com>.
Hi Manoj
You can get the file in a readable format using
hadoop fs -text <fileName>
Provided you have lzo codec within the property 'io.compression.codecs' in core-site.xml
A 'hadoop fs -ls' command would itself display the file size.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Manoj Babu <ma...@gmail.com>
Date: Sun, 21 Oct 2012 13:10:55
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: extracting lzo compressed files
Hi,
Is there any option to extract the lzo compressed file in HDFS from
command line and any option to find the original size of the compressed
file.
Thanks in Advance!
Cheers!
Manoj.
Re: extracting lzo compressed files
Posted by Bejoy KS <be...@gmail.com>.
Hi Manoj
You can get the file in a readable format using
hadoop fs -text <fileName>
Provided you have lzo codec within the property 'io.compression.codecs' in core-site.xml
A 'hadoop fs -ls' command would itself display the file size.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Manoj Babu <ma...@gmail.com>
Date: Sun, 21 Oct 2012 13:10:55
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: extracting lzo compressed files
Hi,
Is there any option to extract the lzo compressed file in HDFS from
command line and any option to find the original size of the compressed
file.
Thanks in Advance!
Cheers!
Manoj.