You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Fabio Pitzolu <fa...@gr-ci.com> on 2013/04/17 16:39:21 UTC
Hadoop fs -getmerge
Hi all,
is there a way to use the "*getmerge*" fs command and not generate the .crc
files in the output local directory?
Thanks,
Fabio Pitzolu**
Re: Hadoop fs -getmerge
Posted by 姚吉龙 <ge...@gmail.com>.
Use this command：hadoop fs -getmerge ＜file in hdfs＞ ＜local＞
—
Sent from Mailbox for iPhone
On Wed, Apr 17, 2013 at 10:40 PM, Fabio Pitzolu <fa...@gr-ci.com>
wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the .crc
> files in the output local directory?
> Thanks,
> Fabio Pitzolu**
Re: Hadoop fs -getmerge
Posted by Daryn Sharp <da...@yahoo-inc.com>.
If -getmerge is updated, specifically deleting the .crc is not necessary. Adding an option that invokes fs.writeChecksum via a cmdline option should do the trick.
On Apr 18, 2013, at 2:41 AM, Fabio Pitzolu wrote:
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter to -getmerge to delete the crc files afterwards?
Fabio Pitzolu
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
Gruppo Consulenza Innovazione - http://www.gr-ci.com<http://www.gr-ci.com/>
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>>
I don't think that is possible. When we use -getmerge, the destination filesystem happens to be a LocalFileSystem which extends from ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>> wrote:
Hi all,
is there a way to use the "getmerge" fs command and not generate the .crc files in the output local directory?
Thanks,
Fabio Pitzolu
Re: Hadoop fs -getmerge
Posted by Daryn Sharp <da...@yahoo-inc.com>.
If -getmerge is updated, specifically deleting the .crc is not necessary. Adding an option that invokes fs.writeChecksum via a cmdline option should do the trick.
On Apr 18, 2013, at 2:41 AM, Fabio Pitzolu wrote:
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter to -getmerge to delete the crc files afterwards?
Fabio Pitzolu
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
Gruppo Consulenza Innovazione - http://www.gr-ci.com<http://www.gr-ci.com/>
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>>
I don't think that is possible. When we use -getmerge, the destination filesystem happens to be a LocalFileSystem which extends from ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>> wrote:
Hi all,
is there a way to use the "getmerge" fs command and not generate the .crc files in the output local directory?
Thanks,
Fabio Pitzolu
Re: Hadoop fs -getmerge
Posted by Daryn Sharp <da...@yahoo-inc.com>.
If -getmerge is updated, specifically deleting the .crc is not necessary. Adding an option that invokes fs.writeChecksum via a cmdline option should do the trick.
On Apr 18, 2013, at 2:41 AM, Fabio Pitzolu wrote:
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter to -getmerge to delete the crc files afterwards?
Fabio Pitzolu
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
Gruppo Consulenza Innovazione - http://www.gr-ci.com<http://www.gr-ci.com/>
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>>
I don't think that is possible. When we use -getmerge, the destination filesystem happens to be a LocalFileSystem which extends from ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>> wrote:
Hi all,
is there a way to use the "getmerge" fs command and not generate the .crc files in the output local directory?
Thanks,
Fabio Pitzolu
Re: Hadoop fs -getmerge
Posted by Daryn Sharp <da...@yahoo-inc.com>.
If -getmerge is updated, specifically deleting the .crc is not necessary. Adding an option that invokes fs.writeChecksum via a cmdline option should do the trick.
On Apr 18, 2013, at 2:41 AM, Fabio Pitzolu wrote:
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter to -getmerge to delete the crc files afterwards?
Fabio Pitzolu
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
Gruppo Consulenza Innovazione - http://www.gr-ci.com<http://www.gr-ci.com/>
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>>
I don't think that is possible. When we use -getmerge, the destination filesystem happens to be a LocalFileSystem which extends from ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>> wrote:
Hi all,
is there a way to use the "getmerge" fs command and not generate the .crc files in the output local directory?
Thanks,
Fabio Pitzolu
Re: Hadoop fs -getmerge
Posted by Fabio Pitzolu <fa...@gr-ci.com>.
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter
to -getmerge to delete the crc files afterwards?
*Fabio Pitzolu*
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
*Gruppo Consulenza Innovazione - http://www.gr-ci.com*
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>
> I don't think that is possible. When we use -getmerge, the destination
> filesystem happens to be a LocalFileSystem which extends from
> ChecksumFileSystem. I believe that's why the CRC files are getting in.
>
> Would it not be possible for you to ignore them, since they have a fixed
> extension ?
>
> Thanks
> Hemanth
>
>
> On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
>
>> Hi all,
>> is there a way to use the "*getmerge*" fs command and not generate the
>> .crc files in the output local directory?
>>
>> Thanks,
>>
>> Fabio Pitzolu**
>>
>
>
Re: Hadoop fs -getmerge
Posted by Fabio Pitzolu <fa...@gr-ci.com>.
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter
to -getmerge to delete the crc files afterwards?
*Fabio Pitzolu*
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
*Gruppo Consulenza Innovazione - http://www.gr-ci.com*
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>
> I don't think that is possible. When we use -getmerge, the destination
> filesystem happens to be a LocalFileSystem which extends from
> ChecksumFileSystem. I believe that's why the CRC files are getting in.
>
> Would it not be possible for you to ignore them, since they have a fixed
> extension ?
>
> Thanks
> Hemanth
>
>
> On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
>
>> Hi all,
>> is there a way to use the "*getmerge*" fs command and not generate the
>> .crc files in the output local directory?
>>
>> Thanks,
>>
>> Fabio Pitzolu**
>>
>
>
Re: Hadoop fs -getmerge
Posted by Fabio Pitzolu <fa...@gr-ci.com>.
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter
to -getmerge to delete the crc files afterwards?
*Fabio Pitzolu*
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
*Gruppo Consulenza Innovazione - http://www.gr-ci.com*
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>
> I don't think that is possible. When we use -getmerge, the destination
> filesystem happens to be a LocalFileSystem which extends from
> ChecksumFileSystem. I believe that's why the CRC files are getting in.
>
> Would it not be possible for you to ignore them, since they have a fixed
> extension ?
>
> Thanks
> Hemanth
>
>
> On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
>
>> Hi all,
>> is there a way to use the "*getmerge*" fs command and not generate the
>> .crc files in the output local directory?
>>
>> Thanks,
>>
>> Fabio Pitzolu**
>>
>
>
Re: Hadoop fs -getmerge
Posted by Fabio Pitzolu <fa...@gr-ci.com>.
Hi Hemanth,
I guess that the only solution is to delete the crc files after the export.
Does anyone of you knows if someone filed a Jira to implement a parameter
to -getmerge to delete the crc files afterwards?
*Fabio Pitzolu*
Consultant - BI & Infrastructure
Mob. +39 3356033776
Telefono 02 87157239
Fax. 02 93664786
*Gruppo Consulenza Innovazione - http://www.gr-ci.com*
2013/4/18 Hemanth Yamijala <yh...@thoughtworks.com>
> I don't think that is possible. When we use -getmerge, the destination
> filesystem happens to be a LocalFileSystem which extends from
> ChecksumFileSystem. I believe that's why the CRC files are getting in.
>
> Would it not be possible for you to ignore them, since they have a fixed
> extension ?
>
> Thanks
> Hemanth
>
>
> On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
>
>> Hi all,
>> is there a way to use the "*getmerge*" fs command and not generate the
>> .crc files in the output local directory?
>>
>> Thanks,
>>
>> Fabio Pitzolu**
>>
>
>
Re: Hadoop fs -getmerge
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
I don't think that is possible. When we use -getmerge, the destination
filesystem happens to be a LocalFileSystem which extends from
ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed
extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the
> .crc files in the output local directory?
>
> Thanks,
>
> Fabio Pitzolu**
>
Re: Hadoop fs -getmerge
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
I don't think that is possible. When we use -getmerge, the destination
filesystem happens to be a LocalFileSystem which extends from
ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed
extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the
> .crc files in the output local directory?
>
> Thanks,
>
> Fabio Pitzolu**
>
Re: Hadoop fs -getmerge
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
I don't think that is possible. When we use -getmerge, the destination
filesystem happens to be a LocalFileSystem which extends from
ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed
extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the
> .crc files in the output local directory?
>
> Thanks,
>
> Fabio Pitzolu**
>
Re: Hadoop fs -getmerge
Posted by 姚吉龙 <ge...@gmail.com>.
Use this command：hadoop fs -getmerge ＜file in hdfs＞ ＜local＞
—
Sent from Mailbox for iPhone
On Wed, Apr 17, 2013 at 10:40 PM, Fabio Pitzolu <fa...@gr-ci.com>
wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the .crc
> files in the output local directory?
> Thanks,
> Fabio Pitzolu**
Re: Hadoop fs -getmerge
Posted by 姚吉龙 <ge...@gmail.com>.
Use this command：hadoop fs -getmerge ＜file in hdfs＞ ＜local＞
—
Sent from Mailbox for iPhone
On Wed, Apr 17, 2013 at 10:40 PM, Fabio Pitzolu <fa...@gr-ci.com>
wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the .crc
> files in the output local directory?
> Thanks,
> Fabio Pitzolu**
Re: Hadoop fs -getmerge
Posted by 姚吉龙 <ge...@gmail.com>.
Use this command：hadoop fs -getmerge ＜file in hdfs＞ ＜local＞
—
Sent from Mailbox for iPhone
On Wed, Apr 17, 2013 at 10:40 PM, Fabio Pitzolu <fa...@gr-ci.com>
wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the .crc
> files in the output local directory?
> Thanks,
> Fabio Pitzolu**
Re: Hadoop fs -getmerge
Posted by Hemanth Yamijala <yh...@thoughtworks.com>.
I don't think that is possible. When we use -getmerge, the destination
filesystem happens to be a LocalFileSystem which extends from
ChecksumFileSystem. I believe that's why the CRC files are getting in.
Would it not be possible for you to ignore them, since they have a fixed
extension ?
Thanks
Hemanth
On Wed, Apr 17, 2013 at 8:09 PM, Fabio Pitzolu <fa...@gr-ci.com>wrote:
> Hi all,
> is there a way to use the "*getmerge*" fs command and not generate the
> .crc files in the output local directory?
>
> Thanks,
>
> Fabio Pitzolu**
>