You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by jamal sasha <ja...@gmail.com> on 2013/08/29 00:37:46 UTC
QUick question
Hi,
A very weird question.
I have data in format 1...
And there is this conversion utility to convert data to format 2 whcih
works like this
restore input > output
and then I want to copy output to hdfs at /user/output
Is tehre a way that i can merge all these commands into single line like
can i directly pipe output to hdfs ?
Re: QUick question
Posted by rtejac <rt...@gmail.com>.
Hello Jamal,
You can do "hadoop fs -getMerge <hdfspath> <local path>" that merges in your local machine, will merge all parts into 1 file. And then you can do -copyFromLocal to HDFS.
On Aug 28, 2013, at 3:37 PM, jamal sasha <ja...@gmail.com> wrote:
> I have data in format 1...
>
> And there is this conversion utility to convert data to format 2 whcih works like this
>
> restore input > output
>
> and then I want to copy output to hdfs at /user/output
>
> Is tehre a way that i can merge all these commands into single line like can i directly pipe output to hdfs ?
Re: QUick question
Posted by rtejac <rt...@gmail.com>.
Hello Jamal,
You can do "hadoop fs -getMerge <hdfspath> <local path>" that merges in your local machine, will merge all parts into 1 file. And then you can do -copyFromLocal to HDFS.
On Aug 28, 2013, at 3:37 PM, jamal sasha <ja...@gmail.com> wrote:
> I have data in format 1...
>
> And there is this conversion utility to convert data to format 2 whcih works like this
>
> restore input > output
>
> and then I want to copy output to hdfs at /user/output
>
> Is tehre a way that i can merge all these commands into single line like can i directly pipe output to hdfs ?
Re: QUick question
Posted by rtejac <rt...@gmail.com>.
Hello Jamal,
You can do "hadoop fs -getMerge <hdfspath> <local path>" that merges in your local machine, will merge all parts into 1 file. And then you can do -copyFromLocal to HDFS.
On Aug 28, 2013, at 3:37 PM, jamal sasha <ja...@gmail.com> wrote:
> I have data in format 1...
>
> And there is this conversion utility to convert data to format 2 whcih works like this
>
> restore input > output
>
> and then I want to copy output to hdfs at /user/output
>
> Is tehre a way that i can merge all these commands into single line like can i directly pipe output to hdfs ?
Re: QUick question
Posted by rtejac <rt...@gmail.com>.
Hello Jamal,
You can do "hadoop fs -getMerge <hdfspath> <local path>" that merges in your local machine, will merge all parts into 1 file. And then you can do -copyFromLocal to HDFS.
On Aug 28, 2013, at 3:37 PM, jamal sasha <ja...@gmail.com> wrote:
> I have data in format 1...
>
> And there is this conversion utility to convert data to format 2 whcih works like this
>
> restore input > output
>
> and then I want to copy output to hdfs at /user/output
>
> Is tehre a way that i can merge all these commands into single line like can i directly pipe output to hdfs ?