You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Grandl Robert <rg...@yahoo.com> on 2012/08/23 05:18:18 UTC
Copy the content of the input splits
Hi,
Do you know how I can copy using Hadoop code the content of input splits into local files ? I saw something like InputSplit class but still not able to figure out how to copy it. Is there somewhere in the code something similar I can look at ?
Thanks,
Robert
Re: Copy the content of the input splits
Posted by Bertrand Dechoux <de...@gmail.com>.
I don't know any helper class doing it but you could simply open a local
file and with your mapper write all the key/value which are provided.
Now the question is why would you want to do that? If it is in preparation
of a map join, you might want to user a higher abstraction tool (cascading,
pig, hive ...). You also may want to check out how DistributedCache works.
Regards
Bertrand
On Thu, Aug 23, 2012 at 5:18 AM, Grandl Robert <rg...@yahoo.com> wrote:
> Hi,
>
> Do you know how I can copy using Hadoop code the content of input splits
> into local files ? I saw something like InputSplit class but still not able
> to figure out how to copy it. Is there somewhere in the code something
> similar I can look at ?
>
> Thanks,
> Robert
>
>
--
Bertrand Dechoux
Re: Copy the content of the input splits
Posted by Bertrand Dechoux <de...@gmail.com>.
I don't know any helper class doing it but you could simply open a local
file and with your mapper write all the key/value which are provided.
Now the question is why would you want to do that? If it is in preparation
of a map join, you might want to user a higher abstraction tool (cascading,
pig, hive ...). You also may want to check out how DistributedCache works.
Regards
Bertrand
On Thu, Aug 23, 2012 at 5:18 AM, Grandl Robert <rg...@yahoo.com> wrote:
> Hi,
>
> Do you know how I can copy using Hadoop code the content of input splits
> into local files ? I saw something like InputSplit class but still not able
> to figure out how to copy it. Is there somewhere in the code something
> similar I can look at ?
>
> Thanks,
> Robert
>
>
--
Bertrand Dechoux
Re: Copy the content of the input splits
Posted by Bertrand Dechoux <de...@gmail.com>.
I don't know any helper class doing it but you could simply open a local
file and with your mapper write all the key/value which are provided.
Now the question is why would you want to do that? If it is in preparation
of a map join, you might want to user a higher abstraction tool (cascading,
pig, hive ...). You also may want to check out how DistributedCache works.
Regards
Bertrand
On Thu, Aug 23, 2012 at 5:18 AM, Grandl Robert <rg...@yahoo.com> wrote:
> Hi,
>
> Do you know how I can copy using Hadoop code the content of input splits
> into local files ? I saw something like InputSplit class but still not able
> to figure out how to copy it. Is there somewhere in the code something
> similar I can look at ?
>
> Thanks,
> Robert
>
>
--
Bertrand Dechoux
Re: Copy the content of the input splits
Posted by Bertrand Dechoux <de...@gmail.com>.
I don't know any helper class doing it but you could simply open a local
file and with your mapper write all the key/value which are provided.
Now the question is why would you want to do that? If it is in preparation
of a map join, you might want to user a higher abstraction tool (cascading,
pig, hive ...). You also may want to check out how DistributedCache works.
Regards
Bertrand
On Thu, Aug 23, 2012 at 5:18 AM, Grandl Robert <rg...@yahoo.com> wrote:
> Hi,
>
> Do you know how I can copy using Hadoop code the content of input splits
> into local files ? I saw something like InputSplit class but still not able
> to figure out how to copy it. Is there somewhere in the code something
> similar I can look at ?
>
> Thanks,
> Robert
>
>
--
Bertrand Dechoux