You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by rakesh kothari <rk...@hotmail.com> on 2010/10/19 22:03:04 UTC

Accessing files from distributed cache

Hi,

What's the way to access files copied to distributed cache from the map tasks ?

e.g.

if I run my M/R job as $hadoop jar my.jar -files hdfs://path/to/my/file.txt, How can I access file.txt in my Map(or reduce) task ?


Thanks,
-Rakesh

Re: Accessing files from distributed cache

Posted by Shrijeet Paliwal <sh...@rocketfuel.com>.

Rakesh,
The API: DistributedCache.getLocalCacheFiles(conf) , returns a list of
files added to dist cache.
More on this topic,

Courtesy : Alexander Behm
http://www.ics.uci.edu/~abehm/hadoop.html#howto_distributed_cache

-Shrijeet
On Tue, Oct 19, 2010 at 1:09 PM, rakesh kothari
<rk...@hotmail.com> wrote:
>
> I am using Hadoop 0.20.1.
>
> -Rakesh
> ________________________________
> From: rkothari_iit@hotmail.com
> To: mapreduce-user@hadoop.apache.org
> Subject: Accessing files from distributed cache
> Date: Tue, 19 Oct 2010 13:03:04 -0700
>
> Hi,
>
> What's the way to access files copied to distributed cache from the map tasks ?
>
> e.g.
>
> if I run my M/R job as $hadoop jar my.jar -files hdfs://path/to/my/file.txt, How can I access file.txt in my Map(or reduce) task ?
>
>
> Thanks,
> -Rakesh

RE: Accessing files from distributed cache

Posted by rakesh kothari <rk...@hotmail.com>.

I am using Hadoop 0.20.1.

-Rakesh
From: rkothari_iit@hotmail.com
To: mapreduce-user@hadoop.apache.org
Subject: Accessing files from distributed cache
Date: Tue, 19 Oct 2010 13:03:04 -0700

Hi,

What's the way to access files copied to distributed cache from the map tasks ?

e.g.

if I run my M/R job as $hadoop jar my.jar -files hdfs://path/to/my/file.txt, How can I access file.txt in my Map(or reduce) task ?

Thanks,
-Rakesh