Re: New Distributed Cache

[Bill Q <bi...@gmail.com>]

Hi Omkar,
Did you find out why the getLocalCacheFiles is deprecated? If it is indeed
deprecated, what would be the alternative to use?

On Thursday, July 11, 2013, Omkar Joshi wrote:

> Yeah Andrew.. there seems to be some problem with context.getCacheFiles()
> api which is returning null..
>
>  Path[] cachedFilePaths =
>
>           context.getLocalCacheFiles(); // I am checking why it is
> deprecated...
>
>       for (Path cachedFilePath : cachedFilePaths) {
>
>         File cachedFile = new File(cachedFilePath.toUri().getRawPath());
>
>         System.out.println("cached fie path >> "
>
>             + cachedFile.getAbsolutePath());
>
>       }
>
> I hope this helps for the time being.. JobContext was suppose to replace
> DistributedCache api (it will be deprecated) however there is some problem
> with that or I am missing something... Will reply if I find the solution to
> it.
>
> context.getCacheFiles will give you the uri used for localizing files...
> (original uri used for adding it to cache)... However you can use
> DistributedCache.getCacheFiles() api till context api is fixed.
>
> context.getLocalCacheFiles .. will give you the actual file path on node
> manager... (after file is localized).
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Thu, Jul 11, 2013 at 8:19 AM, Botelho, Andrew <An...@emc.com>wrote:
>
> So in my driver code, I try to store the file in the cache with this line
> of code:
>
>
>
> job.addCacheFile(new URI("file location"));
>
>
>
> Then in my Mapper code, I do this to try and access the cached file:
>
>
>
> URI[] localPaths = context.getCacheFiles();
>
> File f = new File(localPaths[0]);
>
>
>
> However, I get a NullPointerException when I do that in the Mapper code.
>
>
>
> Any suggesstions?
>
>
>
> Andrew
>
>
>
> *From:* Shahab Yunus [mailto:shahab.yunus@gmail.com]
> *Sent:* Wednesday, July 10, 2013 9:43 PM
> *To:* user@hadoop.apache.org
> *Subject:* Re: New Distributed Cache
>
>
>
> Also, once you have the array of URIs after calling getCacheFiles  you
> can iterate over them using File class or Path (
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/Path.html#Path(java.net.URI)
> )
>
>
>
> Regards,
>
> Shahab
>
>
>
> On Wed, Jul 10, 2013 at 5:08 PM, Omkar Joshi <oj...@hortonworks.com>
> wrote:
>
> did you try JobContext.getCacheFiles() ?
>
>
>
>
> Thanks,
>
> Omkar Joshi
>
> <http://www.hortonworks.com>
>
>

-- 
Many thanks.


Bill