You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Vyacheslav Zholudev <vy...@gmail.com> on 2011/11/25 13:26:36 UTC

Archives in a distributed cache problem

Hi,

I have a problem with adding a zip archive to a dist cache.
I'm adding it during job setup with:
DistributedCache.addCacheArchive(archiveURI, jobConf);

where archiveURI is of type file:///

and later on reading it in Mapper.configure() with:
Path[] paths = DistributedCache.getLocalCacheArchives(jobConf);
LOGGER.info(paths[0].getFileSystem(jobConf).exists(paths[0]))

On my machine it runs fine, and a logger outputs "true".
On another the path does not exist (a logger outputs "false") and I also get a WARNING:
WARNING: Cannot find size and number of subdirectories of baseDir: /tmp/hadoop-jenkins/mapred/local
which I suspect is somehow related to an issue.

In both cases I'm running a hadoop job in the local mode. 
Does anybody have an idea what can be wrong?

P.S. My hadoop version is 0.20.2

Thanks in advance,
Vyacheslav