Stale job-cache - any experiences with this?

[Jiaqi Tan <ta...@gmail.com>]

Hi,

I seem to be having a problem with the job cache on the JobTracker of
my (single-node, DataNode, NameNode, JobTracker, TaskTracker all on
same machine) Hadoop cluster. I've been recompiling my code multiple
times, and the class files in my jarfile are correct (unzipped,
checked for new string constants added in println's to check that the
new version was correctly compiled), but the class files in the
mapred.local directory in the taskTracker/jobcache/<job id>/jars
directory are the ones from the old version.

Has anyone encountered a similar problem where your newly compiled
code just refuses to run and every time you submit the new jar file
some older version ends up getting executed?

Thanks,
Jiaqi

Re: Stale job-cache - any experiences with this?

[jason hadoop <ja...@gmail.com>]

You must have the classes in a jar that is already in your class path, and
that jar is getting picked up and distributed.

On Tue, Jun 2, 2009 at 9:53 AM, Jiaqi Tan <ta...@gmail.com> wrote:

> Hi,
>
> I seem to be having a problem with the job cache on the JobTracker of
> my (single-node, DataNode, NameNode, JobTracker, TaskTracker all on
> same machine) Hadoop cluster. I've been recompiling my code multiple
> times, and the class files in my jarfile are correct (unzipped,
> checked for new string constants added in println's to check that the
> new version was correctly compiled), but the class files in the
> mapred.local directory in the taskTracker/jobcache/<job id>/jars
> directory are the ones from the old version.
>
> Has anyone encountered a similar problem where your newly compiled
> code just refuses to run and every time you submit the new jar file
> some older version ends up getting executed?
>
> Thanks,
> Jiaqi
>



-- 
Alpha Chapters of my book on Hadoop are available
http://www.apress.com/book/view/9781430219422
www.prohadoopbook.com a community for Hadoop Professionals