You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Marshall Schor <ms...@schor.com> on 2009/05/01 00:22:22 UTC

classpath for finding Key classes

In hadoop, one can define the classes to be used for Keys and Values.  I
do this.  When I make my giant Jar file holding everything needed for
running my application, I include these classes.

However, I've discovered that that is not enough it seems (in 0.19.1
version - in case that matters :-) ).  The job start up processes is
reading the configuration and finding the names of my Key classes, and
tries to load them.  But it is not using the giant Jar for my job,
(yet), so it doesn't find them.

A work-around that I've found is to include my giant Jar as the argument
to -libjars - that seems to get the class path set up so the startup /
validation code can find my classes.  This seems wasteful - having the
giant jar in two places...

Is there a best practices way to do this that's better than this?

Thanks. -Marshall Schor