HADOOP-189 and the context class loader

[Benson Margulies <bi...@gmail.com>]

We have fairly good evidence that, as of 0.20.2, hadoop does not set
the thread context class loader to the class loader than includes all
the .jar files from the lib subdirectory of a job jar.

Code we wrote (which is sitting in the 'main' part of the job jar)
calls a class in Mahout (which is sitting in a jar in the 'lib'
directory). That code calls loadClass on
Thread.currentThread().getContextClassLoader() looking for the lucence
StandardAnalyzer. *That* class is sitting in the lucene-core jar,
which is also in the job jar's lib dir.

The result of a class not found exception.

I've checked that we didn't somehow mistakenly unpack lucene in to the
main area (thus ending up with two copies, but I'll check again).

So, the naive interpretation of this is that nothing calls
setContextClassLoader, but I haven't gone reading hadoop source to
check that; I'm asking here instead.