Question about avoiding reflection with isolated class loader

[Guozhang Wang <wa...@gmail.com>]

Hi,

I have a question that is probably related to MAPREDUCE-1700 / 5751 / 5957
<https://issues.apache.org/jira/browse/SPARK-1870>. Basically I have also
encountered the issue that with separate classloaders while developing a
programming framework where I have to use reflection inside the application
code.

To simply the question, let's say my framework code depends on a jar with
an older version and the application code depends on the same jar with a
newer version (for example Avro 1.4 and 1.7). And let's say that jar has a
class named CommonClass. I used a customized post-delegation class loader
like Hadoop does so that these two version can exist separately into
separate class loader. And if I do sth. like:

    public static void main(String arg[]) throws Exception {
        // Say this is framework code
        new CommonClass().printJarVersion();

        // Set the customized class loader as thread context class loader
        Thread thread = Thread.currentThread();
        ClassLoader oldClassLoader = thread.getContextClassLoader();
        File appJar = new File("/dir/to/app/class/path");
        URL[] classpath = new URL[] { appJar.toURI().toURL() };
        PostDelegationClassLoader newClassLoader = new
PostDelegationClassLoader(classpath);
        thread.setContextClassLoader(newClassLoader);

        try {
            // Say this is application code, like process().
            new CommonClass().printJarVersion();

thread.getContextClassLoader().loadClass("CommonClass").newInstance().printJarVersion();
        } finally {
            thread.setContextClassLoader(oldClassLoader);
        }
    }

-----------------------

It will print:

CommonClass: version 1
CommonClass: version 1
CommonClass: version 2

As one can see I have to use reflection to explicitly specify the
customized class loader if I want to create the class with the new version.
This is definitely bad for the users. I saw there are some discussions
around 1700 and am wondering how Hadoop resolved this issue? Any help is
greatly appreciated.

-- Guozhang