Re: [drlvm][design] class unloading: Secondary root set

[Salikh Zakirov <Sa...@Intel.com>]

> Salikh Zakirov wrote:
>> I think you have missed one point: after retracing from secondary root
>> set once,
>> more classloaders may be found reachable, so this step needs to be
>> repeated until
>> convergence (to obtain the closure of reachability with additional
>> links Object->Class,
>> served through vtable marks).

Robin Garner wrote:
> My proposal doesn't require steps (2) although VM->ClassLaoder
> references are weak, and (5), because the trace from the vtable roots is
> no different fromthe standard GC trace.

Okay, It looks like you found a way to avoid repetitive retracing from added roots,
and doing it in one step. If this is the case, I see how one can do
with just enumerating vtables, and without "unload list".

However, I do not understand how can correctness and completeness can be achieved
with just one retracing. Java allows for arbitrary implementation of user-defined
class loaders, including chained (think of java application server loaded as an
application to another application server). 

> You could alternately say that I'm simply refining your approach.  Yes,
> they are structurally very similar - if you agree with my refinements,
> feel free to merge them.

I will gladly merge your ideas as soon as I understand them,
but unfortunately I cannot see how can algorithm be correct without
transitive classloader revival.

It looks to me like one-step approach at deciding whether on unloading a classloader 
can produce incorrect results in the case of multiple chained classloaders.
For example, the test below will unload two of the classloaders incorrectly
at the first System.gc(). Note, that the test works correctly on Sun Hotspot:

$ java -showversion -verbose:gc UnloadTwice
java version "1.5.0_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_05-b05)
Java HotSpot(TM) Client VM (build 1.5.0_05-b05, mixed mode)
loading UnloadTwice.class
loading UnloadTwice.class
loading UnloadTwice.class
[Full GC 284K->131K(1984K), 0.0100915 secs]  <<<<<<<< it does not unload any classloader on the first GC
[Full GC[Unloading class UnloadTwice]        <<<<<<<< it unloads all 3 classloaders at the second GC
[Unloading class UnloadTwice]
[Unloading class UnloadTwice]
 131K->131K(1984K), 0.0092772 secs]
ok

////////////////////////////////////////////////////////////////////////////

import java.io.*;
import java.lang.reflect.*;

public class UnloadTwice extends ClassLoader {

    static Object o;

    public static void init() {
        try {
            UnloadTwice cl1 = new UnloadTwice();
            ClassLoader scl = ClassLoader.getSystemClassLoader();
            cl1.setParent(scl);
            Class c1 = cl1.loadClass("UnloadTwice");
            Method sp1 = c1.getMethod("setParent", ClassLoader.class);
            ClassLoader cl2 = (ClassLoader)c1.newInstance();
            sp1.invoke(cl2, new Object[] { scl });
            Class c2 = cl2.loadClass("UnloadTwice");
            Method sp2 = c2.getMethod("setParent", ClassLoader.class);
            ClassLoader cl3 = (ClassLoader)c2.newInstance();
            sp2.invoke(cl3, new Object[] { scl });
            Class c3 = cl3.loadClass("UnloadTwice");
            o = c3.newInstance();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) {
        init();
        System.gc(); // we still have third-level-chained object o live
        o = null;
        System.gc(); // now o is gone, class unloading should happen
        System.out.println("ok");
    }

    ClassLoader parent;

    public void setParent(ClassLoader parent) {
        this.parent = parent;
    }

    public Class loadClass(String name) {
        try {
            if (!"UnloadTwice".equals(name)) return parent.loadClass(name);
            System.out.println("loading " + name + ".class");
            InputStream in = parent.getResourceAsStream(name + ".class");
            byte bytes[] = new byte[in.available()];
            in.read(bytes);
            return defineClass(bytes, 0, bytes.length);
        } catch (Exception e) { throw new RuntimeException(e); }
    }
}

Re: [drlvm][design] class unloading: Secondary root set

[Robin Garner <ro...@anu.edu.au>]

Salikh Zakirov wrote:
>> Salikh Zakirov wrote:
>>> I think you have missed one point: after retracing from secondary root
>>> set once,
>>> more classloaders may be found reachable, so this step needs to be
>>> repeated until
>>> convergence (to obtain the closure of reachability with additional
>>> links Object->Class,
>>> served through vtable marks).
> 
> Robin Garner wrote:
>> My proposal doesn't require steps (2) although VM->ClassLaoder
>> references are weak, and (5), because the trace from the vtable roots is
>> no different fromthe standard GC trace.
> 
> Okay, It looks like you found a way to avoid repetitive retracing from added roots,
> and doing it in one step. If this is the case, I see how one can do
> with just enumerating vtables, and without "unload list".
> 
> However, I do not understand how can correctness and completeness can be achieved
> with just one retracing. Java allows for arbitrary implementation of user-defined
> class loaders, including chained (think of java application server loaded as an
> application to another application server). 

Because: By tracing from the live vtables, you are using the complete 
transitive closure mechanism of the GC.  Once a classloader becomes 
reachable, its pointers will be enumerated, all the classes that belong 
to it will be traced, and anything reachable from them will be live, 
including other classloaders.

Or am I missing something ?

Wait, I suppose if j.l.ClassLoaders can point to ordinary heap objects 
then they in turn could revive vtables which could revive classloaders - 
yes, you might need to iterate until no new vtables are revived.

>> You could alternately say that I'm simply refining your approach.  Yes,
>> they are structurally very similar - if you agree with my refinements,
>> feel free to merge them.
> 
> I will gladly merge your ideas as soon as I understand them,
> but unfortunately I cannot see how can algorithm be correct without
> transitive classloader revival.
> 
> It looks to me like one-step approach at deciding whether on unloading a classloader 
> can produce incorrect results in the case of multiple chained classloaders.
> For example, the test below will unload two of the classloaders incorrectly
> at the first System.gc(). Note, that the test works correctly on Sun Hotspot:
> 
> $ java -showversion -verbose:gc UnloadTwice
> java version "1.5.0_05"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_05-b05)
> Java HotSpot(TM) Client VM (build 1.5.0_05-b05, mixed mode)
> loading UnloadTwice.class
> loading UnloadTwice.class
> loading UnloadTwice.class
> [Full GC 284K->131K(1984K), 0.0100915 secs]  <<<<<<<< it does not unload any classloader on the first GC
> [Full GC[Unloading class UnloadTwice]        <<<<<<<< it unloads all 3 classloaders at the second GC
> [Unloading class UnloadTwice]
> [Unloading class UnloadTwice]
>  131K->131K(1984K), 0.0092772 secs]
> ok
> 
> ////////////////////////////////////////////////////////////////////////////
> 
> import java.io.*;
> import java.lang.reflect.*;
> 
> public class UnloadTwice extends ClassLoader {
> 
>     static Object o;
> 
>     public static void init() {
>         try {
>             UnloadTwice cl1 = new UnloadTwice();
>             ClassLoader scl = ClassLoader.getSystemClassLoader();
>             cl1.setParent(scl);
>             Class c1 = cl1.loadClass("UnloadTwice");
>             Method sp1 = c1.getMethod("setParent", ClassLoader.class);
>             ClassLoader cl2 = (ClassLoader)c1.newInstance();
>             sp1.invoke(cl2, new Object[] { scl });
>             Class c2 = cl2.loadClass("UnloadTwice");
>             Method sp2 = c2.getMethod("setParent", ClassLoader.class);
>             ClassLoader cl3 = (ClassLoader)c2.newInstance();
>             sp2.invoke(cl3, new Object[] { scl });
>             Class c3 = cl3.loadClass("UnloadTwice");
>             o = c3.newInstance();
>         } catch (Exception e) {
>             e.printStackTrace();
>         }
>     }
> 
>     public static void main(String[] args) {
>         init();
>         System.gc(); // we still have third-level-chained object o live
>         o = null;
>         System.gc(); // now o is gone, class unloading should happen
>         System.out.println("ok");
>     }
> 
>     ClassLoader parent;
> 
>     public void setParent(ClassLoader parent) {
>         this.parent = parent;
>     }
> 
>     public Class loadClass(String name) {
>         try {
>             if (!"UnloadTwice".equals(name)) return parent.loadClass(name);
>             System.out.println("loading " + name + ".class");
>             InputStream in = parent.getResourceAsStream(name + ".class");
>             byte bytes[] = new byte[in.available()];
>             in.read(bytes);
>             return defineClass(bytes, 0, bytes.length);
>         } catch (Exception e) { throw new RuntimeException(e); }
>     }
> }
> 
> 


-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/