IsolatedExecutor

[Peter Firmstone <ji...@zeus.net.au>]

Background:

The java platform has three major issues for River:

   1. Single shared memory heap.
   2. No process isolation, we have to rely solely on the SecurityManager.
   3. Class visibility - ClassLoader inheritance hierarchy.


You might remember me experimenting recently with isolated threads and 
memory error handling.  Well that experiment did bare some fruit:

    * When a memory error occurs, you can catch it, free some memory and
      interrupt the current thread, allowing it to die.
    * The jvm hasn't necessarily completely run out of memory, it just
      wasn't able to allocate the last object.
    * Once a memory error occurs (in IsolatedExecutor), the task that
      was running terminates and the thread that caused the error is
      interrupted.

Caveats:

    * It only works with Executors.
    * It doesn't work for threads outside the Executor, if a non
      Executor thread receives the memory error first, then it's bye bye
      jvm.

Extension ClassLoader default permission is AllPermission, but you can 
change it to a minimal set of permissions for each CodeSource.

Steps toward using a subprocess JVM as an Executor:

    * Make your task and it's result Serializable.
    * Only run tasks in IsolatedExecutors.
    * Make discovery a task
    * Create Jeri Endpoint's for Subprocess streams.
    * Give each task it's own ClassLoader, including jini platform classes.
    * Jeri must be packaged separately in it's own jar - belongs in the
      subprocess boot kernel, load in the Applicaton ClassLoader or
      Extension ClassLoader.
    * DynamicPolicy is installed in the extension ClassLoader.
    * DownloadPermission must be checked by default.

Benefits:

   1. Tasks don't share classes (except for jeri and IsolatedExecutor)
      and don't have visibility into other executing tasks.
   2. If the memory available isn't sufficient for a task, the memory
      error will be caught, the task won't bring down the entire
      subprocess jvm.
   3. The local jvm gets defensive copies of all results due to
      serialization.

What's left to figure out:

   1. Tasks must be written to export a reflective proxy, leaving the
      result in the subprocess jvm.
   2. How to handle reflective proxy's that are transferred to other
      remote nodes, but refer to a smart proxy in the local subprocess
      jvm.  How do we replace the reflective proxy with the smart proxy?


Cheers,

Peter.