Implementation Decision

[Peter Firmstone <ji...@zeus.net.au>]

Hi,

In the version package I'm writing, to manage dynamic class file 
upgrades for applications / services / distributed objects in a djinn, 
I'm having to make a choice.

At present, I've decided to upgrade as a package level transaction, 
rather than at the individual class file level, these are my reasons:

 1. MobileObjectDelegate (A local proxy that remains in one ClassLoader 
to delegate methods to an Object subject to Type change due to movement 
between ClassLoaders during class file upgrades) only have to represent 
Public API and Public Classes. Package private classes and methods don't 
need to be represented, reducing programming complexity.

 2. It allows objects to be migrated from one ClassLoader to another, 
allowing the retirement of the earlier ClassLoader.

 3. If migrating all objects in a Package from one ClassLoader to 
another, it allows one to roll back if Exceptions occur as all old 
objects still exist until strong references are released by the 
MobileObjectDelegates.

There are some caveats in this decision however:

 1. When a package is notified of a new class file version during 
unmarshalling of a remote object, where an old version of that class 
exists locally, it would trigger a package update, it couldn't commence 
using
the object, until all existing objects from the package are migrated, 
causing a delay related directly to the number of objects and 
implementation of Serializable itself.  The delay might not be 
acceptable.  This process might also be triggered by an object within 
the package receiving an ObsoleteMethodException from an external method 
call.

 2. All MobileObjectDelegates would have to be allowed to finish their 
processing, locks acquired on all and any Exported MobileObjectDelegates 
unexported.  This would then be followed by a producer consumer pattern, 
taking advantage of the Executor framework to serialize all object 
instances to their new ClassLoader.  Once this process has completed, 
the MobileObjectDelegate locks would be freed and the program allowed to 
continue.

The alternative scenario is that all Classes in a package have a 
corresponding MobileObjectDelegate class, I haven't got my head around 
whether MobileObjectDelegates could be generated instead of written by 
the programmer similar to Exporter.  Any ideas?  If all package Objects 
had corresponding MobileObjectDelegate Objects, then the package could 
be migrated while still processing other requests.  However this also 
puts more restrictions on the programmer as objects referenced by 
MobileObjectDelegates can't have ordinary constructors.  Any ideas how 
to implement ordinary constructors with auto generated 
MobileObjectDelegates?

Best Regards,

Peter.

Re: Implementation Decision

[Peter Firmstone <ji...@zeus.net.au>]

I'm a tad disappointed no one's asking why I'm not using Uuid for object 
identity?

Firstly, I'd like to.

Let me explain:

The purpose of versioning is to keep important long lived Objects alive 
while code evolves.  One of the projects I had recently was related to 
charge out rates & contracts for several different employment agreements...

Currently, you have to shut down your djinn if you wish to upgrade your 
code and codebases can move etc, need to use something like hibernate or 
JDBC to persist your objects and tackle the whole object relational 
mapping impedance problem, ie, splitting an object into a number of 
tables & reconstituting .  This isn't suitable if we're ever going to 
expand River out into the web.

Not all object's need to persist over upgrades.

Back to the question, the problem, Uuid is a class who's objects may be 
required to live for long periods of time, it is precisely the type of 
object that should be preserved.  Why haven't I chosen to use it yet?  I 
get the feeling I need to.  Currently I have been thinking of River as a 
library, such that if it received an upgrade, the new version would be 
loaded into another ClassLoader to avoid conflicts.  That's the problem, 
Uuid now has two Types, and it's equals method returns false for the 
same Uuid object if using separate Type's.

For this to work there needs to be a small core or base that doesn't 
change, the other packages or jar files can change or be used as 
libraries, however any River classes that produce long lived objects, 
either need to support versioning or not change or be installed locally 
and backward compatible, just like the JRE API itself.

It is very important to keep the version API simple and mostly 
restricted to interfaces or classes that don't change, hence the need to 
have you all review and comment.  I'm not the most experienced but I'm 
having a go.  I'm hoping that those among you who have considered this 
problem before can look at the code and make suggestions.

Best Regards,

Peter.

Re: Implementation Decision

[Peter Firmstone <ji...@zeus.net.au>]

Clarifications:

The package upgrade process is a local JVM process.
Each VersionedPublicClass (VPC) is only Accessible by one 
MobileObjectDelegate (MOD), it's reference doesn't escape.
All versioning interfaces resides in a parent ClassLoader, accessible to 
both packages.  All interfaces implemented by a package would be common 
to both versions and exist in a separate package for interfaces only, 
interface checksum's don't change, so the new interfaces would be loaded 
into the same interface ClassRealm available to all Package ClassRealm's

During a Package Upgrade, Locks wouldn't need to be acquired on VPC's by 
all MOD's, there is opportunity for optimisation based on the following 
Categories of Object:

1.  An AtomicReference (the VPC field inside the MOD) could be used in 
place of a lock provided no Object methods modify global state or state 
of shared package private objects, such that the replacement VPC copy 
Object can be created without loss of information or state while the 
original is still in use.  Definition: Immutable Object

2. Acquiring a lock on the VPC reference field inside the MOD could be 
delayed until serialization, allowing the class to remain available, 
provided that it only modifies its own state, it does not modify shared 
state, no other objects can directly access any fields within that 
object (no references are published or escape, eg defensive copies are 
returned etc).  Definition: Mutable Object no Class (static) methods, no 
references escape.

3. The remaining MOD's that have mutable shared global state and shared 
package private mutable objects (excluding package external public API 
state) would need to aquire locks on the VPC's they represent. (You 
can't unscramble an egg!)    If a package can minimise mutable shared 
package private state and shared class state, then the upgrade process 
may go unnoticed in the background.   The Object trees of these VPC's 
would be moved first and wouldn't commence until their entire group has 
acquired locks.  A ReentrantReadWriteLock that stops any new Read 
requests (method calls of VPC's), hopefully this wouldn't cause any 
deadlocks!  A WriteLock would be held on the reference to the VPC.   
This could be done by a method on the MOD called by the 
DynamicVersionedPublicClassLoader (DVPCL) that causes the MOD to aquire 
a WriteLock and submit the VPC to a que managed by their DVPCL for 
processing, updating the the reference with the new VPC on return.  Once 
these have been processed the MOD's can resume servicing requests.

All MOD's of category 2 could submit a request (to their DVPCL) to be 
notified when a thread becomes available (the MobileObjectDelegate is 
first notified that it needs to request an upgrade) and delay aquiring a 
lock on the VPC reference until that time.

MOD's of category 1 would submit a request to be notified when a thread 
becomes available and update their VPC reference Atomically, these would 
be processed last.

Peter Firmstone wrote:
> Hi,
>
> In the version package I'm writing, to manage dynamic class file 
> upgrades for applications / services / distributed objects in a djinn, 
> I'm having to make a choice.
>
> At present, I've decided to upgrade as a package level transaction, 
> rather than at the individual class file level, these are my reasons:
>
> 1. MobileObjectDelegate (A local proxy that remains in one ClassLoader 
> to delegate methods to an Object subject to Type change due to 
> movement between ClassLoaders during class file upgrades) only have to 
> represent Public API and Public Classes. Package private classes and 
> methods don't need to be represented, reducing programming complexity.
>
> 2. It allows objects to be migrated from one ClassLoader to another, 
> allowing the retirement of the earlier ClassLoader.
>
> 3. If migrating all objects in a Package from one ClassLoader to 
> another, it allows one to roll back if Exceptions occur as all old 
> objects still exist until strong references are released by the 
> MobileObjectDelegates.
>
> There are some caveats in this decision however:
>
> 1. When a package is notified of a new class file version during 
> unmarshalling of a remote object, where an old version of that class 
> exists locally, it would trigger a package update, it couldn't 
> commence using
> the object, until all existing objects from the package are migrated, 
> causing a delay related directly to the number of objects and 
> implementation of Serializable itself.  The delay might not be 
> acceptable.  This process might also be triggered by an object within 
> the package receiving an ObsoleteMethodException from an external 
> method call.
>
> 2. All MobileObjectDelegates would have to be allowed to finish their 
> processing, locks acquired on all and any Exported 
> MobileObjectDelegates unexported.  This would then be followed by a 
> producer consumer pattern, taking advantage of the Executor framework 
> to serialize all object instances to their new ClassLoader.  Once this 
> process has completed, the MobileObjectDelegate locks would be freed 
> and the program allowed to continue.
>
> The alternative scenario is that all Classes in a package have a 
> corresponding MobileObjectDelegate class, I haven't got my head around 
> whether MobileObjectDelegates could be generated instead of written by 
> the programmer similar to Exporter.  Any ideas?  If all package 
> Objects had corresponding MobileObjectDelegate Objects, then the 
> package could be migrated while still processing other requests.  
> However this also puts more restrictions on the programmer as objects 
> referenced by MobileObjectDelegates can't have ordinary constructors.  
> Any ideas how to implement ordinary constructors with auto generated 
> MobileObjectDelegates?
>
> Best Regards,
>
> Peter.
>
>