Re: Distributed Garbage Collection & Security - InvocationConstraints

[Peter Firmstone <ji...@zeus.net.au>]

Peter Jones wrote:
> On Jun 19, 2011, at 5:37 AM, Peter Firmstone wrote:
>   
>> The easiest way to set DGC constraints would be via configuration.
>>
>> Perhaps the reason this hasn't been implemented previously is, the constraints would apply to all services that use DGC, so if you've set Authentication and Integrity as minimal constraints, then this would apply to all services.
>>     
>
> As mentioned offline, I'm not sure how much of this I can page into my consciousness at the moment.
>
> Configuring the server-side constraints for DGC could probably be supported with additional parameters to the export process (and thus set via configuration).  The bigger issue, I think, is what client constraints to apply, and what client subject to use, when JERI's client-side DGC system (defined within BasicObjectEndpoint) makes "dirty" and "clean" calls on behalf of the application.  In the traditional RMI DGC model, those calls happen implicitly as remote references appear and disappear from a JVM.  But in the JERI security model, the client application controls the security behavior of remote calls by explicitly (with respect to the standard JERI layers) specifying constraints and controlling the current subject.
>
> So when the system wants to make a "dirty" or "clean" call for a given remote reference (forget batching for the moment), what constraints to apply, or what subject to use?  There didn't seem to be an answer, without requiring the client application to interact with the DGC system more explicitly, which would be a significant change from the RMI DGC model-- and, I think, not something that seemed worth investing effort on at the time, especially given that Jini services didn't seem to make use of RMI's DGC functionality in practice anyway (instead they used higher-level leasing mechanisms to detect client "failure", and most interest was around just being disable DGC for Jini services).
>   

Having thought about it, something with similar behaviour to a local 
JVM's gc would be appropriate, for example, if code with higher 
privilege allows a reference to escape into less privileged code, the 
garbage collector won't collect it until it becomes unreferenced by all 
code.

So it would seem appropriate to cache the first Subject (or Subjects) 
who've called methods on the remote object and use it (them) for the DGC 
client Subject.  Different Target's might have different Subject's, so 
we could perform Subject orientated batching.

The unprivileged code, wouldn't be able to call methods that had 
constraints, similar to how an object with method guards works, however 
any code that holds a strong reference would prevent that object from 
being garbage collected.

I think this would be useful in an internet environment, where a 
developer wants to export an object and hand it as a parameter to 
another service, and have it unexported automatically when no longer 
required.

Cheers,

Peter.


>   
>> Exporter's javadoc has the following statement regarding the force parameter in unexport:
>>
>> <QUOTE>
>>
>> The |force| parameter serves to indicate whether or not the caller desires the unexport to occur even if there are known remote calls pending or in progress to the remote object that were made possible by this |Exporter|:
>>
>>   * If |force| is |true|, then the remote object will be forcibly
>>     unexported even if there are remote calls pending or in progress,
>>     and this method will return |true|.
>>   * If |force| is |false|, then this acts as a hint to the
>>     implementation that the remote object should not be unexported if
>>     there are known remote calls pending or in progress, and this
>>     method will either unexport the remote object and return |true| or
>>     not unexport the remote object and return |false|. If the
>>     implementation detects that there are indeed remote calls pending
>>     or in progress, then it should return |false|; otherwise, it must
>>     return |true|. If the implementation does not support being able
>>     to unexport conditionally based on knowledge of remote calls
>>     pending or in progress, then it must implement this method as if
>>     |force| were always |true|.
>>
>> If the remote object is unexported as a result of this method, then the implementation may (and should, if possible) prevent remote calls in progress from being able to communicate their results successfully.
>>
>> </QUOTE>
>>
>> I've updated the class Target that implements this functionality for BasicJeriExporter, Target's unexport method now uses thread interruption to attempt to interrupt and abort in process calls if force is specified.  Interruption has been successful with the current jeri qa tests and can be seen in the exception output for some tests.
>>     
>
> FWIW, aborting the execution of in-progress calls, such as via thread interruption, wasn't really the intent of that last sentence-- it was more that an implementation should feel free (or encouraged) to prevent communicating the eventual result of such a call, when control of the dispatching thread is returned to this layer of the system.
>
> Cheers,
>
> -- Peter
>
>
>   

Re: Distributed Garbage Collection & Security - InvocationConstraints

[Peter Firmstone <ji...@zeus.net.au>]

Peter wrote:
> ----- Original message -----
>   
>> On 7/31/2011 5:35 AM, Peter Firmstone wrote:
>>     
>>> I think this would be useful in an internet environment, where a developer
>>> wants to export an object and hand it as a parameter to another service, and
>>> have it unexported automatically when no longer required.
>>>       
>> This is why I make my smart proxy classes be InvocationHandlers and then embed
>> the real remote object and a lease into the marshalled object.  With deferred
>> unmarshalling, then others who unmarshall it for use, apply a
>> LeaseRenewalManager on the lease as well so that everyone using it is making
>> sure the server is not releasing it.
>>
>> Endpoints as they exist today, are not mobile because of code contamination and
>> lost-code-base issues that occur when a remote object is remarshalled to send
>> across the wire.
>>
>> Multiple unmarshalling activities in the same JVM might not be safe if there is
>> static class initialization that should of been per-instance.
>>
>> I think this is an area that needs some investigation for sure.
>>
>> Gregg
>>     
>
> Spot on, I think we can fix this by defining the jini platform that should be available on the classpath then stick to it.  Anything else required (apart from service api) must go into child classloaders.
>
> Anything that's serializable in the platform must remain backward compatible (serialized form) and must be present in all nodes.
>
> We should attempt to put implementation into a child classloader, so it isn't resolved prior to proxy classloader class resolution, allowing proxy's to have their own implementations.
>
> Instead of using the current preferred classes mechanism, the order of class resolution would be:
>
> Proxy classloader, delegates first to the application classloader. (classpath), which delegates to the extension and system classloaders, then finally it try's the proxy classloader own codebase.
>
> If a class is still unresolved then we could try the Platform Implementation classloader, which could be obtained by a static class reference.  Alternatively, platform implementation classes shouldn't be serializable.
>
> Services shouldn't be visible in the application classloader, only their service api.
>
> We can provide abstract classes and interfaces for the platform, for various implementations to cooperate with, while preserving implementation codebase annotations and proxy and service implementations separated in their own classloaders (because these classes don't exist in the parent classloader).  
>
> Different implementations can coexist in separate classloaders and cooperate using common superclasses and interfaces in the application, extension or system classloaders. 
>
> Shared state can be managed in abstract classes using protected static methods with synchronization and security checks, all static fields private.  Abstract object state can be similarly manged with protected methods and private fields.
>
> We have the advantage that service api already utilise interfaces.  
>
> Along with a rename of the com.sun namespace, this is an opportunity to fix some long standing issues that have discouraged wider adoption.
>
> It should be possible to pass on distributed objects to any node without issues, this is the fundamental value of Jini over java rmi.
>
> Ideally: There is no dependency on implementation, only api.  Implementation shouldn't be known until runtime.
>
> This isn't an easy exercise, but I think that Dennis' recent work with Rio might provide some insight.
>
> Separating the platform from service implementations, and providing libraries for service implementers to deploy and developing these separately will make it clearer what developers can change without worrying about backward compatibility. Eg libraries should be free to change.  The platform should be stable and evolve with backward compatibility in mind.
>
> This is difficult, but possible, it will take time and cooperation to figure out the right solutions, on a case by case basis.
>
> If we can figure out these fundamental issues, we just possibly might hold the future of distributed computing in our hands.
>
> Cheers,
>
> Peter.
>   

Another point: the application should also be in it's own child 
classloader.  The River implementation of the Jini Platform, is similar 
to Java, it's a platform, so why not load River applications into a 
child classloader, similar to Java, a platform where most application 
classes are loaded by a child classloader (Known as the application 
classloader or classpath) and where the extension classloader is 
provided to extend the platform.  Without realising it, we extend the 
Jini Platform by loading application classes into the same classloader 
as the platform.

Perhaps the Jini Platform needs it's own application classpath.  
Application classes shouldn't be visible with the jini platform classes, 
since this could lead to codebase annotation loss.  Only Service API 
classes and interfaces should be visible in the jini platform.

Cheers,

Peter.

Re: Distributed Garbage Collection & Security - InvocationConstraints

[Peter <ji...@zeus.net.au>]

----- Original message -----
> On 7/31/2011 5:35 AM, Peter Firmstone wrote:
> > I think this would be useful in an internet environment, where a developer
> > wants to export an object and hand it as a parameter to another service, and
> > have it unexported automatically when no longer required.
>
> This is why I make my smart proxy classes be InvocationHandlers and then embed
> the real remote object and a lease into the marshalled object.  With deferred
> unmarshalling, then others who unmarshall it for use, apply a
> LeaseRenewalManager on the lease as well so that everyone using it is making
> sure the server is not releasing it.
>
> Endpoints as they exist today, are not mobile because of code contamination and
> lost-code-base issues that occur when a remote object is remarshalled to send
> across the wire.
>
> Multiple unmarshalling activities in the same JVM might not be safe if there is
> static class initialization that should of been per-instance.
>
> I think this is an area that needs some investigation for sure.
>
> Gregg

Spot on, I think we can fix this by defining the jini platform that should be available on the classpath then stick to it.  Anything else required (apart from service api) must go into child classloaders.

Anything that's serializable in the platform must remain backward compatible (serialized form) and must be present in all nodes.

We should attempt to put implementation into a child classloader, so it isn't resolved prior to proxy classloader class resolution, allowing proxy's to have their own implementations.

Instead of using the current preferred classes mechanism, the order of class resolution would be:

Proxy classloader, delegates first to the application classloader. (classpath), which delegates to the extension and system classloaders, then finally it try's the proxy classloader own codebase.

If a class is still unresolved then we could try the Platform Implementation classloader, which could be obtained by a static class reference.  Alternatively, platform implementation classes shouldn't be serializable.

Services shouldn't be visible in the application classloader, only their service api.

We can provide abstract classes and interfaces for the platform, for various implementations to cooperate with, while preserving implementation codebase annotations and proxy and service implementations separated in their own classloaders (because these classes don't exist in the parent classloader).  

Different implementations can coexist in separate classloaders and cooperate using common superclasses and interfaces in the application, extension or system classloaders. 

Shared state can be managed in abstract classes using protected static methods with synchronization and security checks, all static fields private.  Abstract object state can be similarly manged with protected methods and private fields.

We have the advantage that service api already utilise interfaces.  

Along with a rename of the com.sun namespace, this is an opportunity to fix some long standing issues that have discouraged wider adoption.

It should be possible to pass on distributed objects to any node without issues, this is the fundamental value of Jini over java rmi.

Ideally: There is no dependency on implementation, only api.  Implementation shouldn't be known until runtime.

This isn't an easy exercise, but I think that Dennis' recent work with Rio might provide some insight.

Separating the platform from service implementations, and providing libraries for service implementers to deploy and developing these separately will make it clearer what developers can change without worrying about backward compatibility. Eg libraries should be free to change.  The platform should be stable and evolve with backward compatibility in mind.

This is difficult, but possible, it will take time and cooperation to figure out the right solutions, on a case by case basis.

If we can figure out these fundamental issues, we just possibly might hold the future of distributed computing in our hands.

Cheers,

Peter.

Re: Distributed Garbage Collection & Security - InvocationConstraints

[Gregg Wonderly <gr...@wonderly.org>]

On 7/31/2011 5:35 AM, Peter Firmstone wrote:
> I think this would be useful in an internet environment, where a developer wants
> to export an object and hand it as a parameter to another service, and have it
> unexported automatically when no longer required.

This is why I make my smart proxy classes be InvocationHandlers and then embed 
the real remote object and a lease into the marshalled object.  With deferred 
unmarshalling, then others who unmarshall it for use, apply a 
LeaseRenewalManager on the lease as well so that everyone using it is making 
sure the server is not releasing it.

Endpoints as they exist today, are not mobile because of code contamination and 
lost-code-base issues that occur when a remote object is remarshalled to send 
across the wire.

Multiple unmarshalling activities in the same JVM might not be safe if there is 
static class initialization that should of been per-instance.

I think this is an area that needs some investigation for sure.

Gregg