Re: Service API, Interfaces, Evolution, Lookup & ClassLoader's again

[Gregg Wonderly <ge...@cox.net>]

Remember, that the serviceUI actually involves the use of a MarshalledObject, 
which is created in some context where an RMIClassLoaderSPI instance will have 
provided some kind of annotation value.  If a service creates the serviceUI 
instance by creating an Array of Entry objects in it's Configuration for 
example, then the annotation will be exactly the same as the service proxy in 
most cases.  But, as you note, the flexibility provided by the use of the 
MarshalledObject and the fact that it exists as an Entry means that it can point 
anywhere for Class resolution.

What is important in the use of ServiceUI, is that you must note the ClassLoader 
that the serviceUI resolves with, and set that up as the context class loader 
for use in the serviceUI activities.  When the service proxy is loaded by a 
different class loader than the serviceUI, then there is some monkey business 
that has to occur in the serviceUI factory.  It has to note the passed 
ServiceItem.service classes ClassLoader and make use of it when calling into the 
proxy methods.  With PreferredClassLoader, this is a vital aspect of maintaining 
the correct class use, both for security as well as correctness.

Gregg Wonderly

Peter Firmstone wrote:
> I realise too that this decision affects ServiceUI.
> 
> In the case of ServiceUI, the ServiceAPI is not known to the client 
> directly, it is known to the ServiceUI, which is known to the Client, so 
> the Service API, must be visible to the ServiceUI, which is visible to 
> the client.
> 
> Since there may be any number of ServiceUI that are platform or client 
> specific, it is the client that knows which ServiceUI is suitable.
> 
> ServiceUI is an implementation detail, it can be displayed into a JFrame 
> or something similar depending on platform.
> 
> If the ServiceAPI only exists in the Proxy ClassLoader, then the 
> ServiceUI must also be loaded into the Proxy ClassLoader. Reading the 
> source for UIDescriptor, this is the intended behaviour for ServiceUI.
> 
> I had originally imagined the dumb proxy's could be loaded into the same 
> ClassLoader as the Service API, each proxy with its own 
> ProtectionDomain, as this would save memory.  However the ServiceUI 
> classes cannot be loaded there.
> 
> In this case the ServiceUI will need it's own child ClassLoader.
> 
> Also there is the case of a dumb proxy (java.lang.reflect.Proxy), where 
> Service API in the client is old and the Proxy has been created with an 
> Extended Service API, that requires new classes.  It will not be 
> possible to unmarshall this dumb proxy, unless it is given it's own 
> class loader in which to load any new Service API class files and the 
> proxy.
> 
> It is very likely that over time, Service API will be extended, this 
> will cause earlier clients to consume additional memory, as they create 
> new ClassLoader's to accommodate additional interfaces required by dumb 
> proxy's.
> 
> It may instead be preferable to provide a single ClassLoader, a child of 
> the AppClassLoader that loads any additional Service API for dumb 
> proxy's and all dumb proxy's.
> 
> dumb proxy's with ServiceUI's will probably need a child classLoader of 
> this for each ServiceUI, in case of implementation variation's between 
> ServiceUI's
> 
> With these additional concerns, I provide a slightly revise class loader 
> tree.
> 
> _______________________________________________________________
> |                                                               |
> |           AppClassLoader                                      |
> |                |_________________________                     |
> |                |                         |                    |
> |                |                  Extended Service API        | 
> Extended Service API
> |                |                   Dumb Proxy ClassLoader     | Only 
> visible to Proxy
> |                |                         |                    | and 
> ServiceUI.
> |                |                   Dumb Proxy ServiceUI       |
> |                |                      ClassLoader's           |
> |                |                                              |
> |                |                                              |
> |          Common Classloader (As Per Dennis' comments)         |
> |       _________|_______________________________________       |
> |      |             |                 |                 |      |
> | Service Imp    Smart Proxy      Parameter Impl   Application  | Child 
> Implementation
> | ClassLoader's  ClassLoader's    ClassLoader's    ClassLoader  | 
> ClassLoader's
> |_______________________________________________________________|
> 
> 
> AppClassLoader - Contains the main() class of the container. Main-Class 
> in manifest points to         com.sun.jini.start.ServiceStarter
> Classpath:  boot.jar, start.jar, jsk-platform.jar, service-api.jar 
> (mulitple Service API jar's allowed, can be untrusted)
> Codebase: none
> 
> CommonClassLoader - Contains the common Rio and Jini technology classes 
> (and other declared common platform JARs) to be made available to its 
> children.
> Classpath: Common JARs such as rio.jar
> Codebase: Context dependent. The codebase returned is the codebase of 
> the specific child CL that is the current context of the request. (Not 
> Sure about the last sentence please explain?)
> 
> Child ClassLoader's - Contains the service specific implementation 
> classes and client application classes.
> Classpath: serviceImpl.jar
> Codebase: "serviceX-dl.jar rio-dl.jar jsk-lib-dl.jar"
> 
> Extended Service API ClassLoader - Contains any Extended Service API not 
> present locally to enable unmarshalling of dumb proxy's that require 
> extended interfaces.  Dumb proxy's can all be loaded into this ClassLoader.
> Extended Service API is not visible to the client.
> Codebase: service-api.jar (only classes missing locally will be loaded).
> 
> Dumb Proxy ServiceUI ClassLoader's - Contains ServiceUI's for dumb 
> proxy's, implementation's may vary and are essentially independent.
> Codebase: serviceUI.jar.
> 
> Note: ServiceUI for smart proxy's will be loaded into the Smart Proxy 
> ClassLoader with the smart proxy.
> 
> If you can see problems I've overlooked with the above approach, please 
> indicate, but also see if you can help with a solution or alternate 
> arrangement.
> 
> Cheers,
> 
> Peter.
> 
> Peter Firmstone wrote:
>> Note this is static Service API, it is not dynamically loaded.
>>
>> I remember now, why I had initially thought about dynamically loadable 
>> Service API, it was for cases where the Service API was extended as it 
>> evolved, to avoid Proxy unmarshalling errors. However as suggested at 
>> the bottom of the previous message, this can be worked around by 
>> including the Service API in the proxy's *-dl.jar, in addition to 
>> including it in the client.  The Service API in the client will be 
>> used in preference, however if anythings missing, the proxy can still 
>> be unmarshalled.
>>
>>
>> Peter Firmstone wrote:
>>> You know something I realised about Evolving code, Interfaces and 
>>> Lookup?
>>>
>>> It's a little difficult to explain, Jini veterans are probably well 
>>> aware of it.
>>>
>>> I'll give you a hint,
>>>
>>>   1. It makes Versioning unnecessary.
>>>   2. It enables easy backward compatibility.
>>>   3. Implementation is isolated from API.
>>>   4. API is easily extended.
>>>
>>> Well it's Jini Lookup semantics.
>>>
>>> Clients lookup Service instances, based on their Service Interface 
>>> (API), so they only discover compatible Services.  Ok that probably 
>>> sounds obvious and "So What", well the Service implementation is free 
>>> to change, it can extend the Service Interface, old clients continue 
>>> to lookup with the earlier interface while new clients will find 
>>> extended service interfaces with added functionality.
>>>
>>> The Service knows nothing of the Client, the Client know's nothing of 
>>> the Service's implementation.  But even better, if we use Interfaces 
>>> for return values and Parameters, neither the client or the service 
>>> need to know anything about each others implementation objects.
>>>
>>> The service is free to change and the client is free to change.  All 
>>> interfaces can be extended, so the service API has flexibility in all 
>>> directions, new methods can be added by extending existing 
>>> interfaces, return values and parameters can change and implement new 
>>> interfaces too.
>>>
>>> This is an extremely flexible communication contract point.  So not 
>>> only can the protocols change but so can the API be extended, while 
>>> remaining fully backward compatible.
>>>
>>> Back to the humble isolated JVM for a moment, I guess the problem 
>>> with using classes without interfaces is, a class has an API and an 
>>> implementation.
>>>
>>> Suddenly the flexibility is gone.
>>>
>>> Take String as a case in point, new methods were added in Java 5, 
>>> this means later code that utilise the new methods, can't run on Java 
>>> 1.4.
>>>
>>> So we're forced to use versioning to work around these issues of 
>>> incompatibility.
>>>
>>> But Service Interfaces don't need to be versioned, they have many 
>>> degrees of freedom in which to maneuver to accommodate change, 
>>> developers use versioning when their implementation would be 
>>> compromised by maintaining complete backward compatibility.
>>>
>>> It gets better though, all implementations in Services, Proxy's and 
>>> Clients can be isolated in their own ClassLoader's and so not step on 
>>> each other's classes if we structure the ClassLoader tree properly.
>>>
>>> Versioning is a concern of implementations.
>>>
>>> We must prevent any implementation from being visible to another - 
>>> seeing the other's classes while running in the same JVM.
>>>
>>> So there is something that concerns me about Application code, 
>>> running in a ClassLoader that is parent to a proxy or a Service.
>>>
>>> Applications could depend on classes, while not part of Service API, 
>>> may be present in other implementation code, eg, the Proxy, but being 
>>> free to vary as implementation classes are, if implementations vary 
>>> (has another version!) and an incompatible class is loaded into a 
>>> Parent ClassLoader, the wrong class will be used, causing runtime 
>>> errors.
>>>
>>> So without fleshing out all the details and leaving as much out as 
>>> possible to allow others to advise of their needs and concerns, I'd 
>>> like to suggest that the following ClassLoader relationship based on 
>>> the Rio ClassLoader structure provided by Dennis, invisibility 
>>> between implementations is fundamental.
>>>
>>>  _______________________________________________________________
>>> |                                                               |
>>> |           AppClassLoader                                      |
>>> |                |                                              |
>>> |          Common Classloader (As Per Dennis' comments)         |
>>> |       _________|_______________________________________       |
>>> |      |             |                 |                 |      |
>>> | Service Imp    Smart Proxy      Parameter Impl   Application  |
>>> | ClassLoader's  ClassLoader's    ClassLoader's    ClassLoader  |
>>> |_______________________________________________________________|
>>>
>>>
>>> AppClassLoader - Contains the main() class of the container. 
>>> Main-Class in manifest points to com.sun.jini.start.ServiceStarter
>>> Classpath:  boot.jar, start.jar, jsk-platform.jar, service-api.jar 
>>> (mulitple Service API jar's allowed, can be untrusted)
>>> Codebase: none
>>>
>>> CommonClassLoader - Contains the common Rio and Jini technology 
>>> classes (and other declared common platform JARs) to be made 
>>> available to its children.
>>> Classpath: Common JARs such as rio.jar
>>> Codebase: Context dependent. The codebase returned is the codebase of 
>>> the specific child CL that is the current context of the request. 
>>> (Not Sure about the last sentence please explain?)
>>>
>>> Child ClassLoader's - Contains the service specific implementation 
>>> classes and client application classes.
>>> Classpath: serviceImpl.jar
>>> Codebase: "serviceX-dl.jar rio-dl.jar jsk-lib-dl.jar"
>>>
>>>
>>> Evolution of Service API is important and to avoid unmarshalling 
>>> errors, due to classes missing from the client's Static Service API, 
>>> when a proxy uses a Service API extending one present in the client.  
>>> For example, when a proxy contains additional Service API, the client 
>>> lacks, it should be loaded into the Proxy's ClassLoader.  I think 
>>> this is why Service API might exist in downloadable archives, for 
>>> safety reasons.
>>>
>>> So perhaps as a protection against unmarshalling error's the proxy's 
>>> codebase should contain the Service API too, however the Service API 
>>> present in the client is given preference due to ClassLoader hierarchy.
>>>
>>> Thoughts?
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>
>>
> 
>