You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Michal Kleczek <mi...@xpro.biz> on 2014/03/02 20:07:23 UTC
River-436 - need some explanation of preferred class provider
Folks,
while woking on the River-436 patch proposal I've came across the
scenario that I am not sure how to handle:
Utility service:
//inteface is NOT preferred
interface Util {...}
//class IS preferred
class UtilProxy implements Util {}
Wrapper service:
//NOT preferred
interface Wrapper {}
//preferred
class WrapperProxy implements Serializable{
//initialized with Util impl from a lookup service
private Util util;
}
Wrapper service codebase includes Util interface but it is _preferred_.
Would deserialization of WrapperProxy end with ClassCastException?
From what I understand UtilProxy is annotated with its codebase. When
deserializing UtilProxy a ClassLoader is going to be created with parent
set to TCCL. It means Util interface is going to be loaded twice by two
ClassLoaders - one for WrapperProxy codebase and another for UtilProxy
codebase.
Am I correct?
And if so: is it desired behavior?
Regards,
--
Michał Kłeczek
XPro Quality Matters
http://www.xpro.biz
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
I think that while it is an approach that could work it has some problems:
1. It requires processing of jar files. Actually the same problem is with
PREFERRED.LIST - there is no way one can use third party off-the-shelf jars
(eg. from Maven Central). It is doable of course but it makes thing _tedious_
2. Although it does not change the format of annotations it _does_ introduce
yet-another-standard that everybody has to understand. While it is not bad
per-se - it is _fixed_ - no way you can change it without changing all the
services/clients (BTW - I do not really like the distinction between services
and clients - the only real difference between them is that services announce
themselves in lookup services. From the security and code downloading
perspective there is no difference)
It would be really good if you take into account River-436. While it is -
again - a new standard, it strives to standardize in such a way that no more
changes in the standard are going to be needed :)
It simply makes creating a proper ClassLoader hierarchy/graph/whatever the
responsibility of the _service_ (the entity that provides code). From the
downloading entity it is irrelevant _how_ classes are created as long as it
can use it (securely of course).
There is one default implementation of a ClassProvider in my patch that shows:
a) how to provide other custom implementations
b) that the approach makes it possible (and in practice quite trivial) to
exchange hierarchies of ClassLoaders and (partially due to limitations in
hierarchical class loading) solve class loading issues presented in this
thread.
Regards,
On Thursday 13 of March 2014 22:40:52 Peter wrote:
> Gregg,
>
> I like your domain explanation.
>
> Each domain belongs to a service and services may use different codebase
> annotation / provisioning mechanisms. While the client owns the local
> domain, this allows some control by the service over it's own domain
> concerns.
>
> Although, I'd be tempted to make the parent, or global domain the api domain
> and place the application in a child domain, to reduce visibility of
> application classes to the service implementations / domains.
>
> Cheers,
>
> Peter.
>
>
>
> ----- Original message -----
>
> > My point is that you have to formalize it in a way that you can then
> > recognize. For both OSGi and Netbeans and many platforms, the knowledge
> > is hardcode into a relationship between the platforms class loading
> > mechanism, and the jar content.
> >
> > I don’t want to always use OSGi. I don’t want to always use Netbeans.
> > I want to use the appropriate mechanism for where the ‘client’ lives,
> > not for ‘how the service is constructed’. This is why
> > PreferredClassLoader has been working so well. It’s something that the
> > ‘client’ dictates and the ‘service jar’ has to standardize on. So, now
> > that you have something ‘new’ that you want to implement, in terms of a
> > client standard (a common client jar that is not in the clients class
> > path), you are going to have to provide a way for that to work.
> >
> > One way that comes to mind, is to take River-336 concepts and go just a
> > bit further by adding the notion of a “domain” for class loader
> > hierarchy. Imagine that every jar could have an additional meta-inf
> > property called “Domain”, which would be prepended by the URL’s path,
> > without the jar file name, that it was loaded from. This would then
> > create a class loading relationship in a graph described by the ‘.’
> > separated components.
> >
> > http://server1/jars/Util.jar => Domain=app
> > http://server1/jars/service1.jar => Domain=app.service1
> > http://server1/jars/service2.jar => Domain=app.service2
> > http://server1/jars/service3.jar => Domain=app.service3
> > http://server1/jars/service4.jar => Domain=app.service4
> >
> > would produce a graph of class loaders with ‘app’ at the parent so that
> > everything in Util.jar or any other associated libraries would be there,
> > and the services would have a parent reference to them.
> >
> > You could add a security permission associated with Domain
> > creation/access so that other services that you had not authorized at
> > server1, could not glue themselves into the class loader hierarchy.
> >
> > This kind of mechanism lets the app developer designate exactly what
> > they want to have happen and control it from the service where it should
> > be controlled.
> >
> > Gregg
> >
> > On Mar 10, 2014, at 1:08 AM, Michał Kłeczek <mi...@xpro.biz>
> >
> > wrote:
> > > Actually it is even worse. Since RMIClassProvider API is stateless the
> > > client has only one list of URLs at a time...
> > >
> > > Regards,
> > >
> > > On Sunday, March 09, 2014 10:54:57 PM Michał Kłeczek wrote:
> > > > The whole point of my example is that the client has no knowledge of
> > > > Util interface - it is simply not interested in it.
> > > >
> > > > The problem is not that the client cannot plug-in RMIClassProvider
> > > > dynamically. It is just that with current format of codebase
> > > > annotation the client cannot do anything. It simply does not have
> > > > enough data to decide what to do - just two lists of URLs without
> > > > any dependency information encoded.
> > > >
> > > > Regards,
> > > >
> > > > On Sunday, March 09, 2014 02:33:03 PM Gregg Wonderly wrote:
> > > > > All you have to provide in the client is a class loading
> > > > > implementation that knows about Util and pins it into a parent
> > > > > class loader from the class loaders that proxies load. I
> > > > > netbeans, this happens because meta data declares that such a
> > > > > relationship exists. In OSGi, this happens because meta data
> > > > > declares that such a relationship exists. All you have to do, is
> > > > > create meta data that specifies that such a relationship exists,
> > > > > and then plug in a River-336 compatible class loading
> > > > > implementation in your client.
> > > > >
> > > > > My point is no that River-336 provides the answer, but rather it
> > > > > provides a
> > > > > mechanism that an application can use. Not every application has
> > > > > such a need, and not every known implementation uses the same
> > > > > model. Thus, there isn’t a single answer that can exist ahead of
> > > > > time.
> > > > >
> > > > > If you want to use OSGi, plug it in. If you want to use Netbeans,
> > > > > plug it in. If you want to use both at the same time, work it
> > > > > out and plug it in.
> > > > >
> > > > > There is room for a single standard to eventually win. But, there
> > > > > isn’t a
> > > > >
> > > > > single standard that is standing alone right now that I see.
> > > > >
> > > > > Gregg Wonderly
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
Gregg,
I like your domain explanation.
Each domain belongs to a service and services may use different codebase annotation / provisioning mechanisms. While the client owns the local domain, this allows some control by the service over it's own domain concerns.
Although, I'd be tempted to make the parent, or global domain the api domain and place the application in a child domain, to reduce visibility of application classes to the service implementations / domains.
Cheers,
Peter.
----- Original message -----
> My point is that you have to formalize it in a way that you can then
> recognize. For both OSGi and Netbeans and many platforms, the knowledge
> is hardcode into a relationship between the platforms class loading
> mechanism, and the jar content.
>
> I don’t want to always use OSGi. I don’t want to always use Netbeans.
> I want to use the appropriate mechanism for where the ‘client’ lives,
> not for ‘how the service is constructed’. This is why
> PreferredClassLoader has been working so well. It’s something that the
> ‘client’ dictates and the ‘service jar’ has to standardize on. So, now
> that you have something ‘new’ that you want to implement, in terms of a
> client standard (a common client jar that is not in the clients class
> path), you are going to have to provide a way for that to work.
>
> One way that comes to mind, is to take River-336 concepts and go just a
> bit further by adding the notion of a “domain” for class loader
> hierarchy. Imagine that every jar could have an additional meta-inf
> property called “Domain”, which would be prepended by the URL’s path,
> without the jar file name, that it was loaded from. This would then
> create a class loading relationship in a graph described by the ‘.’
> separated components.
>
> http://server1/jars/Util.jar => Domain=app
> http://server1/jars/service1.jar => Domain=app.service1
> http://server1/jars/service2.jar => Domain=app.service2
> http://server1/jars/service3.jar => Domain=app.service3
> http://server1/jars/service4.jar => Domain=app.service4
>
> would produce a graph of class loaders with ‘app’ at the parent so that
> everything in Util.jar or any other associated libraries would be there,
> and the services would have a parent reference to them.
>
> You could add a security permission associated with Domain
> creation/access so that other services that you had not authorized at
> server1, could not glue themselves into the class loader hierarchy.
>
> This kind of mechanism lets the app developer designate exactly what
> they want to have happen and control it from the service where it should
> be controlled.
>
> Gregg
>
> On Mar 10, 2014, at 1:08 AM, Michał Kłeczek <mi...@xpro.biz>
> wrote:
>
> > Actually it is even worse. Since RMIClassProvider API is stateless the
> > client has only one list of URLs at a time...
> >
> > Regards,
> >
> > On Sunday, March 09, 2014 10:54:57 PM Michał Kłeczek wrote:
> > > The whole point of my example is that the client has no knowledge of
> > > Util interface - it is simply not interested in it.
> > >
> > > The problem is not that the client cannot plug-in RMIClassProvider
> > > dynamically. It is just that with current format of codebase
> > > annotation the client cannot do anything. It simply does not have
> > > enough data to decide what to do - just two lists of URLs without
> > > any dependency information encoded.
> > >
> > > Regards,
> > >
> > > On Sunday, March 09, 2014 02:33:03 PM Gregg Wonderly wrote:
> > > > All you have to provide in the client is a class loading
> > > > implementation that knows about Util and pins it into a parent
> > > > class loader from the class loaders that proxies load. I
> > > > netbeans, this happens because meta data declares that such a
> > > > relationship exists. In OSGi, this happens because meta data
> > > > declares that such a relationship exists. All you have to do, is
> > > > create meta data that specifies that such a relationship exists,
> > > > and then plug in a River-336 compatible class loading
> > > > implementation in your client.
> > > >
> > > > My point is no that River-336 provides the answer, but rather it
> > > > provides a
> > > > mechanism that an application can use. Not every application has
> > > > such a need, and not every known implementation uses the same
> > > > model. Thus, there isn’t a single answer that can exist ahead of
> > > > time.
> > > >
> > > > If you want to use OSGi, plug it in. If you want to use Netbeans,
> > > > plug it in. If you want to use both at the same time, work it
> > > > out and plug it in.
> > > >
> > > > There is room for a single standard to eventually win. But, there
> > > > isn’t a
> > > >
> > > > single standard that is standing alone right now that I see.
> > > >
> > > > Gregg Wonderly
> >
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska
>
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <gr...@wonderly.org>.
So, for example, lets saw that you might add the permission class, ClassLoaderDomainPermission with the object name being the domain. Anytime that a jar is loaded, if that URL/Codesource doesn’t have an associated instance of ClassLoaderDomainPermission(“domain”), a security exception is thrown. If security passes, then a recursive map of PreferredClassLoader instances is consulted/constructed using String.split(domain,”.”) elements. Finally a new PreferredClassLoader is created with the URL of the jar file and the last PreferredClassLoader found/constructed in that path as it’s parent. Now, you have the ability to create your hierarchy. Clearly, there are some resolution items to manage in terms of getting all of the jars being visible in all of the parent domains. But, that’s really not a huge deal since the existing annotation can provide multiple jar files that can be inspected and homed into their respective domains.
Gregg Wonderly
On Mar 10, 2014, at 11:35 AM, Gregg Wonderly <gr...@wonderly.org> wrote:
> My point is that you have to formalize it in a way that you can then recognize. For both OSGi and Netbeans and many platforms, the knowledge is hardcode into a relationship between the platforms class loading mechanism, and the jar content.
>
> I don’t want to always use OSGi. I don’t want to always use Netbeans. I want to use the appropriate mechanism for where the ‘client’ lives, not for ‘how the service is constructed’. This is why PreferredClassLoader has been working so well. It’s something that the ‘client’ dictates and the ‘service jar’ has to standardize on. So, now that you have something ‘new’ that you want to implement, in terms of a client standard (a common client jar that is not in the clients class path), you are going to have to provide a way for that to work.
>
> One way that comes to mind, is to take River-336 concepts and go just a bit further by adding the notion of a “domain” for class loader hierarchy. Imagine that every jar could have an additional meta-inf property called “Domain”, which would be prepended by the URL’s path, without the jar file name, that it was loaded from. This would then create a class loading relationship in a graph described by the ‘.’ separated components.
>
> http://server1/jars/Util.jar => Domain=app
> http://server1/jars/service1.jar => Domain=app.service1
> http://server1/jars/service2.jar => Domain=app.service2
> http://server1/jars/service3.jar => Domain=app.service3
> http://server1/jars/service4.jar => Domain=app.service4
>
> would produce a graph of class loaders with ‘app’ at the parent so that everything in Util.jar or any other associated libraries would be there, and the services would have a parent reference to them.
>
> You could add a security permission associated with Domain creation/access so that other services that you had not authorized at server1, could not glue themselves into the class loader hierarchy.
>
> This kind of mechanism lets the app developer designate exactly what they want to have happen and control it from the service where it should be controlled.
>
> Gregg
>
> On Mar 10, 2014, at 1:08 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
>
>> Actually it is even worse. Since RMIClassProvider API is stateless the client
>> has only one list of URLs at a time...
>>
>> Regards,
>>
>> On Sunday, March 09, 2014 10:54:57 PM Michał Kłeczek wrote:
>>> The whole point of my example is that the client has no knowledge of Util
>>> interface - it is simply not interested in it.
>>>
>>> The problem is not that the client cannot plug-in RMIClassProvider
>>> dynamically. It is just that with current format of codebase annotation the
>>> client cannot do anything. It simply does not have enough data to decide
>>> what to do - just two lists of URLs without any dependency information
>>> encoded.
>>>
>>> Regards,
>>>
>>> On Sunday, March 09, 2014 02:33:03 PM Gregg Wonderly wrote:
>>>> All you have to provide in the client is a class loading implementation
>>>> that knows about Util and pins it into a parent class loader from the
>>>> class loaders that proxies load. I netbeans, this happens because meta
>>>> data declares that such a relationship exists. In OSGi, this happens
>>>> because meta data declares that such a relationship exists. All you have
>>>> to do, is create meta data that specifies that such a relationship
>>>> exists, and then plug in a River-336 compatible class loading
>>>> implementation in your client.
>>>>
>>>> My point is no that River-336 provides the answer, but rather it provides
>>>> a
>>>> mechanism that an application can use. Not every application has such a
>>>> need, and not every known implementation uses the same model. Thus, there
>>>> isn’t a single answer that can exist ahead of time.
>>>>
>>>> If you want to use OSGi, plug it in. If you want to use Netbeans, plug it
>>>> in. If you want to use both at the same time, work it out and plug it
>>>> in.
>>>>
>>>> There is room for a single standard to eventually win. But, there isn’t
>>>> a
>>>>
>>>> single standard that is standing alone right now that I see.
>>>>
>>>> Gregg Wonderly
>>
>> --
>> Michał Kłeczek
>> XPro Sp. z o. o.
>> ul. Borowskiego 2
>> 03-475 Warszawa
>> Polska
>> --
>> Michał Kłeczek
>> XPro Sp. z o. o.
>> ul. Borowskiego 2
>> 03-475 Warszawa
>> Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <gr...@wonderly.org>.
My point is that you have to formalize it in a way that you can then recognize. For both OSGi and Netbeans and many platforms, the knowledge is hardcode into a relationship between the platforms class loading mechanism, and the jar content.
I don’t want to always use OSGi. I don’t want to always use Netbeans. I want to use the appropriate mechanism for where the ‘client’ lives, not for ‘how the service is constructed’. This is why PreferredClassLoader has been working so well. It’s something that the ‘client’ dictates and the ‘service jar’ has to standardize on. So, now that you have something ‘new’ that you want to implement, in terms of a client standard (a common client jar that is not in the clients class path), you are going to have to provide a way for that to work.
One way that comes to mind, is to take River-336 concepts and go just a bit further by adding the notion of a “domain” for class loader hierarchy. Imagine that every jar could have an additional meta-inf property called “Domain”, which would be prepended by the URL’s path, without the jar file name, that it was loaded from. This would then create a class loading relationship in a graph described by the ‘.’ separated components.
http://server1/jars/Util.jar => Domain=app
http://server1/jars/service1.jar => Domain=app.service1
http://server1/jars/service2.jar => Domain=app.service2
http://server1/jars/service3.jar => Domain=app.service3
http://server1/jars/service4.jar => Domain=app.service4
would produce a graph of class loaders with ‘app’ at the parent so that everything in Util.jar or any other associated libraries would be there, and the services would have a parent reference to them.
You could add a security permission associated with Domain creation/access so that other services that you had not authorized at server1, could not glue themselves into the class loader hierarchy.
This kind of mechanism lets the app developer designate exactly what they want to have happen and control it from the service where it should be controlled.
Gregg
On Mar 10, 2014, at 1:08 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
> Actually it is even worse. Since RMIClassProvider API is stateless the client
> has only one list of URLs at a time...
>
> Regards,
>
> On Sunday, March 09, 2014 10:54:57 PM Michał Kłeczek wrote:
>> The whole point of my example is that the client has no knowledge of Util
>> interface - it is simply not interested in it.
>>
>> The problem is not that the client cannot plug-in RMIClassProvider
>> dynamically. It is just that with current format of codebase annotation the
>> client cannot do anything. It simply does not have enough data to decide
>> what to do - just two lists of URLs without any dependency information
>> encoded.
>>
>> Regards,
>>
>> On Sunday, March 09, 2014 02:33:03 PM Gregg Wonderly wrote:
>>> All you have to provide in the client is a class loading implementation
>>> that knows about Util and pins it into a parent class loader from the
>>> class loaders that proxies load. I netbeans, this happens because meta
>>> data declares that such a relationship exists. In OSGi, this happens
>>> because meta data declares that such a relationship exists. All you have
>>> to do, is create meta data that specifies that such a relationship
>>> exists, and then plug in a River-336 compatible class loading
>>> implementation in your client.
>>>
>>> My point is no that River-336 provides the answer, but rather it provides
>>> a
>>> mechanism that an application can use. Not every application has such a
>>> need, and not every known implementation uses the same model. Thus, there
>>> isn’t a single answer that can exist ahead of time.
>>>
>>> If you want to use OSGi, plug it in. If you want to use Netbeans, plug it
>>> in. If you want to use both at the same time, work it out and plug it
>>> in.
>>>
>>> There is room for a single standard to eventually win. But, there isn’t
>>> a
>>>
>>> single standard that is standing alone right now that I see.
>>>
>>> Gregg Wonderly
>
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Actually it is even worse. Since RMIClassProvider API is stateless the client
has only one list of URLs at a time...
Regards,
On Sunday, March 09, 2014 10:54:57 PM Michał Kłeczek wrote:
> The whole point of my example is that the client has no knowledge of Util
> interface - it is simply not interested in it.
>
> The problem is not that the client cannot plug-in RMIClassProvider
> dynamically. It is just that with current format of codebase annotation the
> client cannot do anything. It simply does not have enough data to decide
> what to do - just two lists of URLs without any dependency information
> encoded.
>
> Regards,
>
> On Sunday, March 09, 2014 02:33:03 PM Gregg Wonderly wrote:
> > All you have to provide in the client is a class loading implementation
> > that knows about Util and pins it into a parent class loader from the
> > class loaders that proxies load. I netbeans, this happens because meta
> > data declares that such a relationship exists. In OSGi, this happens
> > because meta data declares that such a relationship exists. All you have
> > to do, is create meta data that specifies that such a relationship
> > exists, and then plug in a River-336 compatible class loading
> > implementation in your client.
> >
> > My point is no that River-336 provides the answer, but rather it provides
> > a
> > mechanism that an application can use. Not every application has such a
> > need, and not every known implementation uses the same model. Thus, there
> > isn’t a single answer that can exist ahead of time.
> >
> > If you want to use OSGi, plug it in. If you want to use Netbeans, plug it
> > in. If you want to use both at the same time, work it out and plug it
> > in.
> >
> > There is room for a single standard to eventually win. But, there isn’t
> > a
> >
> > single standard that is standing alone right now that I see.
> >
> > Gregg Wonderly
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
The whole point of my example is that the client has no knowledge of Util
interface - it is simply not interested in it.
The problem is not that the client cannot plug-in RMIClassProvider
dynamically. It is just that with current format of codebase annotation the
client cannot do anything. It simply does not have enough data to decide what
to do - just two lists of URLs without any dependency information encoded.
Regards,
On Sunday, March 09, 2014 02:33:03 PM Gregg Wonderly wrote:
> All you have to provide in the client is a class loading implementation that
> knows about Util and pins it into a parent class loader from the class
> loaders that proxies load. I netbeans, this happens because meta data
> declares that such a relationship exists. In OSGi, this happens because
> meta data declares that such a relationship exists. All you have to do, is
> create meta data that specifies that such a relationship exists, and then
> plug in a River-336 compatible class loading implementation in your client.
>
> My point is no that River-336 provides the answer, but rather it provides a
> mechanism that an application can use. Not every application has such a
> need, and not every known implementation uses the same model. Thus, there
> isn’t a single answer that can exist ahead of time.
>
> If you want to use OSGi, plug it in. If you want to use Netbeans, plug it
> in. If you want to use both at the same time, work it out and plug it in.
> There is room for a single standard to eventually win. But, there isn’t a
> single standard that is standing alone right now that I see.
>
> Gregg Wonderly
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
Gregg,
Sim's RiverClassLoading work in trunk is based on your work on River-336.
This could be backported to the 2.2 branch.
Hopefully qa-refactor will get some interested reviewers when it stabilises for merging with trunk.
I'm bogged down with paid work presently, so haven't had much time to focus on qa-refactor's remaining issues. ServiceDiscoveryManager is still failing a couple of tests and there are tests for Reggie that expect listeners to be notified in the same order that listeners register, forcing me to make the notifier's executor single threaded to comply. When the executor is multi threaded, each listener is notified with events arriving in the correct order, but the order between listeners is asynchronous, causing test failure. Personally I think there's something wrong with expecting listeners to be notified in order of registration, but users will be able to configure their executors to be multi threaded if they wish.
Regards,
Peter.
----- Original message -----
> All you have to provide in the client is a class loading implementation
> that knows about Util and pins it into a parent class loader from the
> class loaders that proxies load. I netbeans, this happens because meta
> data declares that such a relationship exists. In OSGi, this happens
> because meta data declares that such a relationship exists. All you
> have to do, is create meta data that specifies that such a relationship
> exists, and then plug in a River-336 compatible class loading
> implementation in your client.
>
> My point is no that River-336 provides the answer, but rather it
> provides a mechanism that an application can use. Not every application
> has such a need, and not every known implementation uses the same model.
> Thus, there isn’t a single answer that can exist ahead of time.
>
> If you want to use OSGi, plug it in. If you want to use Netbeans, plug
> it in. If you want to use both at the same time, work it out and plug
> it in. There is room for a single standard to eventually win. But,
> there isn’t a single standard that is standing alone right now that I
> see.
>
> Gregg Wonderly
>
> On Mar 7, 2014, at 12:17 PM, Michał Kłeczek <mi...@xpro.biz>
> wrote:
>
> > Greg, please look at my example in the first message of this thread.
> > And tell my how the client can decide what ClassLoader should load Util
> > interface assuming it does not have it in it's classpath.
> >
> > Regards,
> > 7 mar 2014 18:51 "Gregg Wonderly" <gr...@wonderly.org> napisał(a):
> >
> > > Okay, I don’t have to reply to all of the exchanges I missed, but I
> > > really want to make it clear, that my class loading changes in
> > > River-336, do in fact fix ALL CLASSLOADING ISSUES! The reason I
> > > “scream” that out, is because it encapsulates every single way that
> > > class loading occurs. If you don’t have a preferred list in your
> > > jar, then preferred class loader is going to always “ask” the parent
> > > to load the class, and the call into the River-336 provided code can
> > > delegate loading in whatever mechanism is appropriate for the
> > > “platform” that the client wants to use.
> > >
> > > This makes it possible to get the class form wherever is needed, and
> > > puts the client in complete control of how class loader resolution
> > > occurs, as well as how class objects are loaded into class loaders
> > > as “owners” of the classes.
> > >
> > > Just because the methods have names indicating “parent” or other
> > > hierarchal relationships doesn’t mean that the actions taken there
> > > have to create any sort of hierarchy.
> > >
> > > Gregg Wonderly
> > >
> > > On Mar 7, 2014, at 10:32 AM, Michał Kłeczek <mi...@xpro.biz>
> > > wrote:
> > >
> > > > Sure there is a need for code downloading for JERI proxies. You
> > > > seem to
> > > assume
> > > > no custom endpoint implementations.
> > > >
> > > > There is really no difference between dynamic proxy and "normal"
> > > > object.
> > > >
> > > > Regards,
> > > >
> > > > On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
> > > > >
> > > > > Now, dynamic proxies are a different story, and JERI already
> > > > > uses the dynamic proxy mechanism. There’s no need, for example
> > > > > to download an implementation class for an object that is
> > > > > directly exported - you only really need the service interface
> > > > > to be available locally.
> > > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Greg Trasuk
> > > >
> > > > --
> > > > Michał Kłeczek
> > > > XPro Sp. z o. o.
> > > > ul. Borowskiego 2
> > > > 03-475 Warszawa
> > > > Polska<Michał Kłeczek (XPro).vcf>
> > >
> > >
>
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <ge...@cox.net>.
All you have to provide in the client is a class loading implementation that knows about Util and pins it into a parent class loader from the class loaders that proxies load. I netbeans, this happens because meta data declares that such a relationship exists. In OSGi, this happens because meta data declares that such a relationship exists. All you have to do, is create meta data that specifies that such a relationship exists, and then plug in a River-336 compatible class loading implementation in your client.
My point is no that River-336 provides the answer, but rather it provides a mechanism that an application can use. Not every application has such a need, and not every known implementation uses the same model. Thus, there isn’t a single answer that can exist ahead of time.
If you want to use OSGi, plug it in. If you want to use Netbeans, plug it in. If you want to use both at the same time, work it out and plug it in. There is room for a single standard to eventually win. But, there isn’t a single standard that is standing alone right now that I see.
Gregg Wonderly
On Mar 7, 2014, at 12:17 PM, Michał Kłeczek <mi...@xpro.biz> wrote:
> Greg, please look at my example in the first message of this thread. And
> tell my how the client can decide what ClassLoader should load Util
> interface assuming it does not have it in it's classpath.
>
> Regards,
> 7 mar 2014 18:51 "Gregg Wonderly" <gr...@wonderly.org> napisał(a):
>
>> Okay, I don’t have to reply to all of the exchanges I missed, but I really
>> want to make it clear, that my class loading changes in River-336, do in
>> fact fix ALL CLASSLOADING ISSUES! The reason I “scream” that out, is
>> because it encapsulates every single way that class loading occurs. If you
>> don’t have a preferred list in your jar, then preferred class loader is
>> going to always “ask” the parent to load the class, and the call into the
>> River-336 provided code can delegate loading in whatever mechanism is
>> appropriate for the “platform” that the client wants to use.
>>
>> This makes it possible to get the class form wherever is needed, and puts
>> the client in complete control of how class loader resolution occurs, as
>> well as how class objects are loaded into class loaders as “owners” of the
>> classes.
>>
>> Just because the methods have names indicating “parent” or other
>> hierarchal relationships doesn’t mean that the actions taken there have to
>> create any sort of hierarchy.
>>
>> Gregg Wonderly
>>
>> On Mar 7, 2014, at 10:32 AM, Michał Kłeczek <mi...@xpro.biz>
>> wrote:
>>
>>> Sure there is a need for code downloading for JERI proxies. You seem to
>> assume
>>> no custom endpoint implementations.
>>>
>>> There is really no difference between dynamic proxy and "normal" object.
>>>
>>> Regards,
>>>
>>> On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
>>>>
>>>> Now, dynamic proxies are a different story, and JERI already uses the
>>>> dynamic proxy mechanism. There’s no need, for example to download an
>>>> implementation class for an object that is directly exported - you only
>>>> really need the service interface to be available locally.
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Greg Trasuk
>>>
>>> --
>>> Michał Kłeczek
>>> XPro Sp. z o. o.
>>> ul. Borowskiego 2
>>> 03-475 Warszawa
>>> Polska<Michał Kłeczek (XPro).vcf>
>>
>>
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Greg, please look at my example in the first message of this thread. And
tell my how the client can decide what ClassLoader should load Util
interface assuming it does not have it in it's classpath.
Regards,
7 mar 2014 18:51 "Gregg Wonderly" <gr...@wonderly.org> napisał(a):
> Okay, I don’t have to reply to all of the exchanges I missed, but I really
> want to make it clear, that my class loading changes in River-336, do in
> fact fix ALL CLASSLOADING ISSUES! The reason I “scream” that out, is
> because it encapsulates every single way that class loading occurs. If you
> don’t have a preferred list in your jar, then preferred class loader is
> going to always “ask” the parent to load the class, and the call into the
> River-336 provided code can delegate loading in whatever mechanism is
> appropriate for the “platform” that the client wants to use.
>
> This makes it possible to get the class form wherever is needed, and puts
> the client in complete control of how class loader resolution occurs, as
> well as how class objects are loaded into class loaders as “owners” of the
> classes.
>
> Just because the methods have names indicating “parent” or other
> hierarchal relationships doesn’t mean that the actions taken there have to
> create any sort of hierarchy.
>
> Gregg Wonderly
>
> On Mar 7, 2014, at 10:32 AM, Michał Kłeczek <mi...@xpro.biz>
> wrote:
>
> > Sure there is a need for code downloading for JERI proxies. You seem to
> assume
> > no custom endpoint implementations.
> >
> > There is really no difference between dynamic proxy and "normal" object.
> >
> > Regards,
> >
> > On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
> >>
> >> Now, dynamic proxies are a different story, and JERI already uses the
> >> dynamic proxy mechanism. There’s no need, for example to download an
> >> implementation class for an object that is directly exported - you only
> >> really need the service interface to be available locally.
> >>
> >>
> >> Cheers,
> >>
> >> Greg Trasuk
> >
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska<Michał Kłeczek (XPro).vcf>
>
>
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <gr...@wonderly.org>.
Okay, I don’t have to reply to all of the exchanges I missed, but I really want to make it clear, that my class loading changes in River-336, do in fact fix ALL CLASSLOADING ISSUES! The reason I “scream” that out, is because it encapsulates every single way that class loading occurs. If you don’t have a preferred list in your jar, then preferred class loader is going to always “ask” the parent to load the class, and the call into the River-336 provided code can delegate loading in whatever mechanism is appropriate for the “platform” that the client wants to use.
This makes it possible to get the class form wherever is needed, and puts the client in complete control of how class loader resolution occurs, as well as how class objects are loaded into class loaders as “owners” of the classes.
Just because the methods have names indicating “parent” or other hierarchal relationships doesn’t mean that the actions taken there have to create any sort of hierarchy.
Gregg Wonderly
On Mar 7, 2014, at 10:32 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
> Sure there is a need for code downloading for JERI proxies. You seem to assume
> no custom endpoint implementations.
>
> There is really no difference between dynamic proxy and "normal" object.
>
> Regards,
>
> On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
>>
>> Now, dynamic proxies are a different story, and JERI already uses the
>> dynamic proxy mechanism. There’s no need, for example to download an
>> implementation class for an object that is directly exported - you only
>> really need the service interface to be available locally.
>>
>>
>> Cheers,
>>
>> Greg Trasuk
>
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska<Michał Kłeczek (XPro).vcf>
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Sure there is a need for code downloading for JERI proxies. You seem to assume
no custom endpoint implementations.
There is really no difference between dynamic proxy and "normal" object.
Regards,
On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
>
> Now, dynamic proxies are a different story, and JERI already uses the
> dynamic proxy mechanism. There’s no need, for example to download an
> implementation class for an object that is directly exported - you only
> really need the service interface to be available locally.
>
>
> Cheers,
>
> Greg Trasuk
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Isn't InvocationHandler a "smart proxy"???
7 mar 2014 18:55 "Gregg Wonderly" <ge...@cox.net> napisał(a):
> Greg is referring to the fact that Jeri provides the ability to ship a
> java.reflection.Proxy object to the report host as the service
> implementation, instead of having to ship the actual class as a
> serializable endpoint. This is a big difference. Adding endpoint
> implementations into this is like apples and oranges. Yes, if there is
> "code" needed, it needs to be downloaded. But, dynamic proxies take away
> the need for code to be downloaded, unless you need a smart proxy.
>
> Gregg Wonderly
>
> On Mar 7, 2014, at 10:32 AM, Michał Kłeczek <mi...@xpro.biz>
> wrote:
>
> > Sure there is a need for code downloading for JERI proxies. You seem to
> assume
> > no custom endpoint implementations.
> >
> > There is really no difference between dynamic proxy and "normal" object.
> >
> > Regards,
> >
> > On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
> >>
> >> Now, dynamic proxies are a different story, and JERI already uses the
> >> dynamic proxy mechanism. There's no need, for example to download an
> >> implementation class for an object that is directly exported - you only
> >> really need the service interface to be available locally.
> >>
> >>
> >> Cheers,
> >>
> >> Greg Trasuk
> >
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska<Michał Kłeczek (XPro).vcf>
>
>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
GlassFish and Jboss do dynamic stub generation for IIOP proxy's too, now that would be cool.
Cheers,
Peter.
----- Original message -----
> Greg is referring to the fact that Jeri provides the ability to ship a
> java.reflection.Proxy object to the report host as the service
> implementation, instead of having to ship the actual class as a
> serializable endpoint. This is a big difference. Adding endpoint
> implementations into this is like apples and oranges. Yes, if there is
> “code” needed, it needs to be downloaded. But, dynamic proxies take
> away the need for code to be downloaded, unless you need a smart proxy.
>
> Gregg Wonderly
>
> On Mar 7, 2014, at 10:32 AM, Michał Kłeczek <mi...@xpro.biz>
> wrote:
>
> > Sure there is a need for code downloading for JERI proxies. You seem
> > to assume no custom endpoint implementations.
> >
> > There is really no difference between dynamic proxy and "normal"
> > object.
> >
> > Regards,
> >
> > On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
> > >
> > > Now, dynamic proxies are a different story, and JERI already uses the
> > > dynamic proxy mechanism. There’s no need, for example to download an
> > > implementation class for an object that is directly exported - you
> > > only really need the service interface to be available locally.
> > >
> > >
> > > Cheers,
> > >
> > > Greg Trasuk
> >
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska<Michał Kłeczek (XPro).vcf>
>
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <ge...@cox.net>.
Greg is referring to the fact that Jeri provides the ability to ship a java.reflection.Proxy object to the report host as the service implementation, instead of having to ship the actual class as a serializable endpoint. This is a big difference. Adding endpoint implementations into this is like apples and oranges. Yes, if there is “code” needed, it needs to be downloaded. But, dynamic proxies take away the need for code to be downloaded, unless you need a smart proxy.
Gregg Wonderly
On Mar 7, 2014, at 10:32 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
> Sure there is a need for code downloading for JERI proxies. You seem to assume
> no custom endpoint implementations.
>
> There is really no difference between dynamic proxy and "normal" object.
>
> Regards,
>
> On Friday, March 07, 2014 09:32:04 AM Greg Trasuk wrote:
>>
>> Now, dynamic proxies are a different story, and JERI already uses the
>> dynamic proxy mechanism. There’s no need, for example to download an
>> implementation class for an object that is directly exported - you only
>> really need the service interface to be available locally.
>>
>>
>> Cheers,
>>
>> Greg Trasuk
>
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska<Michał Kłeczek (XPro).vcf>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
Ok, I've looked into the latest code, previously a lambda was only Serializable if explicitly defined, now it looks like all lambda's are and the expert group has decided to make lambda serialization like inner classes so users know what to expect.
I would have much preferred limited Serialization that's simple, rather than everything's Serializable but broken.
So you're right, magic gone, at least for distributed programming.
I'll do some more investigation, but this looks bleak.
Regards,
Peter.
----- Original message -----
>
>
> Doesn’t look like there’s any magic. I performed a test, which can be
> seen at https://github.com/trasukg/LambdaSerialization
>
> Basically, I serialize a lambda function in one project, then copy it
> into another project, so that the original class is not in the class
> path of the new project. Sure enough, we get a ClassNotFoundException
> on the attempt to deserialize.
>
> I think what Goetz means when he says “the meta factory lazily creates
> the byte code..” is that the way the lambda is instantiated is subject
> to overwriting a run time, however the code contained _within_ the
> lambda function remains resident in the class that defines it.
>
> So, I think it’s fair to say that codebase annotation and dynamic
> loading _will_ be required on a lambda function. More critically, if a
> client were to serialize a lambda function, it would have to make its
> implementation jar available. In other words, the client would still be
> subject to a division of its code into a separate “proxy” or “download”
> layer. And this “-proxy.jar” has to be subject to the same type of
> versioning that a service’s “-proxy.jar” needs to follow, which is why
> caching dynamically loaded code has always been a problem.
>
> Cheers,
>
> Greg Trasuk.
>
> On Mar 7, 2014, at 2:21 PM, Tom Hobbs <tv...@googlemail.com> wrote:
>
> > I'm certainly no expert, but my understanding is that the so called
> > "meta factory" lazily creates the byte code to service the lambda
> > during first call. So you are correct, they are not compiled to anon
> > inner classes, they do end up as private methods, but indy creates
> > uses this factory as a receipe for how to create the lambda, relying
> > on the runtime to follow that recipe as it sees fit.
> >
> > From your link.
> >
> > "Instead of generating bytecode to create the object that implements
> > the lambda expression (such as calling a constructor for an inner
> > class), we describe a recipe for constructing the lambda, and delegate
> > the actual construction to the language runtime. That recipe is
> > encoded in the static and dynamic argument lists of an invokedynamic
> > instruction. The use of invokedynamic lets us defer the selection of
> > a translation strategy until run time."
> >
> > They can also relinked/created/etc should subsequent calls to them be
> > made with different argument types. [2] is also an interesting read.
> >
> > [1]
> > http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html
> > [2]
> > http://www.slideshare.net/jaxlondon2012/lambda-a-peek-under-the-hood-brian-goetz
> >
> >
> > On Fri, Mar 7, 2014 at 6:55 PM, Michał Kłeczek
> > <mi...@xpro.biz>wrote:
> >
> > > Although lambdas are not compiled to anonymous classes the code IS
> > > NOT generated at runtime. They end up as methods and bound in
> > > runtime using invokeDynamic.
> > >
> > > See:
> > > http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html
> > >
> > > Regards
> > > 7 mar 2014 19:22 "Peter" <ji...@zeus.net.au> napisał(a):
> > >
> > > > ----- Original message -----
> > > > >
> > > > > On Mar 7, 2014, at 4:18 AM, Peter <ji...@zeus.net.au> wrote:
> > > > >
> > > > > >
> > > > > > > Also... Could you elaborate more on code dynamically
> > > > > > > generated at
> > > the
> > > > > > > server and how does it differ from code downloading?
> > > > > >
> > > > > > Bytecode for lambda expressions is generated at runtime by the
> > > > > > jvm, lambda's use a single serialization proxy class, necessary
> > > > > > implementation bytecode is dynamically compiled upon
> > > > > > deserialisation.
> > > > > >
> > > > > > How lambdas change
> > > > > > > anything (since lambdas are only Java _language_ level
> > > > constructs I
> > > > > > > really don't see how).
> > > > > >
> > > > > > Dynamically compiled code doesn't need a codebase annotation,
> > > > > > it's created as needed and it's visibility is correct for each
> > > > > > use case, it's very specific, implementing a public interface
> > > > > > and interacting through this.
> > > > > >
> > > > >
> > > > > Um…are you sure about that?
> > > >
> > > > Yes, 100%, they're definitely not syntactic sugar, it does
> > > > serialize the final field arguments to the lambda expression, so
> > > > if these are smart proxy's, yes it will need to marshall those
> > > > arguments but no, it's not an internal anonymous class and doesn't
> > > > need to send its enclosing class.
> > > >
> > > > Go have a look at Brian Goetz's presentations, check out some code
> > > > and read the development mail lists, it's pretty cool.
> > > >
> > > > Cheers,
> > > >
> > > > Peter.
> > > >
> > > > I confess that I haven’t tried it in code,
> > > > > but from reading JSR335, I get the impression that Java lambdas
> > > > > are not LISP lambdas, i.e. it’s not so much dynamic byte code
> > > > > generation as syntactic sugar that avoids the need to define an
> > > > > interface for what is effectively an inner class (it’s not quite
> > > > > an inner class, as it
> > > doesn’t
> > > > > redefine ‘this’, but is otherwise pretty close. For example,
> > > > > there aren’t real closures; local variables can only be captured
> > > > > if they are effectively final). There also seems to be a lot
> > > > > of compile-time type inference going on. I suspect that
> > > > > passing a lambda as a marshalled object will still require the
> > > > > containing class to be loaded.
> > > > >
> > > > > Now, dynamic proxies are a different story, and JERI already
> > > > > uses the dynamic proxy mechanism. There’s no need, for example
> > > > > to download an implementation class for an object that is
> > > > > directly exported - you only really need the service interface
> > > > > to be available locally. Having
> > > said
> > > > > that, I think most of us are in the habit of treating
> > > > > “java.rmi.Remote” as an implementation detail that shouldn’t be
> > > > > reflected in the service interface, so for instance, we’ll have
> > > > > a service interface, “Hello” which doesn’t extend Remote, and
> > > > > whose methods may throw IOException, and then we’ll have a proxy
> > > > > interface (which goes into the ‘-dl’ or in Dennis’ terms
> > > > > ‘-proxy’ jar) called HelloRemote that extends Hello and Remote.
> > > > > The client will download HelloRemote from the codebase
> > > > > annotation.
> > > > >
> > > > > In any case, I don’t think Java lambdas are the magic bullet
> > > > > you’re thinking they are. I’d be happy to be proven wrong
> > > > > though, because
> > > what
> > > > > you’re talking about would be pretty cool.
> > > > >
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Greg Trasuk
> > > >
> > > >
> > >
>
Re: River-436 - need some explanation of preferred class provider
Posted by Greg Trasuk <tr...@stratuscom.com>.
Doesn’t look like there’s any magic. I performed a test, which can be seen at
https://github.com/trasukg/LambdaSerialization
Basically, I serialize a lambda function in one project, then copy it into another project, so that the original class is not in the class path of the new project. Sure enough, we get a ClassNotFoundException on the attempt to deserialize.
I think what Goetz means when he says “the meta factory lazily creates the byte code..” is that the way the lambda is instantiated is subject to overwriting a run time, however the code contained _within_ the lambda function remains resident in the class that defines it.
So, I think it’s fair to say that codebase annotation and dynamic loading _will_ be required on a lambda function. More critically, if a client were to serialize a lambda function, it would have to make its implementation jar available. In other words, the client would still be subject to a division of its code into a separate “proxy” or “download” layer. And this “-proxy.jar” has to be subject to the same type of versioning that a service’s “-proxy.jar” needs to follow, which is why caching dynamically loaded code has always been a problem.
Cheers,
Greg Trasuk.
On Mar 7, 2014, at 2:21 PM, Tom Hobbs <tv...@googlemail.com> wrote:
> I'm certainly no expert, but my understanding is that the so called "meta
> factory" lazily creates the byte code to service the lambda during first
> call. So you are correct, they are not compiled to anon inner classes,
> they do end up as private methods, but indy creates uses this factory as a
> receipe for how to create the lambda, relying on the runtime to follow that
> recipe as it sees fit.
>
> From your link.
>
> "Instead of generating bytecode to create the object that implements the
> lambda expression (such as calling a constructor for an inner class), we
> describe a recipe for constructing the lambda, and delegate the actual
> construction to the language runtime. That recipe is encoded in the static
> and dynamic argument lists of an invokedynamic instruction. The use of
> invokedynamic lets us defer the selection of a translation strategy until
> run time."
>
> They can also relinked/created/etc should subsequent calls to them be made
> with different argument types. [2] is also an interesting read.
>
> [1] http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html
> [2]
> http://www.slideshare.net/jaxlondon2012/lambda-a-peek-under-the-hood-brian-goetz
>
>
> On Fri, Mar 7, 2014 at 6:55 PM, Michał Kłeczek <mi...@xpro.biz>wrote:
>
>> Although lambdas are not compiled to anonymous classes the code IS NOT
>> generated at runtime. They end up as methods and bound in runtime using
>> invokeDynamic.
>>
>> See:
>> http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html
>>
>> Regards
>> 7 mar 2014 19:22 "Peter" <ji...@zeus.net.au> napisał(a):
>>
>>> ----- Original message -----
>>>>
>>>> On Mar 7, 2014, at 4:18 AM, Peter <ji...@zeus.net.au> wrote:
>>>>
>>>>>
>>>>>> Also... Could you elaborate more on code dynamically generated at
>> the
>>>>>> server and how does it differ from code downloading?
>>>>>
>>>>> Bytecode for lambda expressions is generated at runtime by the jvm,
>>>>> lambda's use a single serialization proxy class, necessary
>>>>> implementation bytecode is dynamically compiled upon deserialisation.
>>>>>
>>>>> How lambdas change
>>>>>> anything (since lambdas are only Java _language_ level
>>> constructs I
>>>>>> really don't see how).
>>>>>
>>>>> Dynamically compiled code doesn't need a codebase annotation, it's
>>>>> created as needed and it's visibility is correct for each use case,
>>>>> it's very specific, implementing a public interface and interacting
>>>>> through this.
>>>>>
>>>>
>>>> Um…are you sure about that?
>>>
>>> Yes, 100%, they're definitely not syntactic sugar, it does serialize the
>>> final field arguments to the lambda expression, so if these are smart
>>> proxy's, yes it will need to marshall those arguments but no, it's not an
>>> internal anonymous class and doesn't need to send its enclosing class.
>>>
>>> Go have a look at Brian Goetz's presentations, check out some code and
>>> read the development mail lists, it's pretty cool.
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>> I confess that I haven’t tried it in code,
>>>> but from reading JSR335, I get the impression that Java lambdas are not
>>>> LISP lambdas, i.e. it’s not so much dynamic byte code generation as
>>>> syntactic sugar that avoids the need to define an interface for what is
>>>> effectively an inner class (it’s not quite an inner class, as it
>> doesn’t
>>>> redefine ‘this’, but is otherwise pretty close. For example, there
>>>> aren’t real closures; local variables can only be captured if they are
>>>> effectively final). There also seems to be a lot of compile-time type
>>>> inference going on. I suspect that passing a lambda as a marshalled
>>>> object will still require the containing class to be loaded.
>>>>
>>>> Now, dynamic proxies are a different story, and JERI already uses the
>>>> dynamic proxy mechanism. There’s no need, for example to download an
>>>> implementation class for an object that is directly exported - you only
>>>> really need the service interface to be available locally. Having
>> said
>>>> that, I think most of us are in the habit of treating “java.rmi.Remote”
>>>> as an implementation detail that shouldn’t be reflected in the service
>>>> interface, so for instance, we’ll have a service interface, “Hello”
>>>> which doesn’t extend Remote, and whose methods may throw IOException,
>>>> and then we’ll have a proxy interface (which goes into the ‘-dl’ or in
>>>> Dennis’ terms ‘-proxy’ jar) called HelloRemote that extends Hello and
>>>> Remote. The client will download HelloRemote from the codebase
>>>> annotation.
>>>>
>>>> In any case, I don’t think Java lambdas are the magic bullet you’re
>>>> thinking they are. I’d be happy to be proven wrong though, because
>> what
>>>> you’re talking about would be pretty cool.
>>>>
>>>>
>>>> Cheers,
>>>>
>>>> Greg Trasuk
>>>
>>>
>>
Re: River-436 - need some explanation of preferred class provider
Posted by Tom Hobbs <tv...@googlemail.com>.
I'm certainly no expert, but my understanding is that the so called "meta
factory" lazily creates the byte code to service the lambda during first
call. So you are correct, they are not compiled to anon inner classes,
they do end up as private methods, but indy creates uses this factory as a
receipe for how to create the lambda, relying on the runtime to follow that
recipe as it sees fit.
>From your link.
"Instead of generating bytecode to create the object that implements the
lambda expression (such as calling a constructor for an inner class), we
describe a recipe for constructing the lambda, and delegate the actual
construction to the language runtime. That recipe is encoded in the static
and dynamic argument lists of an invokedynamic instruction. The use of
invokedynamic lets us defer the selection of a translation strategy until
run time."
They can also relinked/created/etc should subsequent calls to them be made
with different argument types. [2] is also an interesting read.
[1] http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html
[2]
http://www.slideshare.net/jaxlondon2012/lambda-a-peek-under-the-hood-brian-goetz
On Fri, Mar 7, 2014 at 6:55 PM, Michał Kłeczek <mi...@xpro.biz>wrote:
> Although lambdas are not compiled to anonymous classes the code IS NOT
> generated at runtime. They end up as methods and bound in runtime using
> invokeDynamic.
>
> See:
> http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html
>
> Regards
> 7 mar 2014 19:22 "Peter" <ji...@zeus.net.au> napisał(a):
>
> > ----- Original message -----
> > >
> > > On Mar 7, 2014, at 4:18 AM, Peter <ji...@zeus.net.au> wrote:
> > >
> > > >
> > > > > Also... Could you elaborate more on code dynamically generated at
> the
> > > > > server and how does it differ from code downloading?
> > > >
> > > > Bytecode for lambda expressions is generated at runtime by the jvm,
> > > > lambda's use a single serialization proxy class, necessary
> > > > implementation bytecode is dynamically compiled upon deserialisation.
> > > >
> > > > How lambdas change
> > > > > anything (since lambdas are only Java _language_ level
> > constructs I
> > > > > really don't see how).
> > > >
> > > > Dynamically compiled code doesn't need a codebase annotation, it's
> > > > created as needed and it's visibility is correct for each use case,
> > > > it's very specific, implementing a public interface and interacting
> > > > through this.
> > > >
> > >
> > > Um…are you sure about that?
> >
> > Yes, 100%, they're definitely not syntactic sugar, it does serialize the
> > final field arguments to the lambda expression, so if these are smart
> > proxy's, yes it will need to marshall those arguments but no, it's not an
> > internal anonymous class and doesn't need to send its enclosing class.
> >
> > Go have a look at Brian Goetz's presentations, check out some code and
> > read the development mail lists, it's pretty cool.
> >
> > Cheers,
> >
> > Peter.
> >
> > I confess that I haven’t tried it in code,
> > > but from reading JSR335, I get the impression that Java lambdas are not
> > > LISP lambdas, i.e. it’s not so much dynamic byte code generation as
> > > syntactic sugar that avoids the need to define an interface for what is
> > > effectively an inner class (it’s not quite an inner class, as it
> doesn’t
> > > redefine ‘this’, but is otherwise pretty close. For example, there
> > > aren’t real closures; local variables can only be captured if they are
> > > effectively final). There also seems to be a lot of compile-time type
> > > inference going on. I suspect that passing a lambda as a marshalled
> > > object will still require the containing class to be loaded.
> > >
> > > Now, dynamic proxies are a different story, and JERI already uses the
> > > dynamic proxy mechanism. There’s no need, for example to download an
> > > implementation class for an object that is directly exported - you only
> > > really need the service interface to be available locally. Having
> said
> > > that, I think most of us are in the habit of treating “java.rmi.Remote”
> > > as an implementation detail that shouldn’t be reflected in the service
> > > interface, so for instance, we’ll have a service interface, “Hello”
> > > which doesn’t extend Remote, and whose methods may throw IOException,
> > > and then we’ll have a proxy interface (which goes into the ‘-dl’ or in
> > > Dennis’ terms ‘-proxy’ jar) called HelloRemote that extends Hello and
> > > Remote. The client will download HelloRemote from the codebase
> > > annotation.
> > >
> > > In any case, I don’t think Java lambdas are the magic bullet you’re
> > > thinking they are. I’d be happy to be proven wrong though, because
> what
> > > you’re talking about would be pretty cool.
> > >
> > >
> > > Cheers,
> > >
> > > Greg Trasuk
> >
> >
>
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Although lambdas are not compiled to anonymous classes the code IS NOT
generated at runtime. They end up as methods and bound in runtime using
invokeDynamic.
See:
http://cr.openjdk.java.net/~briangoetz/lambda/lambda-translation.html
Regards
7 mar 2014 19:22 "Peter" <ji...@zeus.net.au> napisał(a):
> ----- Original message -----
> >
> > On Mar 7, 2014, at 4:18 AM, Peter <ji...@zeus.net.au> wrote:
> >
> > >
> > > > Also... Could you elaborate more on code dynamically generated at the
> > > > server and how does it differ from code downloading?
> > >
> > > Bytecode for lambda expressions is generated at runtime by the jvm,
> > > lambda's use a single serialization proxy class, necessary
> > > implementation bytecode is dynamically compiled upon deserialisation.
> > >
> > > How lambdas change
> > > > anything (since lambdas are only Java _language_ level
> constructs I
> > > > really don't see how).
> > >
> > > Dynamically compiled code doesn't need a codebase annotation, it's
> > > created as needed and it's visibility is correct for each use case,
> > > it's very specific, implementing a public interface and interacting
> > > through this.
> > >
> >
> > Um…are you sure about that?
>
> Yes, 100%, they're definitely not syntactic sugar, it does serialize the
> final field arguments to the lambda expression, so if these are smart
> proxy's, yes it will need to marshall those arguments but no, it's not an
> internal anonymous class and doesn't need to send its enclosing class.
>
> Go have a look at Brian Goetz's presentations, check out some code and
> read the development mail lists, it's pretty cool.
>
> Cheers,
>
> Peter.
>
> I confess that I haven’t tried it in code,
> > but from reading JSR335, I get the impression that Java lambdas are not
> > LISP lambdas, i.e. it’s not so much dynamic byte code generation as
> > syntactic sugar that avoids the need to define an interface for what is
> > effectively an inner class (it’s not quite an inner class, as it doesn’t
> > redefine ‘this’, but is otherwise pretty close. For example, there
> > aren’t real closures; local variables can only be captured if they are
> > effectively final). There also seems to be a lot of compile-time type
> > inference going on. I suspect that passing a lambda as a marshalled
> > object will still require the containing class to be loaded.
> >
> > Now, dynamic proxies are a different story, and JERI already uses the
> > dynamic proxy mechanism. There’s no need, for example to download an
> > implementation class for an object that is directly exported - you only
> > really need the service interface to be available locally. Having said
> > that, I think most of us are in the habit of treating “java.rmi.Remote”
> > as an implementation detail that shouldn’t be reflected in the service
> > interface, so for instance, we’ll have a service interface, “Hello”
> > which doesn’t extend Remote, and whose methods may throw IOException,
> > and then we’ll have a proxy interface (which goes into the ‘-dl’ or in
> > Dennis’ terms ‘-proxy’ jar) called HelloRemote that extends Hello and
> > Remote. The client will download HelloRemote from the codebase
> > annotation.
> >
> > In any case, I don’t think Java lambdas are the magic bullet you’re
> > thinking they are. I’d be happy to be proven wrong though, because what
> > you’re talking about would be pretty cool.
> >
> >
> > Cheers,
> >
> > Greg Trasuk
>
>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
----- Original message -----
>
> On Mar 7, 2014, at 4:18 AM, Peter <ji...@zeus.net.au> wrote:
>
> >
> > > Also... Could you elaborate more on code dynamically generated at the
> > > server and how does it differ from code downloading?
> >
> > Bytecode for lambda expressions is generated at runtime by the jvm,
> > lambda's use a single serialization proxy class, necessary
> > implementation bytecode is dynamically compiled upon deserialisation.
> >
> > How lambdas change
> > > anything (since lambdas are only Java _language_ level constructs I
> > > really don't see how).
> >
> > Dynamically compiled code doesn't need a codebase annotation, it's
> > created as needed and it's visibility is correct for each use case,
> > it's very specific, implementing a public interface and interacting
> > through this.
> >
>
> Um…are you sure about that?
Yes, 100%, they're definitely not syntactic sugar, it does serialize the final field arguments to the lambda expression, so if these are smart proxy's, yes it will need to marshall those arguments but no, it's not an internal anonymous class and doesn't need to send its enclosing class.
Go have a look at Brian Goetz's presentations, check out some code and read the development mail lists, it's pretty cool.
Cheers,
Peter.
I confess that I haven’t tried it in code,
> but from reading JSR335, I get the impression that Java lambdas are not
> LISP lambdas, i.e. it’s not so much dynamic byte code generation as
> syntactic sugar that avoids the need to define an interface for what is
> effectively an inner class (it’s not quite an inner class, as it doesn’t
> redefine ‘this’, but is otherwise pretty close. For example, there
> aren’t real closures; local variables can only be captured if they are
> effectively final). There also seems to be a lot of compile-time type
> inference going on. I suspect that passing a lambda as a marshalled
> object will still require the containing class to be loaded.
>
> Now, dynamic proxies are a different story, and JERI already uses the
> dynamic proxy mechanism. There’s no need, for example to download an
> implementation class for an object that is directly exported - you only
> really need the service interface to be available locally. Having said
> that, I think most of us are in the habit of treating “java.rmi.Remote”
> as an implementation detail that shouldn’t be reflected in the service
> interface, so for instance, we’ll have a service interface, “Hello”
> which doesn’t extend Remote, and whose methods may throw IOException,
> and then we’ll have a proxy interface (which goes into the ‘-dl’ or in
> Dennis’ terms ‘-proxy’ jar) called HelloRemote that extends Hello and
> Remote. The client will download HelloRemote from the codebase
> annotation.
>
> In any case, I don’t think Java lambdas are the magic bullet you’re
> thinking they are. I’d be happy to be proven wrong though, because what
> you’re talking about would be pretty cool.
>
>
> Cheers,
>
> Greg Trasuk
Re: River-436 - need some explanation of preferred class provider
Posted by Greg Trasuk <tr...@stratuscom.com>.
On Mar 7, 2014, at 4:18 AM, Peter <ji...@zeus.net.au> wrote:
>
>> Also... Could you elaborate more on code dynamically generated at the
>> server and how does it differ from code downloading?
>
> Bytecode for lambda expressions is generated at runtime by the jvm, lambda's use a single serialization proxy class, necessary implementation bytecode is dynamically compiled upon deserialisation.
>
> How lambdas change
>> anything (since lambdas are only Java _language_ level constructs I
>> really don't see how).
>
> Dynamically compiled code doesn't need a codebase annotation, it's created as needed and it's visibility is correct for each use case, it's very specific, implementing a public interface and interacting through this.
>
Um…are you sure about that? I confess that I haven’t tried it in code, but from reading JSR335, I get the impression that Java lambdas are not LISP lambdas, i.e. it’s not so much dynamic byte code generation as syntactic sugar that avoids the need to define an interface for what is effectively an inner class (it’s not quite an inner class, as it doesn’t redefine ‘this’, but is otherwise pretty close. For example, there aren’t real closures; local variables can only be captured if they are effectively final). There also seems to be a lot of compile-time type inference going on. I suspect that passing a lambda as a marshalled object will still require the containing class to be loaded.
Now, dynamic proxies are a different story, and JERI already uses the dynamic proxy mechanism. There’s no need, for example to download an implementation class for an object that is directly exported - you only really need the service interface to be available locally. Having said that, I think most of us are in the habit of treating “java.rmi.Remote” as an implementation detail that shouldn’t be reflected in the service interface, so for instance, we’ll have a service interface, “Hello” which doesn’t extend Remote, and whose methods may throw IOException, and then we’ll have a proxy interface (which goes into the ‘-dl’ or in Dennis’ terms ‘-proxy’ jar) called HelloRemote that extends Hello and Remote. The client will download HelloRemote from the codebase annotation.
In any case, I don’t think Java lambdas are the magic bullet you’re thinking they are. I’d be happy to be proven wrong though, because what you’re talking about would be pretty cool.
Cheers,
Greg Trasuk
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
----- Original message -----
> I am not saying OSGI is prevented. I am saying OSGI class loading model
> _could_ be used to fix class loading issues. Because current River class
> loading is simply broken.
>
> If you look at my example - the Wrapper service is simple and sharing
> only Wrapper interface with its client. And still it is not possible to
> use it!!!
PreferredClassLoader may not be the right tool for more complex class relationships.
>
> As per fixing security - may I ask how it is going to be accomplished?
>
By initially placing limitations on deserialisation until trust is established.
These are:
1. Place a limit of the size of the stream, avoids DOS.
2. Before using / loading any class during deserialisation, check it is trusted, against a list of trusted classes, this also means avoiding any codebase downloads.
3. After trust is established, remove restrictions.
To do this may require a buffer, where a stream is inspected prior to deserialisation, but where we are looking for a ProxyTrustVerifier proxy in the stream, after authenticating the proxy, we then ask that proxy for a list of classes it trusts to be deserialised, and we add these to the trusted class list.
Any classes that appear in the stream that aren't trusted aren't deserialised.
Following deserialisation the ProxyTrustVerifier proxy can then be asked to verify the smart proxy and if verified, deserialisation restrictions can be lifted, any constraints will remain in place.
Presently I'm working on fixing race conditions and visibility bugs, SDM is proving quite challenging.
> Also... Could you elaborate more on code dynamically generated at the
> server and how does it differ from code downloading?
Bytecode for lambda expressions is generated at runtime by the jvm, lambda's use a single serialization proxy class, necessary implementation bytecode is dynamically compiled upon deserialisation.
How lambdas change
> anything (since lambdas are only Java _language_ level constructs I
> really don't see how).
Dynamically compiled code doesn't need a codebase annotation, it's created as needed and it's visibility is correct for each use case, it's very specific, implementing a public interface and interacting through this.
>
> The solution to class loading issues is really simple... Use class
> loading model that is good enough :-) - hierarchical is not.
>
> I am actually thinking about jboss-modules library. From their website:
> "It implements a thread-safe, fast, and highly concurrent delegating
> class loader model, coupled to an extensible module resolution system,
> which combine to form a unique, simple and powerful system for
> application execution and distribution."
Looks interesting, appears to be AL2 licensed?
>
> Regards,
>
> On Thursday, March 06, 2014 07:56:13 PM Peter wrote:
> > River doesn't prevent using OSGi, in fact, we are making changes to
> > enable it, we also want to support Maven, but we don't currently
> > mandate using either.
> >
> > I'd reccomment keeping services simple and ensure you only share common
> > public interfaces and classes (service api) and ensure these shared
> > classes are installed locally on your clients.
> >
> > As for security that will be fixed in a following release and we are
> > making steps towards that.
> >
> > Even for OSGi, without River, deserialisation is complex beast.
> >
> > With Java 8, we'll be looking at lambda based services where code is
> > dynamically generated at the server, it's an inversion of control that
> > will allow much more flexibility. In fact I'd reccommend avoiding
> > downloaded code, if it can be avoided, or at least keep it simple and
> > ensure all shared classes are installed locally.
> >
> > If you have a solution to ClassLoader issues, I'm all ears, honestly
> > though, ClassLoaders are a Java issue, we do our best to work around
> > them, I'm not aware of a silver bullet. Beware of complex ClassLoader
> > relationships, as these can lead to deadlock.
> >
> > Cheers,
> >
> > Peter.
> >
>
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
<Attachment> Michał Kłeczek (XPro).vcf
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
I am not saying OSGI is prevented. I am saying OSGI class loading model
_could_ be used to fix class loading issues. Because current River class
loading is simply broken.
If you look at my example - the Wrapper service is simple and sharing only
Wrapper interface with its client. And still it is not possible to use it!!!
As per fixing security - may I ask how it is going to be accomplished?
Also... Could you elaborate more on code dynamically generated at the server
and how does it differ from code downloading? How lambdas change anything
(since lambdas are only Java _language_ level constructs I really don't see
how).
The solution to class loading issues is really simple... Use class loading
model that is good enough :-) - hierarchical is not.
I am actually thinking about jboss-modules library. From their website:
"It implements a thread-safe, fast, and highly concurrent delegating class
loader model, coupled to an extensible module resolution system, which combine
to form a unique, simple and powerful system for application execution and
distribution."
Regards,
On Thursday, March 06, 2014 07:56:13 PM Peter wrote:
> River doesn't prevent using OSGi, in fact, we are making changes to enable
> it, we also want to support Maven, but we don't currently mandate using
> either.
>
> I'd reccomment keeping services simple and ensure you only share common
> public interfaces and classes (service api) and ensure these shared classes
> are installed locally on your clients.
>
> As for security that will be fixed in a following release and we are making
> steps towards that.
>
> Even for OSGi, without River, deserialisation is complex beast.
>
> With Java 8, we'll be looking at lambda based services where code is
> dynamically generated at the server, it's an inversion of control that will
> allow much more flexibility. In fact I'd reccommend avoiding downloaded
> code, if it can be avoided, or at least keep it simple and ensure all
> shared classes are installed locally.
>
> If you have a solution to ClassLoader issues, I'm all ears, honestly though,
> ClassLoaders are a Java issue, we do our best to work around them, I'm not
> aware of a silver bullet. Beware of complex ClassLoader relationships, as
> these can lead to deadlock.
>
> Cheers,
>
> Peter.
>
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
Actually, Distributed could dovetail nicely with a modular service provider, by decoupling implementation from state, object state can be migrated to alternate implementations, remotely, or even locally within the same jvm.
Cheers,
Peter.
----- Original message -----
> ----- Original message -----
> > Peter,
> >
> > I must say that Distributable looks really cool - simple and elegant.
>
> It'd be nice to see it catch on!
>
> > The only thing that I would actually remove is setting up an
> > untrusted object security context in
> > SerialReflectionFactory.create(). It is not really needed _iff_ the
> > code you are executing is already trusted :)
>
> I wanted to prevent constructors being used on ClassLoader, Sockets and
> also prevent using private or package private constructors or basically
> creating objects that shouldn't be serialised. The reason is, the
> object that's about to be created has no ProtectionDomian yet on the
> stack. An object can of course perform a PrivilegedAction from within
> its constructor, so this seemed like a reasonable compromise.
>
>
> Cheers,
>
> Peter.
>
> >
> > Regards,
> >
> > On Thursday, March 06, 2014 09:42:50 PM Peter wrote:
> > > Also worth noting is an interface called Distributed, this is an
> > > alternative to Serializable. Underneath the covers it uses
> > > Externalizable, so it's fast, it also uses constructors, so objects
> > > can use final fields and preserve invarients and subclass non
> > > serializable objects without no arg constructors. Internal state of
> > > Distributed objects are not published and objects can evolve freely,
> > > provided they maintain constructor method signatures. In fact, you
> > > can use interfaces, static factory methods and builder objects, so
> > > your Distributed implementation can be package private and be
> > > completely replaced with another implementation even with a totally
> > > different class name, think providers
> > >
> > > Total and complete freedom from Serializable warts, but completely
> > > compatible, leveraging Object Input and Output streams and the entire
> > > existing Serializable infrastructure.
> > >
> > > Combined with remote reflection proxies and lambda based services
> > > with dynamic remote code generation, we'll be on the doorstep of
> > > distributed compute nirvana.
> > >
> > > Downloaded code may soon be irrelevant to distributed computing and
> > > with it versioning and ClassLoader issues.
> > >
> > > Takes a little time to sink in though.
> > >
> > > The next question for those daring enough to ask "Where will the
> > > future take us?"
> > >
> > > Cheers,
> > >
> > > Peter.
> > >
> > > ----- Original message -----
> > >
> > > > River doesn't prevent using OSGi, in fact, we are making changes to
> > > > enable it, we also want to support Maven, but we don't currently
> > > > mandate using either.
> > > >
> > > > I'd reccomment keeping services simple and ensure you only share
> > > > common public interfaces and classes (service api) and ensure these
> > > > shared classes are installed locally on your clients.
> > > >
> > > > As for security that will be fixed in a following release and we
> > > > are making steps towards that.
> > > >
> > > > Even for OSGi, without River, deserialisation is complex beast.
> > > >
> > > > With Java 8, we'll be looking at lambda based services where code
> > > > is dynamically generated at the server, it's an inversion of
> > > > control that will allow much more flexibility. In fact I'd
> > > > reccommend avoiding downloaded code, if it can be avoided, or at
> > > > least keep it simple and ensure all shared classes are installed
> > > > locally.
> > > >
> > > > If you have a solution to ClassLoader issues, I'm all ears,
> > > > honestly though, ClassLoaders are a Java issue, we do our best to
> > > > work around them, I'm not aware of a silver bullet. Beware of
> > > > complex ClassLoader relationships, as these can lead to deadlock.
> > > >
> > > > Cheers,
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > >
> > > > > Greg,
> > > > >
> > > > > I cannot agree more with the statement that mobile code is the
> > > > > cornerstone of River. I do not want to send just "data"
> > > > > (XML/JSON/whatever). I want to exchange _objects_ (data +
> > > > > code - well... in the end it is all data :-) ). And I want
> > > > > it to be _secure_ and ubiquitous. IMHO we should just make it
> > > > > work - because it simply does not work at this moment - as
> > > > > shown in my example.
> > > > >
> > > > > What I'm arguing is that it is not fixable with hierarchical
> > > > > class loading. The problem is that the client simply does
> > > > > not know what should be the "parent" when deserializing
> > > > > objects. Your River-336 patch is not going to change it
> > > > > (while still important since RMIClassLoader dependency is really
> > > > > a PITA).
> > > > >
> > > > > I am not proposing any particular solution at this moment - BUT I
> > > > > would really like the community to have a second look at
> > > > > existing solutions before coming up with yet another one.
> > > > >
> > > > > The requirements - as I see them right now would be:
> > > > >
> > > > > 1. Make it possible to exchange arbitrary object graphs composed
> > > > > of objects which classes are from _different_ codebases.
> > > > > That would make it possible to create smart proxies wrapping
> > > > > other services' smart proxies (which in theory is possible
> > > > > today... but in practice is not)
> > > > >
> > > > > 2. Do not allow yet untrusted code to execute. Ever.
> > > > > It basically means that from security perspective current
> > > > > algorithm: - deserialize an object
> > > > > - ask it for secure proxy
> > > > > - ask the secure proxy for a verifier
> > > > > - verify the original object
> > > > > is just simply wrong :)
> > > > > What's more - if we only allow trusted code to execute then the
> > > > > whole proxy verification problem becomes much simpler - all
> > > > > those complicated UntrustedObjectSecurityContext tricks are
> > > > > no longer necessary - it is enough to just ask an object
> > > > > to verify itself (placing apropriate invocation
> > > > > constraints on it first).
> > > > >
> > > > > The River-436 patch is going to (mostly) solve 2. The only
> > > > > potential holes left are in unicast response parsing where
> > > > > ObjectInputStream is created to read MarshalledInstance
> > > > > from the network - opening possibility of DOS or other
> > > > > attacks exploiting local code vulnerabilities. It is also going
> > > > > to open a possibility to solve 1 because it is going to make
> > > > > class loading algorithms extensible/dynamic.
> > > > >
> > > > > The big question is whether it is possible to fix 1 without
> > > > > imposing any "container" or "class loading scheme" on the
> > > > > client/service implementor.
> > > > >
> > > > > OTOH - while we do not provide any "container" today - we do
> > > > > provide a "platform". There is no way any client can work without
> > > > > having jsk- platform.jar in its classpath and without specifying
> > > > > PreferredClassProvider as RMIClassProvider and configuring
> > > > > httpmd url handler.
> > > > >
> > > > > Regards,
> > > > >
> > > > > On Tuesday, March 04, 2014 08:03:13 PM Gregg Wonderly wrote:
> > > > > > One of the greatest things about Java is serialization and
> > > > > > mobile code! One of the most limiting aspects of any
> > > > > > language is Serialization!
> > > > > >
> > > > > > If you have an interface or data class that two classes need to
> > > > > > access, there is no choice but to have a common parent class
> > > > > > loader. Your client can institute such as class loading scheme
> > > > > > completely independently of Jini’s use of some other class
> > > > > > loading scheme, provided that you at least allow the “parent
> > > > > > load this please” mechanism of hierarchical class loading to
> > > > > > occur.
> > > > > >
> > > > > > My changes to remove the explicit reliance/dependence on
> > > > > > RMIClassLoader in River-336, and instead allow you to plug in
> > > > > > how the “parent load this please” call out to work, is how you
> > > > > > can solve this so that it actually works for your clients
> > > > > > special needs. The best thing is that this mechanism can
> > > > > > be plugged into, at runtime, whereas RMIClassLoaderSPI is a
> > > > > > onetime thing that requires access to the app class loader,
> > > > > > which might not be possible in some clients. All that is
> > > > > > required is a security grant that allows a particular codebase
> > > > > > to plugin a specific class name.
> > > > > >
> > > > > > Gregg Wonderly
> > > > > >
> > > > > > On Mar 4, 2014, at 7:39 PM, Gregg Wonderly <ge...@cox.net>
> > > > > > wrote:
> > > > > > > > On Mar 4, 2014, at 12:02 AM, Michał Kłeczek
> > > > > > > > <mi...@xpro.biz> wrote:
> > > > > > > >
> > > > > > > > The real problem is that Util interface is in two
> > > > > > > > codebases. It should be in a single codebase shared
> > > > > > > > between UtilProxy and WrapperProxy. But to make it
> > > > > > > > possible we would need to have peer class loading like in
> > > > > > > > ClassWorlds or OSGI. It is not solvable in a standard
> > > > > > > > hierarchical class loading scheme.
> > > > > > >
> > > > > > > This is one of the good examples of where hierarchical
> > > > > > > loading can present challenges.
> > > > > > >
> > > > > > > But the question really is, can an arbitrary client really
> > > > > > > expect for arbitrary services to interact correctly?
> > > > > > > If you want them to do this, it has been shown over and over
> > > > > > > that global types are the best, least troublesome choice.
> > > > > > >
> > > > > > > If you want ubiquitous interactions why not use string based
> > > > > > > values such as XML or better yet, JSON?
> > > > > > >
> > > > > > > Then code and data is immune to class loading snafus and not
> > > > > > > bound to a container or hosting "standard"!
> > > > > > >
> > > > > > > Gregg
> > > > > > >
> > > > > > > > Anyway... It is not really River-436 problem so my patch
> > > > > > > > proposal is going
> > > > > > > > to have the same issue since it is just a replacement for
> > > > > > > > String annotations and not change in class loading scheme.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Michal
> > > > > > > >
> > > > > > > > 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz>
> > > > > > > >
> > > > > > > > napisał(a):
> > > > > > > > > 1. The problem is there is no such thing as "the service
> > > > > > > > > interface". It is
> > > > > > > > > context dependent. What is "the service interface" for
> > > > > > > > > service browser?
> > > > > > > > >
> > > > > > > > > 2. In this particular case Util interface is an
> > > > > > > > > implementation detail of WrapperProxy. It is Wrapper
> > > > > > > > > interface the client is interested in. So I would say it
> > > > > > > > > should be preferred in WrapperProxy codebase.
> > > > > > > > >
> > > > > > > > > 3. Even if Util is not preferred in WrapperProxy codebase
> > > > > > > > > we still have ClassCastException if the client does not
> > > > > > > > > have Util in its classpath. Why
> > > > > > > > > should it? it is interested in Wrapper not in Util. So
> > > > > > > > > either a. We always get ClassCastException if Util is
> > > > > > > > > preferred in WrapperProxy codebase, or
> > > > > > > > > b. We get ClassCastException anyway if a client does not
> > > > > > > > > have Util in its
> > > > > > > > > classpath.
> > > > > > > > > Let's say I want to register RemoteEventListener that
> > > > > > > > > wraps a Javaspace proxy to write events in a space. Does
> > > > > > > > > that mean the service "event source"
> > > > > > > > > has to be aware of Javaspace interface??? That would be
> > > > > > > > > absurd...
> > > > > > > > >
> > > > > > > > > It all does not have anything to do with codebase
> > > > > > > > > services.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Michal
> > > > > > > > >
> > > > > > > > > 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> > > > > > > > > > The Util interface should not be preferred.
> > > > > > > > > > Implementations of Util can
> > > > > > > > > > be preferred but not Util itself.
> > > > > > > > > >
> > > > > > > > > > Services need a common api that all implementations and
> > > > > > > > > > clients can use to interract, even if this is a kind of
> > > > > > > > > > codebase service.
> > > > > > > > > >
> > > > > > > > > > Modifying an interface is generally considered bad
> > > > > > > > > > practise but now Java
> > > > > > > > > > 8 makes it possible to add default methods for added
> > > > > > > > > > functionality, that
> > > > > > > > > > line blurs somewhat. What can you do if an
> > > > > > > > > > earlier interface is loaded by
> > > > > > > > > > a parent ClassLoader and you need a later version, make
> > > > > > > > > > it preferred?
> > > > > > > > > >
> > > > > > > > > > My thoughts are that interfaces should never be
> > > > > > > > > > preferred and all classes
> > > > > > > > > > defined in their methods shouldn't be preferred either.
> > > > > > > > > >
> > > > > > > > > > It would be relatively easy to write a new
> > > > > > > > > > implementation that ensures that interfaces are loaded
> > > > > > > > > > into their own ProtectionDomain in a parent
> > > > > > > > > > ClassLoader. But that would be confusing as
> > > > > > > > > > dynamic policy grants are made to ClassLoader's not
> > > > > > > > > > ProtectionDomains.
> > > > > > > > > >
> > > > > > > > > > But using ProtectionDomains in this manner, preserves
> > > > > > > > > > security, ensures maximum visibility and avoids
> > > > > > > > > > codebase annotation loss, if we ask the
> > > > > > > > > > ProtectionDomain for the annotation, instead of the
> > > > > > > > > > ClassLoader. But this is not how we do things
> > > > > > > > > > presently.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > >
> > > > > > > > > > Peter.
> > > > > > > > > >
> > > > > > > > > > ----- Original message -----
> > > > > > > > > >
> > > > > > > > > > > But it will also be loaded by WrapperProxy
> > > > > > > > > > > ClassLoader, since it is preferred there. So it will
> > > > > > > > > > > end up with ClassCastException, right?
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Michal
> > > > > > > > > > >
> > > > > > > > > > > If Util is installed locally, it will only be loaded
> > > > > > > > > > > by the application
> > > > > > > > > > > ClassLoader, since it isn't preferred.
> > > > > > > > > > >
> > > > > > > > > > > Peter.
> > > > > > > > > > >
> > > > > > > > > > > ----- Original message -----
> > > > > > > > > > >
> > > > > > > > > > > > Folks,
> > > > > > > > > > > > while woking on the River-436 patch proposal I've
> > > > > > > > > > > > came across the scenario that I am not sure how to
> > > > > > > > > > > > handle:
> > > > > > > > > > > >
> > > > > > > > > > > > Utility service:
> > > > > > > > > > > > //inteface is NOT preferred
> > > > > > > > > > > > interface Util {...}
> > > > > > > > > > > > //class IS preferred
> > > > > > > > > > > > class UtilProxy implements Util {}
> > > > > > > > > > > >
> > > > > > > > > > > > Wrapper service:
> > > > > > > > > > > > //NOT preferred
> > > > > > > > > > > > interface Wrapper {}
> > > > > > > > > > > > //preferred
> > > > > > > > > > > > class WrapperProxy implements Serializable{
> > > > > > > > > > > > //initialized with Util impl from a lookup service
> > > > > > > > > > > > private Util util;
> > > > > > > > > > > > }
> > > > > > > > > > > >
> > > > > > > > > > > > Wrapper service codebase includes Util interface
> > > > > > > > > > > > but it is
> > > > > > > > > >
> > > > > > > > > > _preferred_.
> > > > > > > > > >
> > > > > > > > > > > > Would deserialization of WrapperProxy end with
> > > > > > > > > > > > ClassCastException? From what I understand
> > > > > > > > > > > > UtilProxy is annotated with its codebase. When
> > > > > > > > > > > > deserializing UtilProxy a ClassLoader is going to
> > > > > > > > > > > > be created with parent set to TCCL. It means Util
> > > > > > > > > > > > interface is going to be loaded twice by two
> > > > > > > > > > > > ClassLoaders - one for WrapperProxy codebase and
> > > > > > > > > > > > another for UtilProxy codebase.
> > > > > > > > > > > >
> > > > > > > > > > > > Am I correct?
> > > > > > > > > > > > And if so: is it desired behavior?
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Michał Kłeczek
> > > > > > > > > > > > XPro Quality Matters
> > > > > > > > > > > > http://www.xpro.biz
> > > >
> > > > <Attachment> Michał Kłeczek (XPro).vcf
> >
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska
>
> <Attachment> Michał Kłeczek (XPro).vcf
>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
----- Original message -----
> Peter,
>
> I must say that Distributable looks really cool - simple and elegant.
It'd be nice to see it catch on!
> The only thing that I would actually remove is setting up an untrusted
> object security context in SerialReflectionFactory.create(). It is not
> really needed _iff_ the code you are executing is already trusted :)
I wanted to prevent constructors being used on ClassLoader, Sockets and also prevent using private or package private constructors or basically creating objects that shouldn't be serialised. The reason is, the object that's about to be created has no ProtectionDomian yet on the stack. An object can of course perform a PrivilegedAction from within its constructor, so this seemed like a reasonable compromise.
Cheers,
Peter.
>
> Regards,
>
> On Thursday, March 06, 2014 09:42:50 PM Peter wrote:
> > Also worth noting is an interface called Distributed, this is an
> > alternative to Serializable. Underneath the covers it uses
> > Externalizable, so it's fast, it also uses constructors, so objects
> > can use final fields and preserve invarients and subclass non
> > serializable objects without no arg constructors. Internal state of
> > Distributed objects are not published and objects can evolve freely,
> > provided they maintain constructor method signatures. In fact, you
> > can use interfaces, static factory methods and builder objects, so
> > your Distributed implementation can be package private and be
> > completely replaced with another implementation even with a totally
> > different class name, think providers
> >
> > Total and complete freedom from Serializable warts, but completely
> > compatible, leveraging Object Input and Output streams and the entire
> > existing Serializable infrastructure.
> >
> > Combined with remote reflection proxies and lambda based services with
> > dynamic remote code generation, we'll be on the doorstep of distributed
> > compute nirvana.
> >
> > Downloaded code may soon be irrelevant to distributed computing and
> > with it versioning and ClassLoader issues.
> >
> > Takes a little time to sink in though.
> >
> > The next question for those daring enough to ask "Where will the
> > future take us?"
> >
> > Cheers,
> >
> > Peter.
> >
> > ----- Original message -----
> >
> > > River doesn't prevent using OSGi, in fact, we are making changes to
> > > enable it, we also want to support Maven, but we don't currently
> > > mandate using either.
> > >
> > > I'd reccomment keeping services simple and ensure you only share
> > > common public interfaces and classes (service api) and ensure these
> > > shared classes are installed locally on your clients.
> > >
> > > As for security that will be fixed in a following release and we are
> > > making steps towards that.
> > >
> > > Even for OSGi, without River, deserialisation is complex beast.
> > >
> > > With Java 8, we'll be looking at lambda based services where code is
> > > dynamically generated at the server, it's an inversion of control
> > > that will allow much more flexibility. In fact I'd reccommend
> > > avoiding downloaded code, if it can be avoided, or at least keep it
> > > simple and ensure all shared classes are installed locally.
> > >
> > > If you have a solution to ClassLoader issues, I'm all ears, honestly
> > > though, ClassLoaders are a Java issue, we do our best to work around
> > > them, I'm not aware of a silver bullet. Beware of complex
> > > ClassLoader relationships, as these can lead to deadlock.
> > >
> > > Cheers,
> > >
> > > Peter.
> > >
> > > ----- Original message -----
> > >
> > > > Greg,
> > > >
> > > > I cannot agree more with the statement that mobile code is the
> > > > cornerstone of River. I do not want to send just "data"
> > > > (XML/JSON/whatever). I want to exchange _objects_ (data + code -
> > > > well... in the end it is all data :-) ). And I want it to be
> > > > _secure_ and ubiquitous. IMHO we should just make it work -
> > > > because it simply does not work at this moment - as shown in my
> > > > example.
> > > >
> > > > What I'm arguing is that it is not fixable with hierarchical class
> > > > loading. The problem is that the client simply does not know
> > > > what should be the "parent" when deserializing objects. Your
> > > > River-336 patch is not going to change it (while still
> > > > important since RMIClassLoader dependency is really a PITA).
> > > >
> > > > I am not proposing any particular solution at this moment - BUT I
> > > > would really like the community to have a second look at
> > > > existing solutions before coming up with yet another one.
> > > >
> > > > The requirements - as I see them right now would be:
> > > >
> > > > 1. Make it possible to exchange arbitrary object graphs composed of
> > > > objects which classes are from _different_ codebases.
> > > > That would make it possible to create smart proxies wrapping other
> > > > services' smart proxies (which in theory is possible today...
> > > > but in practice is not)
> > > >
> > > > 2. Do not allow yet untrusted code to execute. Ever.
> > > > It basically means that from security perspective current
> > > > algorithm: - deserialize an object
> > > > - ask it for secure proxy
> > > > - ask the secure proxy for a verifier
> > > > - verify the original object
> > > > is just simply wrong :)
> > > > What's more - if we only allow trusted code to execute then the
> > > > whole proxy verification problem becomes much simpler - all
> > > > those complicated UntrustedObjectSecurityContext tricks are no
> > > > longer necessary - it is enough to just ask an object to verify
> > > > itself (placing apropriate invocation constraints on it first).
> > > >
> > > > The River-436 patch is going to (mostly) solve 2. The only
> > > > potential holes left are in unicast response parsing where
> > > > ObjectInputStream is created to read MarshalledInstance from
> > > > the network - opening possibility of DOS or other attacks
> > > > exploiting local code vulnerabilities. It is also going to open a
> > > > possibility to solve 1 because it is going to make class
> > > > loading algorithms extensible/dynamic.
> > > >
> > > > The big question is whether it is possible to fix 1 without
> > > > imposing any "container" or "class loading scheme" on the
> > > > client/service implementor.
> > > >
> > > > OTOH - while we do not provide any "container" today - we do
> > > > provide a "platform". There is no way any client can work without
> > > > having jsk- platform.jar in its classpath and without specifying
> > > > PreferredClassProvider as RMIClassProvider and configuring
> > > > httpmd url handler.
> > > >
> > > > Regards,
> > > >
> > > > On Tuesday, March 04, 2014 08:03:13 PM Gregg Wonderly wrote:
> > > > > One of the greatest things about Java is serialization and mobile
> > > > > code! One of the most limiting aspects of any language is
> > > > > Serialization!
> > > > >
> > > > > If you have an interface or data class that two classes need to
> > > > > access, there is no choice but to have a common parent class
> > > > > loader. Your client can institute such as class loading scheme
> > > > > completely independently of Jini’s use of some other class
> > > > > loading scheme, provided that you at least allow the “parent
> > > > > load this please” mechanism of hierarchical class loading to
> > > > > occur.
> > > > >
> > > > > My changes to remove the explicit reliance/dependence on
> > > > > RMIClassLoader in River-336, and instead allow you to plug in
> > > > > how the “parent load this please” call out to work, is how you
> > > > > can solve this so that it actually works for your clients
> > > > > special needs. The best thing is that this mechanism can be
> > > > > plugged into, at runtime, whereas RMIClassLoaderSPI is a onetime
> > > > > thing that requires access to the app class loader, which might
> > > > > not be possible in some clients. All that is required is a
> > > > > security grant that allows a particular codebase to plugin a
> > > > > specific class name.
> > > > >
> > > > > Gregg Wonderly
> > > > >
> > > > > On Mar 4, 2014, at 7:39 PM, Gregg Wonderly <ge...@cox.net> wrote:
> > > > > > > On Mar 4, 2014, at 12:02 AM, Michał Kłeczek
> > > > > > > <mi...@xpro.biz> wrote:
> > > > > > >
> > > > > > > The real problem is that Util interface is in two codebases.
> > > > > > > It should be in a single codebase shared between UtilProxy
> > > > > > > and WrapperProxy. But to make it possible we would need to
> > > > > > > have peer class loading like in ClassWorlds or OSGI.
> > > > > > > It is not solvable in a standard hierarchical class loading
> > > > > > > scheme.
> > > > > >
> > > > > > This is one of the good examples of where hierarchical loading
> > > > > > can present challenges.
> > > > > >
> > > > > > But the question really is, can an arbitrary client really
> > > > > > expect for arbitrary services to interact correctly? If you
> > > > > > want them to do this, it has been shown over and over that
> > > > > > global types are the best, least troublesome choice.
> > > > > >
> > > > > > If you want ubiquitous interactions why not use string based
> > > > > > values such as XML or better yet, JSON?
> > > > > >
> > > > > > Then code and data is immune to class loading snafus and not
> > > > > > bound to a container or hosting "standard"!
> > > > > >
> > > > > > Gregg
> > > > > >
> > > > > > > Anyway... It is not really River-436 problem so my patch
> > > > > > > proposal is going
> > > > > > > to have the same issue since it is just a replacement for
> > > > > > > String annotations and not change in class loading scheme.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Michal
> > > > > > >
> > > > > > > 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz>
> > > > > > >
> > > > > > > napisał(a):
> > > > > > > > 1. The problem is there is no such thing as "the service
> > > > > > > > interface". It is
> > > > > > > > context dependent. What is "the service interface" for
> > > > > > > > service browser?
> > > > > > > >
> > > > > > > > 2. In this particular case Util interface is an
> > > > > > > > implementation detail of WrapperProxy. It is Wrapper
> > > > > > > > interface the client is interested in. So I would say it
> > > > > > > > should be preferred in WrapperProxy codebase.
> > > > > > > >
> > > > > > > > 3. Even if Util is not preferred in WrapperProxy codebase
> > > > > > > > we still have ClassCastException if the client does not
> > > > > > > > have Util in its classpath. Why
> > > > > > > > should it? it is interested in Wrapper not in Util. So
> > > > > > > > either a. We always get ClassCastException if Util is
> > > > > > > > preferred in WrapperProxy codebase, or
> > > > > > > > b. We get ClassCastException anyway if a client does not
> > > > > > > > have Util in its
> > > > > > > > classpath.
> > > > > > > > Let's say I want to register RemoteEventListener that
> > > > > > > > wraps a Javaspace proxy to write events in a space. Does
> > > > > > > > that mean the service "event source"
> > > > > > > > has to be aware of Javaspace interface??? That would be
> > > > > > > > absurd...
> > > > > > > >
> > > > > > > > It all does not have anything to do with codebase services.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Michal
> > > > > > > >
> > > > > > > > 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> > > > > > > > > The Util interface should not be preferred.
> > > > > > > > > Implementations of Util can
> > > > > > > > > be preferred but not Util itself.
> > > > > > > > >
> > > > > > > > > Services need a common api that all implementations and
> > > > > > > > > clients can use to interract, even if this is a kind of
> > > > > > > > > codebase service.
> > > > > > > > >
> > > > > > > > > Modifying an interface is generally considered bad
> > > > > > > > > practise but now Java
> > > > > > > > > 8 makes it possible to add default methods for added
> > > > > > > > > functionality, that
> > > > > > > > > line blurs somewhat. What can you do if an earlier
> > > > > > > > > interface is loaded by
> > > > > > > > > a parent ClassLoader and you need a later version, make
> > > > > > > > > it preferred?
> > > > > > > > >
> > > > > > > > > My thoughts are that interfaces should never be
> > > > > > > > > preferred and all classes
> > > > > > > > > defined in their methods shouldn't be preferred either.
> > > > > > > > >
> > > > > > > > > It would be relatively easy to write a new implementation
> > > > > > > > > that ensures that interfaces are loaded into their own
> > > > > > > > > ProtectionDomain in a parent ClassLoader. But that
> > > > > > > > > would be confusing as dynamic policy grants are made
> > > > > > > > > to ClassLoader's not ProtectionDomains.
> > > > > > > > >
> > > > > > > > > But using ProtectionDomains in this manner, preserves
> > > > > > > > > security, ensures maximum visibility and avoids codebase
> > > > > > > > > annotation loss, if we ask the ProtectionDomain for the
> > > > > > > > > annotation, instead of the ClassLoader. But this
> > > > > > > > > is not how we do things presently.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > Peter.
> > > > > > > > >
> > > > > > > > > ----- Original message -----
> > > > > > > > >
> > > > > > > > > > But it will also be loaded by WrapperProxy ClassLoader,
> > > > > > > > > > since it is preferred there. So it will end up with
> > > > > > > > > > ClassCastException, right?
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Michal
> > > > > > > > > >
> > > > > > > > > > If Util is installed locally, it will only be loaded
> > > > > > > > > > by the application
> > > > > > > > > > ClassLoader, since it isn't preferred.
> > > > > > > > > >
> > > > > > > > > > Peter.
> > > > > > > > > >
> > > > > > > > > > ----- Original message -----
> > > > > > > > > >
> > > > > > > > > > > Folks,
> > > > > > > > > > > while woking on the River-436 patch proposal I've
> > > > > > > > > > > came across the scenario that I am not sure how to
> > > > > > > > > > > handle:
> > > > > > > > > > >
> > > > > > > > > > > Utility service:
> > > > > > > > > > > //inteface is NOT preferred
> > > > > > > > > > > interface Util {...}
> > > > > > > > > > > //class IS preferred
> > > > > > > > > > > class UtilProxy implements Util {}
> > > > > > > > > > >
> > > > > > > > > > > Wrapper service:
> > > > > > > > > > > //NOT preferred
> > > > > > > > > > > interface Wrapper {}
> > > > > > > > > > > //preferred
> > > > > > > > > > > class WrapperProxy implements Serializable{
> > > > > > > > > > > //initialized with Util impl from a lookup service
> > > > > > > > > > > private Util util;
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > Wrapper service codebase includes Util interface but
> > > > > > > > > > > it is
> > > > > > > > >
> > > > > > > > > _preferred_.
> > > > > > > > >
> > > > > > > > > > > Would deserialization of WrapperProxy end with
> > > > > > > > > > > ClassCastException? From what I understand UtilProxy
> > > > > > > > > > > is annotated with its codebase. When deserializing
> > > > > > > > > > > UtilProxy a ClassLoader is going to be created with
> > > > > > > > > > > parent set to TCCL. It means Util interface is going
> > > > > > > > > > > to be loaded twice by two ClassLoaders - one for
> > > > > > > > > > > WrapperProxy codebase and another for UtilProxy
> > > > > > > > > > > codebase.
> > > > > > > > > > >
> > > > > > > > > > > Am I correct?
> > > > > > > > > > > And if so: is it desired behavior?
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Michał Kłeczek
> > > > > > > > > > > XPro Quality Matters
> > > > > > > > > > > http://www.xpro.biz
> > >
> > > <Attachment> Michał Kłeczek (XPro).vcf
>
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
<Attachment> Michał Kłeczek (XPro).vcf
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Peter,
I must say that Distributable looks really cool - simple and elegant. The only
thing that I would actually remove is setting up an untrusted object security
context in SerialReflectionFactory.create(). It is not really needed _iff_ the
code you are executing is already trusted :)
Regards,
On Thursday, March 06, 2014 09:42:50 PM Peter wrote:
> Also worth noting is an interface called Distributed, this is an alternative
> to Serializable. Underneath the covers it uses Externalizable, so it's
> fast, it also uses constructors, so objects can use final fields and
> preserve invarients and subclass non serializable objects without no arg
> constructors. Internal state of Distributed objects are not published and
> objects can evolve freely, provided they maintain constructor method
> signatures. In fact, you can use interfaces, static factory methods and
> builder objects, so your Distributed implementation can be package private
> and be completely replaced with another implementation even with a totally
> different class name, think providers
>
> Total and complete freedom from Serializable warts, but completely
> compatible, leveraging Object Input and Output streams and the entire
> existing Serializable infrastructure.
>
> Combined with remote reflection proxies and lambda based services with
> dynamic remote code generation, we'll be on the doorstep of distributed
> compute nirvana.
>
> Downloaded code may soon be irrelevant to distributed computing and with it
> versioning and ClassLoader issues.
>
> Takes a little time to sink in though.
>
> The next question for those daring enough to ask "Where will the future take
> us?"
>
> Cheers,
>
> Peter.
>
> ----- Original message -----
>
> > River doesn't prevent using OSGi, in fact, we are making changes to
> > enable it, we also want to support Maven, but we don't currently mandate
> > using either.
> >
> > I'd reccomment keeping services simple and ensure you only share common
> > public interfaces and classes (service api) and ensure these shared
> > classes are installed locally on your clients.
> >
> > As for security that will be fixed in a following release and we are
> > making steps towards that.
> >
> > Even for OSGi, without River, deserialisation is complex beast.
> >
> > With Java 8, we'll be looking at lambda based services where code is
> > dynamically generated at the server, it's an inversion of control that
> > will allow much more flexibility. In fact I'd reccommend avoiding
> > downloaded code, if it can be avoided, or at least keep it simple and
> > ensure all shared classes are installed locally.
> >
> > If you have a solution to ClassLoader issues, I'm all ears, honestly
> > though, ClassLoaders are a Java issue, we do our best to work around
> > them, I'm not aware of a silver bullet. Beware of complex ClassLoader
> > relationships, as these can lead to deadlock.
> >
> > Cheers,
> >
> > Peter.
> >
> > ----- Original message -----
> >
> > > Greg,
> > >
> > > I cannot agree more with the statement that mobile code is the
> > > cornerstone of River. I do not want to send just "data"
> > > (XML/JSON/whatever). I want to exchange _objects_ (data + code -
> > > well... in the end it is all data :-) ). And I want it to be _secure_
> > > and ubiquitous. IMHO we should just make it work - because it simply
> > > does not work at this moment - as shown in my example.
> > >
> > > What I'm arguing is that it is not fixable with hierarchical class
> > > loading. The problem is that the client simply does not know what
> > > should be the "parent" when deserializing objects. Your River-336
> > > patch is not going to change it (while still important since
> > > RMIClassLoader dependency is really a PITA).
> > >
> > > I am not proposing any particular solution at this moment - BUT I would
> > > really like the community to have a second look at existing solutions
> > > before coming up with yet another one.
> > >
> > > The requirements - as I see them right now would be:
> > >
> > > 1. Make it possible to exchange arbitrary object graphs composed of
> > > objects which classes are from _different_ codebases.
> > > That would make it possible to create smart proxies wrapping other
> > > services' smart proxies (which in theory is possible today... but in
> > > practice is not)
> > >
> > > 2. Do not allow yet untrusted code to execute. Ever.
> > > It basically means that from security perspective current algorithm:
> > > - deserialize an object
> > > - ask it for secure proxy
> > > - ask the secure proxy for a verifier
> > > - verify the original object
> > > is just simply wrong :)
> > > What's more - if we only allow trusted code to execute then the whole
> > > proxy verification problem becomes much simpler - all those
> > > complicated UntrustedObjectSecurityContext tricks are no longer
> > > necessary - it is enough to just ask an object to verify itself
> > > (placing apropriate invocation constraints on it first).
> > >
> > > The River-436 patch is going to (mostly) solve 2. The only potential
> > > holes left are in unicast response parsing where ObjectInputStream is
> > > created to read MarshalledInstance from the network - opening
> > > possibility of DOS or other attacks exploiting local code
> > > vulnerabilities. It is also going to open a possibility to solve 1
> > > because it is going to make class loading algorithms
> > > extensible/dynamic.
> > >
> > > The big question is whether it is possible to fix 1 without imposing
> > > any "container" or "class loading scheme" on the client/service
> > > implementor.
> > >
> > > OTOH - while we do not provide any "container" today - we do provide a
> > > "platform". There is no way any client can work without having jsk-
> > > platform.jar in its classpath and without specifying
> > > PreferredClassProvider as RMIClassProvider and configuring httpmd url
> > > handler.
> > >
> > > Regards,
> > >
> > > On Tuesday, March 04, 2014 08:03:13 PM Gregg Wonderly wrote:
> > > > One of the greatest things about Java is serialization and mobile
> > > > code! One of the most limiting aspects of any language is
> > > > Serialization!
> > > >
> > > > If you have an interface or data class that two classes need to
> > > > access, there is no choice but to have a common parent class loader.
> > > > Your client can institute such as class loading scheme completely
> > > > independently of Jini’s use of some other class loading scheme,
> > > > provided that you at least allow the “parent load this please”
> > > > mechanism of hierarchical class loading to occur.
> > > >
> > > > My changes to remove the explicit reliance/dependence on
> > > > RMIClassLoader in River-336, and instead allow you to plug in how the
> > > > “parent load this please” call out to work, is how you can solve this
> > > > so that it actually works for your clients special needs. The best
> > > > thing is that this mechanism can be plugged into, at runtime, whereas
> > > > RMIClassLoaderSPI is a onetime thing that requires access to the app
> > > > class loader, which might not be possible in some clients. All that
> > > > is required is a security grant that allows a particular codebase to
> > > > plugin a specific class name.
> > > >
> > > > Gregg Wonderly
> > > >
> > > > On Mar 4, 2014, at 7:39 PM, Gregg Wonderly <ge...@cox.net> wrote:
> > > > > > On Mar 4, 2014, at 12:02 AM, Michał Kłeczek
> > > > > > <mi...@xpro.biz> wrote:
> > > > > >
> > > > > > The real problem is that Util interface is in two codebases. It
> > > > > > should be in a single codebase shared between UtilProxy and
> > > > > > WrapperProxy. But to make it possible we would need to have peer
> > > > > > class loading like in ClassWorlds or OSGI.
> > > > > > It is not solvable in a standard hierarchical class loading
> > > > > > scheme.
> > > > >
> > > > > This is one of the good examples of where hierarchical loading can
> > > > > present challenges.
> > > > >
> > > > > But the question really is, can an arbitrary client really expect
> > > > > for arbitrary services to interact correctly? If you want them
> > > > > to do this, it has been shown over and over that global types are
> > > > > the best, least troublesome choice.
> > > > >
> > > > > If you want ubiquitous interactions why not use string based values
> > > > > such as XML or better yet, JSON?
> > > > >
> > > > > Then code and data is immune to class loading snafus and not bound
> > > > > to a container or hosting "standard"!
> > > > >
> > > > > Gregg
> > > > >
> > > > > > Anyway... It is not really River-436 problem so my patch proposal
> > > > > > is going
> > > > > > to have the same issue since it is just a replacement for String
> > > > > > annotations and not change in class loading scheme.
> > > > > >
> > > > > > Thanks,
> > > > > > Michal
> > > > > >
> > > > > > 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz>
> > > > > >
> > > > > > napisał(a):
> > > > > > > 1. The problem is there is no such thing as "the service
> > > > > > > interface". It is
> > > > > > > context dependent. What is "the service interface" for service
> > > > > > > browser?
> > > > > > >
> > > > > > > 2. In this particular case Util interface is an implementation
> > > > > > > detail of WrapperProxy. It is Wrapper interface the client is
> > > > > > > interested in. So I would say it should be preferred in
> > > > > > > WrapperProxy codebase.
> > > > > > >
> > > > > > > 3. Even if Util is not preferred in WrapperProxy codebase we
> > > > > > > still have ClassCastException if the client does not have Util
> > > > > > > in its classpath. Why
> > > > > > > should it? it is interested in Wrapper not in Util. So either
> > > > > > > a. We always get ClassCastException if Util is preferred in
> > > > > > > WrapperProxy codebase, or
> > > > > > > b. We get ClassCastException anyway if a client does not have
> > > > > > > Util in its
> > > > > > > classpath.
> > > > > > > Let's say I want to register RemoteEventListener that wraps a
> > > > > > > Javaspace proxy to write events in a space. Does that mean the
> > > > > > > service "event source"
> > > > > > > has to be aware of Javaspace interface??? That would be
> > > > > > > absurd...
> > > > > > >
> > > > > > > It all does not have anything to do with codebase services.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Michal
> > > > > > >
> > > > > > > 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> > > > > > > > The Util interface should not be preferred. Implementations
> > > > > > > > of Util can
> > > > > > > > be preferred but not Util itself.
> > > > > > > >
> > > > > > > > Services need a common api that all implementations and
> > > > > > > > clients can use to interract, even if this is a kind of
> > > > > > > > codebase service.
> > > > > > > >
> > > > > > > > Modifying an interface is generally considered bad practise
> > > > > > > > but now Java
> > > > > > > > 8 makes it possible to add default methods for added
> > > > > > > > functionality, that
> > > > > > > > line blurs somewhat. What can you do if an earlier
> > > > > > > > interface is loaded by
> > > > > > > > a parent ClassLoader and you need a later version, make it
> > > > > > > > preferred?
> > > > > > > >
> > > > > > > > My thoughts are that interfaces should never be preferred and
> > > > > > > > all classes
> > > > > > > > defined in their methods shouldn't be preferred either.
> > > > > > > >
> > > > > > > > It would be relatively easy to write a new implementation
> > > > > > > > that ensures that interfaces are loaded into their own
> > > > > > > > ProtectionDomain in a parent ClassLoader. But that would be
> > > > > > > > confusing as dynamic policy grants are made
> > > > > > > > to ClassLoader's not ProtectionDomains.
> > > > > > > >
> > > > > > > > But using ProtectionDomains in this manner, preserves
> > > > > > > > security, ensures maximum visibility and avoids codebase
> > > > > > > > annotation loss, if we ask the ProtectionDomain for the
> > > > > > > > annotation, instead of the ClassLoader. But this
> > > > > > > > is not how we do things presently.
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Peter.
> > > > > > > >
> > > > > > > > ----- Original message -----
> > > > > > > >
> > > > > > > > > But it will also be loaded by WrapperProxy ClassLoader,
> > > > > > > > > since it is preferred there. So it will end up with
> > > > > > > > > ClassCastException, right?
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Michal
> > > > > > > > >
> > > > > > > > > If Util is installed locally, it will only be loaded by the
> > > > > > > > > application
> > > > > > > > > ClassLoader, since it isn't preferred.
> > > > > > > > >
> > > > > > > > > Peter.
> > > > > > > > >
> > > > > > > > > ----- Original message -----
> > > > > > > > >
> > > > > > > > > > Folks,
> > > > > > > > > > while woking on the River-436 patch proposal I've came
> > > > > > > > > > across the scenario that I am not sure how to handle:
> > > > > > > > > >
> > > > > > > > > > Utility service:
> > > > > > > > > > //inteface is NOT preferred
> > > > > > > > > > interface Util {...}
> > > > > > > > > > //class IS preferred
> > > > > > > > > > class UtilProxy implements Util {}
> > > > > > > > > >
> > > > > > > > > > Wrapper service:
> > > > > > > > > > //NOT preferred
> > > > > > > > > > interface Wrapper {}
> > > > > > > > > > //preferred
> > > > > > > > > > class WrapperProxy implements Serializable{
> > > > > > > > > > //initialized with Util impl from a lookup service
> > > > > > > > > > private Util util;
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > Wrapper service codebase includes Util interface but it
> > > > > > > > > > is
> > > > > > > >
> > > > > > > > _preferred_.
> > > > > > > >
> > > > > > > > > > Would deserialization of WrapperProxy end with
> > > > > > > > > > ClassCastException? From what I understand UtilProxy is
> > > > > > > > > > annotated with its codebase. When deserializing UtilProxy
> > > > > > > > > > a ClassLoader is going to be created with parent set to
> > > > > > > > > > TCCL. It means Util interface is going to be loaded twice
> > > > > > > > > > by two ClassLoaders - one for WrapperProxy codebase and
> > > > > > > > > > another for UtilProxy codebase.
> > > > > > > > > >
> > > > > > > > > > Am I correct?
> > > > > > > > > > And if so: is it desired behavior?
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Michał Kłeczek
> > > > > > > > > > XPro Quality Matters
> > > > > > > > > > http://www.xpro.biz
> >
> > <Attachment> Michał Kłeczek (XPro).vcf
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
Also worth noting is an interface called Distributed, this is an alternative to Serializable. Underneath the covers it uses Externalizable, so it's fast, it also uses constructors, so objects can use final fields and preserve invarients and subclass non serializable objects without no arg constructors. Internal state of Distributed objects are not published and objects can evolve freely, provided they maintain constructor method signatures. In fact, you can use interfaces, static factory methods and builder objects, so your Distributed implementation can be package private and be completely replaced with another implementation even with a totally different class name, think providers
Total and complete freedom from Serializable warts, but completely compatible, leveraging Object Input and Output streams and the entire existing Serializable infrastructure.
Combined with remote reflection proxies and lambda based services with dynamic remote code generation, we'll be on the doorstep of distributed compute nirvana.
Downloaded code may soon be irrelevant to distributed computing and with it versioning and ClassLoader issues.
Takes a little time to sink in though.
The next question for those daring enough to ask "Where will the future take us?"
Cheers,
Peter.
----- Original message -----
> River doesn't prevent using OSGi, in fact, we are making changes to
> enable it, we also want to support Maven, but we don't currently mandate
> using either.
>
> I'd reccomment keeping services simple and ensure you only share common
> public interfaces and classes (service api) and ensure these shared
> classes are installed locally on your clients.
>
> As for security that will be fixed in a following release and we are
> making steps towards that.
>
> Even for OSGi, without River, deserialisation is complex beast.
>
> With Java 8, we'll be looking at lambda based services where code is
> dynamically generated at the server, it's an inversion of control that
> will allow much more flexibility. In fact I'd reccommend avoiding
> downloaded code, if it can be avoided, or at least keep it simple and
> ensure all shared classes are installed locally.
>
> If you have a solution to ClassLoader issues, I'm all ears, honestly
> though, ClassLoaders are a Java issue, we do our best to work around
> them, I'm not aware of a silver bullet. Beware of complex ClassLoader
> relationships, as these can lead to deadlock.
>
> Cheers,
>
> Peter.
>
> ----- Original message -----
> > Greg,
> >
> > I cannot agree more with the statement that mobile code is the
> > cornerstone of River. I do not want to send just "data"
> > (XML/JSON/whatever). I want to exchange _objects_ (data + code -
> > well... in the end it is all data :-) ). And I want it to be _secure_
> > and ubiquitous. IMHO we should just make it work - because it simply
> > does not work at this moment - as shown in my example.
> >
> > What I'm arguing is that it is not fixable with hierarchical class
> > loading. The problem is that the client simply does not know what
> > should be the "parent" when deserializing objects. Your River-336
> > patch is not going to change it (while still important since
> > RMIClassLoader dependency is really a PITA).
> >
> > I am not proposing any particular solution at this moment - BUT I would
> > really like the community to have a second look at existing solutions
> > before coming up with yet another one.
> >
> > The requirements - as I see them right now would be:
> >
> > 1. Make it possible to exchange arbitrary object graphs composed of
> > objects which classes are from _different_ codebases.
> > That would make it possible to create smart proxies wrapping other
> > services' smart proxies (which in theory is possible today... but in
> > practice is not)
> >
> > 2. Do not allow yet untrusted code to execute. Ever.
> > It basically means that from security perspective current algorithm:
> > - deserialize an object
> > - ask it for secure proxy
> > - ask the secure proxy for a verifier
> > - verify the original object
> > is just simply wrong :)
> > What's more - if we only allow trusted code to execute then the whole
> > proxy verification problem becomes much simpler - all those
> > complicated UntrustedObjectSecurityContext tricks are no longer
> > necessary - it is enough to just ask an object to verify itself
> > (placing apropriate invocation constraints on it first).
> >
> > The River-436 patch is going to (mostly) solve 2. The only potential
> > holes left are in unicast response parsing where ObjectInputStream is
> > created to read MarshalledInstance from the network - opening
> > possibility of DOS or other attacks exploiting local code
> > vulnerabilities. It is also going to open a possibility to solve 1
> > because it is going to make class loading algorithms
> > extensible/dynamic.
> >
> > The big question is whether it is possible to fix 1 without imposing
> > any "container" or "class loading scheme" on the client/service
> > implementor.
> >
> > OTOH - while we do not provide any "container" today - we do provide a
> > "platform". There is no way any client can work without having jsk-
> > platform.jar in its classpath and without specifying
> > PreferredClassProvider as RMIClassProvider and configuring httpmd url
> > handler.
> >
> > Regards,
> >
> > On Tuesday, March 04, 2014 08:03:13 PM Gregg Wonderly wrote:
> > > One of the greatest things about Java is serialization and mobile
> > > code! One of the most limiting aspects of any language is
> > > Serialization!
> > >
> > > If you have an interface or data class that two classes need to
> > > access, there is no choice but to have a common parent class loader.
> > > Your client can institute such as class loading scheme completely
> > > independently of Jini’s use of some other class loading scheme,
> > > provided that you at least allow the “parent load this please”
> > > mechanism of hierarchical class loading to occur.
> > >
> > > My changes to remove the explicit reliance/dependence on
> > > RMIClassLoader in River-336, and instead allow you to plug in how the
> > > “parent load this please” call out to work, is how you can solve this
> > > so that it actually works for your clients special needs. The best
> > > thing is that this mechanism can be plugged into, at runtime, whereas
> > > RMIClassLoaderSPI is a onetime thing that requires access to the app
> > > class loader, which might not be possible in some clients. All that
> > > is required is a security grant that allows a particular codebase to
> > > plugin a specific class name.
> > >
> > > Gregg Wonderly
> > >
> > > On Mar 4, 2014, at 7:39 PM, Gregg Wonderly <ge...@cox.net> wrote:
> > > > > On Mar 4, 2014, at 12:02 AM, Michał Kłeczek
> > > > > <mi...@xpro.biz> wrote:
> > > > >
> > > > > The real problem is that Util interface is in two codebases. It
> > > > > should be in a single codebase shared between UtilProxy and
> > > > > WrapperProxy. But to make it possible we would need to have peer
> > > > > class loading like in ClassWorlds or OSGI.
> > > > > It is not solvable in a standard hierarchical class loading
> > > > > scheme.
> > > >
> > > > This is one of the good examples of where hierarchical loading can
> > > > present challenges.
> > > >
> > > > But the question really is, can an arbitrary client really expect
> > > > for arbitrary services to interact correctly? If you want them
> > > > to do this, it has been shown over and over that global types are
> > > > the best, least troublesome choice.
> > > >
> > > > If you want ubiquitous interactions why not use string based values
> > > > such as XML or better yet, JSON?
> > > >
> > > > Then code and data is immune to class loading snafus and not bound
> > > > to a container or hosting "standard"!
> > > >
> > > > Gregg
> > > >
> > > > > Anyway... It is not really River-436 problem so my patch proposal
> > > > > is going
> > > > > to have the same issue since it is just a replacement for String
> > > > > annotations and not change in class loading scheme.
> > > > >
> > > > > Thanks,
> > > > > Michal
> > > > >
> > > > > 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz>
> > > > > napisał(a):
> > > > > > 1. The problem is there is no such thing as "the service
> > > > > > interface". It is
> > > > > > context dependent. What is "the service interface" for service
> > > > > > browser?
> > > > > >
> > > > > > 2. In this particular case Util interface is an implementation
> > > > > > detail of WrapperProxy. It is Wrapper interface the client is
> > > > > > interested in. So I would say it should be preferred in
> > > > > > WrapperProxy codebase.
> > > > > >
> > > > > > 3. Even if Util is not preferred in WrapperProxy codebase we
> > > > > > still have ClassCastException if the client does not have Util
> > > > > > in its classpath. Why
> > > > > > should it? it is interested in Wrapper not in Util. So either
> > > > > > a. We always get ClassCastException if Util is preferred in
> > > > > > WrapperProxy codebase, or
> > > > > > b. We get ClassCastException anyway if a client does not have
> > > > > > Util in its
> > > > > > classpath.
> > > > > > Let's say I want to register RemoteEventListener that wraps a
> > > > > > Javaspace proxy to write events in a space. Does that mean the
> > > > > > service "event source"
> > > > > > has to be aware of Javaspace interface??? That would be
> > > > > > absurd...
> > > > > >
> > > > > > It all does not have anything to do with codebase services.
> > > > > >
> > > > > > Thanks,
> > > > > > Michal
> > > > > >
> > > > > > 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> > > > > > > The Util interface should not be preferred. Implementations
> > > > > > > of Util can
> > > > > > > be preferred but not Util itself.
> > > > > > >
> > > > > > > Services need a common api that all implementations and
> > > > > > > clients can use to interract, even if this is a kind of
> > > > > > > codebase service.
> > > > > > >
> > > > > > > Modifying an interface is generally considered bad practise
> > > > > > > but now Java
> > > > > > > 8 makes it possible to add default methods for added
> > > > > > > functionality, that
> > > > > > > line blurs somewhat. What can you do if an earlier
> > > > > > > interface is loaded by
> > > > > > > a parent ClassLoader and you need a later version, make it
> > > > > > > preferred?
> > > > > > >
> > > > > > > My thoughts are that interfaces should never be preferred and
> > > > > > > all classes
> > > > > > > defined in their methods shouldn't be preferred either.
> > > > > > >
> > > > > > > It would be relatively easy to write a new implementation
> > > > > > > that ensures that interfaces are loaded into their own
> > > > > > > ProtectionDomain in a parent ClassLoader. But that would be
> > > > > > > confusing as dynamic policy grants are made
> > > > > > > to ClassLoader's not ProtectionDomains.
> > > > > > >
> > > > > > > But using ProtectionDomains in this manner, preserves
> > > > > > > security, ensures maximum visibility and avoids codebase
> > > > > > > annotation loss, if we ask the ProtectionDomain for the
> > > > > > > annotation, instead of the ClassLoader. But this
> > > > > > > is not how we do things presently.
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Peter.
> > > > > > >
> > > > > > > ----- Original message -----
> > > > > > >
> > > > > > > > But it will also be loaded by WrapperProxy ClassLoader,
> > > > > > > > since it is preferred there. So it will end up with
> > > > > > > > ClassCastException, right?
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Michal
> > > > > > > >
> > > > > > > > If Util is installed locally, it will only be loaded by the
> > > > > > > > application
> > > > > > > > ClassLoader, since it isn't preferred.
> > > > > > > >
> > > > > > > > Peter.
> > > > > > > >
> > > > > > > > ----- Original message -----
> > > > > > > >
> > > > > > > > > Folks,
> > > > > > > > > while woking on the River-436 patch proposal I've came
> > > > > > > > > across the scenario that I am not sure how to handle:
> > > > > > > > >
> > > > > > > > > Utility service:
> > > > > > > > > //inteface is NOT preferred
> > > > > > > > > interface Util {...}
> > > > > > > > > //class IS preferred
> > > > > > > > > class UtilProxy implements Util {}
> > > > > > > > >
> > > > > > > > > Wrapper service:
> > > > > > > > > //NOT preferred
> > > > > > > > > interface Wrapper {}
> > > > > > > > > //preferred
> > > > > > > > > class WrapperProxy implements Serializable{
> > > > > > > > > //initialized with Util impl from a lookup service
> > > > > > > > > private Util util;
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > Wrapper service codebase includes Util interface but it
> > > > > > > > > is
> > > > > > >
> > > > > > > _preferred_.
> > > > > > >
> > > > > > > > > Would deserialization of WrapperProxy end with
> > > > > > > > > ClassCastException? From what I understand UtilProxy is
> > > > > > > > > annotated with its codebase. When deserializing UtilProxy
> > > > > > > > > a ClassLoader is going to be created with parent set to
> > > > > > > > > TCCL. It means Util interface is going to be loaded twice
> > > > > > > > > by two ClassLoaders - one for WrapperProxy codebase and
> > > > > > > > > another for UtilProxy codebase.
> > > > > > > > >
> > > > > > > > > Am I correct?
> > > > > > > > > And if so: is it desired behavior?
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Michał Kłeczek
> > > > > > > > > XPro Quality Matters
> > > > > > > > > http://www.xpro.biz
> >
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska
> > --
> > Michał Kłeczek
> > XPro Sp. z o. o.
> > ul. Borowskiego 2
> > 03-475 Warszawa
> > Polska
>
> <Attachment> Michał Kłeczek (XPro).vcf
>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
River doesn't prevent using OSGi, in fact, we are making changes to enable it, we also want to support Maven, but we don't currently mandate using either.
I'd reccomment keeping services simple and ensure you only share common public interfaces and classes (service api) and ensure these shared classes are installed locally on your clients.
As for security that will be fixed in a following release and we are making steps towards that.
Even for OSGi, without River, deserialisation is complex beast.
With Java 8, we'll be looking at lambda based services where code is dynamically generated at the server, it's an inversion of control that will allow much more flexibility. In fact I'd reccommend avoiding downloaded code, if it can be avoided, or at least keep it simple and ensure all shared classes are installed locally.
If you have a solution to ClassLoader issues, I'm all ears, honestly though, ClassLoaders are a Java issue, we do our best to work around them, I'm not aware of a silver bullet. Beware of complex ClassLoader relationships, as these can lead to deadlock.
Cheers,
Peter.
----- Original message -----
> Greg,
>
> I cannot agree more with the statement that mobile code is the
> cornerstone of River. I do not want to send just "data"
> (XML/JSON/whatever). I want to exchange _objects_ (data + code -
> well... in the end it is all data :-) ). And I want it to be _secure_
> and ubiquitous. IMHO we should just make it work - because it simply
> does not work at this moment - as shown in my example.
>
> What I'm arguing is that it is not fixable with hierarchical class
> loading. The problem is that the client simply does not know what
> should be the "parent" when deserializing objects. Your River-336 patch
> is not going to change it (while still important since RMIClassLoader
> dependency is really a PITA).
>
> I am not proposing any particular solution at this moment - BUT I would
> really like the community to have a second look at existing solutions
> before coming up with yet another one.
>
> The requirements - as I see them right now would be:
>
> 1. Make it possible to exchange arbitrary object graphs composed of
> objects which classes are from _different_ codebases.
> That would make it possible to create smart proxies wrapping other
> services' smart proxies (which in theory is possible today... but in
> practice is not)
>
> 2. Do not allow yet untrusted code to execute. Ever.
> It basically means that from security perspective current algorithm:
> - deserialize an object
> - ask it for secure proxy
> - ask the secure proxy for a verifier
> - verify the original object
> is just simply wrong :)
> What's more - if we only allow trusted code to execute then the whole
> proxy verification problem becomes much simpler - all those complicated
> UntrustedObjectSecurityContext tricks are no longer necessary - it is
> enough to just ask an object to verify itself (placing apropriate
> invocation constraints on it first).
>
> The River-436 patch is going to (mostly) solve 2. The only potential
> holes left are in unicast response parsing where ObjectInputStream is
> created to read MarshalledInstance from the network - opening
> possibility of DOS or other attacks exploiting local code
> vulnerabilities. It is also going to open a possibility to solve 1
> because it is going to make class loading algorithms extensible/dynamic.
>
> The big question is whether it is possible to fix 1 without imposing any
> "container" or "class loading scheme" on the client/service implementor.
>
> OTOH - while we do not provide any "container" today - we do provide a
> "platform". There is no way any client can work without having jsk-
> platform.jar in its classpath and without specifying
> PreferredClassProvider as RMIClassProvider and configuring httpmd url
> handler.
>
> Regards,
>
> On Tuesday, March 04, 2014 08:03:13 PM Gregg Wonderly wrote:
> > One of the greatest things about Java is serialization and mobile
> > code! One of the most limiting aspects of any language is
> > Serialization!
> >
> > If you have an interface or data class that two classes need to access,
> > there is no choice but to have a common parent class loader. Your
> > client can institute such as class loading scheme completely
> > independently of Jini’s use of some other class loading scheme,
> > provided that you at least allow the “parent load this please”
> > mechanism of hierarchical class loading to occur.
> >
> > My changes to remove the explicit reliance/dependence on
> > RMIClassLoader in River-336, and instead allow you to plug in how the
> > “parent load this please” call out to work, is how you can solve this
> > so that it actually works for your clients special needs. The best
> > thing is that this mechanism can be plugged into, at runtime, whereas
> > RMIClassLoaderSPI is a onetime thing that requires access to the app
> > class loader, which might not be possible in some clients. All that
> > is required is a security grant that allows a particular codebase to
> > plugin a specific class name.
> >
> > Gregg Wonderly
> >
> > On Mar 4, 2014, at 7:39 PM, Gregg Wonderly <ge...@cox.net> wrote:
> > > > On Mar 4, 2014, at 12:02 AM, Michał Kłeczek
> > > > <mi...@xpro.biz> wrote:
> > > >
> > > > The real problem is that Util interface is in two codebases. It
> > > > should be in a single codebase shared between UtilProxy and
> > > > WrapperProxy. But to make it possible we would need to have peer
> > > > class loading like in ClassWorlds or OSGI.
> > > > It is not solvable in a standard hierarchical class loading scheme.
> > >
> > > This is one of the good examples of where hierarchical loading can
> > > present challenges.
> > >
> > > But the question really is, can an arbitrary client really expect for
> > > arbitrary services to interact correctly? If you want them to do
> > > this, it has been shown over and over that global types are the
> > > best, least troublesome choice.
> > >
> > > If you want ubiquitous interactions why not use string based values
> > > such as XML or better yet, JSON?
> > >
> > > Then code and data is immune to class loading snafus and not bound
> > > to a container or hosting "standard"!
> > >
> > > Gregg
> > >
> > > > Anyway... It is not really River-436 problem so my patch proposal
> > > > is going
> > > > to have the same issue since it is just a replacement for String
> > > > annotations and not change in class loading scheme.
> > > >
> > > > Thanks,
> > > > Michal
> > > >
> > > > 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz>
> > > > napisał(a):
> > > > > 1. The problem is there is no such thing as "the service
> > > > > interface". It is
> > > > > context dependent. What is "the service interface" for service
> > > > > browser?
> > > > >
> > > > > 2. In this particular case Util interface is an implementation
> > > > > detail of WrapperProxy. It is Wrapper interface the client is
> > > > > interested in. So I would say it should be preferred in
> > > > > WrapperProxy codebase.
> > > > >
> > > > > 3. Even if Util is not preferred in WrapperProxy codebase we
> > > > > still have ClassCastException if the client does not have Util
> > > > > in its classpath. Why
> > > > > should it? it is interested in Wrapper not in Util. So either
> > > > > a. We always get ClassCastException if Util is preferred in
> > > > > WrapperProxy codebase, or
> > > > > b. We get ClassCastException anyway if a client does not have
> > > > > Util in its
> > > > > classpath.
> > > > > Let's say I want to register RemoteEventListener that wraps a
> > > > > Javaspace proxy to write events in a space. Does that mean the
> > > > > service "event source"
> > > > > has to be aware of Javaspace interface??? That would be absurd...
> > > > >
> > > > > It all does not have anything to do with codebase services.
> > > > >
> > > > > Thanks,
> > > > > Michal
> > > > >
> > > > > 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> > > > > > The Util interface should not be preferred. Implementations
> > > > > > of Util can
> > > > > > be preferred but not Util itself.
> > > > > >
> > > > > > Services need a common api that all implementations and
> > > > > > clients can use to interract, even if this is a kind of
> > > > > > codebase service.
> > > > > >
> > > > > > Modifying an interface is generally considered bad practise
> > > > > > but now Java
> > > > > > 8 makes it possible to add default methods for added
> > > > > > functionality, that
> > > > > > line blurs somewhat. What can you do if an earlier interface
> > > > > > is loaded by
> > > > > > a parent ClassLoader and you need a later version, make it
> > > > > > preferred?
> > > > > >
> > > > > > My thoughts are that interfaces should never be preferred and
> > > > > > all classes
> > > > > > defined in their methods shouldn't be preferred either.
> > > > > >
> > > > > > It would be relatively easy to write a new implementation that
> > > > > > ensures that interfaces are loaded into their own
> > > > > > ProtectionDomain in a parent ClassLoader. But that would be
> > > > > > confusing as dynamic policy grants are made
> > > > > > to ClassLoader's not ProtectionDomains.
> > > > > >
> > > > > > But using ProtectionDomains in this manner, preserves
> > > > > > security, ensures maximum visibility and avoids codebase
> > > > > > annotation loss, if we ask the ProtectionDomain for the
> > > > > > annotation, instead of the ClassLoader. But this
> > > > > > is not how we do things presently.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Peter.
> > > > > >
> > > > > > ----- Original message -----
> > > > > >
> > > > > > > But it will also be loaded by WrapperProxy ClassLoader,
> > > > > > > since it is preferred there. So it will end up with
> > > > > > > ClassCastException, right?
> > > > > > >
> > > > > > > Regards,
> > > > > > > Michal
> > > > > > >
> > > > > > > If Util is installed locally, it will only be loaded by the
> > > > > > > application
> > > > > > > ClassLoader, since it isn't preferred.
> > > > > > >
> > > > > > > Peter.
> > > > > > >
> > > > > > > ----- Original message -----
> > > > > > >
> > > > > > > > Folks,
> > > > > > > > while woking on the River-436 patch proposal I've came
> > > > > > > > across the scenario that I am not sure how to handle:
> > > > > > > >
> > > > > > > > Utility service:
> > > > > > > > //inteface is NOT preferred
> > > > > > > > interface Util {...}
> > > > > > > > //class IS preferred
> > > > > > > > class UtilProxy implements Util {}
> > > > > > > >
> > > > > > > > Wrapper service:
> > > > > > > > //NOT preferred
> > > > > > > > interface Wrapper {}
> > > > > > > > //preferred
> > > > > > > > class WrapperProxy implements Serializable{
> > > > > > > > //initialized with Util impl from a lookup service
> > > > > > > > private Util util;
> > > > > > > > }
> > > > > > > >
> > > > > > > > Wrapper service codebase includes Util interface but it is
> > > > > >
> > > > > > _preferred_.
> > > > > >
> > > > > > > > Would deserialization of WrapperProxy end with
> > > > > > > > ClassCastException? From what I understand UtilProxy is
> > > > > > > > annotated with its codebase. When deserializing UtilProxy
> > > > > > > > a ClassLoader is going to be created with parent set to
> > > > > > > > TCCL. It means Util interface is going to be loaded twice
> > > > > > > > by two ClassLoaders - one for WrapperProxy codebase and
> > > > > > > > another for UtilProxy codebase.
> > > > > > > >
> > > > > > > > Am I correct?
> > > > > > > > And if so: is it desired behavior?
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > --
> > > > > > > > Michał Kłeczek
> > > > > > > > XPro Quality Matters
> > > > > > > > http://www.xpro.biz
>
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
<Attachment> Michał Kłeczek (XPro).vcf
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Greg,
I cannot agree more with the statement that mobile code is the cornerstone of
River. I do not want to send just "data" (XML/JSON/whatever). I want to
exchange _objects_ (data + code - well... in the end it is all data :-) ). And
I want it to be _secure_ and ubiquitous.
IMHO we should just make it work - because it simply does not work at this
moment - as shown in my example.
What I'm arguing is that it is not fixable with hierarchical class loading. The
problem is that the client simply does not know what should be the "parent"
when deserializing objects. Your River-336 patch is not going to change it
(while still important since RMIClassLoader dependency is really a PITA).
I am not proposing any particular solution at this moment - BUT I would really
like the community to have a second look at existing solutions before coming
up with yet another one.
The requirements - as I see them right now would be:
1. Make it possible to exchange arbitrary object graphs composed of objects
which classes are from _different_ codebases.
That would make it possible to create smart proxies wrapping other services'
smart proxies (which in theory is possible today... but in practice is not)
2. Do not allow yet untrusted code to execute. Ever.
It basically means that from security perspective current algorithm:
- deserialize an object
- ask it for secure proxy
- ask the secure proxy for a verifier
- verify the original object
is just simply wrong :)
What's more - if we only allow trusted code to execute then the whole proxy
verification problem becomes much simpler - all those complicated
UntrustedObjectSecurityContext tricks are no longer necessary - it is enough
to just ask an object to verify itself (placing apropriate invocation
constraints on it first).
The River-436 patch is going to (mostly) solve 2. The only potential holes
left are in unicast response parsing where ObjectInputStream is created to
read MarshalledInstance from the network - opening possibility of DOS or other
attacks exploiting local code vulnerabilities.
It is also going to open a possibility to solve 1 because it is going to make
class loading algorithms extensible/dynamic.
The big question is whether it is possible to fix 1 without imposing any
"container" or "class loading scheme" on the client/service implementor.
OTOH - while we do not provide any "container" today - we do provide a
"platform". There is no way any client can work without having jsk-
platform.jar in its classpath and without specifying PreferredClassProvider as
RMIClassProvider and configuring httpmd url handler.
Regards,
On Tuesday, March 04, 2014 08:03:13 PM Gregg Wonderly wrote:
> One of the greatest things about Java is serialization and mobile code! One
> of the most limiting aspects of any language is Serialization!
>
> If you have an interface or data class that two classes need to access,
> there is no choice but to have a common parent class loader. Your client
> can institute such as class loading scheme completely independently of
> Jini’s use of some other class loading scheme, provided that you at least
> allow the “parent load this please” mechanism of hierarchical class loading
> to occur.
>
> My changes to remove the explicit reliance/dependence on RMIClassLoader in
> River-336, and instead allow you to plug in how the “parent load this
> please” call out to work, is how you can solve this so that it actually
> works for your clients special needs. The best thing is that this
> mechanism can be plugged into, at runtime, whereas RMIClassLoaderSPI is a
> onetime thing that requires access to the app class loader, which might not
> be possible in some clients. All that is required is a security grant that
> allows a particular codebase to plugin a specific class name.
>
> Gregg Wonderly
>
> On Mar 4, 2014, at 7:39 PM, Gregg Wonderly <ge...@cox.net> wrote:
> >> On Mar 4, 2014, at 12:02 AM, Michał Kłeczek <mi...@xpro.biz>
> >> wrote:
> >>
> >> The real problem is that Util interface is in two codebases. It should be
> >> in a single codebase shared between UtilProxy and WrapperProxy.
> >> But to make it possible we would need to have peer class loading like in
> >> ClassWorlds or OSGI.
> >> It is not solvable in a standard hierarchical class loading scheme.
> >
> > This is one of the good examples of where hierarchical loading can present
> > challenges.
> >
> > But the question really is, can an arbitrary client really expect for
> > arbitrary services to interact correctly? If you want them to do this,
> > it has been shown over and over that global types are the best, least
> > troublesome choice.
> >
> > If you want ubiquitous interactions why not use string based values such
> > as XML or better yet, JSON?
> >
> > Then code and data is immune to class loading snafus and not bound to a
> > container or hosting "standard"!
> >
> > Gregg
> >
> >> Anyway... It is not really River-436 problem so my patch proposal is
> >> going
> >> to have the same issue since it is just a replacement for String
> >> annotations and not change in class loading scheme.
> >>
> >> Thanks,
> >> Michal
> >>
> >> 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz> napisał(a):
> >>> 1. The problem is there is no such thing as "the service interface". It
> >>> is
> >>> context dependent. What is "the service interface" for service browser?
> >>>
> >>> 2. In this particular case Util interface is an implementation detail of
> >>> WrapperProxy. It is Wrapper interface the client is interested in. So I
> >>> would say it should be preferred in WrapperProxy codebase.
> >>>
> >>> 3. Even if Util is not preferred in WrapperProxy codebase we still have
> >>> ClassCastException if the client does not have Util in its classpath.
> >>> Why
> >>> should it? it is interested in Wrapper not in Util. So either
> >>> a. We always get ClassCastException if Util is preferred in WrapperProxy
> >>> codebase, or
> >>> b. We get ClassCastException anyway if a client does not have Util in
> >>> its
> >>> classpath.
> >>> Let's say I want to register RemoteEventListener that wraps a Javaspace
> >>> proxy to write events in a space. Does that mean the service "event
> >>> source"
> >>> has to be aware of Javaspace interface??? That would be absurd...
> >>>
> >>> It all does not have anything to do with codebase services.
> >>>
> >>> Thanks,
> >>> Michal
> >>>
> >>> 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> >>>> The Util interface should not be preferred. Implementations of Util
> >>>> can
> >>>> be preferred but not Util itself.
> >>>>
> >>>> Services need a common api that all implementations and clients can use
> >>>> to interract, even if this is a kind of codebase service.
> >>>>
> >>>> Modifying an interface is generally considered bad practise but now
> >>>> Java
> >>>> 8 makes it possible to add default methods for added functionality,
> >>>> that
> >>>> line blurs somewhat. What can you do if an earlier interface is loaded
> >>>> by
> >>>> a parent ClassLoader and you need a later version, make it preferred?
> >>>>
> >>>> My thoughts are that interfaces should never be preferred and all
> >>>> classes
> >>>> defined in their methods shouldn't be preferred either.
> >>>>
> >>>> It would be relatively easy to write a new implementation that ensures
> >>>> that interfaces are loaded into their own ProtectionDomain in a parent
> >>>> ClassLoader. But that would be confusing as dynamic policy grants are
> >>>> made
> >>>> to ClassLoader's not ProtectionDomains.
> >>>>
> >>>> But using ProtectionDomains in this manner, preserves security, ensures
> >>>> maximum visibility and avoids codebase annotation loss, if we ask the
> >>>> ProtectionDomain for the annotation, instead of the ClassLoader. But
> >>>> this
> >>>> is not how we do things presently.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Peter.
> >>>>
> >>>> ----- Original message -----
> >>>>
> >>>>> But it will also be loaded by WrapperProxy ClassLoader, since it is
> >>>>> preferred there. So it will end up with ClassCastException, right?
> >>>>>
> >>>>> Regards,
> >>>>> Michal
> >>>>>
> >>>>> If Util is installed locally, it will only be loaded by the
> >>>>> application
> >>>>> ClassLoader, since it isn't preferred.
> >>>>>
> >>>>> Peter.
> >>>>>
> >>>>> ----- Original message -----
> >>>>>
> >>>>>> Folks,
> >>>>>> while woking on the River-436 patch proposal I've came across the
> >>>>>> scenario that I am not sure how to handle:
> >>>>>>
> >>>>>> Utility service:
> >>>>>> //inteface is NOT preferred
> >>>>>> interface Util {...}
> >>>>>> //class IS preferred
> >>>>>> class UtilProxy implements Util {}
> >>>>>>
> >>>>>> Wrapper service:
> >>>>>> //NOT preferred
> >>>>>> interface Wrapper {}
> >>>>>> //preferred
> >>>>>> class WrapperProxy implements Serializable{
> >>>>>> //initialized with Util impl from a lookup service
> >>>>>> private Util util;
> >>>>>> }
> >>>>>>
> >>>>>> Wrapper service codebase includes Util interface but it is
> >>>>
> >>>> _preferred_.
> >>>>
> >>>>>> Would deserialization of WrapperProxy end with ClassCastException?
> >>>>>> From what I understand UtilProxy is annotated with its codebase. When
> >>>>>> deserializing UtilProxy a ClassLoader is going to be created with
> >>>>>> parent set to TCCL. It means Util interface is going to be loaded
> >>>>>> twice by two ClassLoaders - one for WrapperProxy codebase and another
> >>>>>> for UtilProxy codebase.
> >>>>>>
> >>>>>> Am I correct?
> >>>>>> And if so: is it desired behavior?
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> --
> >>>>>> Michał Kłeczek
> >>>>>> XPro Quality Matters
> >>>>>> http://www.xpro.biz
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <ge...@cox.net>.
In Jini/River, there are two ways that any class is resolved. First, it is resolved because a class under construction has a reference to it. Second is that it is resolved by a client class that needs it to specify an interface/class to use for service discovery/usage.
In the first case, we don’t care where it is resolved, if the second case doesn’t occur. For the first case, every service JAR file should carry the definition of every single class that it depends on, marking those that will never be encountered for the Second case as preferred.
This is just a simple fact of how mobile code works. If the “jar” that the service uses to resolve classes for the first case can be the same jar as the client uses to resolve the classes in the second case (OSGI, Maven and other non-mobile code, jar distribution mechanisms), then you can have one class loader having view of the class.
The Preferred class loader will do the right thing, automatically as long as you create the correct preferred list. If you don’t do that correctly, then you can encounter problems. Without a preferred list, the PreferredClassLoader is going to always look in the parent class loader, and that is usually the right thing to do. Most people encounter problems when the list something as preferred which is then passed around to another “proxy” that has another instance of the same class that is not resolved to the first class loader and then you see class cast exception.
The preferred list should only really ever contain the names of classes that are not publicly visible in the API or any reference those public API classes have.
The basic reason to not use Maven or OSGi or other static class resolution mechanisms is that it provides one the flexibility to have a much more dynamically evolving runtime environment including test scenarios where it doesn’t make sense to “publish” a jar file that others may then have access to.
Gregg Wonderly
On Mar 4, 2014, at 12:02 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
> The real problem is that Util interface is in two codebases. It should be
> in a single codebase shared between UtilProxy and WrapperProxy.
> But to make it possible we would need to have peer class loading like in
> ClassWorlds or OSGI.
> It is not solvable in a standard hierarchical class loading scheme.
>
> Anyway... It is not really River-436 problem so my patch proposal is going
> to have the same issue since it is just a replacement for String
> annotations and not change in class loading scheme.
>
> Thanks,
> Michal
> 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz> napisał(a):
>
>> 1. The problem is there is no such thing as "the service interface". It is
>> context dependent. What is "the service interface" for service browser?
>>
>> 2. In this particular case Util interface is an implementation detail of
>> WrapperProxy. It is Wrapper interface the client is interested in. So I
>> would say it should be preferred in WrapperProxy codebase.
>>
>> 3. Even if Util is not preferred in WrapperProxy codebase we still have
>> ClassCastException if the client does not have Util in its classpath. Why
>> should it? it is interested in Wrapper not in Util. So either
>> a. We always get ClassCastException if Util is preferred in WrapperProxy
>> codebase, or
>> b. We get ClassCastException anyway if a client does not have Util in its
>> classpath.
>> Let's say I want to register RemoteEventListener that wraps a Javaspace
>> proxy to write events in a space. Does that mean the service "event source"
>> has to be aware of Javaspace interface??? That would be absurd...
>>
>> It all does not have anything to do with codebase services.
>>
>> Thanks,
>> Michal
>> 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
>>
>>> The Util interface should not be preferred. Implementations of Util can
>>> be preferred but not Util itself.
>>>
>>> Services need a common api that all implementations and clients can use
>>> to interract, even if this is a kind of codebase service.
>>>
>>> Modifying an interface is generally considered bad practise but now Java
>>> 8 makes it possible to add default methods for added functionality, that
>>> line blurs somewhat. What can you do if an earlier interface is loaded by
>>> a parent ClassLoader and you need a later version, make it preferred?
>>>
>>> My thoughts are that interfaces should never be preferred and all classes
>>> defined in their methods shouldn't be preferred either.
>>>
>>> It would be relatively easy to write a new implementation that ensures
>>> that interfaces are loaded into their own ProtectionDomain in a parent
>>> ClassLoader. But that would be confusing as dynamic policy grants are made
>>> to ClassLoader's not ProtectionDomains.
>>>
>>> But using ProtectionDomains in this manner, preserves security, ensures
>>> maximum visibility and avoids codebase annotation loss, if we ask the
>>> ProtectionDomain for the annotation, instead of the ClassLoader. But this
>>> is not how we do things presently.
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>> ----- Original message -----
>>>> But it will also be loaded by WrapperProxy ClassLoader, since it is
>>>> preferred there. So it will end up with ClassCastException, right?
>>>>
>>>> Regards,
>>>> Michal
>>>>
>>>> If Util is installed locally, it will only be loaded by the application
>>>> ClassLoader, since it isn't preferred.
>>>>
>>>> Peter.
>>>>
>>>> ----- Original message -----
>>>>> Folks,
>>>>> while woking on the River-436 patch proposal I've came across the
>>>>> scenario that I am not sure how to handle:
>>>>>
>>>>> Utility service:
>>>>> //inteface is NOT preferred
>>>>> interface Util {...}
>>>>> //class IS preferred
>>>>> class UtilProxy implements Util {}
>>>>>
>>>>> Wrapper service:
>>>>> //NOT preferred
>>>>> interface Wrapper {}
>>>>> //preferred
>>>>> class WrapperProxy implements Serializable{
>>>>> //initialized with Util impl from a lookup service
>>>>> private Util util;
>>>>> }
>>>>>
>>>>> Wrapper service codebase includes Util interface but it is
>>> _preferred_.
>>>>>
>>>>> Would deserialization of WrapperProxy end with ClassCastException?
>>>>> From what I understand UtilProxy is annotated with its codebase. When
>>>>> deserializing UtilProxy a ClassLoader is going to be created with
>>>>> parent set to TCCL. It means Util interface is going to be loaded
>>>>> twice by two ClassLoaders - one for WrapperProxy codebase and another
>>>>> for UtilProxy codebase.
>>>>>
>>>>> Am I correct?
>>>>> And if so: is it desired behavior?
>>>>>
>>>>> Regards,
>>>>>
>>>>> --
>>>>> Michał Kłeczek
>>>>> XPro Quality Matters
>>>>> http://www.xpro.biz
>>>>>
>>>
>>>
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <gr...@wonderly.org>.
> On Mar 4, 2014, at 11:18 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
>
> ClassLoader complexity and dependency management is something that was
> understood long time ago and solved several times (OSGI or NetBeans module
> system come to mind first).
First thing to understand is that these two along with others each have a specific focused problem that they are addressing.
Netbeans is about isolation, not sharing.
OSGi is about versioning and isolation based on versioning.
OSGi supporters, proponents and developers have fought a long to try and create a remoting specification which worked with what they already had.
> To be honest I really do not understand why River community does not make use
> of existing solutions and tries to either reinvent the wheel or live with
> broken architecture and develop ad-hoc fixes not really addressing the root
> causes.
>
> It may sound harsh but for me it looks like NIH syndrome.
It's certainly possible to feel like that's the way it is, but remember that RMI and JRMP existed before these did.
Is it a great and perfect thing? By no means would I say that.
But, there are good things amongst all if these and rather then mash together all of it into a new thing, why can't we just provide pluggable interfaces and implementations that let them all coexist?
There are, in my mind, some pretty powerful abstractions that should make it possible.
Gregg
>
> We are discussing service packaging and container implementation while we have
> _basic_ stuff not working:
> a) we have a huge security issue with allowing untrusted code to execute
> b) we have a class loading issue that makes it impossible to use River for
> simple service composition - as I've shown in my example.
> c) we have a whole lot of concurrency issues in the implementation from which
> a lot were fixed by Peter and it is still not decided whether it is worth
> incorporating those fixes!!!.
>
> On the other hand I hear voices that we should drop/deprecate some of River
> components:
> a) Phoenix
> IMHO _any_ River service container should be _based_ on Phoenix. It has
> capabilities not available in most other containers (JEE containers
> specifically):
> - ability to deploy/execute components in _isolation_ (different virtual
> machines - instantiated on demand)
> - watchdog functionality
> - easily accessible API (since Phoenix _is_ a River service)
> - on-demand component activation (which I think is underestimated - see below)
>
> b) Norm/Mercury/Fiddler
> These are crucial for creating activatable sevices.
> Activation is important in service composition scenarios where a service is
> actually implemented only as a client proxy wrapping other services (no server
> side logic). I would even say we are missing some (simple but important)
> services to make it possible:
> - CodeRepository service (I remember one implementation created some time ago)
> - ProxyTrustDelegate service (that would allow a service to delegate proxy
> verification logic to another service)
>
> Just from the top of my head... :-)
>
> Regards,
>
>> On Tuesday, March 04, 2014 09:17:12 PM Peter wrote:
>> Thanks Michal,
>>
>> Welcome to ClassLoader complexity.
>>
>> We've more recently encouraged the separation of Service API, from
>> implementation at the development stage, instead of relying on tools like
>> classdep. Rio, uses these conventions.
>>
>> This is an important first step.
>>
>> Basically Service API, are interfaces and classes that implementations use
>> to communicate with each other. In this case, because your Util interface
>> needs to be shared, it correlates to service api.
>>
>> If all application code was loaded into URLClassLoader instances as
>> suggested previously some time ago by Nic on this list, then we could
>> ensure that all Service API is loaded into it's own ProtectionDomain in the
>> main application ClassLoader (a URLClassLoader instance that proxy's use as
>> their parent loader)
>>
>> To do so however requires new conventions for codebase annotations.
>>
>> One restriction is that service api cannot be changed after deployment.
>>
>> We could allow Service API to be loaded on demand after deployment, if it
>> doesn't already exist at the client, but again it cannot be changed after
>> deployment, only added to.
>>
>> Cheers,
>>
>> Peter.
>
> --
> Michał Kłeczek
> XPro Sp. z o. o.
> ul. Borowskiego 2
> 03-475 Warszawa
> Polska
> <Michał Kłeczek (XPro).vcf>
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
Peter,
I'm going to make some more general comments here.
Actually my question was about making sure I understand how
PreferredClassLoader works and that there is no "magic" somewhere that I am
not aware of.
ClassLoader complexity and dependency management is something that was
understood long time ago and solved several times (OSGI or NetBeans module
system come to mind first).
To be honest I really do not understand why River community does not make use
of existing solutions and tries to either reinvent the wheel or live with
broken architecture and develop ad-hoc fixes not really addressing the root
causes.
It may sound harsh but for me it looks like NIH syndrome.
We are discussing service packaging and container implementation while we have
_basic_ stuff not working:
a) we have a huge security issue with allowing untrusted code to execute
b) we have a class loading issue that makes it impossible to use River for
simple service composition - as I've shown in my example.
c) we have a whole lot of concurrency issues in the implementation from which
a lot were fixed by Peter and it is still not decided whether it is worth
incorporating those fixes!!!.
On the other hand I hear voices that we should drop/deprecate some of River
components:
a) Phoenix
IMHO _any_ River service container should be _based_ on Phoenix. It has
capabilities not available in most other containers (JEE containers
specifically):
- ability to deploy/execute components in _isolation_ (different virtual
machines - instantiated on demand)
- watchdog functionality
- easily accessible API (since Phoenix _is_ a River service)
- on-demand component activation (which I think is underestimated - see below)
b) Norm/Mercury/Fiddler
These are crucial for creating activatable sevices.
Activation is important in service composition scenarios where a service is
actually implemented only as a client proxy wrapping other services (no server
side logic). I would even say we are missing some (simple but important)
services to make it possible:
- CodeRepository service (I remember one implementation created some time ago)
- ProxyTrustDelegate service (that would allow a service to delegate proxy
verification logic to another service)
Just from the top of my head... :-)
Regards,
On Tuesday, March 04, 2014 09:17:12 PM Peter wrote:
> Thanks Michal,
>
> Welcome to ClassLoader complexity.
>
> We've more recently encouraged the separation of Service API, from
> implementation at the development stage, instead of relying on tools like
> classdep. Rio, uses these conventions.
>
> This is an important first step.
>
> Basically Service API, are interfaces and classes that implementations use
> to communicate with each other. In this case, because your Util interface
> needs to be shared, it correlates to service api.
>
> If all application code was loaded into URLClassLoader instances as
> suggested previously some time ago by Nic on this list, then we could
> ensure that all Service API is loaded into it's own ProtectionDomain in the
> main application ClassLoader (a URLClassLoader instance that proxy's use as
> their parent loader)
>
> To do so however requires new conventions for codebase annotations.
>
> One restriction is that service api cannot be changed after deployment.
>
> We could allow Service API to be loaded on demand after deployment, if it
> doesn't already exist at the client, but again it cannot be changed after
> deployment, only added to.
>
> Cheers,
>
> Peter.
>
>
>
>
--
Michał Kłeczek
XPro Sp. z o. o.
ul. Borowskiego 2
03-475 Warszawa
Polska
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
Thanks Michal,
Welcome to ClassLoader complexity.
We've more recently encouraged the separation of Service API, from implementation at the development stage, instead of relying on tools like classdep. Rio, uses these conventions.
This is an important first step.
Basically Service API, are interfaces and classes that implementations use to communicate with each other. In this case, because your Util interface needs to be shared, it correlates to service api.
If all application code was loaded into URLClassLoader instances as suggested previously some time ago by Nic on this list, then we could ensure that all Service API is loaded into it's own ProtectionDomain in the main application ClassLoader (a URLClassLoader instance that proxy's use as their parent loader)
To do so however requires new conventions for codebase annotations.
One restriction is that service api cannot be changed after deployment.
We could allow Service API to be loaded on demand after deployment, if it doesn't already exist at the client, but again it cannot be changed after deployment, only added to.
Cheers,
Peter.
----- Original message -----
> The real problem is that Util interface is in two codebases. It should be
> in a single codebase shared between UtilProxy and WrapperProxy.
> But to make it possible we would need to have peer class loading like in
> ClassWorlds or OSGI.
> It is not solvable in a standard hierarchical class loading scheme.
>
> Anyway... It is not really River-436 problem so my patch proposal is
> going to have the same issue since it is just a replacement for String
> annotations and not change in class loading scheme.
>
> Thanks,
> Michal
> 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz> napisał(a):
>
> > 1. The problem is there is no such thing as "the service interface".
> > It is context dependent. What is "the service interface" for service
> > browser?
> >
> > 2. In this particular case Util interface is an implementation detail
> > of WrapperProxy. It is Wrapper interface the client is interested in.
> > So I would say it should be preferred in WrapperProxy codebase.
> >
> > 3. Even if Util is not preferred in WrapperProxy codebase we still have
> > ClassCastException if the client does not have Util in its classpath.
> > Why should it? it is interested in Wrapper not in Util. So either
> > a. We always get ClassCastException if Util is preferred in
> > WrapperProxy codebase, or
> > b. We get ClassCastException anyway if a client does not have Util in
> > its classpath.
> > Let's say I want to register RemoteEventListener that wraps a Javaspace
> > proxy to write events in a space. Does that mean the service "event
> > source" has to be aware of Javaspace interface??? That would be
> > absurd...
> >
> > It all does not have anything to do with codebase services.
> >
> > Thanks,
> > Michal
> > 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> >
> > > The Util interface should not be preferred. Implementations of Util
> > > can be preferred but not Util itself.
> > >
> > > Services need a common api that all implementations and clients can
> > > use to interract, even if this is a kind of codebase service.
> > >
> > > Modifying an interface is generally considered bad practise but now
> > > Java 8 makes it possible to add default methods for added
> > > functionality, that line blurs somewhat. What can you do if an
> > > earlier interface is loaded by a parent ClassLoader and you need a
> > > later version, make it preferred?
> > >
> > > My thoughts are that interfaces should never be preferred and all
> > > classes defined in their methods shouldn't be preferred either.
> > >
> > > It would be relatively easy to write a new implementation that
> > > ensures that interfaces are loaded into their own ProtectionDomain
> > > in a parent ClassLoader. But that would be confusing as dynamic
> > > policy grants are made to ClassLoader's not ProtectionDomains.
> > >
> > > But using ProtectionDomains in this manner, preserves security,
> > > ensures maximum visibility and avoids codebase annotation loss, if
> > > we ask the ProtectionDomain for the annotation, instead of the
> > > ClassLoader. But this is not how we do things presently.
> > >
> > > Cheers,
> > >
> > > Peter.
> > >
> > > ----- Original message -----
> > > > But it will also be loaded by WrapperProxy ClassLoader, since it is
> > > > preferred there. So it will end up with ClassCastException, right?
> > > >
> > > > Regards,
> > > > Michal
> > > >
> > > > If Util is installed locally, it will only be loaded by the
> > > > application ClassLoader, since it isn't preferred.
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > > > Folks,
> > > > > while woking on the River-436 patch proposal I've came across the
> > > > > scenario that I am not sure how to handle:
> > > > >
> > > > > Utility service:
> > > > > //inteface is NOT preferred
> > > > > interface Util {...}
> > > > > //class IS preferred
> > > > > class UtilProxy implements Util {}
> > > > >
> > > > > Wrapper service:
> > > > > //NOT preferred
> > > > > interface Wrapper {}
> > > > > //preferred
> > > > > class WrapperProxy implements Serializable{
> > > > > //initialized with Util impl from a lookup service
> > > > > private Util util;
> > > > > }
> > > > >
> > > > > Wrapper service codebase includes Util interface but it is
> > > _preferred_.
> > > > >
> > > > > Would deserialization of WrapperProxy end with
> > > > > ClassCastException? From what I understand UtilProxy is
> > > > > annotated with its codebase. When deserializing UtilProxy a
> > > > > ClassLoader is going to be created with parent set to TCCL. It
> > > > > means Util interface is going to be loaded twice by two
> > > > > ClassLoaders - one for WrapperProxy codebase and another for
> > > > > UtilProxy codebase.
> > > > >
> > > > > Am I correct?
> > > > > And if so: is it desired behavior?
> > > > >
> > > > > Regards,
> > > > >
> > > > > --
> > > > > Michał Kłeczek
> > > > > XPro Quality Matters
> > > > > http://www.xpro.biz
> > > > >
> > >
> > >
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
The real problem is that Util interface is in two codebases. It should be
in a single codebase shared between UtilProxy and WrapperProxy.
But to make it possible we would need to have peer class loading like in
ClassWorlds or OSGI.
It is not solvable in a standard hierarchical class loading scheme.
Anyway... It is not really River-436 problem so my patch proposal is going
to have the same issue since it is just a replacement for String
annotations and not change in class loading scheme.
Thanks,
Michal
4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz> napisał(a):
> 1. The problem is there is no such thing as "the service interface". It is
> context dependent. What is "the service interface" for service browser?
>
> 2. In this particular case Util interface is an implementation detail of
> WrapperProxy. It is Wrapper interface the client is interested in. So I
> would say it should be preferred in WrapperProxy codebase.
>
> 3. Even if Util is not preferred in WrapperProxy codebase we still have
> ClassCastException if the client does not have Util in its classpath. Why
> should it? it is interested in Wrapper not in Util. So either
> a. We always get ClassCastException if Util is preferred in WrapperProxy
> codebase, or
> b. We get ClassCastException anyway if a client does not have Util in its
> classpath.
> Let's say I want to register RemoteEventListener that wraps a Javaspace
> proxy to write events in a space. Does that mean the service "event source"
> has to be aware of Javaspace interface??? That would be absurd...
>
> It all does not have anything to do with codebase services.
>
> Thanks,
> Michal
> 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
>
>> The Util interface should not be preferred. Implementations of Util can
>> be preferred but not Util itself.
>>
>> Services need a common api that all implementations and clients can use
>> to interract, even if this is a kind of codebase service.
>>
>> Modifying an interface is generally considered bad practise but now Java
>> 8 makes it possible to add default methods for added functionality, that
>> line blurs somewhat. What can you do if an earlier interface is loaded by
>> a parent ClassLoader and you need a later version, make it preferred?
>>
>> My thoughts are that interfaces should never be preferred and all classes
>> defined in their methods shouldn't be preferred either.
>>
>> It would be relatively easy to write a new implementation that ensures
>> that interfaces are loaded into their own ProtectionDomain in a parent
>> ClassLoader. But that would be confusing as dynamic policy grants are made
>> to ClassLoader's not ProtectionDomains.
>>
>> But using ProtectionDomains in this manner, preserves security, ensures
>> maximum visibility and avoids codebase annotation loss, if we ask the
>> ProtectionDomain for the annotation, instead of the ClassLoader. But this
>> is not how we do things presently.
>>
>> Cheers,
>>
>> Peter.
>>
>> ----- Original message -----
>> > But it will also be loaded by WrapperProxy ClassLoader, since it is
>> > preferred there. So it will end up with ClassCastException, right?
>> >
>> > Regards,
>> > Michal
>> >
>> > If Util is installed locally, it will only be loaded by the application
>> > ClassLoader, since it isn't preferred.
>> >
>> > Peter.
>> >
>> > ----- Original message -----
>> > > Folks,
>> > > while woking on the River-436 patch proposal I've came across the
>> > > scenario that I am not sure how to handle:
>> > >
>> > > Utility service:
>> > > //inteface is NOT preferred
>> > > interface Util {...}
>> > > //class IS preferred
>> > > class UtilProxy implements Util {}
>> > >
>> > > Wrapper service:
>> > > //NOT preferred
>> > > interface Wrapper {}
>> > > //preferred
>> > > class WrapperProxy implements Serializable{
>> > > //initialized with Util impl from a lookup service
>> > > private Util util;
>> > > }
>> > >
>> > > Wrapper service codebase includes Util interface but it is
>> _preferred_.
>> > >
>> > > Would deserialization of WrapperProxy end with ClassCastException?
>> > > From what I understand UtilProxy is annotated with its codebase. When
>> > > deserializing UtilProxy a ClassLoader is going to be created with
>> > > parent set to TCCL. It means Util interface is going to be loaded
>> > > twice by two ClassLoaders - one for WrapperProxy codebase and another
>> > > for UtilProxy codebase.
>> > >
>> > > Am I correct?
>> > > And if so: is it desired behavior?
>> > >
>> > > Regards,
>> > >
>> > > --
>> > > Michał Kłeczek
>> > > XPro Quality Matters
>> > > http://www.xpro.biz
>> > >
>>
>>
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <ge...@cox.net>.
One of the greatest things about Java is serialization and mobile code! One of the most limiting aspects of any language is Serialization!
If you have an interface or data class that two classes need to access, there is no choice but to have a common parent class loader. Your client can institute such as class loading scheme completely independently of Jini’s use of some other class loading scheme, provided that you at least allow the “parent load this please” mechanism of hierarchical class loading to occur.
My changes to remove the explicit reliance/dependence on RMIClassLoader in River-336, and instead allow you to plug in how the “parent load this please” call out to work, is how you can solve this so that it actually works for your clients special needs. The best thing is that this mechanism can be plugged into, at runtime, whereas RMIClassLoaderSPI is a onetime thing that requires access to the app class loader, which might not be possible in some clients. All that is required is a security grant that allows a particular codebase to plugin a specific class name.
Gregg Wonderly
On Mar 4, 2014, at 7:39 PM, Gregg Wonderly <ge...@cox.net> wrote:
>
>
>> On Mar 4, 2014, at 12:02 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
>>
>> The real problem is that Util interface is in two codebases. It should be
>> in a single codebase shared between UtilProxy and WrapperProxy.
>> But to make it possible we would need to have peer class loading like in
>> ClassWorlds or OSGI.
>> It is not solvable in a standard hierarchical class loading scheme.
>
> This is one of the good examples of where hierarchical loading can present challenges.
>
> But the question really is, can an arbitrary client really expect for arbitrary services to interact correctly? If you want them to do this, it has been shown over and over that global types are the best, least troublesome choice.
>
> If you want ubiquitous interactions why not use string based values such as XML or better yet, JSON?
>
> Then code and data is immune to class loading snafus and not bound to a container or hosting "standard"!
>
> Gregg
>
>> Anyway... It is not really River-436 problem so my patch proposal is going
>> to have the same issue since it is just a replacement for String
>> annotations and not change in class loading scheme.
>>
>> Thanks,
>> Michal
>> 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz> napisał(a):
>>
>>> 1. The problem is there is no such thing as "the service interface". It is
>>> context dependent. What is "the service interface" for service browser?
>>>
>>> 2. In this particular case Util interface is an implementation detail of
>>> WrapperProxy. It is Wrapper interface the client is interested in. So I
>>> would say it should be preferred in WrapperProxy codebase.
>>>
>>> 3. Even if Util is not preferred in WrapperProxy codebase we still have
>>> ClassCastException if the client does not have Util in its classpath. Why
>>> should it? it is interested in Wrapper not in Util. So either
>>> a. We always get ClassCastException if Util is preferred in WrapperProxy
>>> codebase, or
>>> b. We get ClassCastException anyway if a client does not have Util in its
>>> classpath.
>>> Let's say I want to register RemoteEventListener that wraps a Javaspace
>>> proxy to write events in a space. Does that mean the service "event source"
>>> has to be aware of Javaspace interface??? That would be absurd...
>>>
>>> It all does not have anything to do with codebase services.
>>>
>>> Thanks,
>>> Michal
>>> 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
>>>
>>>> The Util interface should not be preferred. Implementations of Util can
>>>> be preferred but not Util itself.
>>>>
>>>> Services need a common api that all implementations and clients can use
>>>> to interract, even if this is a kind of codebase service.
>>>>
>>>> Modifying an interface is generally considered bad practise but now Java
>>>> 8 makes it possible to add default methods for added functionality, that
>>>> line blurs somewhat. What can you do if an earlier interface is loaded by
>>>> a parent ClassLoader and you need a later version, make it preferred?
>>>>
>>>> My thoughts are that interfaces should never be preferred and all classes
>>>> defined in their methods shouldn't be preferred either.
>>>>
>>>> It would be relatively easy to write a new implementation that ensures
>>>> that interfaces are loaded into their own ProtectionDomain in a parent
>>>> ClassLoader. But that would be confusing as dynamic policy grants are made
>>>> to ClassLoader's not ProtectionDomains.
>>>>
>>>> But using ProtectionDomains in this manner, preserves security, ensures
>>>> maximum visibility and avoids codebase annotation loss, if we ask the
>>>> ProtectionDomain for the annotation, instead of the ClassLoader. But this
>>>> is not how we do things presently.
>>>>
>>>> Cheers,
>>>>
>>>> Peter.
>>>>
>>>> ----- Original message -----
>>>>> But it will also be loaded by WrapperProxy ClassLoader, since it is
>>>>> preferred there. So it will end up with ClassCastException, right?
>>>>>
>>>>> Regards,
>>>>> Michal
>>>>>
>>>>> If Util is installed locally, it will only be loaded by the application
>>>>> ClassLoader, since it isn't preferred.
>>>>>
>>>>> Peter.
>>>>>
>>>>> ----- Original message -----
>>>>>> Folks,
>>>>>> while woking on the River-436 patch proposal I've came across the
>>>>>> scenario that I am not sure how to handle:
>>>>>>
>>>>>> Utility service:
>>>>>> //inteface is NOT preferred
>>>>>> interface Util {...}
>>>>>> //class IS preferred
>>>>>> class UtilProxy implements Util {}
>>>>>>
>>>>>> Wrapper service:
>>>>>> //NOT preferred
>>>>>> interface Wrapper {}
>>>>>> //preferred
>>>>>> class WrapperProxy implements Serializable{
>>>>>> //initialized with Util impl from a lookup service
>>>>>> private Util util;
>>>>>> }
>>>>>>
>>>>>> Wrapper service codebase includes Util interface but it is
>>>> _preferred_.
>>>>>>
>>>>>> Would deserialization of WrapperProxy end with ClassCastException?
>>>>>> From what I understand UtilProxy is annotated with its codebase. When
>>>>>> deserializing UtilProxy a ClassLoader is going to be created with
>>>>>> parent set to TCCL. It means Util interface is going to be loaded
>>>>>> twice by two ClassLoaders - one for WrapperProxy codebase and another
>>>>>> for UtilProxy codebase.
>>>>>>
>>>>>> Am I correct?
>>>>>> And if so: is it desired behavior?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> --
>>>>>> Michał Kłeczek
>>>>>> XPro Quality Matters
>>>>>> http://www.xpro.biz
>>>>
>>>>
Re: River-436 - need some explanation of preferred class provider
Posted by Gregg Wonderly <ge...@cox.net>.
> On Mar 4, 2014, at 12:02 AM, Michał Kłeczek <mi...@xpro.biz> wrote:
>
> The real problem is that Util interface is in two codebases. It should be
> in a single codebase shared between UtilProxy and WrapperProxy.
> But to make it possible we would need to have peer class loading like in
> ClassWorlds or OSGI.
> It is not solvable in a standard hierarchical class loading scheme.
This is one of the good examples of where hierarchical loading can present challenges.
But the question really is, can an arbitrary client really expect for arbitrary services to interact correctly? If you want them to do this, it has been shown over and over that global types are the best, least troublesome choice.
If you want ubiquitous interactions why not use string based values such as XML or better yet, JSON?
Then code and data is immune to class loading snafus and not bound to a container or hosting "standard"!
Gregg
> Anyway... It is not really River-436 problem so my patch proposal is going
> to have the same issue since it is just a replacement for String
> annotations and not change in class loading scheme.
>
> Thanks,
> Michal
> 4 mar 2014 06:38 "Michał Kłeczek" <mi...@xpro.biz> napisał(a):
>
>> 1. The problem is there is no such thing as "the service interface". It is
>> context dependent. What is "the service interface" for service browser?
>>
>> 2. In this particular case Util interface is an implementation detail of
>> WrapperProxy. It is Wrapper interface the client is interested in. So I
>> would say it should be preferred in WrapperProxy codebase.
>>
>> 3. Even if Util is not preferred in WrapperProxy codebase we still have
>> ClassCastException if the client does not have Util in its classpath. Why
>> should it? it is interested in Wrapper not in Util. So either
>> a. We always get ClassCastException if Util is preferred in WrapperProxy
>> codebase, or
>> b. We get ClassCastException anyway if a client does not have Util in its
>> classpath.
>> Let's say I want to register RemoteEventListener that wraps a Javaspace
>> proxy to write events in a space. Does that mean the service "event source"
>> has to be aware of Javaspace interface??? That would be absurd...
>>
>> It all does not have anything to do with codebase services.
>>
>> Thanks,
>> Michal
>> 4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
>>
>>> The Util interface should not be preferred. Implementations of Util can
>>> be preferred but not Util itself.
>>>
>>> Services need a common api that all implementations and clients can use
>>> to interract, even if this is a kind of codebase service.
>>>
>>> Modifying an interface is generally considered bad practise but now Java
>>> 8 makes it possible to add default methods for added functionality, that
>>> line blurs somewhat. What can you do if an earlier interface is loaded by
>>> a parent ClassLoader and you need a later version, make it preferred?
>>>
>>> My thoughts are that interfaces should never be preferred and all classes
>>> defined in their methods shouldn't be preferred either.
>>>
>>> It would be relatively easy to write a new implementation that ensures
>>> that interfaces are loaded into their own ProtectionDomain in a parent
>>> ClassLoader. But that would be confusing as dynamic policy grants are made
>>> to ClassLoader's not ProtectionDomains.
>>>
>>> But using ProtectionDomains in this manner, preserves security, ensures
>>> maximum visibility and avoids codebase annotation loss, if we ask the
>>> ProtectionDomain for the annotation, instead of the ClassLoader. But this
>>> is not how we do things presently.
>>>
>>> Cheers,
>>>
>>> Peter.
>>>
>>> ----- Original message -----
>>>> But it will also be loaded by WrapperProxy ClassLoader, since it is
>>>> preferred there. So it will end up with ClassCastException, right?
>>>>
>>>> Regards,
>>>> Michal
>>>>
>>>> If Util is installed locally, it will only be loaded by the application
>>>> ClassLoader, since it isn't preferred.
>>>>
>>>> Peter.
>>>>
>>>> ----- Original message -----
>>>>> Folks,
>>>>> while woking on the River-436 patch proposal I've came across the
>>>>> scenario that I am not sure how to handle:
>>>>>
>>>>> Utility service:
>>>>> //inteface is NOT preferred
>>>>> interface Util {...}
>>>>> //class IS preferred
>>>>> class UtilProxy implements Util {}
>>>>>
>>>>> Wrapper service:
>>>>> //NOT preferred
>>>>> interface Wrapper {}
>>>>> //preferred
>>>>> class WrapperProxy implements Serializable{
>>>>> //initialized with Util impl from a lookup service
>>>>> private Util util;
>>>>> }
>>>>>
>>>>> Wrapper service codebase includes Util interface but it is
>>> _preferred_.
>>>>>
>>>>> Would deserialization of WrapperProxy end with ClassCastException?
>>>>> From what I understand UtilProxy is annotated with its codebase. When
>>>>> deserializing UtilProxy a ClassLoader is going to be created with
>>>>> parent set to TCCL. It means Util interface is going to be loaded
>>>>> twice by two ClassLoaders - one for WrapperProxy codebase and another
>>>>> for UtilProxy codebase.
>>>>>
>>>>> Am I correct?
>>>>> And if so: is it desired behavior?
>>>>>
>>>>> Regards,
>>>>>
>>>>> --
>>>>> Michał Kłeczek
>>>>> XPro Quality Matters
>>>>> http://www.xpro.biz
>>>
>>>
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
1. The problem is there is no such thing as "the service interface". It is
context dependent. What is "the service interface" for service browser?
2. In this particular case Util interface is an implementation detail of
WrapperProxy. It is Wrapper interface the client is interested in. So I
would say it should be preferred in WrapperProxy codebase.
3. Even if Util is not preferred in WrapperProxy codebase we still have
ClassCastException if the client does not have Util in its classpath. Why
should it? it is interested in Wrapper not in Util. So either
a. We always get ClassCastException if Util is preferred in WrapperProxy
codebase, or
b. We get ClassCastException anyway if a client does not have Util in its
classpath.
Let's say I want to register RemoteEventListener that wraps a Javaspace
proxy to write events in a space. Does that mean the service "event source"
has to be aware of Javaspace interface??? That would be absurd...
It all does not have anything to do with codebase services.
Thanks,
Michal
4 mar 2014 00:09 "Peter" <ji...@zeus.net.au> napisał(a):
> The Util interface should not be preferred. Implementations of Util can
> be preferred but not Util itself.
>
> Services need a common api that all implementations and clients can use to
> interract, even if this is a kind of codebase service.
>
> Modifying an interface is generally considered bad practise but now Java 8
> makes it possible to add default methods for added functionality, that line
> blurs somewhat. What can you do if an earlier interface is loaded by a
> parent ClassLoader and you need a later version, make it preferred?
>
> My thoughts are that interfaces should never be preferred and all classes
> defined in their methods shouldn't be preferred either.
>
> It would be relatively easy to write a new implementation that ensures
> that interfaces are loaded into their own ProtectionDomain in a parent
> ClassLoader. But that would be confusing as dynamic policy grants are made
> to ClassLoader's not ProtectionDomains.
>
> But using ProtectionDomains in this manner, preserves security, ensures
> maximum visibility and avoids codebase annotation loss, if we ask the
> ProtectionDomain for the annotation, instead of the ClassLoader. But this
> is not how we do things presently.
>
> Cheers,
>
> Peter.
>
> ----- Original message -----
> > But it will also be loaded by WrapperProxy ClassLoader, since it is
> > preferred there. So it will end up with ClassCastException, right?
> >
> > Regards,
> > Michal
> >
> > If Util is installed locally, it will only be loaded by the application
> > ClassLoader, since it isn't preferred.
> >
> > Peter.
> >
> > ----- Original message -----
> > > Folks,
> > > while woking on the River-436 patch proposal I've came across the
> > > scenario that I am not sure how to handle:
> > >
> > > Utility service:
> > > //inteface is NOT preferred
> > > interface Util {...}
> > > //class IS preferred
> > > class UtilProxy implements Util {}
> > >
> > > Wrapper service:
> > > //NOT preferred
> > > interface Wrapper {}
> > > //preferred
> > > class WrapperProxy implements Serializable{
> > > //initialized with Util impl from a lookup service
> > > private Util util;
> > > }
> > >
> > > Wrapper service codebase includes Util interface but it is _preferred_.
> > >
> > > Would deserialization of WrapperProxy end with ClassCastException?
> > > From what I understand UtilProxy is annotated with its codebase. When
> > > deserializing UtilProxy a ClassLoader is going to be created with
> > > parent set to TCCL. It means Util interface is going to be loaded
> > > twice by two ClassLoaders - one for WrapperProxy codebase and another
> > > for UtilProxy codebase.
> > >
> > > Am I correct?
> > > And if so: is it desired behavior?
> > >
> > > Regards,
> > >
> > > --
> > > Michał Kłeczek
> > > XPro Quality Matters
> > > http://www.xpro.biz
> > >
>
>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
----- Original message -----
> The Util interface should not be preferred. Implementations of Util can
> be preferred but not Util itself.
>
> Services need a common api that all implementations and clients can use
> to interract, even if this is a kind of codebase service.
>
> Modifying an interface is generally considered bad practise but now Java
> 8 makes it possible to add default methods for added functionality, that
> line blurs somewhat. What can you do if an earlier interface is loaded
> by a parent ClassLoader and you need a later version, make it preferred?
>
> My thoughts are that interfaces should never be preferred and all
> classes defined in their methods shouldn't be preferred either.
>
> It would be relatively easy to write a new implementation that ensures
> that interfaces are loaded into their own ProtectionDomain in a parent
> ClassLoader. But that would be confusing as dynamic policy grants are
> made to ClassLoader's not ProtectionDomains.
>
> But using ProtectionDomains in this manner, preserves security, ensures
> maximum visibility and avoids codebase annotation loss, if we ask the
> ProtectionDomain for the annotation, instead of the ClassLoader. But
> this is not how we do things presently.
>
To clarify, codebase annotation loss occurrs where identical classes with different annotations are loaded into the same ClassLoader. ProtectionDomains do allow different classes with different codebase annotations to be loaded into the same ClassLoader.
Service api shouldn't be changed after deployment. So if it didn't exist locally, it would be safe from a security perspective to load it into its own ProtectionDomain in the application ClassLoader for maximum visibility and interoperability, rather than the proxy's child ClassLoader.
> Cheers,
>
> Peter.
>
> ----- Original message -----
> > But it will also be loaded by WrapperProxy ClassLoader, since it is
> > preferred there. So it will end up with ClassCastException, right?
> >
> > Regards,
> > Michal
> >
> > If Util is installed locally, it will only be loaded by the application
> > ClassLoader, since it isn't preferred.
> >
> > Peter.
> >
> > ----- Original message -----
> > > Folks,
> > > while woking on the River-436 patch proposal I've came across the
> > > scenario that I am not sure how to handle:
> > >
> > > Utility service:
> > > //inteface is NOT preferred
> > > interface Util {...}
> > > //class IS preferred
> > > class UtilProxy implements Util {}
> > >
> > > Wrapper service:
> > > //NOT preferred
> > > interface Wrapper {}
> > > //preferred
> > > class WrapperProxy implements Serializable{
> > > //initialized with Util impl from a lookup service
> > > private Util util;
> > > }
> > >
> > > Wrapper service codebase includes Util interface but it is
> > > _preferred_.
> > >
> > > Would deserialization of WrapperProxy end with ClassCastException?
> > > From what I understand UtilProxy is annotated with its codebase. When
> > > deserializing UtilProxy a ClassLoader is going to be created with
> > > parent set to TCCL. It means Util interface is going to be loaded
> > > twice by two ClassLoaders - one for WrapperProxy codebase and another
> > > for UtilProxy codebase.
> > >
> > > Am I correct?
> > > And if so: is it desired behavior?
> > >
> > > Regards,
> > >
> > > --
> > > Michał Kłeczek
> > > XPro Quality Matters
> > > http://www.xpro.biz
> > >
>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
The Util interface should not be preferred. Implementations of Util can be preferred but not Util itself.
Services need a common api that all implementations and clients can use to interract, even if this is a kind of codebase service.
Modifying an interface is generally considered bad practise but now Java 8 makes it possible to add default methods for added functionality, that line blurs somewhat. What can you do if an earlier interface is loaded by a parent ClassLoader and you need a later version, make it preferred?
My thoughts are that interfaces should never be preferred and all classes defined in their methods shouldn't be preferred either.
It would be relatively easy to write a new implementation that ensures that interfaces are loaded into their own ProtectionDomain in a parent ClassLoader. But that would be confusing as dynamic policy grants are made to ClassLoader's not ProtectionDomains.
But using ProtectionDomains in this manner, preserves security, ensures maximum visibility and avoids codebase annotation loss, if we ask the ProtectionDomain for the annotation, instead of the ClassLoader. But this is not how we do things presently.
Cheers,
Peter.
----- Original message -----
> But it will also be loaded by WrapperProxy ClassLoader, since it is
> preferred there. So it will end up with ClassCastException, right?
>
> Regards,
> Michal
>
> If Util is installed locally, it will only be loaded by the application
> ClassLoader, since it isn't preferred.
>
> Peter.
>
> ----- Original message -----
> > Folks,
> > while woking on the River-436 patch proposal I've came across the
> > scenario that I am not sure how to handle:
> >
> > Utility service:
> > //inteface is NOT preferred
> > interface Util {...}
> > //class IS preferred
> > class UtilProxy implements Util {}
> >
> > Wrapper service:
> > //NOT preferred
> > interface Wrapper {}
> > //preferred
> > class WrapperProxy implements Serializable{
> > //initialized with Util impl from a lookup service
> > private Util util;
> > }
> >
> > Wrapper service codebase includes Util interface but it is _preferred_.
> >
> > Would deserialization of WrapperProxy end with ClassCastException?
> > From what I understand UtilProxy is annotated with its codebase. When
> > deserializing UtilProxy a ClassLoader is going to be created with
> > parent set to TCCL. It means Util interface is going to be loaded
> > twice by two ClassLoaders - one for WrapperProxy codebase and another
> > for UtilProxy codebase.
> >
> > Am I correct?
> > And if so: is it desired behavior?
> >
> > Regards,
> >
> > --
> > Michał Kłeczek
> > XPro Quality Matters
> > http://www.xpro.biz
> >
Re: River-436 - need some explanation of preferred class provider
Posted by Michał Kłeczek <mi...@xpro.biz>.
But it will also be loaded by WrapperProxy ClassLoader, since it is
preferred there. So it will end up with ClassCastException, right?
Regards,
Michal
If Util is installed locally, it will only be loaded by the application
ClassLoader, since it isn't preferred.
Peter.
----- Original message -----
> Folks,
> while woking on the River-436 patch proposal I've came across the
> scenario that I am not sure how to handle:
>
> Utility service:
> //inteface is NOT preferred
> interface Util {...}
> //class IS preferred
> class UtilProxy implements Util {}
>
> Wrapper service:
> //NOT preferred
> interface Wrapper {}
> //preferred
> class WrapperProxy implements Serializable{
> //initialized with Util impl from a lookup service
> private Util util;
> }
>
> Wrapper service codebase includes Util interface but it is _preferred_.
>
> Would deserialization of WrapperProxy end with ClassCastException?
> From what I understand UtilProxy is annotated with its codebase. When
> deserializing UtilProxy a ClassLoader is going to be created with parent
> set to TCCL. It means Util interface is going to be loaded twice by two
> ClassLoaders - one for WrapperProxy codebase and another for UtilProxy
> codebase.
>
> Am I correct?
> And if so: is it desired behavior?
>
> Regards,
>
> --
> Michał Kłeczek
> XPro Quality Matters
> http://www.xpro.biz
>
Re: River-436 - need some explanation of preferred class provider
Posted by Peter <ji...@zeus.net.au>.
If Util is installed locally, it will only be loaded by the application ClassLoader, since it isn't preferred.
Peter.
----- Original message -----
> Folks,
> while woking on the River-436 patch proposal I've came across the
> scenario that I am not sure how to handle:
>
> Utility service:
> //inteface is NOT preferred
> interface Util {...}
> //class IS preferred
> class UtilProxy implements Util {}
>
> Wrapper service:
> //NOT preferred
> interface Wrapper {}
> //preferred
> class WrapperProxy implements Serializable{
> //initialized with Util impl from a lookup service
> private Util util;
> }
>
> Wrapper service codebase includes Util interface but it is _preferred_.
>
> Would deserialization of WrapperProxy end with ClassCastException?
> From what I understand UtilProxy is annotated with its codebase. When
> deserializing UtilProxy a ClassLoader is going to be created with parent
> set to TCCL. It means Util interface is going to be loaded twice by two
> ClassLoaders - one for WrapperProxy codebase and another for UtilProxy
> codebase.
>
> Am I correct?
> And if so: is it desired behavior?
>
> Regards,
>
> --
> Michał Kłeczek
> XPro Quality Matters
> http://www.xpro.biz
>