River Core Platform [was: Re: A Question about PermissionDomain's ClassLoader's and User Authentication]

[Peter Firmstone <ji...@zeus.net.au>]

Hi Tom,

I've had some more time to consider your response,  I was thinking of 
creating an branch for River for Internet or Untrusted Networks 
utilising OSGi and later merging this if it was to everyones 
satisfaction.  (A different branch will be difficult in light of the 
namespace changes, anyway I prefer continuous integration) However I 
figure that I'm missing the underlying problem and I think we must come 
to some sort of recognition of it in order to prevent the stagnation of 
River's development.  From what I've seen over time on the malling list, 
I believe that there are two groups, both just as interested in River:

   1. Those happy with the platforms existing features, that just want
      the platform to be supported and evolve to become more user
      friendly.  This group is not particularly concerned with Security,
      their use for the technology is on Trusted Intranets, where
      codebases are controlled by one company.  Security and Codebase
      evolution don't have a large impact on this group.  They
      successfully deploy the software and are in the main happy with
      the current feature set.  If anything some of the security
      features are seen as impeding performance.
   2. The other group can see the potential for River to expand onto the
      Internet, or out into Untrusted Networks.  Jini version 2 was a
      big step in this direction with JAAS, Proxy Trust and the new Java
      2 Security Model ( extended specifically with Jini in mind). 
      However even Jim Waldo has his doubts as to whether the current
      security model or even Jini is sufficient for untrusted
      distributed networks.

So what we have are two groups with different needs, this is why I think 
it is important that we define a Core platform, and break up River into 
the Following Components (Anyone please feel free to suggest other ways 
to break it up).

    * Core Platform
    * Legacy Platform (optional)
    * Security, Modularity & Versioning Extensions
    * Platform Services (These would need to be capable of supporting
      the Modularity and Security components, although not necessarily
      rely on them.)

Anyone have any other ideas?

Is the alternative is to have two separate codebase release artifacts, 
one for Trusted Networks, the other for the Internet or Untrusted 
Networks, that share some common components?

Break up River like this Perhaps:

    * Core Platform (packaged with both release versions)
    * Trusted Intranet Release (contains the core above)
    * Internet or Untrusted Network Release (contains the core above)


Cheers,

Peter.

N.B. In the Internet release it would be simple to solve the dependency 
download hell you describe using Codebase Services, combined with 
dependency resolution.  The lack of OSGi would cause a serious roadblock 
(or at least a reinvention of the wheel) for the Internet version, I 
have looked into Project Jigsaw, however JDK7 support would be some time 
away. The Trusted Intranet release wouldn't need modularity, versioning 
and additional security concerns.

Tom Hobbs wrote:
> Hi Peter,
>
> Thanks for the recent emails.  It wasn't my intention to try and block what
> you're doing, as you say having a disagreement about how the code has /been/
> implemented puts us in a better argument to have than disagreeing about how
> it /should/ be implemented
>
> >From what I understood, you wanted to write a Secure End Point
> implementation which would have a fixed dependency on OSGi sources.  I have
> a read a lot of articles about people complaining about all sorts of things
> (from Scala, Clojure to Ruby/Rails) and how they "just wanted to do this"
> but they ended up having to download dependency after dependency.  It was my
> concern that people wanting to use River would somehow be forced to download
> OSGi also, even if they didn't want those bits of functionality.
>
> All that I was trying to say was, the idea of a Secure End Point is a good
> one, but if we can provide that End Point with some kind of "Security
> Module", which just happens to be implemented with OSGi bits then users
> always have the option of specifying an alternative implementation.  From
> what I can work out, the exact implementation could be specified in one of
> the Jini config files, maybe?
>
> I remember when I asked about what mocking library to use, someone (Jukka?)
> said that the first one to do it gets to choose.  If other's don't like it
> then they can discuss that once it's done and maybe change it themselves if
> it's such a big deal.  I think a similar thing is happening here.  If I
> "don't like" how you implement this End Point then we can discuss that once
> it's done and if I feel strongly enough about it I can put in the
> refactoring work.  But the last thing that should happen is you feeling
> blocked because I'm voicing concerns.  It may well be that my concerns are
> unfounded.  Until we see the code we'll never know for sure.
>
>   
>> Yes my implementation would depend on OSGi, is that bad?
>>     
>
> That particular implementation depending on it, no.  But River depending on
> it, yes.  Isn't it?
>
>   
>> If it wasn't for your encouragement that got me involved in River...
>>     
>
> Are you thanking me, or blaming me?  :-)
>
>   
>> if you don't have it, it's very hard to add or learn from and improve
>>     
> upon. No?
>
> Absolutely right!  Like I said, I'm asking questions and voicing concerns;
> not trying to stop you.  Actually, I don't know enough about the details of
> what you've got in mind to feel qualified to try and stop you!
>
> My apologies if I've come across as being discouraging, that wasn't my
> intention.  It'd be great to see some ideas get turned into code.
>
>   
>> If you do find the time to give me an example of that Proxy wrapper I
>>     
> would appreciate it.
>
> I'll see what I can do.
>
> Cheers,
>
> Tom
>
>
>
> On Thu, Feb 4, 2010 at 10:13 PM, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>   
>> Hi Tom,
>>
>> This is new functionality not present in River / Jini, I liked the
>> terminology Conditions, because you can simply say: "Conditions have changed
>> so were not going to allow that Permission now."
>>
>> The project Neuromancer report available here talks about some of the
>> issues of Proxy trust after disconnection:
>>
>> http://research.sun.com/techrep/2007/abstract-162.html
>>
>> I'd also like to solve the problems associated with ClassLoader visibility,
>> so we can utilise Jini on the internet.
>>
>> At present, the only mechanism for ensuring proxy's are utilising the same
>> bytecode, are the same runtime Type and thus interchangeable, is to use the
>> Preferred Classloader mechanisms, as useful as this is, it doesn't address
>> the issue of change over time on networks where there are multiple entities
>> controlling code.  It would lock in implementations that are shared over the
>> web, creating all sorts of nasty hard to diagnose bugs when the wrong
>> bytecode is loaded.  Or the alternative, a huge memory consumption
>> explosion.
>>
>> I was going to utilse Classworlds to assist with arranging packages into
>> separate classloaders, however Classworlds doesn't deal with versioning over
>> time and I needed a mechanism for that.  The problems I'm trying to solve
>> are very challenging and I need as much help as I can get because I don't
>> have all the answers.  The OSGi people have spent a lot of years working on
>> that particular problem.
>>
>> I'll try to name my classes as expressively as possible so their intent is
>> clear, my gut feel is that OSGi on the front of my Endpoint name would cause
>> confusion, so I would naturally resist this.
>>
>> How do I expose my new functionality so it can be used, but at the same
>> time abstracting it, so it isn't seen?  Isn't it already an abstraction as
>> an Endpoint?
>>
>> Yes my implementation would depend on OSGi, is that bad?
>>
>> Instead of trying to hide the new functionality behind an old interface,
>> how about instead we expose the new functionality, work out the best
>> interface for it, then work out how to map the old functionality to the new
>> with a wrapper, then depreciate the old?
>>
>> How about I just say, look, let me start by writing some code, lets get our
>> heads around it, play and experiment.  If it turns out to be the wrong
>> decision, then lets remove it and try something else, lets not barricade the
>> door before we've had a chance to investigate what's on the other side.
>>
>> If you do find the time to give me an example of that Proxy wrapper I would
>> appreciate it.
>>
>> If it wasn't for your encouragement that got me involved in River, Tim
>> Blackman wouldn't have donated his code before Oracle bought out Sun and I
>> wouldn't have replaced the ClassDep implementation and we wouldn't have
>> dependency support for annotations and generics.  Code can be removed, but
>> if you don't have it, it's very hard to add or learn from and improve upon.
>> No?
>>
>>
>> Cheers,
>>
>> Peter.
>>
>> Tom Hobbs wrote:
>>
>>     
>>> Hi Peter,
>>>
>>>
>>>
>>>       
>>>> Nice to have you back
>>>>
>>>>
>>>>         
>>> I'm always reading the list, rarely have time to do anything about it
>>> though.  I don't suppose my situation is unique...
>>>
>>>
>>>
>>>       
>>>> got an example of your proxy wrapper?
>>>>
>>>>
>>>>         
>>> Sadly not, it was written for my previous employer and therefore the code
>>> is
>>> protected as part of their IP.  I don't see any reason why I can re-invent
>>> it though.
>>>
>>> Thanks for the explanation about what you're up to.  My concerns about
>>> introducing a dependency/coupling on OSGi through this approach still
>>> stands
>>> though.  I think it should either be named as an OSGi End Point (or
>>> whatever
>>> it may be) to avoid suprises, or you should abstract away the OSGi classes
>>> so people can provide their own implementations of this security
>>> behaviour.
>>>
>>> I'd be in favour of the abstraction way, since it would match the other
>>> pluggable components that Jini/River users can choose.
>>>
>>> I'm sure you've already thought of that, though.  I know it seems like I'm
>>> choosing the curtains and wallpaper before you've built the house, but I
>>> think this is worth bearing in mind.
>>>
>>> Cheers,
>>>
>>> Tom
>>>
>>>
>>>
>>> On Thu, Feb 4, 2010 at 11:00 AM, Peter Firmstone <ji...@zeus.net.au>
>>> wrote:
>>>
>>>
>>>
>>>       
>>>> Hi Tom,
>>>>
>>>> Nice to have you back, got an example of your proxy wrapper?
>>>>
>>>> The Endpoint, would be based on an SSLEndpoint, it is for traversing
>>>> NAT's
>>>> for Services that we'd like to make public but can't due to the private
>>>> IP
>>>> address being invisible.
>>>>
>>>> Some NAT's change their ports dynamically, it appears the sockets can
>>>> deal
>>>> with the dynamic ports (although I haven't confirmed this) however, it is
>>>> very likely that the connection will be subject to breakage and will have
>>>> to
>>>> be renegotiated again by a public mediator service.
>>>>
>>>> Unfortunately due to Jini's security model, you can dynamically grant a
>>>> permission, but once, I implemented a RevokeablePermission class which
>>>> dynamically removes a Permission after has been granted, but I found the
>>>> OSGi Conditions model more elegant.  Conditions allow you to have a
>>>> number
>>>> of dynamic tests in place that confirm it is safe to grant a Permission.
>>>>  In
>>>> this case, after we have been disconnected there is no way of knowing we
>>>> have the same service apart from questioning the proxy for it's identity,
>>>> however we cannot trust the proxy anymore and we need to repeat the proxy
>>>> trust process.  If the connection cannot be re started within a short
>>>> time
>>>> frame, I'd re-throw the exception.  That would enable you to handle the
>>>> loss
>>>> of service in various other ways.
>>>>
>>>> The problem for my Revokeable permission is that it revokes permission
>>>> from
>>>> the entire PermissionDomain, not very fine grained at all, like using a
>>>> Sledge Hammer to swat a fly!  If there are other services in the same
>>>> ClassLoader (for memory efficiency reasons) we need to distinguish them
>>>> by
>>>> their Principal, so only the Services that experienced a temporary
>>>> disconnection is denied Permissions until it is re verified.
>>>>
>>>> Once the connection was re established, a permission exception would be
>>>> thrown until the proxy trust is verified again.
>>>>
>>>> Cheers,
>>>>
>>>> Peter.
>>>>
>>>>
>>>> Tom Hobbs wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Not being a guru of any description I can't help you with the specific
>>>>> OSGi
>>>>> questions, but your post makes me assume a few things about your
>>>>> intentions.
>>>>>  Can you clarify, if only for my benefit, please?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Or does the JERI Endpoint catch the exception and try to find it
>>>>>> itself?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> The Endpoint finds the service based on identity, it has moved, we must
>>>>> re
>>>>> verify our Trust because the Conditions have changed.
>>>>>
>>>>> This suggests to me, and I've been wrong before so it's no surprise if I
>>>>> am
>>>>> again, that you're starting to couple the OSGi security mechanisms into
>>>>> this
>>>>> custom JERI Endpoint.  If that assumption is correct, can you start
>>>>> referring to it as an OSGi Endpoint (which should be able to swap
>>>>> between
>>>>> JERI and JRMP, maybe)?  Or a Secure JERI Endpoint that is configured to
>>>>> use
>>>>> the OSGi security mechanism although another, as yet unspecified one,
>>>>> could
>>>>> be swapped in instead (which means hiding the OSGi stuff behind a common
>>>>> interface).
>>>>>
>>>>> I've been involved with wrapping proxies to provide auto-discovery
>>>>> behaviour
>>>>> before and it works very well.  The business code only ever gets
>>>>> RemoteExceptions when a service really can't be found which means a lot
>>>>> less
>>>>> "wait-and-retry" boilerplate code has to be written.
>>>>>
>>>>> Barring someone categorically declaring that the EndPoint is the Wrong
>>>>> Place
>>>>> to do it because the Jini spec says "xyz", it's probably just a matter
>>>>> of
>>>>> taste which approach is the 'right' one.  Personally speaking, it
>>>>> doesn't
>>>>> "seem" like it's the EndPoint's job to go away and find a replacement
>>>>> for
>>>>> a
>>>>> broken/missing service, as far as I'm aware no other EndPoint does that
>>>>> (again, I'm happy to be corrected).  If you do this, do you now call it
>>>>> a
>>>>> Service Refinding Secure JERI EndPoint?  Which makes it sound like the
>>>>> one
>>>>> area of code is now doing a lot of different jobs...
>>>>>
>>>>> Sorry I can't help with the OSGi specific questions.
>>>>>
>>>>> Tom
>>>>>
>>>>>
>>>>> On Thu, Feb 4, 2010 at 6:23 AM, Peter Firmstone <ji...@zeus.net.au>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> I want to load Marshalled Objects from different locations with
>>>>>> identical
>>>>>> bytecode into the same ClassLoader, I want to do this using codebase
>>>>>> services and OSGi, which gives me another level of indirection and a
>>>>>> richer
>>>>>> security model with Condition's
>>>>>>
>>>>>> PermissionDomain has a 1:1 relationship with ClassLoader
>>>>>>
>>>>>> Trust Verifiication.
>>>>>>
>>>>>> Identity and Authentication.
>>>>>>
>>>>>> What distinguishes different services (from different remote locations
>>>>>> and
>>>>>> domains) utilising the same ClassLoader, PermissionDomain and bytecode?
>>>>>>
>>>>>> Principals.
>>>>>>
>>>>>> I wonder if Conditions can apply to Principals? Such that when a proxy
>>>>>> has
>>>>>> lost connection with its service node, the Condition denies a Principal
>>>>>> relating to that proxy any Permission, until the Service can be asked
>>>>>> to
>>>>>> verify it's proxy again?  I was thinking that the proxy itself doesn't
>>>>>> attempt to find its lost service, instead that would be done by a new
>>>>>> JERI
>>>>>> Endpoint implementation, to do that the Endpoint needs to know the
>>>>>> Service's
>>>>>> Identity.
>>>>>>
>>>>>> Any OSGi guru's out there?
>>>>>>
>>>>>> So we have Permissions being granted at different levels and we have
>>>>>> Conditions that make those Permissions Dynamic:
>>>>>>
>>>>>> Principal - Can I trust the service?  Can I grant it a Permission?
>>>>>>  What
>>>>>> are the current Conditions?
>>>>>> PermissionDomain & ClassLoader - relates to the Code Signer (which may
>>>>>> be
>>>>>> different to the Service Principal).  Do I trust the bytecode?  If I
>>>>>> do,
>>>>>> the
>>>>>> bundle tells me the Permissions required to execute that bytecode.
>>>>>>
>>>>>> Lets imagine for a moment, were happily using a Service, and suddenly
>>>>>> it
>>>>>> throws a RemoteException, after some a wait period, we retry, the
>>>>>> service
>>>>>> hasn't returned, so do we ask the JERI Endpoint to find the service?
>>>>>>  Or
>>>>>> does the JERI Endpoint catch the exception and try to find it itself?
>>>>>> The
>>>>>> Endpoint finds the service based on identity, it has moved, we must re
>>>>>> verify our Trust because the Conditions have changed.  So now we must
>>>>>> catch
>>>>>> a Permission denied exception, and go through the re-verification
>>>>>> process.
>>>>>>
>>>>>> Does Bob Scheifler still watch the list?  Anyone know where I can find
>>>>>> Bob?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>
>>>>>           
>>>>         
>>>
>>>       
>>     
>
>