You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Peter <ji...@zeus.net.au> on 2013/12/19 01:42:45 UTC
VOTE: add Startable to Jini specification
+1 Peter.
I've been going over the Exporter implementations and Exporter interface.
The Exporter interface specifies that export and unexport behaviour is defined by the implementation.
That means it's up to the implementation to specify whether a happens before edge occures before export.
I propose we document in Exporter, that if export is called during object construction then that object's implementation cannot safely use final fields.
We should also provide a @see reference to Startable to reccommend its use when final fields are desirable and include Startable in the spec.
We don't need to rewrite the example of exporting in net.jini.config as export isn't performed during construction.
This is a minor update to the Jini Specification.
The FINE log message is an implementation detail and won't be included.
Regards,
Peter.
----- Original message -----
> No, an effectively immutable field is one whose reference doesn't change
> after the service objects safe publication.
>
> An immutable field is final.
>
> It would be Jeri's responsibility to safely publish the service during
> export to ensure a happens before event occurs, ensuring all non
> volatile fields the service contains are safely published and visible.
>
> It's possible to have mutable fields guarded by synchronized methods, or
> volatile fields.
>
> But it isn't possible to have final fields when exporting within the
> constructor.
>
> If the service contains effectively immutable field references, it must
> be safely published or if another thread accesses those fields through
> unsynchronized methods, the thread may only see their default value.
>
> All our service implementations have final fields. All our services had
> some unsynchronized access to non volatile, non final fields.
>
> Trouble is, when you can't have final fields, it's very difficult to
> determine which fields are effectively immutable and the original
> programmers design intent, if there's no comment.
>
> Our service implementations all had unsynchronized init() methods called
> from with the constructor that started threads and exported Some fields
> were final.
>
> For me, a service that contains final fields and is exported after
> construction is easier to maintain and understand.
>
> I suspect, like our Jini service imlementations, most people get it
> wrong most of the time.
>
> I get that others won't want to use this method, however it would be
> detrimental if we don't have such a method for those who would.
>
> Regards,
>
> Peter.
>
> ----- Original message -----
> >
> > I feel like we’re going down a rabbit hole here when you start talking
> > about exporting immutable objects. Wouldn’t it be kind of silly to
> > export an immutable service? Isn’t the whole point that you interact
> > with the service (i.e. alter its state) over a remote interface?
> >
> > Perhaps it’s better to say that exported services need to be
> > thread-safe (which should be fairly obvious). Yes, immutable objects
> > are inherently thread-safe, so for sharing information inside a VM, it
> > makes some sense to minimize the number of mutable objects you pass
> > around. But fundamentally, we’re talking about shared-state systems
> > here.
> >
> > Cheers,
> >
> > Greg.
> >
> > On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
> >
> > > Pat your comment about non final fields is interesting.
> > >
> > > Isn't it also the case that we need to safely publish an effectively
> > > immutable object to share it among threads? That usually means
> > > copying it to a thread safe collection or shared via a synchronized
> > > method, volatile field, or final field in another object?
> > >
> > > So we should also make sure that Jeri uses safe publication during
> > > export.
> > >
> > > That would allow a service that has no final fields to start threads,
> > > then export from within a constructor safely, provided all operations
> > > on non final fields happen before starting threads and exporting.
> > >
> > > All our services have final fields, so Starter is more appropriate
> > > for River's own services.
> > >
> > > Regards,
> > >
> > > Peter.
> > >
> > > ----- Original message -----
> > > > Hmm, good point, Startable, makes more sense.
> > > >
> > > > An object can be exported using Startable.
> > > >
> > > > I think we should have a policy to strongly discourage exporting
> > > > from constructors.
> > > >
> > > > Regards,
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > > > As far as I can tell, the special properties of completing a
> > > > > constructor in the JLS memory model are:
> > > > >
> > > > > 1. A happens-before edge from the end of the constructor to the
> > > > > start of a finalizer. (17.4.5)
> > > > >
> > > > > 2. The guarantee that any thread that only sees a reference to an
> > > > > object after the end of the constructor will see the
> > > > > correctly initialized values of all final fields. (17.5)
> > > > >
> > > > > The special issue with final fields is that implementations have
> > > > > freedom to optimize access to final fields in ways that are
> > > > > not permitted for non-final fields. Strategies for thread
> > > > > safety that work for non-final fields do not necessarily
> > > > > work for final fields. The requirement for final field
> > > > > safety is that the constructor end before another thread
> > > > > accesses the newly constructed object.
> > > > >
> > > > > Calling a start() method after construction if the class
> > > > > implements a new interface seems to me to be harmless,
> > > > > backwards compatible, and useful. It enables the simplest and
> > > > > most direct way of preventing access to the new object by
> > > > > another thread during construction.
> > > > >
> > > > > The roadmap issue is whether it should be required, and if so the
> > > > > level of enforcement. For example, there is no reason to
> > > > > require it if the class does not declare any final fields.
> > > > >
> > > > > Incidentally, and as a detail, "Commission" does not immediately
> > > > > make me think of having a start() method that should be
> > > > > called after construction. If you do go this way, the name
> > > > > needs thought. "Startable" would be more obvious, more
> > > > > memorable, more likely to be found on searches, and more
> > > > > compatible with familiar interface names such as "Cloneable"
> > > > > and "Iterable".
> > > > >
> > > > > Patricia
> > > > >
> > > > >
> > > > > On 12/18/2013 2:18 AM, Peter wrote:
> > > > > > Well, now seems like a good time to have the conversation.
> > > > > >
> > > > > > Yes there are other ways, but I haven't seen one safe
> > > > > > implementation yet, so...
> > > > > >
> > > > > > Does someone have a better way to solve this problem, has
> > > > > > someone already solved this problem I'm unaware of that we can
> > > > > > adopt, or is there a way that's more satisfactory?
> > > > > >
> > > > > > If not, is there something objectionable with the Commission
> > > > > > interface and if so, how can we fix it?
> > > > > >
> > > > > > The SEVERE log message is logged by the River start package,
> > > > > > other containers or frameworks can choose whether or not to do
> > > > > > so, but I'd encourage them to do something similar, yes we can
> > > > > > change it to WARN.
> > > > > >
> > > > > > A much harsher option is to throw an exception during export
> > > > > > which breaks backward compatibility.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Peter.
> > > > > >
> > > > > > ----- Original message -----
> > > > > > > "org.apache.river.api.util.Commission is an interface
> > > > > > > services should implement"
> > > > > > >
> > > > > > > If it's a SHOULD, not a MUST, chucking out a SEVERE is
> > > > > > > incorrect logger behaviour IMO. You could issue a WARN if you
> > > > > > > like but for even that I'd say you need to provide a roadmap
> > > > > > > explaining why the warning and what you intend to do in
> > > > > > > future and what you expect of service writers such as myself.
> > > > > > >
> > > > > > > Commission, at least from my point of view, is your means
> > > > > > > (maybe the River community's - did you ask us?) for
> > > > > > > satisfying your needs in respect of the JMM. As we've
> > > > > > > discussed previously, there are other ways too and they work
> > > > > > > and they are safe if you know what you're doing. Your
> > > > > > > contention was that most don't know what they're doing
> > > > > > > hence, presumably, Commission.
> > > > > > >
> > > > > > > So the thing is, you are seemingly on a road to asserting
> > > > > > > more structure (gosh, a standard?) on the way people write
> > > > > > > their services. If so, you'd best start flagging that
> > > > > > > honestly and openly via a roadmap, deprecation and
> > > > > > > such/whatever rather than sticking out logger messages with
> > > > > > > no clear guidance and at the cost of a certain amount of
> > > > > > > nuisance (no admin I know likes SEVERE's being logged for
> > > > > > > something which isn't critical cos it's noise they don't
> > > > > > > want in log files).
> > > > > > >
> > > > > > > And of course, we all know that when some entity asserts a
> > > > > > > standard or requirement on others for entry, they may choose
> > > > > > > not to enter. Does this help your community or hinder it? The
> > > > > > > answer to that is, it depends. On what? Have you asked or
> > > > > > > tested? How have you tested? What would be considered
> > > > > > > validation or lack of support?
> > > > > > >
> > > > > > > I am not out to flame or troll rather I want to see this
> > > > > > > community demonstrating good behaviour and I'm not feeling
> > > > > > > like what's going on around Commission (what is that big
> > > > > > > change in version number really saying?) is such.
> > > > > > >
> > > > > > > On 18 December 2013 08:52, Peter <ji...@zeus.net.au> wrote:
> > > > > > >
> > > > > > > > Just to clarify org.apache.river.api.util.Commission is an
> > > > > > > > interface services should implement, I would encourage all
> > > > > > > > container projects to pick up the interface and make
> > > > > > > > suggestions for improvement if there are any issues.
> > > > > > > >
> > > > > > > > Interface Commission {
> > > > > > > > void start () throws Exception;
> > > > > > > > }
> > > > > > > >
> > > > > > > > It's called after JMM safe construction to allow the
> > > > > > > > service to start any threads and be exported.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Peter.
> > > > > > > >
> > > > > > > > ----- Original message -----
> > > > > > > > >
> > > > > > > > > The way that services are instantiated and setup is an
> > > > > > > > > implementation detail. When I think of
> > > > > > > > > compatibility I think of the API and the lookup methods.
> > > > > > > > > We think of compatibility from a
> > > > > > > > > client point of view. From the client
> > > > > > > > > point of view, using a service looks like this:
> > > > > > > > >
> > > > > > > > > - Use multicast of unicast discovery to find one or more
> > > > > > > > > ServiceRegistrar instances -
> > > > > > > > > Call lookup(…) on one or more of these instances to get
> > > > > > > > > a set of service candidates - Choose a candidate and
> > > > > > > > > prepare() it using a ProxyPreparer, to yield a usable
> > > > > > > > > service proxy. - Make calls on
> > > > > > > > > it. Ideally hang on to this proxy
> > > > > > > > > instance, so you can skip the discovery and lookup next
> > > > > > > > > time you need it. - If the
> > > > > > > > > call fails, repeat the lookup (and possibly discovery)
> > > > > > > > > til you get a proxy that works.
> > > > > > > > >
> > > > > > > > > Nowhere does the client need to know whether the service
> > > > > > > > > instance is started up using the “com.sun.jini.start”
> > > > > > > > > mechanism, your Commission interface, some other IOC
> > > > > > > > > container (Rio, Harvester, Seven or RiverContainer) or
> > > > > > > > > some unknown mechanism that starts with a static main()
> > > > > > > > > method.
> > > > > > > > >
> > > > > > > > > JSK2.0 was 2.0 because of the introduction of the proxy
> > > > > > > > > verification mechanisms, as well as JERI.
> > > > > > > > > Absent some new client usage mechanism, River doesn’t
> > > > > > > > > need to go to 3.0.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > Greg.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Dec 17, 2013, at 1:58 PM, Peter <ji...@zeus.net.au>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > I think changing services to use safe construction
> > > > > > > > > > techniques is enough to cause the version jump.
> > > > > > > > > >
> > > > > > > > > > At this point I've allowed services to continue unsafe
> > > > > > > > > > construction practices, while logging a SEVERE warning
> > > > > > > > > > when the Commission interface isn't implemented, rather
> > > > > > > > > > than fail.
> > > > > > > > > >
> > > > > > > > > > This is a fundamental change to the way services are
> > > > > > > > > > written.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Peter.
> > > > > > > > > >
> > > > > > > > > > ----- Original message -----
> > > > > > > > > > >
> > > > > > > > > > > Assuming that there aren’t major incompatibilities, I
> > > > > > > > > > > think that would be a “minor” version change
> > > > > > > > > > > according to our versioning policy, so we’d be
> > > > > > > > > > > looking at the “2.3” branch rather than a “3.0”
> > > > > > > > > > > release.
> > > > > > > > > > >
> > > > > > > > > > > I’m still unnerved by the massive amounts of changes
> > > > > > > > > > > to both code and tests in the qa_refactor branch, as
> > > > > > > > > > > well as the apparent instability of the code,
> > > > > > > > > > > although that seems to be improving.
> > > > > > > > > > > In the next few weeks I’m going
> > > > > > > > > > > to try and setup a cross-test case, to see what the
> > > > > > > > > > > “2.2” tests say about the potential “2.3” release
> > > > > > > > > > > and vice-versa.
> > > > > > > > > > >
> > > > > > > > > > > I think what I’d really like to see is an incremental
> > > > > > > > > > > approach where we update limited components of the
> > > > > > > > > > > “2.2” branch, one at a time. Is there anything that
> > > > > > > > > > > we could pull out piecemeal? Maybe it
> > > > > > > > would
> > > > > > > > > > > make sense to split out the infrastructure services,
> > > > > > > > > > > like Reggie, Mahalo, and Outrigger into different
> > > > > > > > > > > sub-projects that could be updated separately?
> > > > > > > > > > >
> > > > > > > > > > > Any thoughts?
> > > > > > > > > > >
> > > > > > > > > > > Greg.
> > > > > > > > > > >
> > > > > > > > > > > On Dec 17, 2013, at 5:03 AM, Peter <ji...@zeus.net.au>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > When the qa_refactor branch stabilises, I plan to
> > > > > > > > > > > > merge trunk and provide a beta release for client
> > > > > > > > > > > > compatibility testing.
> > > > > > > > > > > >
> > > > > > > > > > > > Changes made have been focused on making our code
> > > > > > > > > > > > thread safe, there are significant changes
> > > > > > > > > > > > internally, the public api remains focused on
> > > > > > > > > > > > backward compatibility, however it is advisable
> > > > > > > > > > > > that client services adopt new safe construction
> > > > > > > > > > > > techniques for services and implement the new
> > > > > > > > > > > > Commission interface.
> > > > > > > > > > > >
> > > > > > > > > > > > What's a suitable test period for client testing?
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > >
> > > > > > > > > > > > Peter.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
Sure:
http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/org/apache/river/api/util/Commission.java?view=markup
The interface name will be changed to Startable as per list comments below.
Regards,
Peter.
----- Original message -----
> Hi Peter,
>
> Could you post a link to Startable? I'm on the road with sporadic
> connectivity.
>
> Thanks
>
> Dennis
>
> Sent from my iPhone
>
> > On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
> >
> > +1 Peter.
> >
> >
> > I've been going over the Exporter implementations and Exporter
> > interface.
> >
> > The Exporter interface specifies that export and unexport behaviour is
> > defined by the implementation.
> >
> > That means it's up to the implementation to specify whether a happens
> > before edge occures before export.
> >
> > I propose we document in Exporter, that if export is called during
> > object construction then that object's implementation cannot safely
> > use final fields.
> >
> > We should also provide a @see reference to Startable to reccommend its
> > use when final fields are desirable and include Startable in the spec.
> >
> > We don't need to rewrite the example of exporting in net.jini.config
> > as export isn't performed during construction.
> >
> > This is a minor update to the Jini Specification.
> >
> > The FINE log message is an implementation detail and won't be included.
> >
> > Regards,
> >
> > Peter.
> >
> > ----- Original message -----
> > > No, an effectively immutable field is one whose reference doesn't
> > > change after the service objects safe publication.
> > >
> > > An immutable field is final.
> > >
> > > It would be Jeri's responsibility to safely publish the service
> > > during export to ensure a happens before event occurs, ensuring all
> > > non volatile fields the service contains are safely published and
> > > visible.
> > >
> > > It's possible to have mutable fields guarded by synchronized
> > > methods, or volatile fields.
> > >
> > > But it isn't possible to have final fields when exporting within the
> > > constructor.
> > >
> > > If the service contains effectively immutable field references, it
> > > must be safely published or if another thread accesses those fields
> > > through unsynchronized methods, the thread may only see their
> > > default value.
> > >
> > > All our service implementations have final fields. All our
> > > services had some unsynchronized access to non volatile, non final
> > > fields.
> > >
> > > Trouble is, when you can't have final fields, it's very difficult to
> > > determine which fields are effectively immutable and the original
> > > programmers design intent, if there's no comment.
> > >
> > > Our service implementations all had unsynchronized init() methods
> > > called from with the constructor that started threads and exported
> > > Some fields were final.
> > >
> > > For me, a service that contains final fields and is exported after
> > > construction is easier to maintain and understand.
> > >
> > > I suspect, like our Jini service imlementations, most people get it
> > > wrong most of the time.
> > >
> > > I get that others won't want to use this method, however it would be
> > > detrimental if we don't have such a method for those who would.
> > >
> > > Regards,
> > >
> > > Peter.
> > >
> > > ----- Original message -----
> > > >
> > > > I feel like we’re going down a rabbit hole here when you start
> > > > talking about exporting immutable objects. Wouldn’t it be kind
> > > > of silly to export an immutable service? Isn’t the whole point
> > > > that you interact with the service (i.e. alter its state) over a
> > > > remote interface?
> > > >
> > > > Perhaps it’s better to say that exported services need to be
> > > > thread-safe (which should be fairly obvious). Yes, immutable
> > > > objects are inherently thread-safe, so for sharing information
> > > > inside a VM, it makes some sense to minimize the number of mutable
> > > > objects you pass around. But fundamentally, we’re talking about
> > > > shared-state systems here.
> > > >
> > > > Cheers,
> > > >
> > > > Greg.
> > > >
> > > > > On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
> > > > >
> > > > > Pat your comment about non final fields is interesting.
> > > > >
> > > > > Isn't it also the case that we need to safely publish an
> > > > > effectively immutable object to share it among threads? That
> > > > > usually means copying it to a thread safe collection or shared
> > > > > via a synchronized method, volatile field, or final field in
> > > > > another object?
> > > > >
> > > > > So we should also make sure that Jeri uses safe publication
> > > > > during export.
> > > > >
> > > > > That would allow a service that has no final fields to start
> > > > > threads, then export from within a constructor safely, provided
> > > > > all operations on non final fields happen before starting
> > > > > threads and exporting.
> > > > >
> > > > > All our services have final fields, so Starter is more
> > > > > appropriate for River's own services.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Peter.
> > > > >
> > > > > ----- Original message -----
> > > > > > Hmm, good point, Startable, makes more sense.
> > > > > >
> > > > > > An object can be exported using Startable.
> > > > > >
> > > > > > I think we should have a policy to strongly discourage
> > > > > > exporting from constructors.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Peter.
> > > > > >
> > > > > > ----- Original message -----
> > > > > > > As far as I can tell, the special properties of completing a
> > > > > > > constructor in the JLS memory model are:
> > > > > > >
> > > > > > > 1. A happens-before edge from the end of the constructor to
> > > > > > > the start of a finalizer. (17.4.5)
> > > > > > >
> > > > > > > 2. The guarantee that any thread that only sees a reference
> > > > > > > to an object after the end of the constructor will see
> > > > > > > the correctly initialized values of all final fields.
> > > > > > > (17.5)
> > > > > > >
> > > > > > > The special issue with final fields is that implementations
> > > > > > > have freedom to optimize access to final fields in ways
> > > > > > > that are not permitted for non-final fields. Strategies
> > > > > > > for thread safety that work for non-final fields do not
> > > > > > > necessarily work for final fields. The requirement for
> > > > > > > final field safety is that the constructor end before
> > > > > > > another thread accesses the newly constructed object.
> > > > > > >
> > > > > > > Calling a start() method after construction if the class
> > > > > > > implements a new interface seems to me to be harmless,
> > > > > > > backwards compatible, and useful. It enables the simplest
> > > > > > > and most direct way of preventing access to the new
> > > > > > > object by another thread during construction.
> > > > > > >
> > > > > > > The roadmap issue is whether it should be required, and if
> > > > > > > so the level of enforcement. For example, there is no
> > > > > > > reason to require it if the class does not declare any
> > > > > > > final fields.
> > > > > > >
> > > > > > > Incidentally, and as a detail, "Commission" does not
> > > > > > > immediately make me think of having a start() method
> > > > > > > that should be called after construction. If you do go
> > > > > > > this way, the name needs thought. "Startable" would be
> > > > > > > more obvious, more memorable, more likely to be found on
> > > > > > > searches, and more compatible with familiar interface names
> > > > > > > such as "Cloneable" and "Iterable".
> > > > > > >
> > > > > > > Patricia
> > > > > > >
> > > > > > >
> > > > > > > > On 12/18/2013 2:18 AM, Peter wrote:
> > > > > > > > Well, now seems like a good time to have the conversation.
> > > > > > > >
> > > > > > > > Yes there are other ways, but I haven't seen one safe
> > > > > > > > implementation yet, so...
> > > > > > > >
> > > > > > > > Does someone have a better way to solve this problem, has
> > > > > > > > someone already solved this problem I'm unaware of that we
> > > > > > > > can adopt, or is there a way that's more satisfactory?
> > > > > > > >
> > > > > > > > If not, is there something objectionable with the
> > > > > > > > Commission interface and if so, how can we fix it?
> > > > > > > >
> > > > > > > > The SEVERE log message is logged by the River start
> > > > > > > > package, other containers or frameworks can choose whether
> > > > > > > > or not to do so, but I'd encourage them to do something
> > > > > > > > similar, yes we can change it to WARN.
> > > > > > > >
> > > > > > > > A much harsher option is to throw an exception during
> > > > > > > > export which breaks backward compatibility.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Peter.
> > > > > > > >
> > > > > > > > ----- Original message -----
> > > > > > > > > "org.apache.river.api.util.Commission is an interface
> > > > > > > > > services should implement"
> > > > > > > > >
> > > > > > > > > If it's a SHOULD, not a MUST, chucking out a SEVERE is
> > > > > > > > > incorrect logger behaviour IMO. You could issue a WARN
> > > > > > > > > if you like but for even that I'd say you need to
> > > > > > > > > provide a roadmap explaining why the warning and what
> > > > > > > > > you intend to do in future and what you expect of
> > > > > > > > > service writers such as myself.
> > > > > > > > >
> > > > > > > > > Commission, at least from my point of view, is your means
> > > > > > > > > (maybe the River community's - did you ask us?) for
> > > > > > > > > satisfying your needs in respect of the JMM. As we've
> > > > > > > > > discussed previously, there are other ways too and they
> > > > > > > > > work and they are safe if you know what you're doing.
> > > > > > > > > Your contention was that most don't know what they're
> > > > > > > > > doing hence, presumably, Commission.
> > > > > > > > >
> > > > > > > > > So the thing is, you are seemingly on a road to asserting
> > > > > > > > > more structure (gosh, a standard?) on the way people
> > > > > > > > > write their services. If so, you'd best start flagging
> > > > > > > > > that honestly and openly via a roadmap, deprecation and
> > > > > > > > > such/whatever rather than sticking out logger messages
> > > > > > > > > with no clear guidance and at the cost of a certain
> > > > > > > > > amount of nuisance (no admin I know likes SEVERE's being
> > > > > > > > > logged for something which isn't critical cos it's noise
> > > > > > > > > they don't want in log files).
> > > > > > > > >
> > > > > > > > > And of course, we all know that when some entity asserts
> > > > > > > > > a standard or requirement on others for entry, they may
> > > > > > > > > choose not to enter. Does this help your community or
> > > > > > > > > hinder it? The answer to that is, it depends. On what?
> > > > > > > > > Have you asked or tested? How have you tested? What
> > > > > > > > > would be considered validation or lack of support?
> > > > > > > > >
> > > > > > > > > I am not out to flame or troll rather I want to see this
> > > > > > > > > community demonstrating good behaviour and I'm not
> > > > > > > > > feeling like what's going on around Commission (what is
> > > > > > > > > that big change in version number really saying?) is
> > > > > > > > > such.
> > > > > > > > >
> > > > > > > > > > On 18 December 2013 08:52, Peter <ji...@zeus.net.au>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Just to clarify org.apache.river.api.util.Commission
> > > > > > > > > > is an interface services should implement, I would
> > > > > > > > > > encourage all container projects to pick up the
> > > > > > > > > > interface and make suggestions for improvement if
> > > > > > > > > > there are any issues.
> > > > > > > > > >
> > > > > > > > > > Interface Commission {
> > > > > > > > > > void start () throws Exception;
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > It's called after JMM safe construction to allow the
> > > > > > > > > > service to start any threads and be exported.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Peter.
> > > > > > > > > >
> > > > > > > > > > ----- Original message -----
> > > > > > > > > > >
> > > > > > > > > > > The way that services are instantiated and setup is
> > > > > > > > > > > an implementation detail. When I
> > > > > > > > > > > think of compatibility I think of the API and the
> > > > > > > > > > > lookup methods. We think of compatibility from a
> > > > > > > > > > > client point of view. From the
> > > > > > > > > > > client point of view, using a service looks like
> > > > > > > > > > > this:
> > > > > > > > > > >
> > > > > > > > > > > - Use multicast of unicast discovery to find one or
> > > > > > > > > > > more ServiceRegistrar instances
> > > > > > > > > > > - Call lookup(…) on one or more of these
> > > > > > > > > > > instances to get a set of service candidates -
> > > > > > > > > > > Choose a candidate and prepare() it using a
> > > > > > > > > > > ProxyPreparer, to yield a usable service proxy.
> > > > > > > > > > > - Make calls on it.
> > > > > > > > > > > Ideally hang on to this proxy instance, so
> > > > > > > > > > > you can skip the discovery and lookup next time you
> > > > > > > > > > > need it. - If the call
> > > > > > > > > > > fails, repeat the lookup (and possibly discovery)
> > > > > > > > > > > til you get a proxy that works.
> > > > > > > > > > >
> > > > > > > > > > > Nowhere does the client need to know whether the
> > > > > > > > > > > service instance is started up using the
> > > > > > > > > > > “com.sun.jini.start” mechanism, your Commission
> > > > > > > > > > > interface, some other IOC container (Rio, Harvester,
> > > > > > > > > > > Seven or RiverContainer) or some unknown mechanism
> > > > > > > > > > > that starts with a static main() method.
> > > > > > > > > > >
> > > > > > > > > > > JSK2.0 was 2.0 because of the introduction of the
> > > > > > > > > > > proxy verification mechanisms, as well as JERI.
> > > > > > > > > > > Absent some new client usage mechanism,
> > > > > > > > > > > River doesn’t need to go to 3.0.
> > > > > > > > > > >
> > > > > > > > > > > Cheers,
> > > > > > > > > > >
> > > > > > > > > > > Greg.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Dec 17, 2013, at 1:58 PM, Peter <ji...@zeus.net.au>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > I think changing services to use safe construction
> > > > > > > > > > > > techniques is enough to cause the version jump.
> > > > > > > > > > > >
> > > > > > > > > > > > At this point I've allowed services to continue
> > > > > > > > > > > > unsafe construction practices, while logging a
> > > > > > > > > > > > SEVERE warning when the Commission interface isn't
> > > > > > > > > > > > implemented, rather than fail.
> > > > > > > > > > > >
> > > > > > > > > > > > This is a fundamental change to the way services
> > > > > > > > > > > > are written.
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > >
> > > > > > > > > > > > Peter.
> > > > > > > > > > > >
> > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > >
> > > > > > > > > > > > > Assuming that there aren’t major
> > > > > > > > > > > > > incompatibilities, I think that would be a
> > > > > > > > > > > > > “minor” version change according to our
> > > > > > > > > > > > > versioning policy, so we’d be looking at the
> > > > > > > > > > > > > “2.3” branch rather than a “3.0” release.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I’m still unnerved by the massive amounts of
> > > > > > > > > > > > > changes to both code and tests in the
> > > > > > > > > > > > > qa_refactor branch, as well as the apparent
> > > > > > > > > > > > > instability of the code, although that seems to
> > > > > > > > > > > > > be improving. In the next few
> > > > > > > > > > > > > weeks I’m going to try and setup a cross-test
> > > > > > > > > > > > > case, to see what the “2.2” tests say about the
> > > > > > > > > > > > > potential “2.3” release and vice-versa.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I think what I’d really like to see is an
> > > > > > > > > > > > > incremental approach where we update limited
> > > > > > > > > > > > > components of the “2.2” branch, one at a time.
> > > > > > > > > > > > > Is there anything that we could pull out
> > > > > > > > > > > > > piecemeal? Maybe it
> > > > > > > > > > would
> > > > > > > > > > > > > make sense to split out the infrastructure
> > > > > > > > > > > > > services, like Reggie, Mahalo, and Outrigger
> > > > > > > > > > > > > into different sub-projects that could be
> > > > > > > > > > > > > updated separately?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Greg.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Dec 17, 2013, at 5:03 AM, Peter
> > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > When the qa_refactor branch stabilises, I plan
> > > > > > > > > > > > > > to merge trunk and provide a beta release for
> > > > > > > > > > > > > > client compatibility testing.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Changes made have been focused on making our
> > > > > > > > > > > > > > code thread safe, there are significant changes
> > > > > > > > > > > > > > internally, the public api remains focused on
> > > > > > > > > > > > > > backward compatibility, however it is advisable
> > > > > > > > > > > > > > that client services adopt new safe
> > > > > > > > > > > > > > construction techniques for services and
> > > > > > > > > > > > > > implement the new Commission interface.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > What's a suitable test period for client
> > > > > > > > > > > > > > testing?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Peter.
> >
Re: VOTE: add Startable to Jini specification
Posted by Dennis Reedy <de...@gmail.com>.
Hi Peter,
Could you post a link to Startable? I'm on the road with sporadic connectivity.
Thanks
Dennis
Sent from my iPhone
> On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
>
> +1 Peter.
>
>
> I've been going over the Exporter implementations and Exporter interface.
>
> The Exporter interface specifies that export and unexport behaviour is defined by the implementation.
>
> That means it's up to the implementation to specify whether a happens before edge occures before export.
>
> I propose we document in Exporter, that if export is called during object construction then that object's implementation cannot safely use final fields.
>
> We should also provide a @see reference to Startable to reccommend its use when final fields are desirable and include Startable in the spec.
>
> We don't need to rewrite the example of exporting in net.jini.config as export isn't performed during construction.
>
> This is a minor update to the Jini Specification.
>
> The FINE log message is an implementation detail and won't be included.
>
> Regards,
>
> Peter.
>
> ----- Original message -----
>> No, an effectively immutable field is one whose reference doesn't change
>> after the service objects safe publication.
>>
>> An immutable field is final.
>>
>> It would be Jeri's responsibility to safely publish the service during
>> export to ensure a happens before event occurs, ensuring all non
>> volatile fields the service contains are safely published and visible.
>>
>> It's possible to have mutable fields guarded by synchronized methods, or
>> volatile fields.
>>
>> But it isn't possible to have final fields when exporting within the
>> constructor.
>>
>> If the service contains effectively immutable field references, it must
>> be safely published or if another thread accesses those fields through
>> unsynchronized methods, the thread may only see their default value.
>>
>> All our service implementations have final fields. All our services had
>> some unsynchronized access to non volatile, non final fields.
>>
>> Trouble is, when you can't have final fields, it's very difficult to
>> determine which fields are effectively immutable and the original
>> programmers design intent, if there's no comment.
>>
>> Our service implementations all had unsynchronized init() methods called
>> from with the constructor that started threads and exported Some fields
>> were final.
>>
>> For me, a service that contains final fields and is exported after
>> construction is easier to maintain and understand.
>>
>> I suspect, like our Jini service imlementations, most people get it
>> wrong most of the time.
>>
>> I get that others won't want to use this method, however it would be
>> detrimental if we don't have such a method for those who would.
>>
>> Regards,
>>
>> Peter.
>>
>> ----- Original message -----
>>>
>>> I feel like we’re going down a rabbit hole here when you start talking
>>> about exporting immutable objects. Wouldn’t it be kind of silly to
>>> export an immutable service? Isn’t the whole point that you interact
>>> with the service (i.e. alter its state) over a remote interface?
>>>
>>> Perhaps it’s better to say that exported services need to be
>>> thread-safe (which should be fairly obvious). Yes, immutable objects
>>> are inherently thread-safe, so for sharing information inside a VM, it
>>> makes some sense to minimize the number of mutable objects you pass
>>> around. But fundamentally, we’re talking about shared-state systems
>>> here.
>>>
>>> Cheers,
>>>
>>> Greg.
>>>
>>>> On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
>>>>
>>>> Pat your comment about non final fields is interesting.
>>>>
>>>> Isn't it also the case that we need to safely publish an effectively
>>>> immutable object to share it among threads? That usually means
>>>> copying it to a thread safe collection or shared via a synchronized
>>>> method, volatile field, or final field in another object?
>>>>
>>>> So we should also make sure that Jeri uses safe publication during
>>>> export.
>>>>
>>>> That would allow a service that has no final fields to start threads,
>>>> then export from within a constructor safely, provided all operations
>>>> on non final fields happen before starting threads and exporting.
>>>>
>>>> All our services have final fields, so Starter is more appropriate
>>>> for River's own services.
>>>>
>>>> Regards,
>>>>
>>>> Peter.
>>>>
>>>> ----- Original message -----
>>>>> Hmm, good point, Startable, makes more sense.
>>>>>
>>>>> An object can be exported using Startable.
>>>>>
>>>>> I think we should have a policy to strongly discourage exporting
>>>>> from constructors.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Peter.
>>>>>
>>>>> ----- Original message -----
>>>>>> As far as I can tell, the special properties of completing a
>>>>>> constructor in the JLS memory model are:
>>>>>>
>>>>>> 1. A happens-before edge from the end of the constructor to the
>>>>>> start of a finalizer. (17.4.5)
>>>>>>
>>>>>> 2. The guarantee that any thread that only sees a reference to an
>>>>>> object after the end of the constructor will see the
>>>>>> correctly initialized values of all final fields. (17.5)
>>>>>>
>>>>>> The special issue with final fields is that implementations have
>>>>>> freedom to optimize access to final fields in ways that are
>>>>>> not permitted for non-final fields. Strategies for thread
>>>>>> safety that work for non-final fields do not necessarily
>>>>>> work for final fields. The requirement for final field
>>>>>> safety is that the constructor end before another thread
>>>>>> accesses the newly constructed object.
>>>>>>
>>>>>> Calling a start() method after construction if the class
>>>>>> implements a new interface seems to me to be harmless,
>>>>>> backwards compatible, and useful. It enables the simplest and
>>>>>> most direct way of preventing access to the new object by
>>>>>> another thread during construction.
>>>>>>
>>>>>> The roadmap issue is whether it should be required, and if so the
>>>>>> level of enforcement. For example, there is no reason to
>>>>>> require it if the class does not declare any final fields.
>>>>>>
>>>>>> Incidentally, and as a detail, "Commission" does not immediately
>>>>>> make me think of having a start() method that should be
>>>>>> called after construction. If you do go this way, the name
>>>>>> needs thought. "Startable" would be more obvious, more
>>>>>> memorable, more likely to be found on searches, and more
>>>>>> compatible with familiar interface names such as "Cloneable"
>>>>>> and "Iterable".
>>>>>>
>>>>>> Patricia
>>>>>>
>>>>>>
>>>>>>> On 12/18/2013 2:18 AM, Peter wrote:
>>>>>>> Well, now seems like a good time to have the conversation.
>>>>>>>
>>>>>>> Yes there are other ways, but I haven't seen one safe
>>>>>>> implementation yet, so...
>>>>>>>
>>>>>>> Does someone have a better way to solve this problem, has
>>>>>>> someone already solved this problem I'm unaware of that we can
>>>>>>> adopt, or is there a way that's more satisfactory?
>>>>>>>
>>>>>>> If not, is there something objectionable with the Commission
>>>>>>> interface and if so, how can we fix it?
>>>>>>>
>>>>>>> The SEVERE log message is logged by the River start package,
>>>>>>> other containers or frameworks can choose whether or not to do
>>>>>>> so, but I'd encourage them to do something similar, yes we can
>>>>>>> change it to WARN.
>>>>>>>
>>>>>>> A much harsher option is to throw an exception during export
>>>>>>> which breaks backward compatibility.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>> ----- Original message -----
>>>>>>>> "org.apache.river.api.util.Commission is an interface
>>>>>>>> services should implement"
>>>>>>>>
>>>>>>>> If it's a SHOULD, not a MUST, chucking out a SEVERE is
>>>>>>>> incorrect logger behaviour IMO. You could issue a WARN if you
>>>>>>>> like but for even that I'd say you need to provide a roadmap
>>>>>>>> explaining why the warning and what you intend to do in
>>>>>>>> future and what you expect of service writers such as myself.
>>>>>>>>
>>>>>>>> Commission, at least from my point of view, is your means
>>>>>>>> (maybe the River community's - did you ask us?) for
>>>>>>>> satisfying your needs in respect of the JMM. As we've
>>>>>>>> discussed previously, there are other ways too and they work
>>>>>>>> and they are safe if you know what you're doing. Your
>>>>>>>> contention was that most don't know what they're doing
>>>>>>>> hence, presumably, Commission.
>>>>>>>>
>>>>>>>> So the thing is, you are seemingly on a road to asserting
>>>>>>>> more structure (gosh, a standard?) on the way people write
>>>>>>>> their services. If so, you'd best start flagging that
>>>>>>>> honestly and openly via a roadmap, deprecation and
>>>>>>>> such/whatever rather than sticking out logger messages with
>>>>>>>> no clear guidance and at the cost of a certain amount of
>>>>>>>> nuisance (no admin I know likes SEVERE's being logged for
>>>>>>>> something which isn't critical cos it's noise they don't
>>>>>>>> want in log files).
>>>>>>>>
>>>>>>>> And of course, we all know that when some entity asserts a
>>>>>>>> standard or requirement on others for entry, they may choose
>>>>>>>> not to enter. Does this help your community or hinder it? The
>>>>>>>> answer to that is, it depends. On what? Have you asked or
>>>>>>>> tested? How have you tested? What would be considered
>>>>>>>> validation or lack of support?
>>>>>>>>
>>>>>>>> I am not out to flame or troll rather I want to see this
>>>>>>>> community demonstrating good behaviour and I'm not feeling
>>>>>>>> like what's going on around Commission (what is that big
>>>>>>>> change in version number really saying?) is such.
>>>>>>>>
>>>>>>>>> On 18 December 2013 08:52, Peter <ji...@zeus.net.au> wrote:
>>>>>>>>>
>>>>>>>>> Just to clarify org.apache.river.api.util.Commission is an
>>>>>>>>> interface services should implement, I would encourage all
>>>>>>>>> container projects to pick up the interface and make
>>>>>>>>> suggestions for improvement if there are any issues.
>>>>>>>>>
>>>>>>>>> Interface Commission {
>>>>>>>>> void start () throws Exception;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> It's called after JMM safe construction to allow the
>>>>>>>>> service to start any threads and be exported.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Peter.
>>>>>>>>>
>>>>>>>>> ----- Original message -----
>>>>>>>>>>
>>>>>>>>>> The way that services are instantiated and setup is an
>>>>>>>>>> implementation detail. When I think of
>>>>>>>>>> compatibility I think of the API and the lookup methods.
>>>>>>>>>> We think of compatibility from a
>>>>>>>>>> client point of view. From the client
>>>>>>>>>> point of view, using a service looks like this:
>>>>>>>>>>
>>>>>>>>>> - Use multicast of unicast discovery to find one or more
>>>>>>>>>> ServiceRegistrar instances -
>>>>>>>>>> Call lookup(…) on one or more of these instances to get
>>>>>>>>>> a set of service candidates - Choose a candidate and
>>>>>>>>>> prepare() it using a ProxyPreparer, to yield a usable
>>>>>>>>>> service proxy. - Make calls on
>>>>>>>>>> it. Ideally hang on to this proxy
>>>>>>>>>> instance, so you can skip the discovery and lookup next
>>>>>>>>>> time you need it. - If the
>>>>>>>>>> call fails, repeat the lookup (and possibly discovery)
>>>>>>>>>> til you get a proxy that works.
>>>>>>>>>>
>>>>>>>>>> Nowhere does the client need to know whether the service
>>>>>>>>>> instance is started up using the “com.sun.jini.start”
>>>>>>>>>> mechanism, your Commission interface, some other IOC
>>>>>>>>>> container (Rio, Harvester, Seven or RiverContainer) or
>>>>>>>>>> some unknown mechanism that starts with a static main()
>>>>>>>>>> method.
>>>>>>>>>>
>>>>>>>>>> JSK2.0 was 2.0 because of the introduction of the proxy
>>>>>>>>>> verification mechanisms, as well as JERI.
>>>>>>>>>> Absent some new client usage mechanism, River doesn’t
>>>>>>>>>> need to go to 3.0.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Greg.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Dec 17, 2013, at 1:58 PM, Peter <ji...@zeus.net.au>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I think changing services to use safe construction
>>>>>>>>>>> techniques is enough to cause the version jump.
>>>>>>>>>>>
>>>>>>>>>>> At this point I've allowed services to continue unsafe
>>>>>>>>>>> construction practices, while logging a SEVERE warning
>>>>>>>>>>> when the Commission interface isn't implemented, rather
>>>>>>>>>>> than fail.
>>>>>>>>>>>
>>>>>>>>>>> This is a fundamental change to the way services are
>>>>>>>>>>> written.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Peter.
>>>>>>>>>>>
>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>>
>>>>>>>>>>>> Assuming that there aren’t major incompatibilities, I
>>>>>>>>>>>> think that would be a “minor” version change
>>>>>>>>>>>> according to our versioning policy, so we’d be
>>>>>>>>>>>> looking at the “2.3” branch rather than a “3.0”
>>>>>>>>>>>> release.
>>>>>>>>>>>>
>>>>>>>>>>>> I’m still unnerved by the massive amounts of changes
>>>>>>>>>>>> to both code and tests in the qa_refactor branch, as
>>>>>>>>>>>> well as the apparent instability of the code,
>>>>>>>>>>>> although that seems to be improving.
>>>>>>>>>>>> In the next few weeks I’m going
>>>>>>>>>>>> to try and setup a cross-test case, to see what the
>>>>>>>>>>>> “2.2” tests say about the potential “2.3” release
>>>>>>>>>>>> and vice-versa.
>>>>>>>>>>>>
>>>>>>>>>>>> I think what I’d really like to see is an incremental
>>>>>>>>>>>> approach where we update limited components of the
>>>>>>>>>>>> “2.2” branch, one at a time. Is there anything that
>>>>>>>>>>>> we could pull out piecemeal? Maybe it
>>>>>>>>> would
>>>>>>>>>>>> make sense to split out the infrastructure services,
>>>>>>>>>>>> like Reggie, Mahalo, and Outrigger into different
>>>>>>>>>>>> sub-projects that could be updated separately?
>>>>>>>>>>>>
>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> Greg.
>>>>>>>>>>>>
>>>>>>>>>>>> On Dec 17, 2013, at 5:03 AM, Peter <ji...@zeus.net.au>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> When the qa_refactor branch stabilises, I plan to
>>>>>>>>>>>>> merge trunk and provide a beta release for client
>>>>>>>>>>>>> compatibility testing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Changes made have been focused on making our code
>>>>>>>>>>>>> thread safe, there are significant changes
>>>>>>>>>>>>> internally, the public api remains focused on
>>>>>>>>>>>>> backward compatibility, however it is advisable
>>>>>>>>>>>>> that client services adopt new safe construction
>>>>>>>>>>>>> techniques for services and implement the new
>>>>>>>>>>>>> Commission interface.
>>>>>>>>>>>>>
>>>>>>>>>>>>> What's a suitable test period for client testing?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Peter.
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
----- Original message -----
> On 12/19/2013 6:18 PM, Greg Trasuk wrote:
<SNIP>
> ...
> > Even talking about performance is
> > not very helpful until the average corporate coder at the City of
> > Wherever can actually use the product.
> >
>
> The last sentence quoted above is basically the problem I had with
> working on River performance.
>
> I'm a veteran of many performance campaigns, but every campaign has
> started from either a specific customer problem, or a benchmark that
> would affect how many computers we could sell. That meant I always had a
> workload I could run, but that needed to run faster.
>
> In that situation, performance changes can target real bottlenecks, and
> each could be tested to see if it was justified by a sufficient gain in
> performance.
>
> Patricia
Don't worry about performance, there's plenty more correctness required :)
Although I have been guilty of fixing some test hot spots, but not at the expense of correctness.
Regards,
Peter.
Re: VOTE: add Startable to Jini specification
Posted by Patricia Shanahan <pa...@acm.org>.
On 12/19/2013 6:18 PM, Greg Trasuk wrote:
...
> I guess that’s one of my frustrations with all this “refactoring” of
> the existing codebase - if it doesn’t make it easier for people to
> use Jini/River to create their own service-oriented architectures,
> then it just seems like a lot of navel-gazing. Similarly, we’ve had
> statements about reorganizing the source code to make it easier to
> develop with. To which I ask, “when was the last time anyone worried
> about the source code of Tomcat, or Websphere, or Weblogic?, or
> whether they used Maven or not?”. Even talking about performance is
> not very helpful until the average corporate coder at the City of
> Wherever can actually use the product.
>
The last sentence quoted above is basically the problem I had with
working on River performance.
I'm a veteran of many performance campaigns, but every campaign has
started from either a specific customer problem, or a benchmark that
would affect how many computers we could sell. That meant I always had a
workload I could run, but that needed to run faster.
In that situation, performance changes can target real bottlenecks, and
each could be tested to see if it was justified by a sufficient gain in
performance.
Patricia
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
Pat,
I'd like to invite you back, even if it's just in an advisory capacity.
I think we can all benefit from your experience.
So far you've managed to explain everything so eloquently, the rest of us look like children fighting over toys.
Regards,
Peter.
----- Original message -----
> On 12/20/2013 1:25 AM, Dan Creswell wrote:
> ...
> > And "serious flaw" - really? How might you judge that as the case?
> > Have we got a serious service-startup problem? Jini 2.x doesn't seem
> > to have such an issue even with the "serious flaw" you claim? Perhaps
> > this "serious flaw" has come into being because of all the other work
> > you've done since Jini 2.x (and I'm not saying the work isn't
> > valuable)? If that is so, perhaps that is because you have changed the
> > contract against which a service is written? If so, I would suggest
> > you would be having a vote on modifying the contract not adding an
> > interface.
> >
> ...
>
> I think there is a big picture question that underlies all this. There
> are two, potentially complementary, approaches to thread safety,
> experiment and theory. Each has advantages and disadvantages.
>
> In the pure experimental approach, the only thread safety issues that
> require change are those that manifest as bugs in some known test. The
> upside is that one can do strictly test driven development, where every
> change fixes a test failure. The downside is that the resulting code can
> be fragile - an apparently irrelevant change can shift a latent bug to
> causing a test failure. The code may run fine on small, lightly loaded
> systems, but fail on larger systems especially if there is contention
> for processors and/or memory.
>
> In a theory driven approach, the objective is to make the code
> thread-safe by design. It requires changes to conform, in the case of
> Java, to the JMM, regardless of whether the issue manifests as a test
> failure. The upside is that the resulting code should be more robust -
> mere timing changes, or shifts in the locations of page and cache line
> boundaries, cannot cause failures. The downside is that it requires
> changes that are not necessarily supported by a failing test case.
>
> My own reading of River code convinced me that it was largely in an
> experimental state, with multi-thread issues that were not being fixed
> because they did not cause known test failures. Comments like `Jini 2.x
> doesn't seem to have such an issue even with the "serious flaw" you
> claim?` are typical of an experimental approach.
>
> As I understand the big picture, Peter is pushing a change to a more
> theory-based approach.
>
> Shifting the balance from experimental towards theory involves a risk of
> cascading problems because making one change that affects the
> relationships between threads can open up a sequence of events that
> triggers an apparently unrelated bug.
>
> I think the PMC should discuss and decide on whether to make the change
> to the theory based approach. If I were still a PMC member I would be
> strongly in favor, but it is not up to me. A lot of decisions, including
> this vote, would be obvious and non-controversial in the light of a
> definite policy, either way, on that issue.
>
> Patricia
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
No problem, don't beat yourself up about it.
----- Original message -----
>
> Peter - I apologize. I had meant to keep my suggestion to cool down
> off-list, but it appears that I failed to remove dev@river.apache.org
> from the recipients when I replied.
>
> Cheers,
>
> Greg.
>
> On Dec 21, 2013, at 3:50 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>
> > I think the real problem is some people haven't been reading up on
> > concurrency and are insufficiently informed to be able to properly
> > discuss the issue.
> >
> > When I don't know something, I either ask someone who does, or I keep
> > my mouth shut so as not to look like a fool.
> >
> > It's more a case of, if you behave like a fool long enough to make it
> > frustrating, that's exactly how you'll be treated.
> >
> > Sadly it's by people who should know better, who while they may lack
> > an understanding of the JMM are still very skilled programmers, that's
> > what makes it frustrating.
> >
> > Final fields are made visible after construction completes, so when
> > other threads receive a copy of the object in an unconstructed state,
> > like they do during export in a constructor, then those threads can
> > continue to see the incomplete object, rather than the fully
> > constructed one.
> >
> > See for yourselves, look at all the final fields in Mercury, then
> > realise that it's reference escaped during construction. The other
> > services are also guilty.
> >
> > That also means that all the remote method invocations wrapped up as
> > runnable tasks and executed on Mercury's methods in an executor pool
> > will have also seen parts of Mercury in a partially constructed state.
> >
> > http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/com/sun/jini/mercury/MailboxImpl.java?r1=545310&r2=1552606
> >
> > If I haven't convinced you, read up, spend some time on the
> > concurrency interest mail list or ask someone you trust who does know.
> >
> > Regards,
> >
> > Peter.
> >
> > On 21/12/2013 11:10 AM, Greg Trasuk wrote:
> > > (off-list)
> > >
> > > Peter:
> > >
> > > You’re sounding unprofessional, and you’re missing the fact that
> > > people are giving reasonable and well-thought-out feedback. You
> > > seem to be taking the conversation personally. Perhaps you should
> > > take a few hours off the list and cool down. Things will still be
> > > here when you’ve regained composure.
> > >
> > > Best regards,
> > >
> > > Greg.
> > >
> > > On Dec 20, 2013, at 7:28 PM, Peter<ji...@zeus.net.au> wrote:
> > >
> > > > You just did.
> > > >
> > > > The blue pill.
> > > >
> > > > I prefer the truth, it's easier in the long run.
> > > >
> > > > Feel free to cast your vote.
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > > > Peter,
> > > > >
> > > > > There's so many things wrong in there, I'm not even going to
> > > > > dignify this with a response.
> >
>
Re: VOTE: add Startable to Jini specification
Posted by Greg Trasuk <tr...@stratuscom.com>.
Peter - I apologize. I had meant to keep my suggestion to cool down off-list, but it appears that I failed to remove dev@river.apache.org from the recipients when I replied.
Cheers,
Greg.
On Dec 21, 2013, at 3:50 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> I think the real problem is some people haven't been reading up on concurrency and are insufficiently informed to be able to properly discuss the issue.
>
> When I don't know something, I either ask someone who does, or I keep my mouth shut so as not to look like a fool.
>
> It's more a case of, if you behave like a fool long enough to make it frustrating, that's exactly how you'll be treated.
>
> Sadly it's by people who should know better, who while they may lack an understanding of the JMM are still very skilled programmers, that's what makes it frustrating.
>
> Final fields are made visible after construction completes, so when other threads receive a copy of the object in an unconstructed state, like they do during export in a constructor, then those threads can continue to see the incomplete object, rather than the fully constructed one.
>
> See for yourselves, look at all the final fields in Mercury, then realise that it's reference escaped during construction. The other services are also guilty.
>
> That also means that all the remote method invocations wrapped up as runnable tasks and executed on Mercury's methods in an executor pool will have also seen parts of Mercury in a partially constructed state.
>
> http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/com/sun/jini/mercury/MailboxImpl.java?r1=545310&r2=1552606
>
> If I haven't convinced you, read up, spend some time on the concurrency interest mail list or ask someone you trust who does know.
>
> Regards,
>
> Peter.
>
> On 21/12/2013 11:10 AM, Greg Trasuk wrote:
>> (off-list)
>>
>> Peter:
>>
>> You’re sounding unprofessional, and you’re missing the fact that people are giving reasonable and well-thought-out feedback. You seem to be taking the conversation personally. Perhaps you should take a few hours off the list and cool down. Things will still be here when you’ve regained composure.
>>
>> Best regards,
>>
>> Greg.
>>
>> On Dec 20, 2013, at 7:28 PM, Peter<ji...@zeus.net.au> wrote:
>>
>>> You just did.
>>>
>>> The blue pill.
>>>
>>> I prefer the truth, it's easier in the long run.
>>>
>>> Feel free to cast your vote.
>>>
>>> Peter.
>>>
>>> ----- Original message -----
>>>> Peter,
>>>>
>>>> There's so many things wrong in there, I'm not even going to dignify this
>>>> with a response.
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
Thank you.
----- Original message -----
> I'll try. I really think there are three strategies that should be
> considered separately:
>
> 1. Fix concurrency bugs in River code, even if the bug can only be
> identified in theory, without causing any known external symptoms.
>
> 2. Externalize some or all of any facilities that are added to implement
> Strategy 1, so that users can apply them in their own code.
>
> 3. Log non-use of Strategy 2 facilities in order to encourage users to
> use them.
>
> I'm going to use Startable as an example to illustrate the differences
> between these strategies.
>
> Adding Startable to a River implementation package, and using it in
> River code, is a Strategy 1 item. It would not call for a separate vote,
> but would be voted on as part of a release candidate that implements
> Strategy 1.
>
> Putting Startable in a package users are expected to use directly,
> documenting it as part of River's external interface, is a Strategy 2
> item. It is an external interface change, and needs to be voted on
> accordingly.
>
> Logging non-use of Startable is a Strategy 3 item, and is also an
> external interface change.
>
> Patricia
>
>
> On 12/21/2013 3:11 AM, Peter wrote:
> > Yes it's a difficult decision, originally when I set out to fix the
> > issues I found, I didn't expect to be here today, on one hand, I'd
> > like to complete the work I started, I've invested too much time to
> > walk away now.
> >
> > On the other hand, "The community" may not want it fixed.
> >
> > Patricia, you're much better at communicating the development
> > concepts, can you do me a favour and put forward a vote proposal, it
> > would be beneficial to all parties to come to some decision on the
> > underlying issue.
> >
> > Regards,
> >
> > Peter.
> >
> >
> > ----- Original message -----
> > > I have not seen any evidence of lack of understanding. I believe
> > > there is a genuine lack of consensus on how to deal with some of
> > > these ordering issues.
> > >
> > > In particular, to what extent should River live with code that will
> > > almost always work?
> > >
> > > The damage from export-from-constructor with final fields comes if
> > > another thread references the object before it is fully constructed.
> > > Generally, the chain of instructions from the export to another
> > > thread seeing the reference is far longer than the chain of
> > > instructions from the export to the end of the constructor. Almost
> > > always, the constructor will finish first and the code will work.
> > >
> > > Personally, I would not want to depend on that, because page faults
> > > and operating system interrupts with resource contention can stretch
> > > out a few instructions to several milliseconds or even seconds.
> > >
> > > However, a project like River depends on reaching consensus and that
> > > in turn depends on discussing issues with respect for all points of
> > > view.
> > >
> > > Patricia
> > >
> > > On 12/21/2013 12:50 AM, Peter Firmstone wrote:
> > > > I think the real problem is some people haven't been reading up on
> > > > concurrency and are insufficiently informed to be able to properly
> > > > discuss the issue.
> > > >
> > > > When I don't know something, I either ask someone who does, or I
> > > > keep my mouth shut so as not to look like a fool.
> > > >
> > > > It's more a case of, if you behave like a fool long enough to make
> > > > it frustrating, that's exactly how you'll be treated.
> > > >
> > > > Sadly it's by people who should know better, who while they may
> > > > lack an understanding of the JMM are still very skilled
> > > > programmers, that's what makes it frustrating.
> > > >
> > > > Final fields are made visible after construction completes, so when
> > > > other threads receive a copy of the object in an unconstructed
> > > > state, like they do during export in a constructor, then those
> > > > threads can continue to see the incomplete object, rather than the
> > > > fully constructed one.
> > > >
> > > > See for yourselves, look at all the final fields in Mercury, then
> > > > realise that it's reference escaped during construction. The
> > > > other services are also guilty.
> > > >
> > > > That also means that all the remote method invocations wrapped up
> > > > as runnable tasks and executed on Mercury's methods in an executor
> > > > pool will have also seen parts of Mercury in a partially
> > > > constructed state.
> > > >
> > > > http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/com/sun/jini/mercury/MailboxImpl.java?r1=545310&r2=1552606
> > > >
> > > >
> > > > If I haven't convinced you, read up, spend some time on the
> > > > concurrency interest mail list or ask someone you trust who does
> > > > know.
> > > >
> > > > Regards,
> > > >
> > > > Peter.
> > > >
> > > > On 21/12/2013 11:10 AM, Greg Trasuk wrote:
> > > > > (off-list)
> > > > >
> > > > > Peter:
> > > > >
> > > > > You’re sounding unprofessional, and you’re missing the fact that
> > > > > people are giving reasonable and well-thought-out feedback.
> > > > > You seem to be taking the conversation personally. Perhaps
> > > > > you should take a few hours off the list and cool down.
> > > > > Things will still be here when you’ve regained composure.
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Greg.
> > > > >
> > > > > On Dec 20, 2013, at 7:28 PM, Peter<ji...@zeus.net.au> wrote:
> > > > >
> > > > > > You just did.
> > > > > >
> > > > > > The blue pill.
> > > > > >
> > > > > > I prefer the truth, it's easier in the long run.
> > > > > >
> > > > > > Feel free to cast your vote.
> > > > > >
> > > > > > Peter.
> > > > > >
> > > > > > ----- Original message -----
> > > > > > > Peter,
> > > > > > >
> > > > > > > There's so many things wrong in there, I'm not even going to
> > > > > > > dignify this
> > > > > > > with a response.
> > > >
> > >
> >
> >
>
Re: VOTE: add Startable to Jini specification
Posted by Patricia Shanahan <pa...@acm.org>.
I'll try. I really think there are three strategies that should be
considered separately:
1. Fix concurrency bugs in River code, even if the bug can only be
identified in theory, without causing any known external symptoms.
2. Externalize some or all of any facilities that are added to implement
Strategy 1, so that users can apply them in their own code.
3. Log non-use of Strategy 2 facilities in order to encourage users to
use them.
I'm going to use Startable as an example to illustrate the differences
between these strategies.
Adding Startable to a River implementation package, and using it in
River code, is a Strategy 1 item. It would not call for a separate vote,
but would be voted on as part of a release candidate that implements
Strategy 1.
Putting Startable in a package users are expected to use directly,
documenting it as part of River's external interface, is a Strategy 2
item. It is an external interface change, and needs to be voted on
accordingly.
Logging non-use of Startable is a Strategy 3 item, and is also an
external interface change.
Patricia
On 12/21/2013 3:11 AM, Peter wrote:
> Yes it's a difficult decision, originally when I set out to fix the issues I found, I didn't expect to be here today, on one hand, I'd like to complete the work I started, I've invested too much time to walk away now.
>
> On the other hand, "The community" may not want it fixed.
>
> Patricia, you're much better at communicating the development concepts, can you do me a favour and put forward a vote proposal, it would be beneficial to all parties to come to some decision on the underlying issue.
>
> Regards,
>
> Peter.
>
>
> ----- Original message -----
>> I have not seen any evidence of lack of understanding. I believe there
>> is a genuine lack of consensus on how to deal with some of these
>> ordering issues.
>>
>> In particular, to what extent should River live with code that will
>> almost always work?
>>
>> The damage from export-from-constructor with final fields comes if
>> another thread references the object before it is fully constructed.
>> Generally, the chain of instructions from the export to another thread
>> seeing the reference is far longer than the chain of instructions from
>> the export to the end of the constructor. Almost always, the constructor
>> will finish first and the code will work.
>>
>> Personally, I would not want to depend on that, because page faults and
>> operating system interrupts with resource contention can stretch out a
>> few instructions to several milliseconds or even seconds.
>>
>> However, a project like River depends on reaching consensus and that in
>> turn depends on discussing issues with respect for all points of view.
>>
>> Patricia
>>
>> On 12/21/2013 12:50 AM, Peter Firmstone wrote:
>>> I think the real problem is some people haven't been reading up on
>>> concurrency and are insufficiently informed to be able to properly
>>> discuss the issue.
>>>
>>> When I don't know something, I either ask someone who does, or I keep
>>> my mouth shut so as not to look like a fool.
>>>
>>> It's more a case of, if you behave like a fool long enough to make it
>>> frustrating, that's exactly how you'll be treated.
>>>
>>> Sadly it's by people who should know better, who while they may lack an
>>> understanding of the JMM are still very skilled programmers, that's
>>> what makes it frustrating.
>>>
>>> Final fields are made visible after construction completes, so when
>>> other threads receive a copy of the object in an unconstructed state,
>>> like they do during export in a constructor, then those threads can
>>> continue to see the incomplete object, rather than the fully
>>> constructed one.
>>>
>>> See for yourselves, look at all the final fields in Mercury, then
>>> realise that it's reference escaped during construction. The other
>>> services are also guilty.
>>>
>>> That also means that all the remote method invocations wrapped up as
>>> runnable tasks and executed on Mercury's methods in an executor pool
>>> will have also seen parts of Mercury in a partially constructed state.
>>>
>>> http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/com/sun/jini/mercury/MailboxImpl.java?r1=545310&r2=1552606
>>>
>>>
>>> If I haven't convinced you, read up, spend some time on the concurrency
>>> interest mail list or ask someone you trust who does know.
>>>
>>> Regards,
>>>
>>> Peter.
>>>
>>> On 21/12/2013 11:10 AM, Greg Trasuk wrote:
>>>> (off-list)
>>>>
>>>> Peter:
>>>>
>>>> You’re sounding unprofessional, and you’re missing the fact that
>>>> people are giving reasonable and well-thought-out feedback. You seem
>>>> to be taking the conversation personally. Perhaps you should take
>>>> a few hours off the list and cool down. Things will still be here
>>>> when you’ve regained composure.
>>>>
>>>> Best regards,
>>>>
>>>> Greg.
>>>>
>>>> On Dec 20, 2013, at 7:28 PM, Peter<ji...@zeus.net.au> wrote:
>>>>
>>>>> You just did.
>>>>>
>>>>> The blue pill.
>>>>>
>>>>> I prefer the truth, it's easier in the long run.
>>>>>
>>>>> Feel free to cast your vote.
>>>>>
>>>>> Peter.
>>>>>
>>>>> ----- Original message -----
>>>>>> Peter,
>>>>>>
>>>>>> There's so many things wrong in there, I'm not even going to
>>>>>> dignify this
>>>>>> with a response.
>>>
>>
>
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
Yes it's a difficult decision, originally when I set out to fix the issues I found, I didn't expect to be here today, on one hand, I'd like to complete the work I started, I've invested too much time to walk away now.
On the other hand, "The community" may not want it fixed.
Patricia, you're much better at communicating the development concepts, can you do me a favour and put forward a vote proposal, it would be beneficial to all parties to come to some decision on the underlying issue.
Regards,
Peter.
----- Original message -----
> I have not seen any evidence of lack of understanding. I believe there
> is a genuine lack of consensus on how to deal with some of these
> ordering issues.
>
> In particular, to what extent should River live with code that will
> almost always work?
>
> The damage from export-from-constructor with final fields comes if
> another thread references the object before it is fully constructed.
> Generally, the chain of instructions from the export to another thread
> seeing the reference is far longer than the chain of instructions from
> the export to the end of the constructor. Almost always, the constructor
> will finish first and the code will work.
>
> Personally, I would not want to depend on that, because page faults and
> operating system interrupts with resource contention can stretch out a
> few instructions to several milliseconds or even seconds.
>
> However, a project like River depends on reaching consensus and that in
> turn depends on discussing issues with respect for all points of view.
>
> Patricia
>
> On 12/21/2013 12:50 AM, Peter Firmstone wrote:
> > I think the real problem is some people haven't been reading up on
> > concurrency and are insufficiently informed to be able to properly
> > discuss the issue.
> >
> > When I don't know something, I either ask someone who does, or I keep
> > my mouth shut so as not to look like a fool.
> >
> > It's more a case of, if you behave like a fool long enough to make it
> > frustrating, that's exactly how you'll be treated.
> >
> > Sadly it's by people who should know better, who while they may lack an
> > understanding of the JMM are still very skilled programmers, that's
> > what makes it frustrating.
> >
> > Final fields are made visible after construction completes, so when
> > other threads receive a copy of the object in an unconstructed state,
> > like they do during export in a constructor, then those threads can
> > continue to see the incomplete object, rather than the fully
> > constructed one.
> >
> > See for yourselves, look at all the final fields in Mercury, then
> > realise that it's reference escaped during construction. The other
> > services are also guilty.
> >
> > That also means that all the remote method invocations wrapped up as
> > runnable tasks and executed on Mercury's methods in an executor pool
> > will have also seen parts of Mercury in a partially constructed state.
> >
> > http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/com/sun/jini/mercury/MailboxImpl.java?r1=545310&r2=1552606
> >
> >
> > If I haven't convinced you, read up, spend some time on the concurrency
> > interest mail list or ask someone you trust who does know.
> >
> > Regards,
> >
> > Peter.
> >
> > On 21/12/2013 11:10 AM, Greg Trasuk wrote:
> > > (off-list)
> > >
> > > Peter:
> > >
> > > You’re sounding unprofessional, and you’re missing the fact that
> > > people are giving reasonable and well-thought-out feedback. You seem
> > > to be taking the conversation personally. Perhaps you should take
> > > a few hours off the list and cool down. Things will still be here
> > > when you’ve regained composure.
> > >
> > > Best regards,
> > >
> > > Greg.
> > >
> > > On Dec 20, 2013, at 7:28 PM, Peter<ji...@zeus.net.au> wrote:
> > >
> > > > You just did.
> > > >
> > > > The blue pill.
> > > >
> > > > I prefer the truth, it's easier in the long run.
> > > >
> > > > Feel free to cast your vote.
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > > > Peter,
> > > > >
> > > > > There's so many things wrong in there, I'm not even going to
> > > > > dignify this
> > > > > with a response.
> >
>
Re: VOTE: add Startable to Jini specification
Posted by Patricia Shanahan <pa...@acm.org>.
I have not seen any evidence of lack of understanding. I believe there
is a genuine lack of consensus on how to deal with some of these
ordering issues.
In particular, to what extent should River live with code that will
almost always work?
The damage from export-from-constructor with final fields comes if
another thread references the object before it is fully constructed.
Generally, the chain of instructions from the export to another thread
seeing the reference is far longer than the chain of instructions from
the export to the end of the constructor. Almost always, the constructor
will finish first and the code will work.
Personally, I would not want to depend on that, because page faults and
operating system interrupts with resource contention can stretch out a
few instructions to several milliseconds or even seconds.
However, a project like River depends on reaching consensus and that in
turn depends on discussing issues with respect for all points of view.
Patricia
On 12/21/2013 12:50 AM, Peter Firmstone wrote:
> I think the real problem is some people haven't been reading up on
> concurrency and are insufficiently informed to be able to properly
> discuss the issue.
>
> When I don't know something, I either ask someone who does, or I keep my
> mouth shut so as not to look like a fool.
>
> It's more a case of, if you behave like a fool long enough to make it
> frustrating, that's exactly how you'll be treated.
>
> Sadly it's by people who should know better, who while they may lack an
> understanding of the JMM are still very skilled programmers, that's what
> makes it frustrating.
>
> Final fields are made visible after construction completes, so when
> other threads receive a copy of the object in an unconstructed state,
> like they do during export in a constructor, then those threads can
> continue to see the incomplete object, rather than the fully constructed
> one.
>
> See for yourselves, look at all the final fields in Mercury, then
> realise that it's reference escaped during construction. The other
> services are also guilty.
>
> That also means that all the remote method invocations wrapped up as
> runnable tasks and executed on Mercury's methods in an executor pool
> will have also seen parts of Mercury in a partially constructed state.
>
> http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/com/sun/jini/mercury/MailboxImpl.java?r1=545310&r2=1552606
>
>
> If I haven't convinced you, read up, spend some time on the concurrency
> interest mail list or ask someone you trust who does know.
>
> Regards,
>
> Peter.
>
> On 21/12/2013 11:10 AM, Greg Trasuk wrote:
>> (off-list)
>>
>> Peter:
>>
>> You’re sounding unprofessional, and you’re missing the fact that
>> people are giving reasonable and well-thought-out feedback. You seem
>> to be taking the conversation personally. Perhaps you should take a
>> few hours off the list and cool down. Things will still be here when
>> you’ve regained composure.
>>
>> Best regards,
>>
>> Greg.
>>
>> On Dec 20, 2013, at 7:28 PM, Peter<ji...@zeus.net.au> wrote:
>>
>>> You just did.
>>>
>>> The blue pill.
>>>
>>> I prefer the truth, it's easier in the long run.
>>>
>>> Feel free to cast your vote.
>>>
>>> Peter.
>>>
>>> ----- Original message -----
>>>> Peter,
>>>>
>>>> There's so many things wrong in there, I'm not even going to dignify
>>>> this
>>>> with a response.
>
Re: VOTE: add Startable to Jini specification
Posted by Peter Firmstone <ji...@zeus.net.au>.
I think the real problem is some people haven't been reading up on
concurrency and are insufficiently informed to be able to properly
discuss the issue.
When I don't know something, I either ask someone who does, or I keep my
mouth shut so as not to look like a fool.
It's more a case of, if you behave like a fool long enough to make it
frustrating, that's exactly how you'll be treated.
Sadly it's by people who should know better, who while they may lack an
understanding of the JMM are still very skilled programmers, that's what
makes it frustrating.
Final fields are made visible after construction completes, so when
other threads receive a copy of the object in an unconstructed state,
like they do during export in a constructor, then those threads can
continue to see the incomplete object, rather than the fully constructed
one.
See for yourselves, look at all the final fields in Mercury, then
realise that it's reference escaped during construction. The other
services are also guilty.
That also means that all the remote method invocations wrapped up as
runnable tasks and executed on Mercury's methods in an executor pool
will have also seen parts of Mercury in a partially constructed state.
http://svn.apache.org/viewvc/river/jtsk/skunk/qa_refactor/trunk/src/com/sun/jini/mercury/MailboxImpl.java?r1=545310&r2=1552606
If I haven't convinced you, read up, spend some time on the concurrency
interest mail list or ask someone you trust who does know.
Regards,
Peter.
On 21/12/2013 11:10 AM, Greg Trasuk wrote:
> (off-list)
>
> Peter:
>
> You’re sounding unprofessional, and you’re missing the fact that people are giving reasonable and well-thought-out feedback. You seem to be taking the conversation personally. Perhaps you should take a few hours off the list and cool down. Things will still be here when you’ve regained composure.
>
> Best regards,
>
> Greg.
>
> On Dec 20, 2013, at 7:28 PM, Peter<ji...@zeus.net.au> wrote:
>
>> You just did.
>>
>> The blue pill.
>>
>> I prefer the truth, it's easier in the long run.
>>
>> Feel free to cast your vote.
>>
>> Peter.
>>
>> ----- Original message -----
>>> Peter,
>>>
>>> There's so many things wrong in there, I'm not even going to dignify this
>>> with a response.
Re: VOTE: add Startable to Jini specification
Posted by Greg Trasuk <tr...@stratuscom.com>.
(off-list)
Peter:
You’re sounding unprofessional, and you’re missing the fact that people are giving reasonable and well-thought-out feedback. You seem to be taking the conversation personally. Perhaps you should take a few hours off the list and cool down. Things will still be here when you’ve regained composure.
Best regards,
Greg.
On Dec 20, 2013, at 7:28 PM, Peter <ji...@zeus.net.au> wrote:
> You just did.
>
> The blue pill.
>
> I prefer the truth, it's easier in the long run.
>
> Feel free to cast your vote.
>
> Peter.
>
> ----- Original message -----
>> Peter,
>>
>> There's so many things wrong in there, I'm not even going to dignify this
>> with a response.
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
You just did.
The blue pill.
I prefer the truth, it's easier in the long run.
Feel free to cast your vote.
Peter.
----- Original message -----
> Peter,
>
> There's so many things wrong in there, I'm not even going to dignify this
> with a response.
Re: VOTE: add Startable to Jini specification
Posted by Dan Creswell <da...@gmail.com>.
Peter,
There's so many things wrong in there, I'm not even going to dignify this
with a response.
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
----- Original message -----
> "Comments like `Jini 2.x doesn't seem to have such an issue even with the
> "serious flaw" you
> claim?` are typical of an experimental approach."
>
> They might well be typical but in this case my statement has nothing to
> do with an experimental approach.
>
> [ In fact, you could have asked me if I'm advocating such a thing
> directly, why didn't you? ]
Perhaps you were unaware or hadn't realised, but Patricia nailed it, you're using an experimental approach.
Now it's up to you to decide, the blue pill or the red pill?
>
> Here's what I'm asking for: A clear statement of what problem we're
> fixing.
I'll do my best to put it simply:
Our starter kit, Phoenix and Activation, all use reflective constructor signatures to create services. Although you can't define an interface for a constructor, these are the closest equivalent
The proposed interface allow our services to use final fields simply by allowing exporting and threads that access internal shared state to be delayed until after the constructor completes.
For examples, use the diff view via our svn website on the qa_refactor branch. All services, including those in the qa suite have been fixed / converted.
The proposed solution is simple and complete.
All I'm hearing at the moment is that if some thread other than
> the constructing thread reads a final before the constructing thread is
> done, bad things can happen. That is entirely true and in alignment with
> the JMM, a statement of the obvious if you will.
>
> BUT
>
> The critical aspect is not releasing the reference to an object to
> another thread before construction completes but that other thread
> READING that final variable using that reference before construction is
> complete.
>
> i.e. there is nothing wrong with releasing a reference, it's about what
> you do with it and when.
>
> SO
>
> What I'm interested in hearing is: What code in the JSTK is getting hold
> of a reference prior to construction completing and then reading from
> it? Or what code could and how? Perhaps, for example, we're worried about
> something an exporter might do (of course you could stipulate that it not
> access the object for any other reason than to dispatch a remote
> request)? Or maybe a JoinManager (and what if I wanted the reference to
> my JoinManager to be final)?
>
> AND
>
> Does the change we're discussing actually constitute a band-aid or a
> complete solution? Is a new interface enough or do we need to go fix up
> some other classes like JoinManager and such to do this properly?
>
> BECAUSE
>
> I care about fixing the right problem the right way...
>
>
>
>
> On 20 December 2013 18:38, Patricia Shanahan <pa...@acm.org> wrote:
>
> > On 12/20/2013 1:25 AM, Dan Creswell wrote:
> > ...
> >
> > And "serious flaw" - really? How might you judge that as the case?
> > Have we
> > > got a serious service-startup problem? Jini 2.x doesn't seem to have
> > > such an issue even with the "serious flaw" you claim? Perhaps this
> > > "serious flaw" has come into being because of all the other work
> > > you've done since Jini 2.x (and I'm not saying the work isn't
> > > valuable)? If that is so, perhaps that is because you have changed
> > > the contract against which a service is written? If so, I would
> > > suggest you would be having a vote on modifying the contract not
> > > adding an interface.
> > >
> > > ...
> >
> > I think there is a big picture question that underlies all this. There
> > are two, potentially complementary, approaches to thread safety,
> > experiment and theory. Each has advantages and disadvantages.
> >
> > In the pure experimental approach, the only thread safety issues that
> > require change are those that manifest as bugs in some known test. The
> > upside is that one can do strictly test driven development, where every
> > change fixes a test failure. The downside is that the resulting code
> > can be fragile - an apparently irrelevant change can shift a latent
> > bug to causing a test failure. The code may run fine on small, lightly
> > loaded systems, but fail on larger systems especially if there is
> > contention for processors and/or memory.
> >
> > In a theory driven approach, the objective is to make the code
> > thread-safe by design. It requires changes to conform, in the case of
> > Java, to the JMM, regardless of whether the issue manifests as a test
> > failure. The upside is that the resulting code should be more robust -
> > mere timing changes, or shifts in the locations of page and cache line
> > boundaries, cannot cause failures. The downside is that it requires
> > changes that are not necessarily supported by a failing test case.
> >
> > My own reading of River code convinced me that it was largely in an
> > experimental state, with multi-thread issues that were not being fixed
> > because they did not cause known test failures. Comments like `Jini 2.x
> >
> > doesn't seem to have such an issue even with the "serious flaw" you
> > claim?` are typical of an experimental approach.
> >
> > As I understand the big picture, Peter is pushing a change to a more
> > theory-based approach.
> >
> > Shifting the balance from experimental towards theory involves a risk
> > of cascading problems because making one change that affects the
> > relationships between threads can open up a sequence of events that
> > triggers an apparently unrelated bug.
> >
> > I think the PMC should discuss and decide on whether to make the change
> > to the theory based approach. If I were still a PMC member I would be
> > strongly in favor, but it is not up to me. A lot of decisions,
> > including this vote, would be obvious and non-controversial in the
> > light of a definite policy, either way, on that issue.
> >
> > Patricia
> >
> >
RESULT: Re: VOTE: add Startable to Jini specification
Posted by Peter Firmstone <ji...@zeus.net.au>.
Vote results in chronical order, after 72 hours:
+1 Peter Firmstone
+1 Simon IJskes
+0 Greg Trasuk
According to our rules that's one vote short for inclusion into the Jini
Specification.
On this occassion there was interest in developing a more comprehensive
standard for starting services, I hope container providers in the River
community are able to collaborate and provide such a solution.
Unless there are any objections (by lazy concensus), I propose we use
Startable for River's service implementations and also allow River users
to implement it, if they so choose, so they too may use final fields safely.
Regards,
Peter.
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
Compiler writers (static and runtime) will make optimisations based on the JMM whether it's acceptable to us or not, these optimisations can change between releases, so something that works ok most of the time now during testing, may not work at all in future ,or fail under load.
JSR133, the java memory model (JMM), is a contract between programmers and compiler writers, it guarantees predictable behaviour for programmers that follow the rules laid out by the JMM. Section 3.2 Final fields, paragraph 4 states: "The usage model for final fields is a simple one. Set the final fields for an object in that object's constructor. Do not write a reference to an object being constructed in a place where another thread can see it before the object's constructor is finished."
I think the JMM is a standard we should consider adopting, to improve the predictability and stability of our code across different java platforms and releases.
I had three choices for creating services that contained final fields in a manner compliant with the JMM:
1. Add a Starter interface with a start method.
2. Use inheritance, where final fields belong in a superclass, and a stateless child class exports and starts any threads.
3. Disallow final fields.
Which method would you choose and why?
----- Original message -----
>
> On Dec 20, 2013, at 2:49 PM, Dan Creswell <da...@gmail.com> wrote:
> >
> > BUT
> >
> > The critical aspect is not releasing the reference to an object to
> > another thread before construction completes but that other thread
> > READING that final variable using that reference before construction
> > is complete.
> >
> > i.e. there is nothing wrong with releasing a reference, it's about
> > what you do with it and when.
>
> Okay, this is where it’s important to make sure that everyone is getting
> what the JMM says. This is not about “when” but the “order”. If we
> have the tasks identified as follows:
>
> 1. T1 starts to construct O1
> 2. T2 gains reference to O1 via T1 handing it a reference
> 3. T1 completes construction of O1
> 4. T2 reads some value from the fields of O1
>
> then one could say that the constructor completed before the reference
> to O1, by T2. What I think is important to understand, is that you
> have to guarantee that will happen with a shared happens before
> relationship between T1 and T2.
>
> Theory based programming is used to make sure this happens. Test base
> programming might result in some delay being placed in T2 such that T1
> should always complete.
>
> I don’t think that there is anyone here that would do such a thing, but
> sometimes it happens by mistake. I.e. what if T2 goes to do something
> on the network in the test, accessing a nonexistent DNS server which
> stalls it and causes the timing to work out okay? What if the 3.5
> second reduction to 2 second reduction that Peter made, opens the window
> to T2 using O1 before T1 completes?
>
> We really do need to fix these things. I’ve made many, many patches to
> my private copy of Jini over the years, specifically related to seeing
> these kinds of things going on. It’s real stuff. As I’ve said before.
> Many of us are being saved by the fences and locks in Vector and
> Hashtable which are causing implicit happens-before events which fix
> these issues. But, those structures just KILL performance. As we
> remove the use of those, as Patricia said, other things are going to
> start breaking, and Peter has found that happening all along his path of
> exploration of how bad things actually are.
>
> Gregg Wonderly
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
People may also find the following interesting:
http://www.azulsystems.com/blog/cliff/2011-10-17-writing-to-final-fields-via-reflection
Regards,
Peter.
----- Original message -----
>
> On Dec 20, 2013, at 2:49 PM, Dan Creswell <da...@gmail.com> wrote:
> >
> > BUT
> >
> > The critical aspect is not releasing the reference to an object to
> > another thread before construction completes but that other thread
> > READING that final variable using that reference before construction
> > is complete.
> >
> > i.e. there is nothing wrong with releasing a reference, it's about
> > what you do with it and when.
>
> Okay, this is where it’s important to make sure that everyone is getting
> what the JMM says. This is not about “when” but the “order”. If we
> have the tasks identified as follows:
>
> 1. T1 starts to construct O1
> 2. T2 gains reference to O1 via T1 handing it a reference
> 3. T1 completes construction of O1
> 4. T2 reads some value from the fields of O1
>
> then one could say that the constructor completed before the reference
> to O1, by T2. What I think is important to understand, is that you
> have to guarantee that will happen with a shared happens before
> relationship between T1 and T2.
>
> Theory based programming is used to make sure this happens. Test base
> programming might result in some delay being placed in T2 such that T1
> should always complete.
>
> I don’t think that there is anyone here that would do such a thing, but
> sometimes it happens by mistake. I.e. what if T2 goes to do something
> on the network in the test, accessing a nonexistent DNS server which
> stalls it and causes the timing to work out okay? What if the 3.5
> second reduction to 2 second reduction that Peter made, opens the window
> to T2 using O1 before T1 completes?
>
> We really do need to fix these things. I’ve made many, many patches to
> my private copy of Jini over the years, specifically related to seeing
> these kinds of things going on. It’s real stuff. As I’ve said before.
> Many of us are being saved by the fences and locks in Vector and
> Hashtable which are causing implicit happens-before events which fix
> these issues. But, those structures just KILL performance. As we
> remove the use of those, as Patricia said, other things are going to
> start breaking, and Peter has found that happening all along his path of
> exploration of how bad things actually are.
>
> Gregg Wonderly
Re: VOTE: add Startable to Jini specification
Posted by Gregg Wonderly <gr...@wonderly.org>.
On Dec 20, 2013, at 2:49 PM, Dan Creswell <da...@gmail.com> wrote:
>
> BUT
>
> The critical aspect is not releasing the reference to an object to another
> thread before construction completes but that other thread READING that
> final variable using that reference before construction is complete.
>
> i.e. there is nothing wrong with releasing a reference, it's about what you
> do with it and when.
Okay, this is where it’s important to make sure that everyone is getting what the JMM says. This is not about “when” but the “order”. If we have the tasks identified as follows:
1. T1 starts to construct O1
2. T2 gains reference to O1 via T1 handing it a reference
3. T1 completes construction of O1
4. T2 reads some value from the fields of O1
then one could say that the constructor completed before the reference to O1, by T2. What I think is important to understand, is that you have to guarantee that will happen with a shared happens before relationship between T1 and T2.
Theory based programming is used to make sure this happens. Test base programming might result in some delay being placed in T2 such that T1 should always complete.
I don’t think that there is anyone here that would do such a thing, but sometimes it happens by mistake. I.e. what if T2 goes to do something on the network in the test, accessing a nonexistent DNS server which stalls it and causes the timing to work out okay? What if the 3.5 second reduction to 2 second reduction that Peter made, opens the window to T2 using O1 before T1 completes?
We really do need to fix these things. I’ve made many, many patches to my private copy of Jini over the years, specifically related to seeing these kinds of things going on.
It’s real stuff. As I’ve said before. Many of us are being saved by the fences and locks in Vector and Hashtable which are causing implicit happens-before events which fix these issues. But, those structures just KILL performance. As we remove the use of those, as Patricia said, other things are going to start breaking, and Peter has found that happening all along his path of exploration of how bad things actually are.
Gregg Wonderly
Re: VOTE: add Startable to Jini specification
Posted by Dan Creswell <da...@gmail.com>.
"Comments like `Jini 2.x doesn't seem to have such an issue even with the
"serious flaw" you
claim?` are typical of an experimental approach."
They might well be typical but in this case my statement has nothing to do
with an experimental approach.
[ In fact, you could have asked me if I'm advocating such a thing directly,
why didn't you? ]
Here's what I'm asking for: A clear statement of what problem we're fixing.
All I'm hearing at the moment is that if some thread other than the
constructing thread reads a final before the constructing thread is done,
bad things can happen. That is entirely true and in alignment with the JMM,
a statement of the obvious if you will.
BUT
The critical aspect is not releasing the reference to an object to another
thread before construction completes but that other thread READING that
final variable using that reference before construction is complete.
i.e. there is nothing wrong with releasing a reference, it's about what you
do with it and when.
SO
What I'm interested in hearing is: What code in the JSTK is getting hold of
a reference prior to construction completing and then reading from it? Or
what code could and how? Perhaps, for example, we're worried about
something an exporter might do (of course you could stipulate that it not
access the object for any other reason than to dispatch a remote request)?
Or maybe a JoinManager (and what if I wanted the reference to my
JoinManager to be final)?
AND
Does the change we're discussing actually constitute a band-aid or a
complete solution? Is a new interface enough or do we need to go fix up
some other classes like JoinManager and such to do this properly?
BECAUSE
I care about fixing the right problem the right way...
On 20 December 2013 18:38, Patricia Shanahan <pa...@acm.org> wrote:
> On 12/20/2013 1:25 AM, Dan Creswell wrote:
> ...
>
> And "serious flaw" - really? How might you judge that as the case? Have we
>> got a serious service-startup problem? Jini 2.x doesn't seem to have such
>> an issue even with the "serious flaw" you claim? Perhaps this "serious
>> flaw" has come into being because of all the other work you've done since
>> Jini 2.x (and I'm not saying the work isn't valuable)? If that is so,
>> perhaps that is because you have changed the contract against which a
>> service is written? If so, I would suggest you would be having a vote on
>> modifying the contract not adding an interface.
>>
>> ...
>
> I think there is a big picture question that underlies all this. There
> are two, potentially complementary, approaches to thread safety,
> experiment and theory. Each has advantages and disadvantages.
>
> In the pure experimental approach, the only thread safety issues that
> require change are those that manifest as bugs in some known test. The
> upside is that one can do strictly test driven development, where every
> change fixes a test failure. The downside is that the resulting code can
> be fragile - an apparently irrelevant change can shift a latent bug to
> causing a test failure. The code may run fine on small, lightly loaded
> systems, but fail on larger systems especially if there is contention
> for processors and/or memory.
>
> In a theory driven approach, the objective is to make the code
> thread-safe by design. It requires changes to conform, in the case of
> Java, to the JMM, regardless of whether the issue manifests as a test
> failure. The upside is that the resulting code should be more robust -
> mere timing changes, or shifts in the locations of page and cache line
> boundaries, cannot cause failures. The downside is that it requires
> changes that are not necessarily supported by a failing test case.
>
> My own reading of River code convinced me that it was largely in an
> experimental state, with multi-thread issues that were not being fixed
> because they did not cause known test failures. Comments like `Jini 2.x
>
> doesn't seem to have such an issue even with the "serious flaw" you
> claim?` are typical of an experimental approach.
>
> As I understand the big picture, Peter is pushing a change to a more
> theory-based approach.
>
> Shifting the balance from experimental towards theory involves a risk of
> cascading problems because making one change that affects the
> relationships between threads can open up a sequence of events that
> triggers an apparently unrelated bug.
>
> I think the PMC should discuss and decide on whether to make the change
> to the theory based approach. If I were still a PMC member I would be
> strongly in favor, but it is not up to me. A lot of decisions, including
> this vote, would be obvious and non-controversial in the light of a
> definite policy, either way, on that issue.
>
> Patricia
>
>
Re: VOTE: add Startable to Jini specification
Posted by Patricia Shanahan <pa...@acm.org>.
On 12/20/2013 1:25 AM, Dan Creswell wrote:
...
> And "serious flaw" - really? How might you judge that as the case? Have we
> got a serious service-startup problem? Jini 2.x doesn't seem to have such
> an issue even with the "serious flaw" you claim? Perhaps this "serious
> flaw" has come into being because of all the other work you've done since
> Jini 2.x (and I'm not saying the work isn't valuable)? If that is so,
> perhaps that is because you have changed the contract against which a
> service is written? If so, I would suggest you would be having a vote on
> modifying the contract not adding an interface.
>
...
I think there is a big picture question that underlies all this. There
are two, potentially complementary, approaches to thread safety,
experiment and theory. Each has advantages and disadvantages.
In the pure experimental approach, the only thread safety issues that
require change are those that manifest as bugs in some known test. The
upside is that one can do strictly test driven development, where every
change fixes a test failure. The downside is that the resulting code can
be fragile - an apparently irrelevant change can shift a latent bug to
causing a test failure. The code may run fine on small, lightly loaded
systems, but fail on larger systems especially if there is contention
for processors and/or memory.
In a theory driven approach, the objective is to make the code
thread-safe by design. It requires changes to conform, in the case of
Java, to the JMM, regardless of whether the issue manifests as a test
failure. The upside is that the resulting code should be more robust -
mere timing changes, or shifts in the locations of page and cache line
boundaries, cannot cause failures. The downside is that it requires
changes that are not necessarily supported by a failing test case.
My own reading of River code convinced me that it was largely in an
experimental state, with multi-thread issues that were not being fixed
because they did not cause known test failures. Comments like `Jini 2.x
doesn't seem to have such an issue even with the "serious flaw" you
claim?` are typical of an experimental approach.
As I understand the big picture, Peter is pushing a change to a more
theory-based approach.
Shifting the balance from experimental towards theory involves a risk of
cascading problems because making one change that affects the
relationships between threads can open up a sequence of events that
triggers an apparently unrelated bug.
I think the PMC should discuss and decide on whether to make the change
to the theory based approach. If I were still a PMC member I would be
strongly in favor, but it is not up to me. A lot of decisions, including
this vote, would be obvious and non-controversial in the light of a
definite policy, either way, on that issue.
Patricia
Re: VOTE: add Startable to Jini specification
Posted by Dan Creswell <da...@gmail.com>.
A few words stand out there:
"we can still cooperate"
You feel that this:
"Get over your fear people, let us develop, we are maintaining backward
compatibility."
is a demonstration of co-operation? No, that's an order.
"so I can fix a serious flaw with our api?"
I? Not we, I - so are you co-operating and working with the community or
doing your own thing?
And "serious flaw" - really? How might you judge that as the case? Have we
got a serious service-startup problem? Jini 2.x doesn't seem to have such
an issue even with the "serious flaw" you claim? Perhaps this "serious
flaw" has come into being because of all the other work you've done since
Jini 2.x (and I'm not saying the work isn't valuable)? If that is so,
perhaps that is because you have changed the contract against which a
service is written? If so, I would suggest you would be having a vote on
modifying the contract not adding an interface.
"This only needs one more +1 vote to pass or -1 for a blocking vote."
Yes, and that voting system is there to ensure quorum amongst your
community, like it or not, that's the Apache game. Discussion, argument,
agreement, moving forward together. So then, you want to split the
community? Okay but how about you fork and take your stuff somewhere else
because this project has other developers on board doing stuff which
wouldn't qualify as maintenance but perhaps would qualify as not aligned
with you.
I appreciate you have a level of frustration, however, you've been here
before and taken the same approach again and again where you attempt to
resolve by force/assertion. Might you consider changing your tactics or are
you just outright correct and we should back off?
On 20 December 2013 04:49, Peter <ji...@zeus.net.au> wrote:
> This only needs one more +1 vote to pass or -1 for a blocking vote.
>
> I only need a single method interface so I can fix a serious flaw with our
> api? It's not even a breaking change!
>
> Get over your fear people, let us develop, we are maintaining backward
> compatibility.
>
> Without development River will go to the Attic, this is supposed to be an
> open source development project, not a Jini code museum.
>
> Get used to the fact there will be a beta release for testing soon.
>
> I have people declining to participate because of the hostility towards
> development that exists on this list. Lets show some determination, ignore
> the fear and get the job done.
>
> The Jini team, they had vision, they might not have delivered what they
> promised, their marketing might have been mixed up, the limitations of Java
> might have gotten in the way, but by God, they had vision. When you reach
> high you risk failure, but without failure there is no success either.
>
> We need to decide, is River a Jini code museum, or are we planning to make
> River work outside local networks and become part of the internet? Are
> there people here with vision? IPv6 is on our doorstep, there are data
> silo's and high powered mobile devices everywhere we could be connecting to
> make useful applications. Or do we live in fear and squander our
> opportunities?
>
> Perhaps we should fork into two projects, one for Jini 2.x and one for
> those who can see a future, have vision and would like to share but keep
> getting shouted down?
>
> Lets create a Jini maintenance project, there's no trademark now, let it
> have it's own lists, and set River free.
>
> Don't take it personal we can still cooperate, I understand people have
> established niche's, but fresh developers have itches to scratch.
>
> And before you go flaming me, food for thought: People can only judge on
> what they know, they cannot judge upon what they don't, fear the unknown,
> or learn.
>
> Regards,
>
> Peter.
>
> ----- Original message -----
> >
> > I don’t really see a need to mess around with the existing
> > infrastructure services. Surely if we can start them as it is and they
> > can be reached through lookup, that’s good enough? I’m more concerned
> > with having an interface that makes sense for developers to write their
> > own services with. I suspect we need to just do a few experiments and
> > see what works.
> >
> > I guess that’s one of my frustrations with all this “refactoring” of the
> > existing codebase - if it doesn’t make it easier for people to use
> > Jini/River to create their own service-oriented architectures, then it
> > just seems like a lot of navel-gazing. Similarly, we’ve had statements
> > about reorganizing the source code to make it easier to develop with.
> > To which I ask, “when was the last time anyone worried about the source
> > code of Tomcat, or Websphere, or Weblogic?, or whether they used Maven
> > or not?”. Even talking about performance is not very helpful until the
> > average corporate coder at the City of Wherever can actually use the
> > product.
> >
> > We need to get to a point where people are building real things on top
> > of River, not messing around with River. That’s not to say that the
> > River code can’t be improved or modernized, but it reached “good enough”
> > back in 2003 or so.
> >
> > Lest I sound like I’m complaining, let me say it’s great that we’ve had
> > this much traffic on the list in the last few weeks! I think we’re
> > having great discussions- keep it up, folks!
> >
> > Greg.
> >
> > On Dec 19, 2013, at 8:54 PM, Peter <ji...@zeus.net.au> wrote:
> >
> > > If you can work put what a common reflective constructor signature you
> > > need is, any lifecycle methods required, I'll include these for every
> > > existing Jini service and we can make a standard.
> > >
> > > We can also support JSR330 which is AL2 licensed.
> > >
> > > Support @Inject for the constructor.
> > >
> > > Provider<JavaSpace05>
> > >
> > > Regards,
> > >
> > > Peter.
> > >
> > >
> > > ----- Original message -----
> > > > Hopefully Startable will get enough votes to get across the line, we
> > > > need to make progress, if you care about River you'll vote for it,
> > > > not because it's the perfect solution, simply because it's an
> > > > acceptable solution that makes sense and it marks a progress
> > > > milestone.
> > > >
> > > > The reflective constructor signature is acceptable and is used by the
> > > > starter kit and Activation.
> > > >
> > > > We've got Groovy Configuration.
> > > >
> > > > The service class can also define a static main method.
> > > >
> > > > If we at least accept these signatures for our services, they could
> > > > run in any container.
> > > >
> > > > Perhaps you also want a public shutdown method for lifecycle
> > > > management? Closeable would also be acceptable?
> > > >
> > > > Is there anything else you need?
> > > >
> > > > Regards,
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > > > "What we need is a discussion on a more developer-friendly service
> > > > > startup convention to replace ServiceStarter."
> > > > >
> > > > > Maybe, that is to say, I'm not anti but I feel there's an
> > > > > additional wider aspect to kickaround as well....
> > > > >
> > > > > Should there be a failure of the type we've discussed so far, where
> > > > > does the responsibility for sorting it out lie? The service writer,
> > > > > they created the bug in the first place. BUT, has it occurred to
> > > > > anyone that export() isn't the only place multi-threading problems
> > > > > can occur in a service implementation?
> > > > >
> > > > > Even if they've got service construction right (with whatever
> > > > > convention or pattern you care to adopt, me, I'm in the camp of if
> > > > > it works because it accounts for the JMM it's fine) have they got
> > > > > all their multi-threading right elsewhere? What about all those
> > > > > service methods exported and advertised by JoinManager?
> > > > >
> > > > > Do we actually think it's possible to address that challenge? (You
> > > > > could do it by single-locking the entire service so only one call
> > > > > can be dispatched at a time, yeah right, that's palatable). What
> > > > > makes export a special case in this context? Why does that matter
> > > > > so when there are many other ways to screw up?
> > > > >
> > > > >
> > > > > On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > +0. Let me explain…
> > > > > >
> > > > > > Having a separate interface to say start() is all very well and
> > > > > > harmless in itself, but:
> > > > > >
> > > > > > (a) It doesn’t really address the issue of how service instances
> > > > > > should be started. We don’t actually have solid
> > > > > > recommendations on this issue, although there have been attempts
> > > > > > in various containers to systematize it.
> > > > > >
> > > > > > (b) the proposed warning addresses a case where a service
> > > > > > instance is doing “exporter.export(this);”, which I’d call bad
> > > > > > practice in any case. It happens that ServiceStarter appears to
> > > > > > encourage this use case because it works by calling a
> > > > > > constructor. But the called constructor could (and should)
> > > > > > construct an instance of some other class, and then export and
> > > > > > register that instance rather than “this”. That style doesn’t
> > > > > > mandate a Startable interface, and having Startable doesn’t
> > > > > > mandate good multi-threaded code. In other words, you could
> > > > > > implement Startable and still have poor thread-safety.
> > > > > >
> > > > > > (c) even where there is an apparent problem, the exposure window
> > > > > > is very small. Consider the last bits of the init(…)
> method in
> > > > > > “com.sun.jini.reggie.ServiceRegistrarImpl”
> > > > > >
> > > > > > myRef = (Registrar) serverExporter.export(this);
> > > > > > proxy = RegistrarProxy.getInstance(myRef, myServiceID);
> > > > > > myLocator = (proxy instanceof RemoteMethodControl) ?
> > > > > > new ConstrainableLookupLocator(
> > > > > > unicastDiscoveryHost, unicaster.port, null) :
> > > > > > new LookupLocator(unicastDiscoveryHost, unicaster.port);
> > > > > > /* register myself */
> > > > > > Item item = new Item(new ServiceItem(myServiceID,
> > > > > > proxy,
> > > > > > lookupAttrs));
> > > > > > SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
> > > > > > addService(reg);
> > > > > > if (log != null) {
> > > > > > log.snapshot();
> > > > > > }
> > > > > >
> > > > > > try {
> > > > > > DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
> > > > > > discoer;
> > > > > > String[] groups = dgm.getGroups();
> > > > > > if (groups == null || groups.length > 0) {
> > > > > > throw new ConfigurationException(
> > > > > > "discoveryManager must be initially configured with " +
> > > > > > "no groups");
> > > > > > }
> > > > > > DiscoveryLocatorManagement dlm =
> > > > > > (DiscoveryLocatorManagement) discoer;
> > > > > > if (dlm.getLocators().length > 0) {
> > > > > > throw new ConfigurationException(
> > > > > > "discoveryManager must be initially configured with " +
> > > > > > "no locators");
> > > > > > }
> > > > > > dgm.setGroups(lookupGroups);
> > > > > > dlm.setLocators(lookupLocators);
> > > > > > } catch (ClassCastException e) {
> > > > > > throw new ConfigurationException(null, e);
> > > > > > }
> > > > > > joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
> > > > > > discoer, null, config);
> > > > > >
> > > > > > /* start up all the daemon threads */
> > > > > > serviceExpirer.start();
> > > > > > eventExpirer.start();
> > > > > > unicaster.start();
> > > > > > multicaster.start();
> > > > > > announcer.start();
> > > > > >
> > > > > > /* Shutdown hook so reggie sends a final announcement
> > > > > > * packet if VM is terminated. If reggie is terminated
> > > > > > * through DestroyAdmin.destroy() this hook will have no effect.
> > > > > > * A timeout on announcer.join() was considered but not deemed
> > > > > > * necessary at this point in time.
> > > > > > */
> > > > > > Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
> > > > > > public void run() {
> > > > > > try {
> > > > > > announcer.interrupt();
> > > > > > announcer.join();
> > > > > > } catch (Throwable t) {
> > > > > > logThrow(Level.FINEST, getClass().getName(),
> > > > > > "run", "exception shutting announcer down",
> > > > > > new Object[]{}, t);
> > > > > > }
> > > > > > }
> > > > > > }));
> > > > > >
> > > > > > snapshotter.start();
> > > > > > if (logger.isLoggable(Level.INFO)) {
> > > > > > logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
> > > > > > new Object[]{ myServiceID,
> > > > > > Arrays.asList(memberGroups),
> > > > > > myLocator });
> > > > > > }
> > > > > > ready.ready();
> > > > > > }
> > > > > >
> > > > > >
> > > > > > The window from when the join manager is started up (ignore the
> > > > > > serverExporter.export(). The only way a client gets to this
> > > > > > endpoint is through a proxy downloaded from the registrar) to
> > > > > > when the init() method exits is effectively 9 lines of code,
> > > > > > during which time a _lot_ of network i/o would have to happen in
> > > > > > order for the object to be used prior to the constructor exit.
> > > > > > As well, the actual remote calls are filtered through a command
> > > > > > interface and through the tasker object, and the command classes
> > > > > > often go through synchronized() blocks, which (I think) act as
> > > > > > happens-before fences.
> > > > > >
> > > > > > The whole thing seems to me a little like the old servlet days,
> > > > > > when we got the SingleThreadModel interface and it only served
> > > > > > to give naive developers a false sense of security.
> > > > > >
> > > > > > What we need is a discussion on a more developer-friendly service
> > > > > > startup convention to replace ServiceStarter.
> > > > > >
> > > > > > Having said that, I’m not overly opposed to having Startable,
> > > > > > given that as Dan suggested, any output from the ServiceStarter
> > > > > > is in the form of a suggestion rather than an error.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Greg.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
> > > > > >
> > > > > > > +1 Peter.
> > > > > > >
> > > > > > >
> > > > > > > I've been going over the Exporter implementations and Exporter
> > > > > > > interface.
> > > > > > >
> > > > > > > The Exporter interface specifies that export and unexport
> > > > > > > behaviour is
> > > > > > defined by the implementation.
> > > > > > >
> > > > > > > That means it's up to the implementation to specify whether a
> > > > > > > happens
> > > > > > before edge occures before export.
> > > > > > >
> > > > > > > I propose we document in Exporter, that if export is called
> > > > > > > during
> > > > > > object construction then that object's implementation cannot
> > > > > > safely use final fields.
> > > > > > >
> > > > > > > We should also provide a @see reference to Startable to
> > > > > > > reccommend its
> > > > > > use when final fields are desirable and include Startable in the
> > > > > > spec.
> > > > > > >
> > > > > > > We don't need to rewrite the example of exporting in
> > > > > > > net.jini.config as
> > > > > > export isn't performed during construction.
> > > > > > >
> > > > > > > This is a minor update to the Jini Specification.
> > > > > > >
> > > > > > > The FINE log message is an implementation detail and won't be
> > > > > > > included.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Peter.
> > > > > > >
> > > > > > > ----- Original message -----
> > > > > > > > No, an effectively immutable field is one whose reference
> > > > > > > > doesn't change after the service objects safe publication.
> > > > > > > >
> > > > > > > > An immutable field is final.
> > > > > > > >
> > > > > > > > It would be Jeri's responsibility to safely publish the
> > > > > > > > service during export to ensure a happens before event
> > > > > > > > occurs, ensuring all non volatile fields the service
> > > > > > > > contains are safely published and visible.
> > > > > > > >
> > > > > > > > It's possible to have mutable fields guarded by synchronized
> > > > > > > > methods, or volatile fields.
> > > > > > > >
> > > > > > > > But it isn't possible to have final fields when exporting
> > > > > > > > within the constructor.
> > > > > > > >
> > > > > > > > If the service contains effectively immutable field
> > > > > > > > references, it must be safely published or if another thread
> > > > > > > > accesses those fields through unsynchronized methods, the
> > > > > > > > thread may only see their default value.
> > > > > > > >
> > > > > > > > All our service implementations have final fields.
> All
> > > > > > > > our services
> > > > > > had
> > > > > > > > some unsynchronized access to non volatile, non final fields.
> > > > > > > >
> > > > > > > > Trouble is, when you can't have final fields, it's very
> > > > > > > > difficult to determine which fields are effectively
> > > > > > > > immutable and the original programmers design intent, if
> > > > > > > > there's no comment.
> > > > > > > >
> > > > > > > > Our service implementations all had unsynchronized init()
> > > > > > > > methods called from with the constructor that started
> > > > > > > > threads and exported Some
> > > > > > fields
> > > > > > > > were final.
> > > > > > > >
> > > > > > > > For me, a service that contains final fields and is exported
> > > > > > > > after construction is easier to maintain and understand.
> > > > > > > >
> > > > > > > > I suspect, like our Jini service imlementations, most people
> > > > > > > > get it wrong most of the time.
> > > > > > > >
> > > > > > > > I get that others won't want to use this method, however it
> > > > > > > > would be detrimental if we don't have such a method for
> > > > > > > > those who would.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Peter.
> > > > > > > >
> > > > > > > > ----- Original message -----
> > > > > > > > >
> > > > > > > > > I feel like we’re going down a rabbit hole here when you
> > > > > > > > > start talking about exporting immutable objects.
> > > > > > > > > Wouldn’t it be kind of silly to export an immutable
> > > > > > > > > service? Isn’t the whole point that you
> interact
> > > > > > > > > with the service (i.e. alter its state) over a remote
> > > > > > > > > interface?
> > > > > > > > >
> > > > > > > > > Perhaps it’s better to say that exported services need to
> > > > > > > > > be thread-safe (which should be fairly obvious).
> > > > > > > > > Yes, immutable objects are inherently thread-safe, so for
> > > > > > > > > sharing information inside a VM, it makes some sense to
> > > > > > > > > minimize the number of mutable objects you pass around.
> > > > > > > > > But fundamentally, we’re talking about shared-state
> > > > > > > > > systems here.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > Greg.
> > > > > > > > >
> > > > > > > > > On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Pat your comment about non final fields is interesting.
> > > > > > > > > >
> > > > > > > > > > Isn't it also the case that we need to safely publish an
> > > > > > > > > > effectively immutable object to share it among threads?
> > > > > > > > > > That usually means copying it to a thread safe
> > > > > > > > > > collection or shared via a synchronized method, volatile
> > > > > > > > > > field, or final field in another object?
> > > > > > > > > >
> > > > > > > > > > So we should also make sure that Jeri uses safe
> > > > > > > > > > publication during export.
> > > > > > > > > >
> > > > > > > > > > That would allow a service that has no final fields to
> > > > > > > > > > start threads, then export from within a constructor
> > > > > > > > > > safely, provided all operations on non final fields
> > > > > > > > > > happen before starting threads and exporting.
> > > > > > > > > >
> > > > > > > > > > All our services have final fields, so Starter is more
> > > > > > > > > > appropriate for River's own services.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Peter.
> > > > > > > > > >
> > > > > > > > > > ----- Original message -----
> > > > > > > > > > > Hmm, good point, Startable, makes more sense.
> > > > > > > > > > >
> > > > > > > > > > > An object can be exported using Startable.
> > > > > > > > > > >
> > > > > > > > > > > I think we should have a policy to strongly discourage
> > > > > > > > > > > exporting from constructors.
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > Peter.
> > > > > > > > > > >
> > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > As far as I can tell, the special properties of
> > > > > > > > > > > > completing a constructor in
> the JLS memory
> > > > > > > > > > > > model are:
> > > > > > > > > > > >
> > > > > > > > > > > > 1. A happens-before edge from the end of the
> > > > > > > > > > > > constructor to the start of a
> finalizer.
> > > > > > > > > > > > (17.4.5)
> > > > > > > > > > > >
> > > > > > > > > > > > 2. The guarantee that any thread that only sees a
> > > > > > > > > > > > reference to an object after
> the end of
> > > > > > > > > > > > the constructor will see the correctly initialized
> > > > > > > > > > > > values of all final fields. (17.5)
> > > > > > > > > > > >
> > > > > > > > > > > > The special issue with final fields is that
> > > > > > > > > > > > implementations have freedom
> to optimize
> > > > > > > > > > > > access to final fields in ways that are not permitted
> > > > > > > > > > > > for non-final fields.
> Strategies for
> > > > > > > > > > > > thread safety that work for non-final
>
> > > > > > > > > > > > fields do not necessarily work for final fields. The
> > > > > > > > > > > > requirement for final field safety is that the
> > > > > > > > > > > > constructor end before another thread accesses the
> > > > > > > > > > > > newly constructed object.
> > > > > > > > > > > >
> > > > > > > > > > > > Calling a start() method after construction if the
> > > > > > > > > > > > class implements a new interface seems
> to me
> > > > > > > > > > > > to be harmless, backwards compatible, and
> > > > > > > > > > > > useful. It enables the simplest and most direct way
> > > > > > > > > > > > of preventing access to the
> new object by
> > > > > > > > > > > > another thread during construction.
> > > > > > > > > > > >
> > > > > > > > > > > > The roadmap issue is whether it should be required,
> > > > > > > > > > > > and if so the level of
> enforcement. For
> > > > > > > > > > > > example, there is no reason to require it if the
> > > > > > > > > > > > class does not declare any final
> fields.
> > > > > > > > > > > >
> > > > > > > > > > > > Incidentally, and as a detail, "Commission" does not
> > > > > > > > > > > > immediately make me think of
> having a
> > > > > > > > > > > > start() method that should be called after
> > > > > > > > > > > > construction. If you do go this way, the name needs
> > > > > > > > > > > > thought. "Startable" would be
> more
> > > > > > > > > > > > obvious, more memorable, more likely to be found on
> > > > > > > > > > > > searches, and more compatible with
> familiar
> > > > > > > > > > > > interface names such as
> "Cloneable" and
> > > > > > > > > > > > "Iterable".
> > > > > > > > > > > >
> > > > > > > > > > > > Patricia
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On 12/18/2013 2:18 AM, Peter wrote:
> > > > > > > > > > > > > Well, now seems like a good time to have the
> > > > > > > > > > > > > conversation.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Yes there are other ways, but I haven't seen one
> > > > > > > > > > > > > safe implementation yet, so...
> > > > > > > > > > > > >
> > > > > > > > > > > > > Does someone have a better way to solve this
> > > > > > > > > > > > > problem, has someone already solved this problem
> > > > > > > > > > > > > I'm unaware of that we can adopt, or is there a
> > > > > > > > > > > > > way that's more satisfactory?
> > > > > > > > > > > > >
> > > > > > > > > > > > > If not, is there something objectionable with the
> > > > > > > > > > > > > Commission interface and if so, how can we fix it?
> > > > > > > > > > > > >
> > > > > > > > > > > > > The SEVERE log message is logged by the River start
> > > > > > > > > > > > > package, other containers or frameworks can choose
> > > > > > > > > > > > > whether or not to do so, but I'd encourage them to
> > > > > > > > > > > > > do something similar, yes we can change it to WARN.
> > > > > > > > > > > > >
> > > > > > > > > > > > > A much harsher option is to throw an exception
> > > > > > > > > > > > > during export which breaks backward compatibility.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Regards,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Peter.
> > > > > > > > > > > > >
> > > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > > "org.apache.river.api.util.Commission is an
> > > > > > > > > > > > > > interface services should implement"
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > If it's a SHOULD, not a MUST, chucking out a
> > > > > > > > > > > > > > SEVERE is incorrect logger behaviour IMO. You
> > > > > > > > > > > > > > could issue a WARN if you like but for even that
> > > > > > > > > > > > > > I'd say you need to provide a roadmap explaining
> > > > > > > > > > > > > > why the warning and what you intend to do in
> > > > > > > > > > > > > > future and what you expect of service writers
> > > > > > > > > > > > > > such as myself.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Commission, at least from my point of view, is
> > > > > > > > > > > > > > your means (maybe the River community's - did
> > > > > > > > > > > > > > you ask us?) for satisfying your needs in
> > > > > > > > > > > > > > respect of the JMM. As we've discussed
> > > > > > > > > > > > > > previously, there are other ways too and they
> > > > > > > > > > > > > > work and they are safe if you know what you're
> > > > > > > > > > > > > > doing. Your contention was that most don't know
> > > > > > > > > > > > > > what they're doing hence, presumably, Commission.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > So the thing is, you are seemingly on a road to
> > > > > > > > > > > > > > asserting more structure (gosh, a standard?) on
> > > > > > > > > > > > > > the way people write their services. If so,
> > > > > > > > > > > > > > you'd best start flagging that honestly and
> > > > > > > > > > > > > > openly via a roadmap, deprecation and
> > > > > > > > > > > > > > such/whatever rather than sticking out logger
> > > > > > > > > > > > > > messages with no clear guidance and at the cost
> > > > > > > > > > > > > > of a certain amount of nuisance (no admin I know
> > > > > > > > > > > > > > likes SEVERE's being logged for something which
> > > > > > > > > > > > > > isn't critical cos it's noise they don't want in
> > > > > > > > > > > > > > log files).
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > And of course, we all know that when some entity
> > > > > > > > > > > > > > asserts a standard or requirement on others for
> > > > > > > > > > > > > > entry, they may choose not to enter. Does this
> > > > > > > > > > > > > > help your community or hinder it? The answer to
> > > > > > > > > > > > > > that is, it depends. On what? Have you asked or
> > > > > > > > > > > > > > tested? How have you tested? What would be
> > > > > > > > > > > > > > considered validation or lack of support?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I am not out to flame or troll rather I want to
> > > > > > > > > > > > > > see this community demonstrating good behaviour
> > > > > > > > > > > > > > and I'm not feeling like what's going on around
> > > > > > > > > > > > > > Commission (what is that big change in version
> > > > > > > > > > > > > > number really saying?) is such.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 18 December 2013 08:52, Peter
> > > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Just to clarify
> > > > > > > > > > > > > > > org.apache.river.api.util.Commission is an
> > > > > > > > > > > > > > > interface services should implement, I would
> > > > > > > > > > > > > > > encourage all container projects to pick up the
> > > > > > > > > > > > > > > interface and make suggestions for improvement
> > > > > > > > > > > > > > > if there are any issues.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Interface Commission {
> > > > > > > > > > > > > > > void start () throws Exception;
> > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > It's called after JMM safe construction to
> > > > > > > > > > > > > > > allow the service to start any threads and be
> > > > > > > > > > > > > > > exported.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The way that services are instantiated and
> > > > > > > > > > > > > > > > setup is an implementation detail.
>
> > > > > > > > > > > > > > > > When I think of
> compatibility
> > > > > > > > > > > > > > > > I think of the API and the lookup methods. We
> > > > > > > > > > > > > > > > think of compatibility from a client point of
> > > > > > > > > > > > > > > > view.
>
> > > > > > > > > > > > > > > > From the client point of view, using a
> > > > > > > > > > > > > > > > service looks like this:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > - Use multicast of unicast discovery to find
> > > > > > > > > > > > > > > > one or more ServiceRegistrar instances
>
> > > > > > > > > > > > > > > > - Call
> > > > > > > > > > > > > > > > lookup(…) on one or more of these instances
> > > > > > > > > > > > > > > > to get a set of service candidates
> -
> > > > > > > > > > > > > > > > Choose a candidate and prepare() it using a
> > > > > > > > > > > > > > > > ProxyPreparer, to yield a usable service
> > > > > > > > > > > > > > > > proxy.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > - Make calls on it.
>
> > > > > > > > > > > > > > > > Ideally hang on to this proxy
> > > > > > > > > > > > > > > > instance, so you can skip the discovery and
> > > > > > > > > > > > > > > > lookup next time you need it.
>
> > > > > > > > > > > > > > > > - If the call
> > > > > > > > > > > > > > > > fails, repeat the lookup (and possibly
> > > > > > > > > > > > > > > > discovery) til you get a proxy that works.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Nowhere does the client need to know whether
> > > > > > > > > > > > > > > > the service instance is started up using the
> > > > > > > > > > > > > > > > “com.sun.jini.start” mechanism, your
> > > > > > > > > > > > > > > > Commission interface, some other IOC
> > > > > > > > > > > > > > > > container (Rio, Harvester, Seven or
> > > > > > > > > > > > > > > > RiverContainer) or some unknown mechanism
> > > > > > > > > > > > > > > > that starts with a static main() method.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > JSK2.0 was 2.0 because of the introduction
> > > > > > > > > > > > > > > > of the proxy verification mechanisms, as
> > > > > > > > > > > > > > > > well as JERI. Absent some new client usage
> > > > > > > > > > > > > > > > mechanism, River doesn’t need to go to 3.0.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Greg.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > On Dec 17, 2013, at 1:58 PM, Peter
> > > > > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I think changing services to use safe
> > > > > > > > > > > > > > > > > construction techniques is enough to cause
> > > > > > > > > > > > > > > > > the version jump.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > At this point I've allowed services to
> > > > > > > > > > > > > > > > > continue unsafe construction practices,
> > > > > > > > > > > > > > > > > while logging a SEVERE warning when the
> > > > > > > > > > > > > > > > > Commission interface isn't implemented,
> > > > > > > > > > > > > > > > > rather than fail.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > This is a fundamental change to the way
> > > > > > > > > > > > > > > > > services are written.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Assuming that there aren’t major
> > > > > > > > > > > > > > > > > > incompatibilities, I think that would be
> > > > > > > > > > > > > > > > > > a “minor” version change according to our
> > > > > > > > > > > > > > > > > > versioning policy, so we’d be looking at
> > > > > > > > > > > > > > > > > > the “2.3” branch rather than a “3.0”
> > > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I’m still unnerved by the massive
> > > > > > > > > > > > > > > > > > amounts of changes to both code and
> > > > > > > > > > > > > > > > > > tests in the qa_refactor branch, as well
> > > > > > > > > > > > > > > > > > as the apparent instability of the code,
> > > > > > > > > > > > > > > > > > although that seems to be improving. In
> > > > > > > > > > > > > > > > > > the next few weeks I’m going to try and
> > > > > > > > > > > > > > > > > > setup a cross-test case, to see what the
> > > > > > > > > > > > > > > > > > “2.2” tests say about the potential
> > > > > > > > > > > > > > > > > > “2.3” release and vice-versa.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > I think what I’d really like to see is an
> > > > > > > > > > > > > > > > > > incremental approach where we update
> > > > > > > > > > > > > > > > > > limited components of the “2.2” branch,
> > > > > > > > > > > > > > > > > > one at a time. Is there anything that we
> > > > > > > > > > > > > > > > > > could pull out piecemeal? Maybe it
> > > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > > make sense to split out the
> > > > > > > > > > > > > > > > > > infrastructure services, like Reggie,
> > > > > > > > > > > > > > > > > > Mahalo, and Outrigger into different
> > > > > > > > > > > > > > > > > > sub-projects that could be updated
> > > > > > > > > > > > > > > > > > separately?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Greg.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > On Dec 17, 2013, at 5:03 AM, Peter
> > > > > > > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > When the qa_refactor branch
> > > > > > > > > > > > > > > > > > > stabilises, I plan to merge trunk and
> > > > > > > > > > > > > > > > > > > provide a beta release for client
> > > > > > > > > > > > > > > > > > > compatibility testing.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Changes made have been focused on
> > > > > > > > > > > > > > > > > > > making our code thread safe, there are
> > > > > > > > > > > > > > > > > > > significant changes internally, the
> > > > > > > > > > > > > > > > > > > public api remains focused on backward
> > > > > > > > > > > > > > > > > > > compatibility, however it is advisable
> > > > > > > > > > > > > > > > > > > that client services adopt new safe
> > > > > > > > > > > > > > > > > > > construction techniques for services
> > > > > > > > > > > > > > > > > > > and implement the new Commission
> > > > > > > > > > > > > > > > > > > interface.
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > What's a suitable test period for
> > > > > > > > > > > > > > > > > > > client testing?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > >
> >
>
>
Re: VOTE: add Startable to Jini specification
Posted by Greg Trasuk <tr...@stratuscom.com>.
Hi Peter, and list:
A couple of things here -
1 - We’re collaborating on an email list, and we’re pretty evenly spread across the global timezones, not to mention that River is a part-time pursuit for most of us. There may be some delay between when a point is made and when anyone else actually reads it. Then it may take some time to digest a statement and craft a well-reasoned reply. Then it’ll be a while before the original sender reads the reply, and so on. So I think we have to accept that it takes some time to reach consensus.
2 - We follow the standard Apache voting conventions at (http://www.apache.org/foundation/voting.html). Code review votes require lazy consensus, i.e. no “-1” votes. The three-vote requirement is for releases. So, you’re actually good to go, although, given the above I’d wait for while. Also, for some issues, it might be a good idea to launch a “Discuss” thread and let that settle out prior to a “vote” thread.
So, following the voting norms, you’re proposal is good-to-go (although given 1, I’d give it a little more time). I haven’t seen any negative votes. Certainly my comment was “+0”. In fact I’ve seen good discussion from people I consider experts in the field. Even if someone was critical of the idea (which I haven’t seen), critics give you good information, and it’s usually a good idea to consider opposing points of view.
Cheers,
Greg.
On Dec 19, 2013, at 11:49 PM, Peter <ji...@zeus.net.au> wrote:
> This only needs one more +1 vote to pass or -1 for a blocking vote.
>
> I only need a single method interface so I can fix a serious flaw with our api? It's not even a breaking change!
>
> Get over your fear people, let us develop, we are maintaining backward compatibility.
>
> Without development River will go to the Attic, this is supposed to be an open source development project, not a Jini code museum.
>
> Get used to the fact there will be a beta release for testing soon.
>
> I have people declining to participate because of the hostility towards development that exists on this list. Lets show some determination, ignore the fear and get the job done.
>
> The Jini team, they had vision, they might not have delivered what they promised, their marketing might have been mixed up, the limitations of Java might have gotten in the way, but by God, they had vision. When you reach high you risk failure, but without failure there is no success either.
>
> We need to decide, is River a Jini code museum, or are we planning to make River work outside local networks and become part of the internet? Are there people here with vision? IPv6 is on our doorstep, there are data silo's and high powered mobile devices everywhere we could be connecting to make useful applications. Or do we live in fear and squander our opportunities?
>
> Perhaps we should fork into two projects, one for Jini 2.x and one for those who can see a future, have vision and would like to share but keep getting shouted down?
>
> Lets create a Jini maintenance project, there's no trademark now, let it have it's own lists, and set River free.
>
> Don't take it personal we can still cooperate, I understand people have established niche's, but fresh developers have itches to scratch.
>
> And before you go flaming me, food for thought: People can only judge on what they know, they cannot judge upon what they don't, fear the unknown, or learn.
>
> Regards,
>
> Peter.
>
> ----- Original message -----
>>
>> I don’t really see a need to mess around with the existing
>> infrastructure services. Surely if we can start them as it is and they
>> can be reached through lookup, that’s good enough? I’m more concerned
>> with having an interface that makes sense for developers to write their
>> own services with. I suspect we need to just do a few experiments and
>> see what works.
>>
>> I guess that’s one of my frustrations with all this “refactoring” of the
>> existing codebase - if it doesn’t make it easier for people to use
>> Jini/River to create their own service-oriented architectures, then it
>> just seems like a lot of navel-gazing. Similarly, we’ve had statements
>> about reorganizing the source code to make it easier to develop with.
>> To which I ask, “when was the last time anyone worried about the source
>> code of Tomcat, or Websphere, or Weblogic?, or whether they used Maven
>> or not?”. Even talking about performance is not very helpful until the
>> average corporate coder at the City of Wherever can actually use the
>> product.
>>
>> We need to get to a point where people are building real things on top
>> of River, not messing around with River. That’s not to say that the
>> River code can’t be improved or modernized, but it reached “good enough”
>> back in 2003 or so.
>>
>> Lest I sound like I’m complaining, let me say it’s great that we’ve had
>> this much traffic on the list in the last few weeks! I think we’re
>> having great discussions- keep it up, folks!
>>
>> Greg.
>>
>> On Dec 19, 2013, at 8:54 PM, Peter <ji...@zeus.net.au> wrote:
>>
>>> If you can work put what a common reflective constructor signature you
>>> need is, any lifecycle methods required, I'll include these for every
>>> existing Jini service and we can make a standard.
>>>
>>> We can also support JSR330 which is AL2 licensed.
>>>
>>> Support @Inject for the constructor.
>>>
>>> Provider<JavaSpace05>
>>>
>>> Regards,
>>>
>>> Peter.
>>>
>>>
>>> ----- Original message -----
>>>> Hopefully Startable will get enough votes to get across the line, we
>>>> need to make progress, if you care about River you'll vote for it,
>>>> not because it's the perfect solution, simply because it's an
>>>> acceptable solution that makes sense and it marks a progress
>>>> milestone.
>>>>
>>>> The reflective constructor signature is acceptable and is used by the
>>>> starter kit and Activation.
>>>>
>>>> We've got Groovy Configuration.
>>>>
>>>> The service class can also define a static main method.
>>>>
>>>> If we at least accept these signatures for our services, they could
>>>> run in any container.
>>>>
>>>> Perhaps you also want a public shutdown method for lifecycle
>>>> management? Closeable would also be acceptable?
>>>>
>>>> Is there anything else you need?
>>>>
>>>> Regards,
>>>>
>>>> Peter.
>>>>
>>>> ----- Original message -----
>>>>> "What we need is a discussion on a more developer-friendly service
>>>>> startup convention to replace ServiceStarter."
>>>>>
>>>>> Maybe, that is to say, I'm not anti but I feel there's an
>>>>> additional wider aspect to kickaround as well....
>>>>>
>>>>> Should there be a failure of the type we've discussed so far, where
>>>>> does the responsibility for sorting it out lie? The service writer,
>>>>> they created the bug in the first place. BUT, has it occurred to
>>>>> anyone that export() isn't the only place multi-threading problems
>>>>> can occur in a service implementation?
>>>>>
>>>>> Even if they've got service construction right (with whatever
>>>>> convention or pattern you care to adopt, me, I'm in the camp of if
>>>>> it works because it accounts for the JMM it's fine) have they got
>>>>> all their multi-threading right elsewhere? What about all those
>>>>> service methods exported and advertised by JoinManager?
>>>>>
>>>>> Do we actually think it's possible to address that challenge? (You
>>>>> could do it by single-locking the entire service so only one call
>>>>> can be dispatched at a time, yeah right, that's palatable). What
>>>>> makes export a special case in this context? Why does that matter
>>>>> so when there are many other ways to screw up?
>>>>>
>>>>>
>>>>> On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> +0. Let me explain…
>>>>>>
>>>>>> Having a separate interface to say start() is all very well and
>>>>>> harmless in itself, but:
>>>>>>
>>>>>> (a) It doesn’t really address the issue of how service instances
>>>>>> should be started. We don’t actually have solid
>>>>>> recommendations on this issue, although there have been attempts
>>>>>> in various containers to systematize it.
>>>>>>
>>>>>> (b) the proposed warning addresses a case where a service
>>>>>> instance is doing “exporter.export(this);”, which I’d call bad
>>>>>> practice in any case. It happens that ServiceStarter appears to
>>>>>> encourage this use case because it works by calling a
>>>>>> constructor. But the called constructor could (and should)
>>>>>> construct an instance of some other class, and then export and
>>>>>> register that instance rather than “this”. That style doesn’t
>>>>>> mandate a Startable interface, and having Startable doesn’t
>>>>>> mandate good multi-threaded code. In other words, you could
>>>>>> implement Startable and still have poor thread-safety.
>>>>>>
>>>>>> (c) even where there is an apparent problem, the exposure window
>>>>>> is very small. Consider the last bits of the init(…) method in
>>>>>> “com.sun.jini.reggie.ServiceRegistrarImpl”
>>>>>>
>>>>>> myRef = (Registrar) serverExporter.export(this);
>>>>>> proxy = RegistrarProxy.getInstance(myRef, myServiceID);
>>>>>> myLocator = (proxy instanceof RemoteMethodControl) ?
>>>>>> new ConstrainableLookupLocator(
>>>>>> unicastDiscoveryHost, unicaster.port, null) :
>>>>>> new LookupLocator(unicastDiscoveryHost, unicaster.port);
>>>>>> /* register myself */
>>>>>> Item item = new Item(new ServiceItem(myServiceID,
>>>>>> proxy,
>>>>>> lookupAttrs));
>>>>>> SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
>>>>>> addService(reg);
>>>>>> if (log != null) {
>>>>>> log.snapshot();
>>>>>> }
>>>>>>
>>>>>> try {
>>>>>> DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
>>>>>> discoer;
>>>>>> String[] groups = dgm.getGroups();
>>>>>> if (groups == null || groups.length > 0) {
>>>>>> throw new ConfigurationException(
>>>>>> "discoveryManager must be initially configured with " +
>>>>>> "no groups");
>>>>>> }
>>>>>> DiscoveryLocatorManagement dlm =
>>>>>> (DiscoveryLocatorManagement) discoer;
>>>>>> if (dlm.getLocators().length > 0) {
>>>>>> throw new ConfigurationException(
>>>>>> "discoveryManager must be initially configured with " +
>>>>>> "no locators");
>>>>>> }
>>>>>> dgm.setGroups(lookupGroups);
>>>>>> dlm.setLocators(lookupLocators);
>>>>>> } catch (ClassCastException e) {
>>>>>> throw new ConfigurationException(null, e);
>>>>>> }
>>>>>> joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
>>>>>> discoer, null, config);
>>>>>>
>>>>>> /* start up all the daemon threads */
>>>>>> serviceExpirer.start();
>>>>>> eventExpirer.start();
>>>>>> unicaster.start();
>>>>>> multicaster.start();
>>>>>> announcer.start();
>>>>>>
>>>>>> /* Shutdown hook so reggie sends a final announcement
>>>>>> * packet if VM is terminated. If reggie is terminated
>>>>>> * through DestroyAdmin.destroy() this hook will have no effect.
>>>>>> * A timeout on announcer.join() was considered but not deemed
>>>>>> * necessary at this point in time.
>>>>>> */
>>>>>> Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
>>>>>> public void run() {
>>>>>> try {
>>>>>> announcer.interrupt();
>>>>>> announcer.join();
>>>>>> } catch (Throwable t) {
>>>>>> logThrow(Level.FINEST, getClass().getName(),
>>>>>> "run", "exception shutting announcer down",
>>>>>> new Object[]{}, t);
>>>>>> }
>>>>>> }
>>>>>> }));
>>>>>>
>>>>>> snapshotter.start();
>>>>>> if (logger.isLoggable(Level.INFO)) {
>>>>>> logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
>>>>>> new Object[]{ myServiceID,
>>>>>> Arrays.asList(memberGroups),
>>>>>> myLocator });
>>>>>> }
>>>>>> ready.ready();
>>>>>> }
>>>>>>
>>>>>>
>>>>>> The window from when the join manager is started up (ignore the
>>>>>> serverExporter.export(). The only way a client gets to this
>>>>>> endpoint is through a proxy downloaded from the registrar) to
>>>>>> when the init() method exits is effectively 9 lines of code,
>>>>>> during which time a _lot_ of network i/o would have to happen in
>>>>>> order for the object to be used prior to the constructor exit.
>>>>>> As well, the actual remote calls are filtered through a command
>>>>>> interface and through the tasker object, and the command classes
>>>>>> often go through synchronized() blocks, which (I think) act as
>>>>>> happens-before fences.
>>>>>>
>>>>>> The whole thing seems to me a little like the old servlet days,
>>>>>> when we got the SingleThreadModel interface and it only served
>>>>>> to give naive developers a false sense of security.
>>>>>>
>>>>>> What we need is a discussion on a more developer-friendly service
>>>>>> startup convention to replace ServiceStarter.
>>>>>>
>>>>>> Having said that, I’m not overly opposed to having Startable,
>>>>>> given that as Dan suggested, any output from the ServiceStarter
>>>>>> is in the form of a suggestion rather than an error.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Greg.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
>>>>>>
>>>>>>> +1 Peter.
>>>>>>>
>>>>>>>
>>>>>>> I've been going over the Exporter implementations and Exporter
>>>>>>> interface.
>>>>>>>
>>>>>>> The Exporter interface specifies that export and unexport
>>>>>>> behaviour is
>>>>>> defined by the implementation.
>>>>>>>
>>>>>>> That means it's up to the implementation to specify whether a
>>>>>>> happens
>>>>>> before edge occures before export.
>>>>>>>
>>>>>>> I propose we document in Exporter, that if export is called
>>>>>>> during
>>>>>> object construction then that object's implementation cannot
>>>>>> safely use final fields.
>>>>>>>
>>>>>>> We should also provide a @see reference to Startable to
>>>>>>> reccommend its
>>>>>> use when final fields are desirable and include Startable in the
>>>>>> spec.
>>>>>>>
>>>>>>> We don't need to rewrite the example of exporting in
>>>>>>> net.jini.config as
>>>>>> export isn't performed during construction.
>>>>>>>
>>>>>>> This is a minor update to the Jini Specification.
>>>>>>>
>>>>>>> The FINE log message is an implementation detail and won't be
>>>>>>> included.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>> ----- Original message -----
>>>>>>>> No, an effectively immutable field is one whose reference
>>>>>>>> doesn't change after the service objects safe publication.
>>>>>>>>
>>>>>>>> An immutable field is final.
>>>>>>>>
>>>>>>>> It would be Jeri's responsibility to safely publish the
>>>>>>>> service during export to ensure a happens before event
>>>>>>>> occurs, ensuring all non volatile fields the service
>>>>>>>> contains are safely published and visible.
>>>>>>>>
>>>>>>>> It's possible to have mutable fields guarded by synchronized
>>>>>>>> methods, or volatile fields.
>>>>>>>>
>>>>>>>> But it isn't possible to have final fields when exporting
>>>>>>>> within the constructor.
>>>>>>>>
>>>>>>>> If the service contains effectively immutable field
>>>>>>>> references, it must be safely published or if another thread
>>>>>>>> accesses those fields through unsynchronized methods, the
>>>>>>>> thread may only see their default value.
>>>>>>>>
>>>>>>>> All our service implementations have final fields. All
>>>>>>>> our services
>>>>>> had
>>>>>>>> some unsynchronized access to non volatile, non final fields.
>>>>>>>>
>>>>>>>> Trouble is, when you can't have final fields, it's very
>>>>>>>> difficult to determine which fields are effectively
>>>>>>>> immutable and the original programmers design intent, if
>>>>>>>> there's no comment.
>>>>>>>>
>>>>>>>> Our service implementations all had unsynchronized init()
>>>>>>>> methods called from with the constructor that started
>>>>>>>> threads and exported Some
>>>>>> fields
>>>>>>>> were final.
>>>>>>>>
>>>>>>>> For me, a service that contains final fields and is exported
>>>>>>>> after construction is easier to maintain and understand.
>>>>>>>>
>>>>>>>> I suspect, like our Jini service imlementations, most people
>>>>>>>> get it wrong most of the time.
>>>>>>>>
>>>>>>>> I get that others won't want to use this method, however it
>>>>>>>> would be detrimental if we don't have such a method for
>>>>>>>> those who would.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Peter.
>>>>>>>>
>>>>>>>> ----- Original message -----
>>>>>>>>>
>>>>>>>>> I feel like we’re going down a rabbit hole here when you
>>>>>>>>> start talking about exporting immutable objects.
>>>>>>>>> Wouldn’t it be kind of silly to export an immutable
>>>>>>>>> service? Isn’t the whole point that you interact
>>>>>>>>> with the service (i.e. alter its state) over a remote
>>>>>>>>> interface?
>>>>>>>>>
>>>>>>>>> Perhaps it’s better to say that exported services need to
>>>>>>>>> be thread-safe (which should be fairly obvious).
>>>>>>>>> Yes, immutable objects are inherently thread-safe, so for
>>>>>>>>> sharing information inside a VM, it makes some sense to
>>>>>>>>> minimize the number of mutable objects you pass around.
>>>>>>>>> But fundamentally, we’re talking about shared-state
>>>>>>>>> systems here.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Greg.
>>>>>>>>>
>>>>>>>>> On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Pat your comment about non final fields is interesting.
>>>>>>>>>>
>>>>>>>>>> Isn't it also the case that we need to safely publish an
>>>>>>>>>> effectively immutable object to share it among threads?
>>>>>>>>>> That usually means copying it to a thread safe
>>>>>>>>>> collection or shared via a synchronized method, volatile
>>>>>>>>>> field, or final field in another object?
>>>>>>>>>>
>>>>>>>>>> So we should also make sure that Jeri uses safe
>>>>>>>>>> publication during export.
>>>>>>>>>>
>>>>>>>>>> That would allow a service that has no final fields to
>>>>>>>>>> start threads, then export from within a constructor
>>>>>>>>>> safely, provided all operations on non final fields
>>>>>>>>>> happen before starting threads and exporting.
>>>>>>>>>>
>>>>>>>>>> All our services have final fields, so Starter is more
>>>>>>>>>> appropriate for River's own services.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> Peter.
>>>>>>>>>>
>>>>>>>>>> ----- Original message -----
>>>>>>>>>>> Hmm, good point, Startable, makes more sense.
>>>>>>>>>>>
>>>>>>>>>>> An object can be exported using Startable.
>>>>>>>>>>>
>>>>>>>>>>> I think we should have a policy to strongly discourage
>>>>>>>>>>> exporting from constructors.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Peter.
>>>>>>>>>>>
>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>> As far as I can tell, the special properties of
>>>>>>>>>>>> completing a constructor in the JLS memory
>>>>>>>>>>>> model are:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. A happens-before edge from the end of the
>>>>>>>>>>>> constructor to the start of a finalizer.
>>>>>>>>>>>> (17.4.5)
>>>>>>>>>>>>
>>>>>>>>>>>> 2. The guarantee that any thread that only sees a
>>>>>>>>>>>> reference to an object after the end of
>>>>>>>>>>>> the constructor will see the correctly initialized
>>>>>>>>>>>> values of all final fields. (17.5)
>>>>>>>>>>>>
>>>>>>>>>>>> The special issue with final fields is that
>>>>>>>>>>>> implementations have freedom to optimize
>>>>>>>>>>>> access to final fields in ways that are not permitted
>>>>>>>>>>>> for non-final fields. Strategies for
>>>>>>>>>>>> thread safety that work for non-final
>>>>>>>>>>>> fields do not necessarily work for final fields. The
>>>>>>>>>>>> requirement for final field safety is that the
>>>>>>>>>>>> constructor end before another thread accesses the
>>>>>>>>>>>> newly constructed object.
>>>>>>>>>>>>
>>>>>>>>>>>> Calling a start() method after construction if the
>>>>>>>>>>>> class implements a new interface seems to me
>>>>>>>>>>>> to be harmless, backwards compatible, and
>>>>>>>>>>>> useful. It enables the simplest and most direct way
>>>>>>>>>>>> of preventing access to the new object by
>>>>>>>>>>>> another thread during construction.
>>>>>>>>>>>>
>>>>>>>>>>>> The roadmap issue is whether it should be required,
>>>>>>>>>>>> and if so the level of enforcement. For
>>>>>>>>>>>> example, there is no reason to require it if the
>>>>>>>>>>>> class does not declare any final fields.
>>>>>>>>>>>>
>>>>>>>>>>>> Incidentally, and as a detail, "Commission" does not
>>>>>>>>>>>> immediately make me think of having a
>>>>>>>>>>>> start() method that should be called after
>>>>>>>>>>>> construction. If you do go this way, the name needs
>>>>>>>>>>>> thought. "Startable" would be more
>>>>>>>>>>>> obvious, more memorable, more likely to be found on
>>>>>>>>>>>> searches, and more compatible with familiar
>>>>>>>>>>>> interface names such as "Cloneable" and
>>>>>>>>>>>> "Iterable".
>>>>>>>>>>>>
>>>>>>>>>>>> Patricia
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 12/18/2013 2:18 AM, Peter wrote:
>>>>>>>>>>>>> Well, now seems like a good time to have the
>>>>>>>>>>>>> conversation.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes there are other ways, but I haven't seen one
>>>>>>>>>>>>> safe implementation yet, so...
>>>>>>>>>>>>>
>>>>>>>>>>>>> Does someone have a better way to solve this
>>>>>>>>>>>>> problem, has someone already solved this problem
>>>>>>>>>>>>> I'm unaware of that we can adopt, or is there a
>>>>>>>>>>>>> way that's more satisfactory?
>>>>>>>>>>>>>
>>>>>>>>>>>>> If not, is there something objectionable with the
>>>>>>>>>>>>> Commission interface and if so, how can we fix it?
>>>>>>>>>>>>>
>>>>>>>>>>>>> The SEVERE log message is logged by the River start
>>>>>>>>>>>>> package, other containers or frameworks can choose
>>>>>>>>>>>>> whether or not to do so, but I'd encourage them to
>>>>>>>>>>>>> do something similar, yes we can change it to WARN.
>>>>>>>>>>>>>
>>>>>>>>>>>>> A much harsher option is to throw an exception
>>>>>>>>>>>>> during export which breaks backward compatibility.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>>
>>>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>>>> "org.apache.river.api.util.Commission is an
>>>>>>>>>>>>>> interface services should implement"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If it's a SHOULD, not a MUST, chucking out a
>>>>>>>>>>>>>> SEVERE is incorrect logger behaviour IMO. You
>>>>>>>>>>>>>> could issue a WARN if you like but for even that
>>>>>>>>>>>>>> I'd say you need to provide a roadmap explaining
>>>>>>>>>>>>>> why the warning and what you intend to do in
>>>>>>>>>>>>>> future and what you expect of service writers
>>>>>>>>>>>>>> such as myself.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Commission, at least from my point of view, is
>>>>>>>>>>>>>> your means (maybe the River community's - did
>>>>>>>>>>>>>> you ask us?) for satisfying your needs in
>>>>>>>>>>>>>> respect of the JMM. As we've discussed
>>>>>>>>>>>>>> previously, there are other ways too and they
>>>>>>>>>>>>>> work and they are safe if you know what you're
>>>>>>>>>>>>>> doing. Your contention was that most don't know
>>>>>>>>>>>>>> what they're doing hence, presumably, Commission.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So the thing is, you are seemingly on a road to
>>>>>>>>>>>>>> asserting more structure (gosh, a standard?) on
>>>>>>>>>>>>>> the way people write their services. If so,
>>>>>>>>>>>>>> you'd best start flagging that honestly and
>>>>>>>>>>>>>> openly via a roadmap, deprecation and
>>>>>>>>>>>>>> such/whatever rather than sticking out logger
>>>>>>>>>>>>>> messages with no clear guidance and at the cost
>>>>>>>>>>>>>> of a certain amount of nuisance (no admin I know
>>>>>>>>>>>>>> likes SEVERE's being logged for something which
>>>>>>>>>>>>>> isn't critical cos it's noise they don't want in
>>>>>>>>>>>>>> log files).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And of course, we all know that when some entity
>>>>>>>>>>>>>> asserts a standard or requirement on others for
>>>>>>>>>>>>>> entry, they may choose not to enter. Does this
>>>>>>>>>>>>>> help your community or hinder it? The answer to
>>>>>>>>>>>>>> that is, it depends. On what? Have you asked or
>>>>>>>>>>>>>> tested? How have you tested? What would be
>>>>>>>>>>>>>> considered validation or lack of support?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am not out to flame or troll rather I want to
>>>>>>>>>>>>>> see this community demonstrating good behaviour
>>>>>>>>>>>>>> and I'm not feeling like what's going on around
>>>>>>>>>>>>>> Commission (what is that big change in version
>>>>>>>>>>>>>> number really saying?) is such.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 18 December 2013 08:52, Peter
>>>>>>>>>>>>>> <ji...@zeus.net.au> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just to clarify
>>>>>>>>>>>>>>> org.apache.river.api.util.Commission is an
>>>>>>>>>>>>>>> interface services should implement, I would
>>>>>>>>>>>>>>> encourage all container projects to pick up the
>>>>>>>>>>>>>>> interface and make suggestions for improvement
>>>>>>>>>>>>>>> if there are any issues.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Interface Commission {
>>>>>>>>>>>>>>> void start () throws Exception;
>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's called after JMM safe construction to
>>>>>>>>>>>>>>> allow the service to start any threads and be
>>>>>>>>>>>>>>> exported.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The way that services are instantiated and
>>>>>>>>>>>>>>>> setup is an implementation detail.
>>>>>>>>>>>>>>>> When I think of compatibility
>>>>>>>>>>>>>>>> I think of the API and the lookup methods. We
>>>>>>>>>>>>>>>> think of compatibility from a client point of
>>>>>>>>>>>>>>>> view.
>>>>>>>>>>>>>>>> From the client point of view, using a
>>>>>>>>>>>>>>>> service looks like this:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Use multicast of unicast discovery to find
>>>>>>>>>>>>>>>> one or more ServiceRegistrar instances
>>>>>>>>>>>>>>>> - Call
>>>>>>>>>>>>>>>> lookup(…) on one or more of these instances
>>>>>>>>>>>>>>>> to get a set of service candidates -
>>>>>>>>>>>>>>>> Choose a candidate and prepare() it using a
>>>>>>>>>>>>>>>> ProxyPreparer, to yield a usable service
>>>>>>>>>>>>>>>> proxy.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Make calls on it.
>>>>>>>>>>>>>>>> Ideally hang on to this proxy
>>>>>>>>>>>>>>>> instance, so you can skip the discovery and
>>>>>>>>>>>>>>>> lookup next time you need it.
>>>>>>>>>>>>>>>> - If the call
>>>>>>>>>>>>>>>> fails, repeat the lookup (and possibly
>>>>>>>>>>>>>>>> discovery) til you get a proxy that works.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Nowhere does the client need to know whether
>>>>>>>>>>>>>>>> the service instance is started up using the
>>>>>>>>>>>>>>>> “com.sun.jini.start” mechanism, your
>>>>>>>>>>>>>>>> Commission interface, some other IOC
>>>>>>>>>>>>>>>> container (Rio, Harvester, Seven or
>>>>>>>>>>>>>>>> RiverContainer) or some unknown mechanism
>>>>>>>>>>>>>>>> that starts with a static main() method.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> JSK2.0 was 2.0 because of the introduction
>>>>>>>>>>>>>>>> of the proxy verification mechanisms, as
>>>>>>>>>>>>>>>> well as JERI. Absent some new client usage
>>>>>>>>>>>>>>>> mechanism, River doesn’t need to go to 3.0.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Greg.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Dec 17, 2013, at 1:58 PM, Peter
>>>>>>>>>>>>>>>> <ji...@zeus.net.au> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think changing services to use safe
>>>>>>>>>>>>>>>>> construction techniques is enough to cause
>>>>>>>>>>>>>>>>> the version jump.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> At this point I've allowed services to
>>>>>>>>>>>>>>>>> continue unsafe construction practices,
>>>>>>>>>>>>>>>>> while logging a SEVERE warning when the
>>>>>>>>>>>>>>>>> Commission interface isn't implemented,
>>>>>>>>>>>>>>>>> rather than fail.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is a fundamental change to the way
>>>>>>>>>>>>>>>>> services are written.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Assuming that there aren’t major
>>>>>>>>>>>>>>>>>> incompatibilities, I think that would be
>>>>>>>>>>>>>>>>>> a “minor” version change according to our
>>>>>>>>>>>>>>>>>> versioning policy, so we’d be looking at
>>>>>>>>>>>>>>>>>> the “2.3” branch rather than a “3.0”
>>>>>>>>>>>>>>>>>> release.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I’m still unnerved by the massive
>>>>>>>>>>>>>>>>>> amounts of changes to both code and
>>>>>>>>>>>>>>>>>> tests in the qa_refactor branch, as well
>>>>>>>>>>>>>>>>>> as the apparent instability of the code,
>>>>>>>>>>>>>>>>>> although that seems to be improving. In
>>>>>>>>>>>>>>>>>> the next few weeks I’m going to try and
>>>>>>>>>>>>>>>>>> setup a cross-test case, to see what the
>>>>>>>>>>>>>>>>>> “2.2” tests say about the potential
>>>>>>>>>>>>>>>>>> “2.3” release and vice-versa.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I think what I’d really like to see is an
>>>>>>>>>>>>>>>>>> incremental approach where we update
>>>>>>>>>>>>>>>>>> limited components of the “2.2” branch,
>>>>>>>>>>>>>>>>>> one at a time. Is there anything that we
>>>>>>>>>>>>>>>>>> could pull out piecemeal? Maybe it
>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> make sense to split out the
>>>>>>>>>>>>>>>>>> infrastructure services, like Reggie,
>>>>>>>>>>>>>>>>>> Mahalo, and Outrigger into different
>>>>>>>>>>>>>>>>>> sub-projects that could be updated
>>>>>>>>>>>>>>>>>> separately?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Greg.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Dec 17, 2013, at 5:03 AM, Peter
>>>>>>>>>>>>>>>>>> <ji...@zeus.net.au> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> When the qa_refactor branch
>>>>>>>>>>>>>>>>>>> stabilises, I plan to merge trunk and
>>>>>>>>>>>>>>>>>>> provide a beta release for client
>>>>>>>>>>>>>>>>>>> compatibility testing.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Changes made have been focused on
>>>>>>>>>>>>>>>>>>> making our code thread safe, there are
>>>>>>>>>>>>>>>>>>> significant changes internally, the
>>>>>>>>>>>>>>>>>>> public api remains focused on backward
>>>>>>>>>>>>>>>>>>> compatibility, however it is advisable
>>>>>>>>>>>>>>>>>>> that client services adopt new safe
>>>>>>>>>>>>>>>>>>> construction techniques for services
>>>>>>>>>>>>>>>>>>> and implement the new Commission
>>>>>>>>>>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What's a suitable test period for
>>>>>>>>>>>>>>>>>>> client testing?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
This only needs one more +1 vote to pass or -1 for a blocking vote.
I only need a single method interface so I can fix a serious flaw with our api? It's not even a breaking change!
Get over your fear people, let us develop, we are maintaining backward compatibility.
Without development River will go to the Attic, this is supposed to be an open source development project, not a Jini code museum.
Get used to the fact there will be a beta release for testing soon.
I have people declining to participate because of the hostility towards development that exists on this list. Lets show some determination, ignore the fear and get the job done.
The Jini team, they had vision, they might not have delivered what they promised, their marketing might have been mixed up, the limitations of Java might have gotten in the way, but by God, they had vision. When you reach high you risk failure, but without failure there is no success either.
We need to decide, is River a Jini code museum, or are we planning to make River work outside local networks and become part of the internet? Are there people here with vision? IPv6 is on our doorstep, there are data silo's and high powered mobile devices everywhere we could be connecting to make useful applications. Or do we live in fear and squander our opportunities?
Perhaps we should fork into two projects, one for Jini 2.x and one for those who can see a future, have vision and would like to share but keep getting shouted down?
Lets create a Jini maintenance project, there's no trademark now, let it have it's own lists, and set River free.
Don't take it personal we can still cooperate, I understand people have established niche's, but fresh developers have itches to scratch.
And before you go flaming me, food for thought: People can only judge on what they know, they cannot judge upon what they don't, fear the unknown, or learn.
Regards,
Peter.
----- Original message -----
>
> I don’t really see a need to mess around with the existing
> infrastructure services. Surely if we can start them as it is and they
> can be reached through lookup, that’s good enough? I’m more concerned
> with having an interface that makes sense for developers to write their
> own services with. I suspect we need to just do a few experiments and
> see what works.
>
> I guess that’s one of my frustrations with all this “refactoring” of the
> existing codebase - if it doesn’t make it easier for people to use
> Jini/River to create their own service-oriented architectures, then it
> just seems like a lot of navel-gazing. Similarly, we’ve had statements
> about reorganizing the source code to make it easier to develop with.
> To which I ask, “when was the last time anyone worried about the source
> code of Tomcat, or Websphere, or Weblogic?, or whether they used Maven
> or not?”. Even talking about performance is not very helpful until the
> average corporate coder at the City of Wherever can actually use the
> product.
>
> We need to get to a point where people are building real things on top
> of River, not messing around with River. That’s not to say that the
> River code can’t be improved or modernized, but it reached “good enough”
> back in 2003 or so.
>
> Lest I sound like I’m complaining, let me say it’s great that we’ve had
> this much traffic on the list in the last few weeks! I think we’re
> having great discussions- keep it up, folks!
>
> Greg.
>
> On Dec 19, 2013, at 8:54 PM, Peter <ji...@zeus.net.au> wrote:
>
> > If you can work put what a common reflective constructor signature you
> > need is, any lifecycle methods required, I'll include these for every
> > existing Jini service and we can make a standard.
> >
> > We can also support JSR330 which is AL2 licensed.
> >
> > Support @Inject for the constructor.
> >
> > Provider<JavaSpace05>
> >
> > Regards,
> >
> > Peter.
> >
> >
> > ----- Original message -----
> > > Hopefully Startable will get enough votes to get across the line, we
> > > need to make progress, if you care about River you'll vote for it,
> > > not because it's the perfect solution, simply because it's an
> > > acceptable solution that makes sense and it marks a progress
> > > milestone.
> > >
> > > The reflective constructor signature is acceptable and is used by the
> > > starter kit and Activation.
> > >
> > > We've got Groovy Configuration.
> > >
> > > The service class can also define a static main method.
> > >
> > > If we at least accept these signatures for our services, they could
> > > run in any container.
> > >
> > > Perhaps you also want a public shutdown method for lifecycle
> > > management? Closeable would also be acceptable?
> > >
> > > Is there anything else you need?
> > >
> > > Regards,
> > >
> > > Peter.
> > >
> > > ----- Original message -----
> > > > "What we need is a discussion on a more developer-friendly service
> > > > startup convention to replace ServiceStarter."
> > > >
> > > > Maybe, that is to say, I'm not anti but I feel there's an
> > > > additional wider aspect to kickaround as well....
> > > >
> > > > Should there be a failure of the type we've discussed so far, where
> > > > does the responsibility for sorting it out lie? The service writer,
> > > > they created the bug in the first place. BUT, has it occurred to
> > > > anyone that export() isn't the only place multi-threading problems
> > > > can occur in a service implementation?
> > > >
> > > > Even if they've got service construction right (with whatever
> > > > convention or pattern you care to adopt, me, I'm in the camp of if
> > > > it works because it accounts for the JMM it's fine) have they got
> > > > all their multi-threading right elsewhere? What about all those
> > > > service methods exported and advertised by JoinManager?
> > > >
> > > > Do we actually think it's possible to address that challenge? (You
> > > > could do it by single-locking the entire service so only one call
> > > > can be dispatched at a time, yeah right, that's palatable). What
> > > > makes export a special case in this context? Why does that matter
> > > > so when there are many other ways to screw up?
> > > >
> > > >
> > > > On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com>
> > > > wrote:
> > > >
> > > > >
> > > > > +0. Let me explain…
> > > > >
> > > > > Having a separate interface to say start() is all very well and
> > > > > harmless in itself, but:
> > > > >
> > > > > (a) It doesn’t really address the issue of how service instances
> > > > > should be started. We don’t actually have solid
> > > > > recommendations on this issue, although there have been attempts
> > > > > in various containers to systematize it.
> > > > >
> > > > > (b) the proposed warning addresses a case where a service
> > > > > instance is doing “exporter.export(this);”, which I’d call bad
> > > > > practice in any case. It happens that ServiceStarter appears to
> > > > > encourage this use case because it works by calling a
> > > > > constructor. But the called constructor could (and should)
> > > > > construct an instance of some other class, and then export and
> > > > > register that instance rather than “this”. That style doesn’t
> > > > > mandate a Startable interface, and having Startable doesn’t
> > > > > mandate good multi-threaded code. In other words, you could
> > > > > implement Startable and still have poor thread-safety.
> > > > >
> > > > > (c) even where there is an apparent problem, the exposure window
> > > > > is very small. Consider the last bits of the init(…) method in
> > > > > “com.sun.jini.reggie.ServiceRegistrarImpl”
> > > > >
> > > > > myRef = (Registrar) serverExporter.export(this);
> > > > > proxy = RegistrarProxy.getInstance(myRef, myServiceID);
> > > > > myLocator = (proxy instanceof RemoteMethodControl) ?
> > > > > new ConstrainableLookupLocator(
> > > > > unicastDiscoveryHost, unicaster.port, null) :
> > > > > new LookupLocator(unicastDiscoveryHost, unicaster.port);
> > > > > /* register myself */
> > > > > Item item = new Item(new ServiceItem(myServiceID,
> > > > > proxy,
> > > > > lookupAttrs));
> > > > > SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
> > > > > addService(reg);
> > > > > if (log != null) {
> > > > > log.snapshot();
> > > > > }
> > > > >
> > > > > try {
> > > > > DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
> > > > > discoer;
> > > > > String[] groups = dgm.getGroups();
> > > > > if (groups == null || groups.length > 0) {
> > > > > throw new ConfigurationException(
> > > > > "discoveryManager must be initially configured with " +
> > > > > "no groups");
> > > > > }
> > > > > DiscoveryLocatorManagement dlm =
> > > > > (DiscoveryLocatorManagement) discoer;
> > > > > if (dlm.getLocators().length > 0) {
> > > > > throw new ConfigurationException(
> > > > > "discoveryManager must be initially configured with " +
> > > > > "no locators");
> > > > > }
> > > > > dgm.setGroups(lookupGroups);
> > > > > dlm.setLocators(lookupLocators);
> > > > > } catch (ClassCastException e) {
> > > > > throw new ConfigurationException(null, e);
> > > > > }
> > > > > joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
> > > > > discoer, null, config);
> > > > >
> > > > > /* start up all the daemon threads */
> > > > > serviceExpirer.start();
> > > > > eventExpirer.start();
> > > > > unicaster.start();
> > > > > multicaster.start();
> > > > > announcer.start();
> > > > >
> > > > > /* Shutdown hook so reggie sends a final announcement
> > > > > * packet if VM is terminated. If reggie is terminated
> > > > > * through DestroyAdmin.destroy() this hook will have no effect.
> > > > > * A timeout on announcer.join() was considered but not deemed
> > > > > * necessary at this point in time.
> > > > > */
> > > > > Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
> > > > > public void run() {
> > > > > try {
> > > > > announcer.interrupt();
> > > > > announcer.join();
> > > > > } catch (Throwable t) {
> > > > > logThrow(Level.FINEST, getClass().getName(),
> > > > > "run", "exception shutting announcer down",
> > > > > new Object[]{}, t);
> > > > > }
> > > > > }
> > > > > }));
> > > > >
> > > > > snapshotter.start();
> > > > > if (logger.isLoggable(Level.INFO)) {
> > > > > logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
> > > > > new Object[]{ myServiceID,
> > > > > Arrays.asList(memberGroups),
> > > > > myLocator });
> > > > > }
> > > > > ready.ready();
> > > > > }
> > > > >
> > > > >
> > > > > The window from when the join manager is started up (ignore the
> > > > > serverExporter.export(). The only way a client gets to this
> > > > > endpoint is through a proxy downloaded from the registrar) to
> > > > > when the init() method exits is effectively 9 lines of code,
> > > > > during which time a _lot_ of network i/o would have to happen in
> > > > > order for the object to be used prior to the constructor exit.
> > > > > As well, the actual remote calls are filtered through a command
> > > > > interface and through the tasker object, and the command classes
> > > > > often go through synchronized() blocks, which (I think) act as
> > > > > happens-before fences.
> > > > >
> > > > > The whole thing seems to me a little like the old servlet days,
> > > > > when we got the SingleThreadModel interface and it only served
> > > > > to give naive developers a false sense of security.
> > > > >
> > > > > What we need is a discussion on a more developer-friendly service
> > > > > startup convention to replace ServiceStarter.
> > > > >
> > > > > Having said that, I’m not overly opposed to having Startable,
> > > > > given that as Dan suggested, any output from the ServiceStarter
> > > > > is in the form of a suggestion rather than an error.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Greg.
> > > > >
> > > > >
> > > > >
> > > > > On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
> > > > >
> > > > > > +1 Peter.
> > > > > >
> > > > > >
> > > > > > I've been going over the Exporter implementations and Exporter
> > > > > > interface.
> > > > > >
> > > > > > The Exporter interface specifies that export and unexport
> > > > > > behaviour is
> > > > > defined by the implementation.
> > > > > >
> > > > > > That means it's up to the implementation to specify whether a
> > > > > > happens
> > > > > before edge occures before export.
> > > > > >
> > > > > > I propose we document in Exporter, that if export is called
> > > > > > during
> > > > > object construction then that object's implementation cannot
> > > > > safely use final fields.
> > > > > >
> > > > > > We should also provide a @see reference to Startable to
> > > > > > reccommend its
> > > > > use when final fields are desirable and include Startable in the
> > > > > spec.
> > > > > >
> > > > > > We don't need to rewrite the example of exporting in
> > > > > > net.jini.config as
> > > > > export isn't performed during construction.
> > > > > >
> > > > > > This is a minor update to the Jini Specification.
> > > > > >
> > > > > > The FINE log message is an implementation detail and won't be
> > > > > > included.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Peter.
> > > > > >
> > > > > > ----- Original message -----
> > > > > > > No, an effectively immutable field is one whose reference
> > > > > > > doesn't change after the service objects safe publication.
> > > > > > >
> > > > > > > An immutable field is final.
> > > > > > >
> > > > > > > It would be Jeri's responsibility to safely publish the
> > > > > > > service during export to ensure a happens before event
> > > > > > > occurs, ensuring all non volatile fields the service
> > > > > > > contains are safely published and visible.
> > > > > > >
> > > > > > > It's possible to have mutable fields guarded by synchronized
> > > > > > > methods, or volatile fields.
> > > > > > >
> > > > > > > But it isn't possible to have final fields when exporting
> > > > > > > within the constructor.
> > > > > > >
> > > > > > > If the service contains effectively immutable field
> > > > > > > references, it must be safely published or if another thread
> > > > > > > accesses those fields through unsynchronized methods, the
> > > > > > > thread may only see their default value.
> > > > > > >
> > > > > > > All our service implementations have final fields. All
> > > > > > > our services
> > > > > had
> > > > > > > some unsynchronized access to non volatile, non final fields.
> > > > > > >
> > > > > > > Trouble is, when you can't have final fields, it's very
> > > > > > > difficult to determine which fields are effectively
> > > > > > > immutable and the original programmers design intent, if
> > > > > > > there's no comment.
> > > > > > >
> > > > > > > Our service implementations all had unsynchronized init()
> > > > > > > methods called from with the constructor that started
> > > > > > > threads and exported Some
> > > > > fields
> > > > > > > were final.
> > > > > > >
> > > > > > > For me, a service that contains final fields and is exported
> > > > > > > after construction is easier to maintain and understand.
> > > > > > >
> > > > > > > I suspect, like our Jini service imlementations, most people
> > > > > > > get it wrong most of the time.
> > > > > > >
> > > > > > > I get that others won't want to use this method, however it
> > > > > > > would be detrimental if we don't have such a method for
> > > > > > > those who would.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Peter.
> > > > > > >
> > > > > > > ----- Original message -----
> > > > > > > >
> > > > > > > > I feel like we’re going down a rabbit hole here when you
> > > > > > > > start talking about exporting immutable objects.
> > > > > > > > Wouldn’t it be kind of silly to export an immutable
> > > > > > > > service? Isn’t the whole point that you interact
> > > > > > > > with the service (i.e. alter its state) over a remote
> > > > > > > > interface?
> > > > > > > >
> > > > > > > > Perhaps it’s better to say that exported services need to
> > > > > > > > be thread-safe (which should be fairly obvious).
> > > > > > > > Yes, immutable objects are inherently thread-safe, so for
> > > > > > > > sharing information inside a VM, it makes some sense to
> > > > > > > > minimize the number of mutable objects you pass around.
> > > > > > > > But fundamentally, we’re talking about shared-state
> > > > > > > > systems here.
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Greg.
> > > > > > > >
> > > > > > > > On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Pat your comment about non final fields is interesting.
> > > > > > > > >
> > > > > > > > > Isn't it also the case that we need to safely publish an
> > > > > > > > > effectively immutable object to share it among threads?
> > > > > > > > > That usually means copying it to a thread safe
> > > > > > > > > collection or shared via a synchronized method, volatile
> > > > > > > > > field, or final field in another object?
> > > > > > > > >
> > > > > > > > > So we should also make sure that Jeri uses safe
> > > > > > > > > publication during export.
> > > > > > > > >
> > > > > > > > > That would allow a service that has no final fields to
> > > > > > > > > start threads, then export from within a constructor
> > > > > > > > > safely, provided all operations on non final fields
> > > > > > > > > happen before starting threads and exporting.
> > > > > > > > >
> > > > > > > > > All our services have final fields, so Starter is more
> > > > > > > > > appropriate for River's own services.
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > >
> > > > > > > > > Peter.
> > > > > > > > >
> > > > > > > > > ----- Original message -----
> > > > > > > > > > Hmm, good point, Startable, makes more sense.
> > > > > > > > > >
> > > > > > > > > > An object can be exported using Startable.
> > > > > > > > > >
> > > > > > > > > > I think we should have a policy to strongly discourage
> > > > > > > > > > exporting from constructors.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Peter.
> > > > > > > > > >
> > > > > > > > > > ----- Original message -----
> > > > > > > > > > > As far as I can tell, the special properties of
> > > > > > > > > > > completing a constructor in the JLS memory
> > > > > > > > > > > model are:
> > > > > > > > > > >
> > > > > > > > > > > 1. A happens-before edge from the end of the
> > > > > > > > > > > constructor to the start of a finalizer.
> > > > > > > > > > > (17.4.5)
> > > > > > > > > > >
> > > > > > > > > > > 2. The guarantee that any thread that only sees a
> > > > > > > > > > > reference to an object after the end of
> > > > > > > > > > > the constructor will see the correctly initialized
> > > > > > > > > > > values of all final fields. (17.5)
> > > > > > > > > > >
> > > > > > > > > > > The special issue with final fields is that
> > > > > > > > > > > implementations have freedom to optimize
> > > > > > > > > > > access to final fields in ways that are not permitted
> > > > > > > > > > > for non-final fields. Strategies for
> > > > > > > > > > > thread safety that work for non-final
> > > > > > > > > > > fields do not necessarily work for final fields. The
> > > > > > > > > > > requirement for final field safety is that the
> > > > > > > > > > > constructor end before another thread accesses the
> > > > > > > > > > > newly constructed object.
> > > > > > > > > > >
> > > > > > > > > > > Calling a start() method after construction if the
> > > > > > > > > > > class implements a new interface seems to me
> > > > > > > > > > > to be harmless, backwards compatible, and
> > > > > > > > > > > useful. It enables the simplest and most direct way
> > > > > > > > > > > of preventing access to the new object by
> > > > > > > > > > > another thread during construction.
> > > > > > > > > > >
> > > > > > > > > > > The roadmap issue is whether it should be required,
> > > > > > > > > > > and if so the level of enforcement. For
> > > > > > > > > > > example, there is no reason to require it if the
> > > > > > > > > > > class does not declare any final fields.
> > > > > > > > > > >
> > > > > > > > > > > Incidentally, and as a detail, "Commission" does not
> > > > > > > > > > > immediately make me think of having a
> > > > > > > > > > > start() method that should be called after
> > > > > > > > > > > construction. If you do go this way, the name needs
> > > > > > > > > > > thought. "Startable" would be more
> > > > > > > > > > > obvious, more memorable, more likely to be found on
> > > > > > > > > > > searches, and more compatible with familiar
> > > > > > > > > > > interface names such as "Cloneable" and
> > > > > > > > > > > "Iterable".
> > > > > > > > > > >
> > > > > > > > > > > Patricia
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On 12/18/2013 2:18 AM, Peter wrote:
> > > > > > > > > > > > Well, now seems like a good time to have the
> > > > > > > > > > > > conversation.
> > > > > > > > > > > >
> > > > > > > > > > > > Yes there are other ways, but I haven't seen one
> > > > > > > > > > > > safe implementation yet, so...
> > > > > > > > > > > >
> > > > > > > > > > > > Does someone have a better way to solve this
> > > > > > > > > > > > problem, has someone already solved this problem
> > > > > > > > > > > > I'm unaware of that we can adopt, or is there a
> > > > > > > > > > > > way that's more satisfactory?
> > > > > > > > > > > >
> > > > > > > > > > > > If not, is there something objectionable with the
> > > > > > > > > > > > Commission interface and if so, how can we fix it?
> > > > > > > > > > > >
> > > > > > > > > > > > The SEVERE log message is logged by the River start
> > > > > > > > > > > > package, other containers or frameworks can choose
> > > > > > > > > > > > whether or not to do so, but I'd encourage them to
> > > > > > > > > > > > do something similar, yes we can change it to WARN.
> > > > > > > > > > > >
> > > > > > > > > > > > A much harsher option is to throw an exception
> > > > > > > > > > > > during export which breaks backward compatibility.
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > >
> > > > > > > > > > > > Peter.
> > > > > > > > > > > >
> > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > "org.apache.river.api.util.Commission is an
> > > > > > > > > > > > > interface services should implement"
> > > > > > > > > > > > >
> > > > > > > > > > > > > If it's a SHOULD, not a MUST, chucking out a
> > > > > > > > > > > > > SEVERE is incorrect logger behaviour IMO. You
> > > > > > > > > > > > > could issue a WARN if you like but for even that
> > > > > > > > > > > > > I'd say you need to provide a roadmap explaining
> > > > > > > > > > > > > why the warning and what you intend to do in
> > > > > > > > > > > > > future and what you expect of service writers
> > > > > > > > > > > > > such as myself.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Commission, at least from my point of view, is
> > > > > > > > > > > > > your means (maybe the River community's - did
> > > > > > > > > > > > > you ask us?) for satisfying your needs in
> > > > > > > > > > > > > respect of the JMM. As we've discussed
> > > > > > > > > > > > > previously, there are other ways too and they
> > > > > > > > > > > > > work and they are safe if you know what you're
> > > > > > > > > > > > > doing. Your contention was that most don't know
> > > > > > > > > > > > > what they're doing hence, presumably, Commission.
> > > > > > > > > > > > >
> > > > > > > > > > > > > So the thing is, you are seemingly on a road to
> > > > > > > > > > > > > asserting more structure (gosh, a standard?) on
> > > > > > > > > > > > > the way people write their services. If so,
> > > > > > > > > > > > > you'd best start flagging that honestly and
> > > > > > > > > > > > > openly via a roadmap, deprecation and
> > > > > > > > > > > > > such/whatever rather than sticking out logger
> > > > > > > > > > > > > messages with no clear guidance and at the cost
> > > > > > > > > > > > > of a certain amount of nuisance (no admin I know
> > > > > > > > > > > > > likes SEVERE's being logged for something which
> > > > > > > > > > > > > isn't critical cos it's noise they don't want in
> > > > > > > > > > > > > log files).
> > > > > > > > > > > > >
> > > > > > > > > > > > > And of course, we all know that when some entity
> > > > > > > > > > > > > asserts a standard or requirement on others for
> > > > > > > > > > > > > entry, they may choose not to enter. Does this
> > > > > > > > > > > > > help your community or hinder it? The answer to
> > > > > > > > > > > > > that is, it depends. On what? Have you asked or
> > > > > > > > > > > > > tested? How have you tested? What would be
> > > > > > > > > > > > > considered validation or lack of support?
> > > > > > > > > > > > >
> > > > > > > > > > > > > I am not out to flame or troll rather I want to
> > > > > > > > > > > > > see this community demonstrating good behaviour
> > > > > > > > > > > > > and I'm not feeling like what's going on around
> > > > > > > > > > > > > Commission (what is that big change in version
> > > > > > > > > > > > > number really saying?) is such.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On 18 December 2013 08:52, Peter
> > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Just to clarify
> > > > > > > > > > > > > > org.apache.river.api.util.Commission is an
> > > > > > > > > > > > > > interface services should implement, I would
> > > > > > > > > > > > > > encourage all container projects to pick up the
> > > > > > > > > > > > > > interface and make suggestions for improvement
> > > > > > > > > > > > > > if there are any issues.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Interface Commission {
> > > > > > > > > > > > > > void start () throws Exception;
> > > > > > > > > > > > > > }
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > It's called after JMM safe construction to
> > > > > > > > > > > > > > allow the service to start any threads and be
> > > > > > > > > > > > > > exported.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > The way that services are instantiated and
> > > > > > > > > > > > > > > setup is an implementation detail.
> > > > > > > > > > > > > > > When I think of compatibility
> > > > > > > > > > > > > > > I think of the API and the lookup methods. We
> > > > > > > > > > > > > > > think of compatibility from a client point of
> > > > > > > > > > > > > > > view.
> > > > > > > > > > > > > > > From the client point of view, using a
> > > > > > > > > > > > > > > service looks like this:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - Use multicast of unicast discovery to find
> > > > > > > > > > > > > > > one or more ServiceRegistrar instances
> > > > > > > > > > > > > > > - Call
> > > > > > > > > > > > > > > lookup(…) on one or more of these instances
> > > > > > > > > > > > > > > to get a set of service candidates -
> > > > > > > > > > > > > > > Choose a candidate and prepare() it using a
> > > > > > > > > > > > > > > ProxyPreparer, to yield a usable service
> > > > > > > > > > > > > > > proxy.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - Make calls on it.
> > > > > > > > > > > > > > > Ideally hang on to this proxy
> > > > > > > > > > > > > > > instance, so you can skip the discovery and
> > > > > > > > > > > > > > > lookup next time you need it.
> > > > > > > > > > > > > > > - If the call
> > > > > > > > > > > > > > > fails, repeat the lookup (and possibly
> > > > > > > > > > > > > > > discovery) til you get a proxy that works.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Nowhere does the client need to know whether
> > > > > > > > > > > > > > > the service instance is started up using the
> > > > > > > > > > > > > > > “com.sun.jini.start” mechanism, your
> > > > > > > > > > > > > > > Commission interface, some other IOC
> > > > > > > > > > > > > > > container (Rio, Harvester, Seven or
> > > > > > > > > > > > > > > RiverContainer) or some unknown mechanism
> > > > > > > > > > > > > > > that starts with a static main() method.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > JSK2.0 was 2.0 because of the introduction
> > > > > > > > > > > > > > > of the proxy verification mechanisms, as
> > > > > > > > > > > > > > > well as JERI. Absent some new client usage
> > > > > > > > > > > > > > > mechanism, River doesn’t need to go to 3.0.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Greg.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Dec 17, 2013, at 1:58 PM, Peter
> > > > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I think changing services to use safe
> > > > > > > > > > > > > > > > construction techniques is enough to cause
> > > > > > > > > > > > > > > > the version jump.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > At this point I've allowed services to
> > > > > > > > > > > > > > > > continue unsafe construction practices,
> > > > > > > > > > > > > > > > while logging a SEVERE warning when the
> > > > > > > > > > > > > > > > Commission interface isn't implemented,
> > > > > > > > > > > > > > > > rather than fail.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This is a fundamental change to the way
> > > > > > > > > > > > > > > > services are written.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Assuming that there aren’t major
> > > > > > > > > > > > > > > > > incompatibilities, I think that would be
> > > > > > > > > > > > > > > > > a “minor” version change according to our
> > > > > > > > > > > > > > > > > versioning policy, so we’d be looking at
> > > > > > > > > > > > > > > > > the “2.3” branch rather than a “3.0”
> > > > > > > > > > > > > > > > > release.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I’m still unnerved by the massive
> > > > > > > > > > > > > > > > > amounts of changes to both code and
> > > > > > > > > > > > > > > > > tests in the qa_refactor branch, as well
> > > > > > > > > > > > > > > > > as the apparent instability of the code,
> > > > > > > > > > > > > > > > > although that seems to be improving. In
> > > > > > > > > > > > > > > > > the next few weeks I’m going to try and
> > > > > > > > > > > > > > > > > setup a cross-test case, to see what the
> > > > > > > > > > > > > > > > > “2.2” tests say about the potential
> > > > > > > > > > > > > > > > > “2.3” release and vice-versa.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > I think what I’d really like to see is an
> > > > > > > > > > > > > > > > > incremental approach where we update
> > > > > > > > > > > > > > > > > limited components of the “2.2” branch,
> > > > > > > > > > > > > > > > > one at a time. Is there anything that we
> > > > > > > > > > > > > > > > > could pull out piecemeal? Maybe it
> > > > > > > > > > > > > > would
> > > > > > > > > > > > > > > > > make sense to split out the
> > > > > > > > > > > > > > > > > infrastructure services, like Reggie,
> > > > > > > > > > > > > > > > > Mahalo, and Outrigger into different
> > > > > > > > > > > > > > > > > sub-projects that could be updated
> > > > > > > > > > > > > > > > > separately?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Greg.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > On Dec 17, 2013, at 5:03 AM, Peter
> > > > > > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > When the qa_refactor branch
> > > > > > > > > > > > > > > > > > stabilises, I plan to merge trunk and
> > > > > > > > > > > > > > > > > > provide a beta release for client
> > > > > > > > > > > > > > > > > > compatibility testing.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Changes made have been focused on
> > > > > > > > > > > > > > > > > > making our code thread safe, there are
> > > > > > > > > > > > > > > > > > significant changes internally, the
> > > > > > > > > > > > > > > > > > public api remains focused on backward
> > > > > > > > > > > > > > > > > > compatibility, however it is advisable
> > > > > > > > > > > > > > > > > > that client services adopt new safe
> > > > > > > > > > > > > > > > > > construction techniques for services
> > > > > > > > > > > > > > > > > > and implement the new Commission
> > > > > > > > > > > > > > > > > > interface.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > What's a suitable test period for
> > > > > > > > > > > > > > > > > > client testing?
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > >
> >
>
Re: VOTE: add Startable to Jini specification
Posted by Dan Creswell <da...@gmail.com>.
Well there's two communities of users, right? Those that use River & those
that want to work on River who've likely been users previously.
In the past we've had some desire to focus on growing the River developer
base to support the River user base which drove the code restructuring
discussion. Fair enough too but, maybe a problem for later when we've got
people complaining about not being able to get their heads into the code?
Right now, I think as users, we've all got complaints about writing service
code to run stuff on River so perhaps it's time to focus there
aggressively. If that's done well, perhaps you gain users and in turn more
developer base?
On 20 December 2013 02:18, Greg Trasuk <tr...@stratuscom.com> wrote:
>
> I don’t really see a need to mess around with the existing infrastructure
> services. Surely if we can start them as it is and they can be reached
> through lookup, that’s good enough? I’m more concerned with having an
> interface that makes sense for developers to write their own services with.
> I suspect we need to just do a few experiments and see what works.
>
> I guess that’s one of my frustrations with all this “refactoring” of the
> existing codebase - if it doesn’t make it easier for people to use
> Jini/River to create their own service-oriented architectures, then it just
> seems like a lot of navel-gazing. Similarly, we’ve had statements about
> reorganizing the source code to make it easier to develop with. To which I
> ask, “when was the last time anyone worried about the source code of
> Tomcat, or Websphere, or Weblogic?, or whether they used Maven or not?”.
> Even talking about performance is not very helpful until the average
> corporate coder at the City of Wherever can actually use the product.
>
> We need to get to a point where people are building real things on top of
> River, not messing around with River. That’s not to say that the River
> code can’t be improved or modernized, but it reached “good enough” back in
> 2003 or so.
>
> Lest I sound like I’m complaining, let me say it’s great that we’ve had
> this much traffic on the list in the last few weeks! I think we’re having
> great discussions- keep it up, folks!
>
> Greg.
>
> On Dec 19, 2013, at 8:54 PM, Peter <ji...@zeus.net.au> wrote:
>
> > If you can work put what a common reflective constructor signature you
> need is, any lifecycle methods required, I'll include these for every
> existing Jini service and we can make a standard.
> >
> > We can also support JSR330 which is AL2 licensed.
> >
> > Support @Inject for the constructor.
> >
> > Provider<JavaSpace05>
> >
> > Regards,
> >
> > Peter.
> >
> >
> > ----- Original message -----
> >> Hopefully Startable will get enough votes to get across the line, we
> >> need to make progress, if you care about River you'll vote for it, not
> >> because it's the perfect solution, simply because it's an acceptable
> >> solution that makes sense and it marks a progress milestone.
> >>
> >> The reflective constructor signature is acceptable and is used by the
> >> starter kit and Activation.
> >>
> >> We've got Groovy Configuration.
> >>
> >> The service class can also define a static main method.
> >>
> >> If we at least accept these signatures for our services, they could run
> >> in any container.
> >>
> >> Perhaps you also want a public shutdown method for lifecycle management?
> >> Closeable would also be acceptable?
> >>
> >> Is there anything else you need?
> >>
> >> Regards,
> >>
> >> Peter.
> >>
> >> ----- Original message -----
> >>> "What we need is a discussion on a more developer-friendly service
> >>> startup convention to replace ServiceStarter."
> >>>
> >>> Maybe, that is to say, I'm not anti but I feel there's an additional
> >>> wider aspect to kickaround as well....
> >>>
> >>> Should there be a failure of the type we've discussed so far, where
> >>> does the responsibility for sorting it out lie? The service writer,
> >>> they created the bug in the first place. BUT, has it occurred to
> >>> anyone that export() isn't the only place multi-threading problems can
> >>> occur in a service implementation?
> >>>
> >>> Even if they've got service construction right (with whatever
> >>> convention or pattern you care to adopt, me, I'm in the camp of if it
> >>> works because it accounts for the JMM it's fine) have they got all
> >>> their multi-threading right elsewhere? What about all those service
> >>> methods exported and advertised by JoinManager?
> >>>
> >>> Do we actually think it's possible to address that challenge? (You
> >>> could do it by single-locking the entire service so only one call can
> >>> be dispatched at a time, yeah right, that's palatable). What makes
> >>> export a special case in this context? Why does that matter so when
> >>> there are many other ways to screw up?
> >>>
> >>>
> >>> On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com> wrote:
> >>>
> >>>>
> >>>> +0. Let me explain…
> >>>>
> >>>> Having a separate interface to say start() is all very well and
> >>>> harmless in itself, but:
> >>>>
> >>>> (a) It doesn’t really address the issue of how service instances
> >>>> should be started. We don’t actually have solid recommendations on
> >>>> this issue, although there have been attempts in various containers
> >>>> to systematize it.
> >>>>
> >>>> (b) the proposed warning addresses a case where a service instance is
> >>>> doing “exporter.export(this);”, which I’d call bad practice in any
> >>>> case. It happens that ServiceStarter appears to encourage this use
> >>>> case because it works by calling a constructor. But the called
> >>>> constructor could (and should) construct an instance of some other
> >>>> class, and then export and register that instance rather than
> >>>> “this”. That style doesn’t mandate a Startable interface, and
> >>>> having Startable doesn’t mandate good multi-threaded code. In
> >>>> other words, you could implement Startable and still have poor
> >>>> thread-safety.
> >>>>
> >>>> (c) even where there is an apparent problem, the exposure window is
> >>>> very small. Consider the last bits of the init(…) method in
> >>>> “com.sun.jini.reggie.ServiceRegistrarImpl”
> >>>>
> >>>> myRef = (Registrar) serverExporter.export(this);
> >>>> proxy = RegistrarProxy.getInstance(myRef, myServiceID);
> >>>> myLocator = (proxy instanceof RemoteMethodControl) ?
> >>>> new ConstrainableLookupLocator(
> >>>> unicastDiscoveryHost, unicaster.port, null) :
> >>>> new LookupLocator(unicastDiscoveryHost, unicaster.port);
> >>>> /* register myself */
> >>>> Item item = new Item(new ServiceItem(myServiceID,
> >>>> proxy,
> >>>> lookupAttrs));
> >>>> SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
> >>>> addService(reg);
> >>>> if (log != null) {
> >>>> log.snapshot();
> >>>> }
> >>>>
> >>>> try {
> >>>> DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
> >>>> discoer;
> >>>> String[] groups = dgm.getGroups();
> >>>> if (groups == null || groups.length > 0) {
> >>>> throw new ConfigurationException(
> >>>> "discoveryManager must be initially configured with " +
> >>>> "no groups");
> >>>> }
> >>>> DiscoveryLocatorManagement dlm =
> >>>> (DiscoveryLocatorManagement) discoer;
> >>>> if (dlm.getLocators().length > 0) {
> >>>> throw new ConfigurationException(
> >>>> "discoveryManager must be initially configured with " +
> >>>> "no locators");
> >>>> }
> >>>> dgm.setGroups(lookupGroups);
> >>>> dlm.setLocators(lookupLocators);
> >>>> } catch (ClassCastException e) {
> >>>> throw new ConfigurationException(null, e);
> >>>> }
> >>>> joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
> >>>> discoer, null, config);
> >>>>
> >>>> /* start up all the daemon threads */
> >>>> serviceExpirer.start();
> >>>> eventExpirer.start();
> >>>> unicaster.start();
> >>>> multicaster.start();
> >>>> announcer.start();
> >>>>
> >>>> /* Shutdown hook so reggie sends a final announcement
> >>>> * packet if VM is terminated. If reggie is terminated
> >>>> * through DestroyAdmin.destroy() this hook will have no effect.
> >>>> * A timeout on announcer.join() was considered but not deemed
> >>>> * necessary at this point in time.
> >>>> */
> >>>> Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
> >>>> public void run() {
> >>>> try {
> >>>> announcer.interrupt();
> >>>> announcer.join();
> >>>> } catch (Throwable t) {
> >>>> logThrow(Level.FINEST, getClass().getName(),
> >>>> "run", "exception shutting announcer down",
> >>>> new Object[]{}, t);
> >>>> }
> >>>> }
> >>>> }));
> >>>>
> >>>> snapshotter.start();
> >>>> if (logger.isLoggable(Level.INFO)) {
> >>>> logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
> >>>> new Object[]{ myServiceID,
> >>>> Arrays.asList(memberGroups),
> >>>> myLocator });
> >>>> }
> >>>> ready.ready();
> >>>> }
> >>>>
> >>>>
> >>>> The window from when the join manager is started up (ignore the
> >>>> serverExporter.export(). The only way a client gets to this
> >>>> endpoint is through a proxy downloaded from the registrar) to when
> >>>> the init() method exits is effectively 9 lines of code, during which
> >>>> time a _lot_ of network i/o would have to happen in order for the
> >>>> object to be used prior to the constructor exit. As well, the
> >>>> actual remote calls are filtered through a command interface and
> >>>> through the tasker object, and the command classes often go through
> >>>> synchronized() blocks, which (I think) act as happens-before fences.
> >>>>
> >>>> The whole thing seems to me a little like the old servlet days, when
> >>>> we got the SingleThreadModel interface and it only served to give
> >>>> naive developers a false sense of security.
> >>>>
> >>>> What we need is a discussion on a more developer-friendly service
> >>>> startup convention to replace ServiceStarter.
> >>>>
> >>>> Having said that, I’m not overly opposed to having Startable, given
> >>>> that as Dan suggested, any output from the ServiceStarter is in the
> >>>> form of a suggestion rather than an error.
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Greg.
> >>>>
> >>>>
> >>>>
> >>>> On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
> >>>>
> >>>>> +1 Peter.
> >>>>>
> >>>>>
> >>>>> I've been going over the Exporter implementations and Exporter
> >>>>> interface.
> >>>>>
> >>>>> The Exporter interface specifies that export and unexport behaviour
> >>>>> is
> >>>> defined by the implementation.
> >>>>>
> >>>>> That means it's up to the implementation to specify whether a
> >>>>> happens
> >>>> before edge occures before export.
> >>>>>
> >>>>> I propose we document in Exporter, that if export is called during
> >>>> object construction then that object's implementation cannot safely
> >>>> use final fields.
> >>>>>
> >>>>> We should also provide a @see reference to Startable to reccommend
> >>>>> its
> >>>> use when final fields are desirable and include Startable in the
> >>>> spec.
> >>>>>
> >>>>> We don't need to rewrite the example of exporting in
> >>>>> net.jini.config as
> >>>> export isn't performed during construction.
> >>>>>
> >>>>> This is a minor update to the Jini Specification.
> >>>>>
> >>>>> The FINE log message is an implementation detail and won't be
> >>>>> included.
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> Peter.
> >>>>>
> >>>>> ----- Original message -----
> >>>>>> No, an effectively immutable field is one whose reference doesn't
> >>>>>> change after the service objects safe publication.
> >>>>>>
> >>>>>> An immutable field is final.
> >>>>>>
> >>>>>> It would be Jeri's responsibility to safely publish the service
> >>>>>> during export to ensure a happens before event occurs, ensuring
> >>>>>> all non volatile fields the service contains are safely published
> >>>>>> and visible.
> >>>>>>
> >>>>>> It's possible to have mutable fields guarded by synchronized
> >>>>>> methods, or volatile fields.
> >>>>>>
> >>>>>> But it isn't possible to have final fields when exporting within
> >>>>>> the constructor.
> >>>>>>
> >>>>>> If the service contains effectively immutable field references,
> >>>>>> it must be safely published or if another thread accesses those
> >>>>>> fields through unsynchronized methods, the thread may only see
> >>>>>> their default value.
> >>>>>>
> >>>>>> All our service implementations have final fields. All our
> >>>>>> services
> >>>> had
> >>>>>> some unsynchronized access to non volatile, non final fields.
> >>>>>>
> >>>>>> Trouble is, when you can't have final fields, it's very difficult
> >>>>>> to determine which fields are effectively immutable and the
> >>>>>> original programmers design intent, if there's no comment.
> >>>>>>
> >>>>>> Our service implementations all had unsynchronized init() methods
> >>>>>> called from with the constructor that started threads and
> >>>>>> exported Some
> >>>> fields
> >>>>>> were final.
> >>>>>>
> >>>>>> For me, a service that contains final fields and is exported
> >>>>>> after construction is easier to maintain and understand.
> >>>>>>
> >>>>>> I suspect, like our Jini service imlementations, most people get
> >>>>>> it wrong most of the time.
> >>>>>>
> >>>>>> I get that others won't want to use this method, however it would
> >>>>>> be detrimental if we don't have such a method for those who
> >>>>>> would.
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> Peter.
> >>>>>>
> >>>>>> ----- Original message -----
> >>>>>>>
> >>>>>>> I feel like we’re going down a rabbit hole here when you start
> >>>>>>> talking about exporting immutable objects. Wouldn’t it be
> >>>>>>> kind of silly to export an immutable service? Isn’t the
> >>>>>>> whole point that you interact with the service (i.e. alter its
> >>>>>>> state) over a remote interface?
> >>>>>>>
> >>>>>>> Perhaps it’s better to say that exported services need to be
> >>>>>>> thread-safe (which should be fairly obvious). Yes,
> >>>>>>> immutable objects are inherently thread-safe, so for sharing
> >>>>>>> information inside a VM, it makes some sense to minimize the
> >>>>>>> number of mutable objects you pass around. But
> >>>>>>> fundamentally, we’re talking about shared-state systems here.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Greg.
> >>>>>>>
> >>>>>>> On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
> >>>>>>>
> >>>>>>>> Pat your comment about non final fields is interesting.
> >>>>>>>>
> >>>>>>>> Isn't it also the case that we need to safely publish an
> >>>>>>>> effectively immutable object to share it among threads?
> >>>>>>>> That usually means copying it to a thread safe collection or
> >>>>>>>> shared via a synchronized method, volatile field, or final
> >>>>>>>> field in another object?
> >>>>>>>>
> >>>>>>>> So we should also make sure that Jeri uses safe publication
> >>>>>>>> during export.
> >>>>>>>>
> >>>>>>>> That would allow a service that has no final fields to start
> >>>>>>>> threads, then export from within a constructor safely,
> >>>>>>>> provided all operations on non final fields happen before
> >>>>>>>> starting threads and exporting.
> >>>>>>>>
> >>>>>>>> All our services have final fields, so Starter is more
> >>>>>>>> appropriate for River's own services.
> >>>>>>>>
> >>>>>>>> Regards,
> >>>>>>>>
> >>>>>>>> Peter.
> >>>>>>>>
> >>>>>>>> ----- Original message -----
> >>>>>>>>> Hmm, good point, Startable, makes more sense.
> >>>>>>>>>
> >>>>>>>>> An object can be exported using Startable.
> >>>>>>>>>
> >>>>>>>>> I think we should have a policy to strongly discourage
> >>>>>>>>> exporting from constructors.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>>
> >>>>>>>>> Peter.
> >>>>>>>>>
> >>>>>>>>> ----- Original message -----
> >>>>>>>>>> As far as I can tell, the special properties of
> >>>>>>>>>> completing a constructor in the JLS memory
> >>>>>>>>>> model are:
> >>>>>>>>>>
> >>>>>>>>>> 1. A happens-before edge from the end of the constructor
> >>>>>>>>>> to the start of a finalizer. (17.4.5)
> >>>>>>>>>>
> >>>>>>>>>> 2. The guarantee that any thread that only sees a
> >>>>>>>>>> reference to an object after the end of the
> >>>>>>>>>> constructor will see the correctly initialized
> >>>>>>>>>> values of all final fields. (17.5)
> >>>>>>>>>>
> >>>>>>>>>> The special issue with final fields is that
> >>>>>>>>>> implementations have freedom to optimize
> >>>>>>>>>> access to final fields in ways that are not permitted
> >>>>>>>>>> for non-final fields. Strategies for thread
> >>>>>>>>>> safety that work for non-final fields do not
> >>>>>>>>>> necessarily work for final fields. The requirement for
> >>>>>>>>>> final field safety is that the constructor end
> >>>>>>>>>> before another thread accesses the newly constructed
> >>>>>>>>>> object.
> >>>>>>>>>>
> >>>>>>>>>> Calling a start() method after construction if the class
> >>>>>>>>>> implements a new interface seems to me to be
> >>>>>>>>>> harmless, backwards compatible, and useful. It
> >>>>>>>>>> enables the simplest and most direct way of preventing
> >>>>>>>>>> access to the new object by another thread
> >>>>>>>>>> during construction.
> >>>>>>>>>>
> >>>>>>>>>> The roadmap issue is whether it should be required, and
> >>>>>>>>>> if so the level of enforcement. For example,
> >>>>>>>>>> there is no reason to require it if the class
> >>>>>>>>>> does not declare any final fields.
> >>>>>>>>>>
> >>>>>>>>>> Incidentally, and as a detail, "Commission" does not
> >>>>>>>>>> immediately make me think of having a start()
> >>>>>>>>>> method that should be called after construction.
> >>>>>>>>>> If you do go this way, the name needs thought.
> >>>>>>>>>> "Startable" would be more obvious, more
> >>>>>>>>>> memorable, more likely to be found on
> >>>>>>>>>> searches, and more compatible with familiar interface
> >>>>>>>>>> names such as "Cloneable" and "Iterable".
> >>>>>>>>>>
> >>>>>>>>>> Patricia
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 12/18/2013 2:18 AM, Peter wrote:
> >>>>>>>>>>> Well, now seems like a good time to have the
> >>>>>>>>>>> conversation.
> >>>>>>>>>>>
> >>>>>>>>>>> Yes there are other ways, but I haven't seen one safe
> >>>>>>>>>>> implementation yet, so...
> >>>>>>>>>>>
> >>>>>>>>>>> Does someone have a better way to solve this problem,
> >>>>>>>>>>> has someone already solved this problem I'm unaware of
> >>>>>>>>>>> that we can adopt, or is there a way that's more
> >>>>>>>>>>> satisfactory?
> >>>>>>>>>>>
> >>>>>>>>>>> If not, is there something objectionable with the
> >>>>>>>>>>> Commission interface and if so, how can we fix it?
> >>>>>>>>>>>
> >>>>>>>>>>> The SEVERE log message is logged by the River start
> >>>>>>>>>>> package, other containers or frameworks can choose
> >>>>>>>>>>> whether or not to do so, but I'd encourage them to do
> >>>>>>>>>>> something similar, yes we can change it to WARN.
> >>>>>>>>>>>
> >>>>>>>>>>> A much harsher option is to throw an exception during
> >>>>>>>>>>> export which breaks backward compatibility.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>>
> >>>>>>>>>>> Peter.
> >>>>>>>>>>>
> >>>>>>>>>>> ----- Original message -----
> >>>>>>>>>>>> "org.apache.river.api.util.Commission is an interface
> >>>>>>>>>>>> services should implement"
> >>>>>>>>>>>>
> >>>>>>>>>>>> If it's a SHOULD, not a MUST, chucking out a SEVERE
> >>>>>>>>>>>> is incorrect logger behaviour IMO. You could issue a
> >>>>>>>>>>>> WARN if you like but for even that I'd say you need
> >>>>>>>>>>>> to provide a roadmap explaining why the warning and
> >>>>>>>>>>>> what you intend to do in future and what you expect
> >>>>>>>>>>>> of service writers such as myself.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Commission, at least from my point of view, is your
> >>>>>>>>>>>> means (maybe the River community's - did you ask us?)
> >>>>>>>>>>>> for satisfying your needs in respect of the JMM. As
> >>>>>>>>>>>> we've discussed previously, there are other ways too
> >>>>>>>>>>>> and they work and they are safe if you know what
> >>>>>>>>>>>> you're doing. Your contention was that most don't
> >>>>>>>>>>>> know what they're doing hence, presumably,
> >>>>>>>>>>>> Commission.
> >>>>>>>>>>>>
> >>>>>>>>>>>> So the thing is, you are seemingly on a road to
> >>>>>>>>>>>> asserting more structure (gosh, a standard?) on the
> >>>>>>>>>>>> way people write their services. If so, you'd best
> >>>>>>>>>>>> start flagging that honestly and openly via a
> >>>>>>>>>>>> roadmap, deprecation and such/whatever rather than
> >>>>>>>>>>>> sticking out logger messages with no clear guidance
> >>>>>>>>>>>> and at the cost of a certain amount of nuisance (no
> >>>>>>>>>>>> admin I know likes SEVERE's being logged for
> >>>>>>>>>>>> something which isn't critical cos it's noise they
> >>>>>>>>>>>> don't want in log files).
> >>>>>>>>>>>>
> >>>>>>>>>>>> And of course, we all know that when some entity
> >>>>>>>>>>>> asserts a standard or requirement on others for
> >>>>>>>>>>>> entry, they may choose not to enter. Does this help
> >>>>>>>>>>>> your community or hinder it? The answer to that is,
> >>>>>>>>>>>> it depends. On what? Have you asked or tested? How
> >>>>>>>>>>>> have you tested? What would be considered validation
> >>>>>>>>>>>> or lack of support?
> >>>>>>>>>>>>
> >>>>>>>>>>>> I am not out to flame or troll rather I want to see
> >>>>>>>>>>>> this community demonstrating good behaviour and I'm
> >>>>>>>>>>>> not feeling like what's going on around Commission
> >>>>>>>>>>>> (what is that big change in version number really
> >>>>>>>>>>>> saying?) is such.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 18 December 2013 08:52, Peter <ji...@zeus.net.au>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Just to clarify
> >>>>>>>>>>>>> org.apache.river.api.util.Commission is an
> >>>>>>>>>>>>> interface services should implement, I would
> >>>>>>>>>>>>> encourage all container projects to pick up the
> >>>>>>>>>>>>> interface and make suggestions for improvement if
> >>>>>>>>>>>>> there are any issues.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Interface Commission {
> >>>>>>>>>>>>> void start () throws Exception;
> >>>>>>>>>>>>> }
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It's called after JMM safe construction to allow
> >>>>>>>>>>>>> the service to start any threads and be exported.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Peter.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> ----- Original message -----
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The way that services are instantiated and setup
> >>>>>>>>>>>>>> is an implementation detail.
> >>>>>>>>>>>>>> When I think of compatibility
> >>>>>>>>>>>>>> I think of the API and the lookup methods. We
> >>>>>>>>>>>>>> think of compatibility from a client point of
> >>>>>>>>>>>>>> view. From
> >>>>>>>>>>>>>> the client point of view, using a service looks
> >>>>>>>>>>>>>> like this:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - Use multicast of unicast discovery to find one
> >>>>>>>>>>>>>> or more ServiceRegistrar instances
> >>>>>>>>>>>>>> - Call
> >>>>>>>>>>>>>> lookup(…) on one or more of these instances to
> >>>>>>>>>>>>>> get a set of service candidates - Choose a
> >>>>>>>>>>>>>> candidate and prepare() it using a
> >>>>>>>>>>>>>> ProxyPreparer, to yield a usable service proxy.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - Make calls on it.
> >>>>>>>>>>>>>> Ideally hang on to this proxy
> >>>>>>>>>>>>>> instance, so you can skip the discovery and
> >>>>>>>>>>>>>> lookup next time you need it.
> >>>>>>>>>>>>>> - If the call
> >>>>>>>>>>>>>> fails, repeat the lookup (and possibly
> >>>>>>>>>>>>>> discovery) til you get a proxy that works.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Nowhere does the client need to know whether the
> >>>>>>>>>>>>>> service instance is started up using the
> >>>>>>>>>>>>>> “com.sun.jini.start” mechanism, your Commission
> >>>>>>>>>>>>>> interface, some other IOC container (Rio,
> >>>>>>>>>>>>>> Harvester, Seven or RiverContainer) or some
> >>>>>>>>>>>>>> unknown mechanism that starts with a static
> >>>>>>>>>>>>>> main() method.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> JSK2.0 was 2.0 because of the introduction of the
> >>>>>>>>>>>>>> proxy verification mechanisms, as well as JERI.
> >>>>>>>>>>>>>> Absent some new client usage mechanism, River
> >>>>>>>>>>>>>> doesn’t need to go to 3.0.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Greg.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Dec 17, 2013, at 1:58 PM, Peter
> >>>>>>>>>>>>>> <ji...@zeus.net.au> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think changing services to use safe
> >>>>>>>>>>>>>>> construction techniques is enough to cause the
> >>>>>>>>>>>>>>> version jump.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> At this point I've allowed services to continue
> >>>>>>>>>>>>>>> unsafe construction practices, while logging a
> >>>>>>>>>>>>>>> SEVERE warning when the Commission interface
> >>>>>>>>>>>>>>> isn't implemented, rather than fail.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> This is a fundamental change to the way
> >>>>>>>>>>>>>>> services are written.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Peter.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> ----- Original message -----
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Assuming that there aren’t major
> >>>>>>>>>>>>>>>> incompatibilities, I think that would be a
> >>>>>>>>>>>>>>>> “minor” version change according to our
> >>>>>>>>>>>>>>>> versioning policy, so we’d be looking at the
> >>>>>>>>>>>>>>>> “2.3” branch rather than a “3.0” release.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I’m still unnerved by the massive amounts of
> >>>>>>>>>>>>>>>> changes to both code and tests in the
> >>>>>>>>>>>>>>>> qa_refactor branch, as well as the apparent
> >>>>>>>>>>>>>>>> instability of the code, although that seems
> >>>>>>>>>>>>>>>> to be improving. In the next few weeks I’m
> >>>>>>>>>>>>>>>> going to try and setup a cross-test case, to
> >>>>>>>>>>>>>>>> see what the “2.2” tests say about the
> >>>>>>>>>>>>>>>> potential “2.3” release and vice-versa.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I think what I’d really like to see is an
> >>>>>>>>>>>>>>>> incremental approach where we update limited
> >>>>>>>>>>>>>>>> components of the “2.2” branch, one at a
> >>>>>>>>>>>>>>>> time. Is there anything that we could pull
> >>>>>>>>>>>>>>>> out piecemeal? Maybe it
> >>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>> make sense to split out the infrastructure
> >>>>>>>>>>>>>>>> services, like Reggie, Mahalo, and Outrigger
> >>>>>>>>>>>>>>>> into different sub-projects that could be
> >>>>>>>>>>>>>>>> updated separately?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Any thoughts?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Greg.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Dec 17, 2013, at 5:03 AM, Peter
> >>>>>>>>>>>>>>>> <ji...@zeus.net.au> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> When the qa_refactor branch stabilises, I
> >>>>>>>>>>>>>>>>> plan to merge trunk and provide a beta
> >>>>>>>>>>>>>>>>> release for client compatibility testing.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Changes made have been focused on making
> >>>>>>>>>>>>>>>>> our code thread safe, there are significant
> >>>>>>>>>>>>>>>>> changes internally, the public api remains
> >>>>>>>>>>>>>>>>> focused on backward compatibility, however
> >>>>>>>>>>>>>>>>> it is advisable that client services adopt
> >>>>>>>>>>>>>>>>> new safe construction techniques for
> >>>>>>>>>>>>>>>>> services and implement the new Commission
> >>>>>>>>>>>>>>>>> interface.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> What's a suitable test period for client
> >>>>>>>>>>>>>>>>> testing?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Peter.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>
> >
>
>
Re: VOTE: add Startable to Jini specification
Posted by Greg Trasuk <tr...@stratuscom.com>.
I don’t really see a need to mess around with the existing infrastructure services. Surely if we can start them as it is and they can be reached through lookup, that’s good enough? I’m more concerned with having an interface that makes sense for developers to write their own services with. I suspect we need to just do a few experiments and see what works.
I guess that’s one of my frustrations with all this “refactoring” of the existing codebase - if it doesn’t make it easier for people to use Jini/River to create their own service-oriented architectures, then it just seems like a lot of navel-gazing. Similarly, we’ve had statements about reorganizing the source code to make it easier to develop with. To which I ask, “when was the last time anyone worried about the source code of Tomcat, or Websphere, or Weblogic?, or whether they used Maven or not?”. Even talking about performance is not very helpful until the average corporate coder at the City of Wherever can actually use the product.
We need to get to a point where people are building real things on top of River, not messing around with River. That’s not to say that the River code can’t be improved or modernized, but it reached “good enough” back in 2003 or so.
Lest I sound like I’m complaining, let me say it’s great that we’ve had this much traffic on the list in the last few weeks! I think we’re having great discussions- keep it up, folks!
Greg.
On Dec 19, 2013, at 8:54 PM, Peter <ji...@zeus.net.au> wrote:
> If you can work put what a common reflective constructor signature you need is, any lifecycle methods required, I'll include these for every existing Jini service and we can make a standard.
>
> We can also support JSR330 which is AL2 licensed.
>
> Support @Inject for the constructor.
>
> Provider<JavaSpace05>
>
> Regards,
>
> Peter.
>
>
> ----- Original message -----
>> Hopefully Startable will get enough votes to get across the line, we
>> need to make progress, if you care about River you'll vote for it, not
>> because it's the perfect solution, simply because it's an acceptable
>> solution that makes sense and it marks a progress milestone.
>>
>> The reflective constructor signature is acceptable and is used by the
>> starter kit and Activation.
>>
>> We've got Groovy Configuration.
>>
>> The service class can also define a static main method.
>>
>> If we at least accept these signatures for our services, they could run
>> in any container.
>>
>> Perhaps you also want a public shutdown method for lifecycle management?
>> Closeable would also be acceptable?
>>
>> Is there anything else you need?
>>
>> Regards,
>>
>> Peter.
>>
>> ----- Original message -----
>>> "What we need is a discussion on a more developer-friendly service
>>> startup convention to replace ServiceStarter."
>>>
>>> Maybe, that is to say, I'm not anti but I feel there's an additional
>>> wider aspect to kickaround as well....
>>>
>>> Should there be a failure of the type we've discussed so far, where
>>> does the responsibility for sorting it out lie? The service writer,
>>> they created the bug in the first place. BUT, has it occurred to
>>> anyone that export() isn't the only place multi-threading problems can
>>> occur in a service implementation?
>>>
>>> Even if they've got service construction right (with whatever
>>> convention or pattern you care to adopt, me, I'm in the camp of if it
>>> works because it accounts for the JMM it's fine) have they got all
>>> their multi-threading right elsewhere? What about all those service
>>> methods exported and advertised by JoinManager?
>>>
>>> Do we actually think it's possible to address that challenge? (You
>>> could do it by single-locking the entire service so only one call can
>>> be dispatched at a time, yeah right, that's palatable). What makes
>>> export a special case in this context? Why does that matter so when
>>> there are many other ways to screw up?
>>>
>>>
>>> On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com> wrote:
>>>
>>>>
>>>> +0. Let me explain…
>>>>
>>>> Having a separate interface to say start() is all very well and
>>>> harmless in itself, but:
>>>>
>>>> (a) It doesn’t really address the issue of how service instances
>>>> should be started. We don’t actually have solid recommendations on
>>>> this issue, although there have been attempts in various containers
>>>> to systematize it.
>>>>
>>>> (b) the proposed warning addresses a case where a service instance is
>>>> doing “exporter.export(this);”, which I’d call bad practice in any
>>>> case. It happens that ServiceStarter appears to encourage this use
>>>> case because it works by calling a constructor. But the called
>>>> constructor could (and should) construct an instance of some other
>>>> class, and then export and register that instance rather than
>>>> “this”. That style doesn’t mandate a Startable interface, and
>>>> having Startable doesn’t mandate good multi-threaded code. In
>>>> other words, you could implement Startable and still have poor
>>>> thread-safety.
>>>>
>>>> (c) even where there is an apparent problem, the exposure window is
>>>> very small. Consider the last bits of the init(…) method in
>>>> “com.sun.jini.reggie.ServiceRegistrarImpl”
>>>>
>>>> myRef = (Registrar) serverExporter.export(this);
>>>> proxy = RegistrarProxy.getInstance(myRef, myServiceID);
>>>> myLocator = (proxy instanceof RemoteMethodControl) ?
>>>> new ConstrainableLookupLocator(
>>>> unicastDiscoveryHost, unicaster.port, null) :
>>>> new LookupLocator(unicastDiscoveryHost, unicaster.port);
>>>> /* register myself */
>>>> Item item = new Item(new ServiceItem(myServiceID,
>>>> proxy,
>>>> lookupAttrs));
>>>> SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
>>>> addService(reg);
>>>> if (log != null) {
>>>> log.snapshot();
>>>> }
>>>>
>>>> try {
>>>> DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
>>>> discoer;
>>>> String[] groups = dgm.getGroups();
>>>> if (groups == null || groups.length > 0) {
>>>> throw new ConfigurationException(
>>>> "discoveryManager must be initially configured with " +
>>>> "no groups");
>>>> }
>>>> DiscoveryLocatorManagement dlm =
>>>> (DiscoveryLocatorManagement) discoer;
>>>> if (dlm.getLocators().length > 0) {
>>>> throw new ConfigurationException(
>>>> "discoveryManager must be initially configured with " +
>>>> "no locators");
>>>> }
>>>> dgm.setGroups(lookupGroups);
>>>> dlm.setLocators(lookupLocators);
>>>> } catch (ClassCastException e) {
>>>> throw new ConfigurationException(null, e);
>>>> }
>>>> joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
>>>> discoer, null, config);
>>>>
>>>> /* start up all the daemon threads */
>>>> serviceExpirer.start();
>>>> eventExpirer.start();
>>>> unicaster.start();
>>>> multicaster.start();
>>>> announcer.start();
>>>>
>>>> /* Shutdown hook so reggie sends a final announcement
>>>> * packet if VM is terminated. If reggie is terminated
>>>> * through DestroyAdmin.destroy() this hook will have no effect.
>>>> * A timeout on announcer.join() was considered but not deemed
>>>> * necessary at this point in time.
>>>> */
>>>> Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
>>>> public void run() {
>>>> try {
>>>> announcer.interrupt();
>>>> announcer.join();
>>>> } catch (Throwable t) {
>>>> logThrow(Level.FINEST, getClass().getName(),
>>>> "run", "exception shutting announcer down",
>>>> new Object[]{}, t);
>>>> }
>>>> }
>>>> }));
>>>>
>>>> snapshotter.start();
>>>> if (logger.isLoggable(Level.INFO)) {
>>>> logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
>>>> new Object[]{ myServiceID,
>>>> Arrays.asList(memberGroups),
>>>> myLocator });
>>>> }
>>>> ready.ready();
>>>> }
>>>>
>>>>
>>>> The window from when the join manager is started up (ignore the
>>>> serverExporter.export(). The only way a client gets to this
>>>> endpoint is through a proxy downloaded from the registrar) to when
>>>> the init() method exits is effectively 9 lines of code, during which
>>>> time a _lot_ of network i/o would have to happen in order for the
>>>> object to be used prior to the constructor exit. As well, the
>>>> actual remote calls are filtered through a command interface and
>>>> through the tasker object, and the command classes often go through
>>>> synchronized() blocks, which (I think) act as happens-before fences.
>>>>
>>>> The whole thing seems to me a little like the old servlet days, when
>>>> we got the SingleThreadModel interface and it only served to give
>>>> naive developers a false sense of security.
>>>>
>>>> What we need is a discussion on a more developer-friendly service
>>>> startup convention to replace ServiceStarter.
>>>>
>>>> Having said that, I’m not overly opposed to having Startable, given
>>>> that as Dan suggested, any output from the ServiceStarter is in the
>>>> form of a suggestion rather than an error.
>>>>
>>>> Cheers,
>>>>
>>>> Greg.
>>>>
>>>>
>>>>
>>>> On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
>>>>
>>>>> +1 Peter.
>>>>>
>>>>>
>>>>> I've been going over the Exporter implementations and Exporter
>>>>> interface.
>>>>>
>>>>> The Exporter interface specifies that export and unexport behaviour
>>>>> is
>>>> defined by the implementation.
>>>>>
>>>>> That means it's up to the implementation to specify whether a
>>>>> happens
>>>> before edge occures before export.
>>>>>
>>>>> I propose we document in Exporter, that if export is called during
>>>> object construction then that object's implementation cannot safely
>>>> use final fields.
>>>>>
>>>>> We should also provide a @see reference to Startable to reccommend
>>>>> its
>>>> use when final fields are desirable and include Startable in the
>>>> spec.
>>>>>
>>>>> We don't need to rewrite the example of exporting in
>>>>> net.jini.config as
>>>> export isn't performed during construction.
>>>>>
>>>>> This is a minor update to the Jini Specification.
>>>>>
>>>>> The FINE log message is an implementation detail and won't be
>>>>> included.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Peter.
>>>>>
>>>>> ----- Original message -----
>>>>>> No, an effectively immutable field is one whose reference doesn't
>>>>>> change after the service objects safe publication.
>>>>>>
>>>>>> An immutable field is final.
>>>>>>
>>>>>> It would be Jeri's responsibility to safely publish the service
>>>>>> during export to ensure a happens before event occurs, ensuring
>>>>>> all non volatile fields the service contains are safely published
>>>>>> and visible.
>>>>>>
>>>>>> It's possible to have mutable fields guarded by synchronized
>>>>>> methods, or volatile fields.
>>>>>>
>>>>>> But it isn't possible to have final fields when exporting within
>>>>>> the constructor.
>>>>>>
>>>>>> If the service contains effectively immutable field references,
>>>>>> it must be safely published or if another thread accesses those
>>>>>> fields through unsynchronized methods, the thread may only see
>>>>>> their default value.
>>>>>>
>>>>>> All our service implementations have final fields. All our
>>>>>> services
>>>> had
>>>>>> some unsynchronized access to non volatile, non final fields.
>>>>>>
>>>>>> Trouble is, when you can't have final fields, it's very difficult
>>>>>> to determine which fields are effectively immutable and the
>>>>>> original programmers design intent, if there's no comment.
>>>>>>
>>>>>> Our service implementations all had unsynchronized init() methods
>>>>>> called from with the constructor that started threads and
>>>>>> exported Some
>>>> fields
>>>>>> were final.
>>>>>>
>>>>>> For me, a service that contains final fields and is exported
>>>>>> after construction is easier to maintain and understand.
>>>>>>
>>>>>> I suspect, like our Jini service imlementations, most people get
>>>>>> it wrong most of the time.
>>>>>>
>>>>>> I get that others won't want to use this method, however it would
>>>>>> be detrimental if we don't have such a method for those who
>>>>>> would.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Peter.
>>>>>>
>>>>>> ----- Original message -----
>>>>>>>
>>>>>>> I feel like we’re going down a rabbit hole here when you start
>>>>>>> talking about exporting immutable objects. Wouldn’t it be
>>>>>>> kind of silly to export an immutable service? Isn’t the
>>>>>>> whole point that you interact with the service (i.e. alter its
>>>>>>> state) over a remote interface?
>>>>>>>
>>>>>>> Perhaps it’s better to say that exported services need to be
>>>>>>> thread-safe (which should be fairly obvious). Yes,
>>>>>>> immutable objects are inherently thread-safe, so for sharing
>>>>>>> information inside a VM, it makes some sense to minimize the
>>>>>>> number of mutable objects you pass around. But
>>>>>>> fundamentally, we’re talking about shared-state systems here.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Greg.
>>>>>>>
>>>>>>> On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
>>>>>>>
>>>>>>>> Pat your comment about non final fields is interesting.
>>>>>>>>
>>>>>>>> Isn't it also the case that we need to safely publish an
>>>>>>>> effectively immutable object to share it among threads?
>>>>>>>> That usually means copying it to a thread safe collection or
>>>>>>>> shared via a synchronized method, volatile field, or final
>>>>>>>> field in another object?
>>>>>>>>
>>>>>>>> So we should also make sure that Jeri uses safe publication
>>>>>>>> during export.
>>>>>>>>
>>>>>>>> That would allow a service that has no final fields to start
>>>>>>>> threads, then export from within a constructor safely,
>>>>>>>> provided all operations on non final fields happen before
>>>>>>>> starting threads and exporting.
>>>>>>>>
>>>>>>>> All our services have final fields, so Starter is more
>>>>>>>> appropriate for River's own services.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Peter.
>>>>>>>>
>>>>>>>> ----- Original message -----
>>>>>>>>> Hmm, good point, Startable, makes more sense.
>>>>>>>>>
>>>>>>>>> An object can be exported using Startable.
>>>>>>>>>
>>>>>>>>> I think we should have a policy to strongly discourage
>>>>>>>>> exporting from constructors.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Peter.
>>>>>>>>>
>>>>>>>>> ----- Original message -----
>>>>>>>>>> As far as I can tell, the special properties of
>>>>>>>>>> completing a constructor in the JLS memory
>>>>>>>>>> model are:
>>>>>>>>>>
>>>>>>>>>> 1. A happens-before edge from the end of the constructor
>>>>>>>>>> to the start of a finalizer. (17.4.5)
>>>>>>>>>>
>>>>>>>>>> 2. The guarantee that any thread that only sees a
>>>>>>>>>> reference to an object after the end of the
>>>>>>>>>> constructor will see the correctly initialized
>>>>>>>>>> values of all final fields. (17.5)
>>>>>>>>>>
>>>>>>>>>> The special issue with final fields is that
>>>>>>>>>> implementations have freedom to optimize
>>>>>>>>>> access to final fields in ways that are not permitted
>>>>>>>>>> for non-final fields. Strategies for thread
>>>>>>>>>> safety that work for non-final fields do not
>>>>>>>>>> necessarily work for final fields. The requirement for
>>>>>>>>>> final field safety is that the constructor end
>>>>>>>>>> before another thread accesses the newly constructed
>>>>>>>>>> object.
>>>>>>>>>>
>>>>>>>>>> Calling a start() method after construction if the class
>>>>>>>>>> implements a new interface seems to me to be
>>>>>>>>>> harmless, backwards compatible, and useful. It
>>>>>>>>>> enables the simplest and most direct way of preventing
>>>>>>>>>> access to the new object by another thread
>>>>>>>>>> during construction.
>>>>>>>>>>
>>>>>>>>>> The roadmap issue is whether it should be required, and
>>>>>>>>>> if so the level of enforcement. For example,
>>>>>>>>>> there is no reason to require it if the class
>>>>>>>>>> does not declare any final fields.
>>>>>>>>>>
>>>>>>>>>> Incidentally, and as a detail, "Commission" does not
>>>>>>>>>> immediately make me think of having a start()
>>>>>>>>>> method that should be called after construction.
>>>>>>>>>> If you do go this way, the name needs thought.
>>>>>>>>>> "Startable" would be more obvious, more
>>>>>>>>>> memorable, more likely to be found on
>>>>>>>>>> searches, and more compatible with familiar interface
>>>>>>>>>> names such as "Cloneable" and "Iterable".
>>>>>>>>>>
>>>>>>>>>> Patricia
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 12/18/2013 2:18 AM, Peter wrote:
>>>>>>>>>>> Well, now seems like a good time to have the
>>>>>>>>>>> conversation.
>>>>>>>>>>>
>>>>>>>>>>> Yes there are other ways, but I haven't seen one safe
>>>>>>>>>>> implementation yet, so...
>>>>>>>>>>>
>>>>>>>>>>> Does someone have a better way to solve this problem,
>>>>>>>>>>> has someone already solved this problem I'm unaware of
>>>>>>>>>>> that we can adopt, or is there a way that's more
>>>>>>>>>>> satisfactory?
>>>>>>>>>>>
>>>>>>>>>>> If not, is there something objectionable with the
>>>>>>>>>>> Commission interface and if so, how can we fix it?
>>>>>>>>>>>
>>>>>>>>>>> The SEVERE log message is logged by the River start
>>>>>>>>>>> package, other containers or frameworks can choose
>>>>>>>>>>> whether or not to do so, but I'd encourage them to do
>>>>>>>>>>> something similar, yes we can change it to WARN.
>>>>>>>>>>>
>>>>>>>>>>> A much harsher option is to throw an exception during
>>>>>>>>>>> export which breaks backward compatibility.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Peter.
>>>>>>>>>>>
>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>> "org.apache.river.api.util.Commission is an interface
>>>>>>>>>>>> services should implement"
>>>>>>>>>>>>
>>>>>>>>>>>> If it's a SHOULD, not a MUST, chucking out a SEVERE
>>>>>>>>>>>> is incorrect logger behaviour IMO. You could issue a
>>>>>>>>>>>> WARN if you like but for even that I'd say you need
>>>>>>>>>>>> to provide a roadmap explaining why the warning and
>>>>>>>>>>>> what you intend to do in future and what you expect
>>>>>>>>>>>> of service writers such as myself.
>>>>>>>>>>>>
>>>>>>>>>>>> Commission, at least from my point of view, is your
>>>>>>>>>>>> means (maybe the River community's - did you ask us?)
>>>>>>>>>>>> for satisfying your needs in respect of the JMM. As
>>>>>>>>>>>> we've discussed previously, there are other ways too
>>>>>>>>>>>> and they work and they are safe if you know what
>>>>>>>>>>>> you're doing. Your contention was that most don't
>>>>>>>>>>>> know what they're doing hence, presumably,
>>>>>>>>>>>> Commission.
>>>>>>>>>>>>
>>>>>>>>>>>> So the thing is, you are seemingly on a road to
>>>>>>>>>>>> asserting more structure (gosh, a standard?) on the
>>>>>>>>>>>> way people write their services. If so, you'd best
>>>>>>>>>>>> start flagging that honestly and openly via a
>>>>>>>>>>>> roadmap, deprecation and such/whatever rather than
>>>>>>>>>>>> sticking out logger messages with no clear guidance
>>>>>>>>>>>> and at the cost of a certain amount of nuisance (no
>>>>>>>>>>>> admin I know likes SEVERE's being logged for
>>>>>>>>>>>> something which isn't critical cos it's noise they
>>>>>>>>>>>> don't want in log files).
>>>>>>>>>>>>
>>>>>>>>>>>> And of course, we all know that when some entity
>>>>>>>>>>>> asserts a standard or requirement on others for
>>>>>>>>>>>> entry, they may choose not to enter. Does this help
>>>>>>>>>>>> your community or hinder it? The answer to that is,
>>>>>>>>>>>> it depends. On what? Have you asked or tested? How
>>>>>>>>>>>> have you tested? What would be considered validation
>>>>>>>>>>>> or lack of support?
>>>>>>>>>>>>
>>>>>>>>>>>> I am not out to flame or troll rather I want to see
>>>>>>>>>>>> this community demonstrating good behaviour and I'm
>>>>>>>>>>>> not feeling like what's going on around Commission
>>>>>>>>>>>> (what is that big change in version number really
>>>>>>>>>>>> saying?) is such.
>>>>>>>>>>>>
>>>>>>>>>>>> On 18 December 2013 08:52, Peter <ji...@zeus.net.au>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Just to clarify
>>>>>>>>>>>>> org.apache.river.api.util.Commission is an
>>>>>>>>>>>>> interface services should implement, I would
>>>>>>>>>>>>> encourage all container projects to pick up the
>>>>>>>>>>>>> interface and make suggestions for improvement if
>>>>>>>>>>>>> there are any issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Interface Commission {
>>>>>>>>>>>>> void start () throws Exception;
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's called after JMM safe construction to allow
>>>>>>>>>>>>> the service to start any threads and be exported.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>>
>>>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The way that services are instantiated and setup
>>>>>>>>>>>>>> is an implementation detail.
>>>>>>>>>>>>>> When I think of compatibility
>>>>>>>>>>>>>> I think of the API and the lookup methods. We
>>>>>>>>>>>>>> think of compatibility from a client point of
>>>>>>>>>>>>>> view. From
>>>>>>>>>>>>>> the client point of view, using a service looks
>>>>>>>>>>>>>> like this:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Use multicast of unicast discovery to find one
>>>>>>>>>>>>>> or more ServiceRegistrar instances
>>>>>>>>>>>>>> - Call
>>>>>>>>>>>>>> lookup(…) on one or more of these instances to
>>>>>>>>>>>>>> get a set of service candidates - Choose a
>>>>>>>>>>>>>> candidate and prepare() it using a
>>>>>>>>>>>>>> ProxyPreparer, to yield a usable service proxy.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Make calls on it.
>>>>>>>>>>>>>> Ideally hang on to this proxy
>>>>>>>>>>>>>> instance, so you can skip the discovery and
>>>>>>>>>>>>>> lookup next time you need it.
>>>>>>>>>>>>>> - If the call
>>>>>>>>>>>>>> fails, repeat the lookup (and possibly
>>>>>>>>>>>>>> discovery) til you get a proxy that works.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Nowhere does the client need to know whether the
>>>>>>>>>>>>>> service instance is started up using the
>>>>>>>>>>>>>> “com.sun.jini.start” mechanism, your Commission
>>>>>>>>>>>>>> interface, some other IOC container (Rio,
>>>>>>>>>>>>>> Harvester, Seven or RiverContainer) or some
>>>>>>>>>>>>>> unknown mechanism that starts with a static
>>>>>>>>>>>>>> main() method.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> JSK2.0 was 2.0 because of the introduction of the
>>>>>>>>>>>>>> proxy verification mechanisms, as well as JERI.
>>>>>>>>>>>>>> Absent some new client usage mechanism, River
>>>>>>>>>>>>>> doesn’t need to go to 3.0.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Greg.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Dec 17, 2013, at 1:58 PM, Peter
>>>>>>>>>>>>>> <ji...@zeus.net.au> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think changing services to use safe
>>>>>>>>>>>>>>> construction techniques is enough to cause the
>>>>>>>>>>>>>>> version jump.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> At this point I've allowed services to continue
>>>>>>>>>>>>>>> unsafe construction practices, while logging a
>>>>>>>>>>>>>>> SEVERE warning when the Commission interface
>>>>>>>>>>>>>>> isn't implemented, rather than fail.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This is a fundamental change to the way
>>>>>>>>>>>>>>> services are written.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Assuming that there aren’t major
>>>>>>>>>>>>>>>> incompatibilities, I think that would be a
>>>>>>>>>>>>>>>> “minor” version change according to our
>>>>>>>>>>>>>>>> versioning policy, so we’d be looking at the
>>>>>>>>>>>>>>>> “2.3” branch rather than a “3.0” release.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I’m still unnerved by the massive amounts of
>>>>>>>>>>>>>>>> changes to both code and tests in the
>>>>>>>>>>>>>>>> qa_refactor branch, as well as the apparent
>>>>>>>>>>>>>>>> instability of the code, although that seems
>>>>>>>>>>>>>>>> to be improving. In the next few weeks I’m
>>>>>>>>>>>>>>>> going to try and setup a cross-test case, to
>>>>>>>>>>>>>>>> see what the “2.2” tests say about the
>>>>>>>>>>>>>>>> potential “2.3” release and vice-versa.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think what I’d really like to see is an
>>>>>>>>>>>>>>>> incremental approach where we update limited
>>>>>>>>>>>>>>>> components of the “2.2” branch, one at a
>>>>>>>>>>>>>>>> time. Is there anything that we could pull
>>>>>>>>>>>>>>>> out piecemeal? Maybe it
>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> make sense to split out the infrastructure
>>>>>>>>>>>>>>>> services, like Reggie, Mahalo, and Outrigger
>>>>>>>>>>>>>>>> into different sub-projects that could be
>>>>>>>>>>>>>>>> updated separately?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Greg.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Dec 17, 2013, at 5:03 AM, Peter
>>>>>>>>>>>>>>>> <ji...@zeus.net.au> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> When the qa_refactor branch stabilises, I
>>>>>>>>>>>>>>>>> plan to merge trunk and provide a beta
>>>>>>>>>>>>>>>>> release for client compatibility testing.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Changes made have been focused on making
>>>>>>>>>>>>>>>>> our code thread safe, there are significant
>>>>>>>>>>>>>>>>> changes internally, the public api remains
>>>>>>>>>>>>>>>>> focused on backward compatibility, however
>>>>>>>>>>>>>>>>> it is advisable that client services adopt
>>>>>>>>>>>>>>>>> new safe construction techniques for
>>>>>>>>>>>>>>>>> services and implement the new Commission
>>>>>>>>>>>>>>>>> interface.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> What's a suitable test period for client
>>>>>>>>>>>>>>>>> testing?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
If you can work put what a common reflective constructor signature you need is, any lifecycle methods required, I'll include these for every existing Jini service and we can make a standard.
We can also support JSR330 which is AL2 licensed.
Support @Inject for the constructor.
Provider<JavaSpace05>
Regards,
Peter.
----- Original message -----
> Hopefully Startable will get enough votes to get across the line, we
> need to make progress, if you care about River you'll vote for it, not
> because it's the perfect solution, simply because it's an acceptable
> solution that makes sense and it marks a progress milestone.
>
> The reflective constructor signature is acceptable and is used by the
> starter kit and Activation.
>
> We've got Groovy Configuration.
>
> The service class can also define a static main method.
>
> If we at least accept these signatures for our services, they could run
> in any container.
>
> Perhaps you also want a public shutdown method for lifecycle management?
> Closeable would also be acceptable?
>
> Is there anything else you need?
>
> Regards,
>
> Peter.
>
> ----- Original message -----
> > "What we need is a discussion on a more developer-friendly service
> > startup convention to replace ServiceStarter."
> >
> > Maybe, that is to say, I'm not anti but I feel there's an additional
> > wider aspect to kickaround as well....
> >
> > Should there be a failure of the type we've discussed so far, where
> > does the responsibility for sorting it out lie? The service writer,
> > they created the bug in the first place. BUT, has it occurred to
> > anyone that export() isn't the only place multi-threading problems can
> > occur in a service implementation?
> >
> > Even if they've got service construction right (with whatever
> > convention or pattern you care to adopt, me, I'm in the camp of if it
> > works because it accounts for the JMM it's fine) have they got all
> > their multi-threading right elsewhere? What about all those service
> > methods exported and advertised by JoinManager?
> >
> > Do we actually think it's possible to address that challenge? (You
> > could do it by single-locking the entire service so only one call can
> > be dispatched at a time, yeah right, that's palatable). What makes
> > export a special case in this context? Why does that matter so when
> > there are many other ways to screw up?
> >
> >
> > On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com> wrote:
> >
> > >
> > > +0. Let me explain…
> > >
> > > Having a separate interface to say start() is all very well and
> > > harmless in itself, but:
> > >
> > > (a) It doesn’t really address the issue of how service instances
> > > should be started. We don’t actually have solid recommendations on
> > > this issue, although there have been attempts in various containers
> > > to systematize it.
> > >
> > > (b) the proposed warning addresses a case where a service instance is
> > > doing “exporter.export(this);”, which I’d call bad practice in any
> > > case. It happens that ServiceStarter appears to encourage this use
> > > case because it works by calling a constructor. But the called
> > > constructor could (and should) construct an instance of some other
> > > class, and then export and register that instance rather than
> > > “this”. That style doesn’t mandate a Startable interface, and
> > > having Startable doesn’t mandate good multi-threaded code. In
> > > other words, you could implement Startable and still have poor
> > > thread-safety.
> > >
> > > (c) even where there is an apparent problem, the exposure window is
> > > very small. Consider the last bits of the init(…) method in
> > > “com.sun.jini.reggie.ServiceRegistrarImpl”
> > >
> > > myRef = (Registrar) serverExporter.export(this);
> > > proxy = RegistrarProxy.getInstance(myRef, myServiceID);
> > > myLocator = (proxy instanceof RemoteMethodControl) ?
> > > new ConstrainableLookupLocator(
> > > unicastDiscoveryHost, unicaster.port, null) :
> > > new LookupLocator(unicastDiscoveryHost, unicaster.port);
> > > /* register myself */
> > > Item item = new Item(new ServiceItem(myServiceID,
> > > proxy,
> > > lookupAttrs));
> > > SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
> > > addService(reg);
> > > if (log != null) {
> > > log.snapshot();
> > > }
> > >
> > > try {
> > > DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
> > > discoer;
> > > String[] groups = dgm.getGroups();
> > > if (groups == null || groups.length > 0) {
> > > throw new ConfigurationException(
> > > "discoveryManager must be initially configured with " +
> > > "no groups");
> > > }
> > > DiscoveryLocatorManagement dlm =
> > > (DiscoveryLocatorManagement) discoer;
> > > if (dlm.getLocators().length > 0) {
> > > throw new ConfigurationException(
> > > "discoveryManager must be initially configured with " +
> > > "no locators");
> > > }
> > > dgm.setGroups(lookupGroups);
> > > dlm.setLocators(lookupLocators);
> > > } catch (ClassCastException e) {
> > > throw new ConfigurationException(null, e);
> > > }
> > > joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
> > > discoer, null, config);
> > >
> > > /* start up all the daemon threads */
> > > serviceExpirer.start();
> > > eventExpirer.start();
> > > unicaster.start();
> > > multicaster.start();
> > > announcer.start();
> > >
> > > /* Shutdown hook so reggie sends a final announcement
> > > * packet if VM is terminated. If reggie is terminated
> > > * through DestroyAdmin.destroy() this hook will have no effect.
> > > * A timeout on announcer.join() was considered but not deemed
> > > * necessary at this point in time.
> > > */
> > > Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
> > > public void run() {
> > > try {
> > > announcer.interrupt();
> > > announcer.join();
> > > } catch (Throwable t) {
> > > logThrow(Level.FINEST, getClass().getName(),
> > > "run", "exception shutting announcer down",
> > > new Object[]{}, t);
> > > }
> > > }
> > > }));
> > >
> > > snapshotter.start();
> > > if (logger.isLoggable(Level.INFO)) {
> > > logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
> > > new Object[]{ myServiceID,
> > > Arrays.asList(memberGroups),
> > > myLocator });
> > > }
> > > ready.ready();
> > > }
> > >
> > >
> > > The window from when the join manager is started up (ignore the
> > > serverExporter.export(). The only way a client gets to this
> > > endpoint is through a proxy downloaded from the registrar) to when
> > > the init() method exits is effectively 9 lines of code, during which
> > > time a _lot_ of network i/o would have to happen in order for the
> > > object to be used prior to the constructor exit. As well, the
> > > actual remote calls are filtered through a command interface and
> > > through the tasker object, and the command classes often go through
> > > synchronized() blocks, which (I think) act as happens-before fences.
> > >
> > > The whole thing seems to me a little like the old servlet days, when
> > > we got the SingleThreadModel interface and it only served to give
> > > naive developers a false sense of security.
> > >
> > > What we need is a discussion on a more developer-friendly service
> > > startup convention to replace ServiceStarter.
> > >
> > > Having said that, I’m not overly opposed to having Startable, given
> > > that as Dan suggested, any output from the ServiceStarter is in the
> > > form of a suggestion rather than an error.
> > >
> > > Cheers,
> > >
> > > Greg.
> > >
> > >
> > >
> > > On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
> > >
> > > > +1 Peter.
> > > >
> > > >
> > > > I've been going over the Exporter implementations and Exporter
> > > > interface.
> > > >
> > > > The Exporter interface specifies that export and unexport behaviour
> > > > is
> > > defined by the implementation.
> > > >
> > > > That means it's up to the implementation to specify whether a
> > > > happens
> > > before edge occures before export.
> > > >
> > > > I propose we document in Exporter, that if export is called during
> > > object construction then that object's implementation cannot safely
> > > use final fields.
> > > >
> > > > We should also provide a @see reference to Startable to reccommend
> > > > its
> > > use when final fields are desirable and include Startable in the
> > > spec.
> > > >
> > > > We don't need to rewrite the example of exporting in
> > > > net.jini.config as
> > > export isn't performed during construction.
> > > >
> > > > This is a minor update to the Jini Specification.
> > > >
> > > > The FINE log message is an implementation detail and won't be
> > > > included.
> > > >
> > > > Regards,
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > > > No, an effectively immutable field is one whose reference doesn't
> > > > > change after the service objects safe publication.
> > > > >
> > > > > An immutable field is final.
> > > > >
> > > > > It would be Jeri's responsibility to safely publish the service
> > > > > during export to ensure a happens before event occurs, ensuring
> > > > > all non volatile fields the service contains are safely published
> > > > > and visible.
> > > > >
> > > > > It's possible to have mutable fields guarded by synchronized
> > > > > methods, or volatile fields.
> > > > >
> > > > > But it isn't possible to have final fields when exporting within
> > > > > the constructor.
> > > > >
> > > > > If the service contains effectively immutable field references,
> > > > > it must be safely published or if another thread accesses those
> > > > > fields through unsynchronized methods, the thread may only see
> > > > > their default value.
> > > > >
> > > > > All our service implementations have final fields. All our
> > > > > services
> > > had
> > > > > some unsynchronized access to non volatile, non final fields.
> > > > >
> > > > > Trouble is, when you can't have final fields, it's very difficult
> > > > > to determine which fields are effectively immutable and the
> > > > > original programmers design intent, if there's no comment.
> > > > >
> > > > > Our service implementations all had unsynchronized init() methods
> > > > > called from with the constructor that started threads and
> > > > > exported Some
> > > fields
> > > > > were final.
> > > > >
> > > > > For me, a service that contains final fields and is exported
> > > > > after construction is easier to maintain and understand.
> > > > >
> > > > > I suspect, like our Jini service imlementations, most people get
> > > > > it wrong most of the time.
> > > > >
> > > > > I get that others won't want to use this method, however it would
> > > > > be detrimental if we don't have such a method for those who
> > > > > would.
> > > > >
> > > > > Regards,
> > > > >
> > > > > Peter.
> > > > >
> > > > > ----- Original message -----
> > > > > >
> > > > > > I feel like we’re going down a rabbit hole here when you start
> > > > > > talking about exporting immutable objects. Wouldn’t it be
> > > > > > kind of silly to export an immutable service? Isn’t the
> > > > > > whole point that you interact with the service (i.e. alter its
> > > > > > state) over a remote interface?
> > > > > >
> > > > > > Perhaps it’s better to say that exported services need to be
> > > > > > thread-safe (which should be fairly obvious). Yes,
> > > > > > immutable objects are inherently thread-safe, so for sharing
> > > > > > information inside a VM, it makes some sense to minimize the
> > > > > > number of mutable objects you pass around. But
> > > > > > fundamentally, we’re talking about shared-state systems here.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Greg.
> > > > > >
> > > > > > On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
> > > > > >
> > > > > > > Pat your comment about non final fields is interesting.
> > > > > > >
> > > > > > > Isn't it also the case that we need to safely publish an
> > > > > > > effectively immutable object to share it among threads?
> > > > > > > That usually means copying it to a thread safe collection or
> > > > > > > shared via a synchronized method, volatile field, or final
> > > > > > > field in another object?
> > > > > > >
> > > > > > > So we should also make sure that Jeri uses safe publication
> > > > > > > during export.
> > > > > > >
> > > > > > > That would allow a service that has no final fields to start
> > > > > > > threads, then export from within a constructor safely,
> > > > > > > provided all operations on non final fields happen before
> > > > > > > starting threads and exporting.
> > > > > > >
> > > > > > > All our services have final fields, so Starter is more
> > > > > > > appropriate for River's own services.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Peter.
> > > > > > >
> > > > > > > ----- Original message -----
> > > > > > > > Hmm, good point, Startable, makes more sense.
> > > > > > > >
> > > > > > > > An object can be exported using Startable.
> > > > > > > >
> > > > > > > > I think we should have a policy to strongly discourage
> > > > > > > > exporting from constructors.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > > Peter.
> > > > > > > >
> > > > > > > > ----- Original message -----
> > > > > > > > > As far as I can tell, the special properties of
> > > > > > > > > completing a constructor in the JLS memory
> > > > > > > > > model are:
> > > > > > > > >
> > > > > > > > > 1. A happens-before edge from the end of the constructor
> > > > > > > > > to the start of a finalizer. (17.4.5)
> > > > > > > > >
> > > > > > > > > 2. The guarantee that any thread that only sees a
> > > > > > > > > reference to an object after the end of the
> > > > > > > > > constructor will see the correctly initialized
> > > > > > > > > values of all final fields. (17.5)
> > > > > > > > >
> > > > > > > > > The special issue with final fields is that
> > > > > > > > > implementations have freedom to optimize
> > > > > > > > > access to final fields in ways that are not permitted
> > > > > > > > > for non-final fields. Strategies for thread
> > > > > > > > > safety that work for non-final fields do not
> > > > > > > > > necessarily work for final fields. The requirement for
> > > > > > > > > final field safety is that the constructor end
> > > > > > > > > before another thread accesses the newly constructed
> > > > > > > > > object.
> > > > > > > > >
> > > > > > > > > Calling a start() method after construction if the class
> > > > > > > > > implements a new interface seems to me to be
> > > > > > > > > harmless, backwards compatible, and useful. It
> > > > > > > > > enables the simplest and most direct way of preventing
> > > > > > > > > access to the new object by another thread
> > > > > > > > > during construction.
> > > > > > > > >
> > > > > > > > > The roadmap issue is whether it should be required, and
> > > > > > > > > if so the level of enforcement. For example,
> > > > > > > > > there is no reason to require it if the class
> > > > > > > > > does not declare any final fields.
> > > > > > > > >
> > > > > > > > > Incidentally, and as a detail, "Commission" does not
> > > > > > > > > immediately make me think of having a start()
> > > > > > > > > method that should be called after construction.
> > > > > > > > > If you do go this way, the name needs thought.
> > > > > > > > > "Startable" would be more obvious, more
> > > > > > > > > memorable, more likely to be found on
> > > > > > > > > searches, and more compatible with familiar interface
> > > > > > > > > names such as "Cloneable" and "Iterable".
> > > > > > > > >
> > > > > > > > > Patricia
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 12/18/2013 2:18 AM, Peter wrote:
> > > > > > > > > > Well, now seems like a good time to have the
> > > > > > > > > > conversation.
> > > > > > > > > >
> > > > > > > > > > Yes there are other ways, but I haven't seen one safe
> > > > > > > > > > implementation yet, so...
> > > > > > > > > >
> > > > > > > > > > Does someone have a better way to solve this problem,
> > > > > > > > > > has someone already solved this problem I'm unaware of
> > > > > > > > > > that we can adopt, or is there a way that's more
> > > > > > > > > > satisfactory?
> > > > > > > > > >
> > > > > > > > > > If not, is there something objectionable with the
> > > > > > > > > > Commission interface and if so, how can we fix it?
> > > > > > > > > >
> > > > > > > > > > The SEVERE log message is logged by the River start
> > > > > > > > > > package, other containers or frameworks can choose
> > > > > > > > > > whether or not to do so, but I'd encourage them to do
> > > > > > > > > > something similar, yes we can change it to WARN.
> > > > > > > > > >
> > > > > > > > > > A much harsher option is to throw an exception during
> > > > > > > > > > export which breaks backward compatibility.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Peter.
> > > > > > > > > >
> > > > > > > > > > ----- Original message -----
> > > > > > > > > > > "org.apache.river.api.util.Commission is an interface
> > > > > > > > > > > services should implement"
> > > > > > > > > > >
> > > > > > > > > > > If it's a SHOULD, not a MUST, chucking out a SEVERE
> > > > > > > > > > > is incorrect logger behaviour IMO. You could issue a
> > > > > > > > > > > WARN if you like but for even that I'd say you need
> > > > > > > > > > > to provide a roadmap explaining why the warning and
> > > > > > > > > > > what you intend to do in future and what you expect
> > > > > > > > > > > of service writers such as myself.
> > > > > > > > > > >
> > > > > > > > > > > Commission, at least from my point of view, is your
> > > > > > > > > > > means (maybe the River community's - did you ask us?)
> > > > > > > > > > > for satisfying your needs in respect of the JMM. As
> > > > > > > > > > > we've discussed previously, there are other ways too
> > > > > > > > > > > and they work and they are safe if you know what
> > > > > > > > > > > you're doing. Your contention was that most don't
> > > > > > > > > > > know what they're doing hence, presumably,
> > > > > > > > > > > Commission.
> > > > > > > > > > >
> > > > > > > > > > > So the thing is, you are seemingly on a road to
> > > > > > > > > > > asserting more structure (gosh, a standard?) on the
> > > > > > > > > > > way people write their services. If so, you'd best
> > > > > > > > > > > start flagging that honestly and openly via a
> > > > > > > > > > > roadmap, deprecation and such/whatever rather than
> > > > > > > > > > > sticking out logger messages with no clear guidance
> > > > > > > > > > > and at the cost of a certain amount of nuisance (no
> > > > > > > > > > > admin I know likes SEVERE's being logged for
> > > > > > > > > > > something which isn't critical cos it's noise they
> > > > > > > > > > > don't want in log files).
> > > > > > > > > > >
> > > > > > > > > > > And of course, we all know that when some entity
> > > > > > > > > > > asserts a standard or requirement on others for
> > > > > > > > > > > entry, they may choose not to enter. Does this help
> > > > > > > > > > > your community or hinder it? The answer to that is,
> > > > > > > > > > > it depends. On what? Have you asked or tested? How
> > > > > > > > > > > have you tested? What would be considered validation
> > > > > > > > > > > or lack of support?
> > > > > > > > > > >
> > > > > > > > > > > I am not out to flame or troll rather I want to see
> > > > > > > > > > > this community demonstrating good behaviour and I'm
> > > > > > > > > > > not feeling like what's going on around Commission
> > > > > > > > > > > (what is that big change in version number really
> > > > > > > > > > > saying?) is such.
> > > > > > > > > > >
> > > > > > > > > > > On 18 December 2013 08:52, Peter <ji...@zeus.net.au>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Just to clarify
> > > > > > > > > > > > org.apache.river.api.util.Commission is an
> > > > > > > > > > > > interface services should implement, I would
> > > > > > > > > > > > encourage all container projects to pick up the
> > > > > > > > > > > > interface and make suggestions for improvement if
> > > > > > > > > > > > there are any issues.
> > > > > > > > > > > >
> > > > > > > > > > > > Interface Commission {
> > > > > > > > > > > > void start () throws Exception;
> > > > > > > > > > > > }
> > > > > > > > > > > >
> > > > > > > > > > > > It's called after JMM safe construction to allow
> > > > > > > > > > > > the service to start any threads and be exported.
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > >
> > > > > > > > > > > > Peter.
> > > > > > > > > > > >
> > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > >
> > > > > > > > > > > > > The way that services are instantiated and setup
> > > > > > > > > > > > > is an implementation detail.
> > > > > > > > > > > > > When I think of compatibility
> > > > > > > > > > > > > I think of the API and the lookup methods. We
> > > > > > > > > > > > > think of compatibility from a client point of
> > > > > > > > > > > > > view. From
> > > > > > > > > > > > > the client point of view, using a service looks
> > > > > > > > > > > > > like this:
> > > > > > > > > > > > >
> > > > > > > > > > > > > - Use multicast of unicast discovery to find one
> > > > > > > > > > > > > or more ServiceRegistrar instances
> > > > > > > > > > > > > - Call
> > > > > > > > > > > > > lookup(…) on one or more of these instances to
> > > > > > > > > > > > > get a set of service candidates - Choose a
> > > > > > > > > > > > > candidate and prepare() it using a
> > > > > > > > > > > > > ProxyPreparer, to yield a usable service proxy.
> > > > > > > > > > > > >
> > > > > > > > > > > > > - Make calls on it.
> > > > > > > > > > > > > Ideally hang on to this proxy
> > > > > > > > > > > > > instance, so you can skip the discovery and
> > > > > > > > > > > > > lookup next time you need it.
> > > > > > > > > > > > > - If the call
> > > > > > > > > > > > > fails, repeat the lookup (and possibly
> > > > > > > > > > > > > discovery) til you get a proxy that works.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Nowhere does the client need to know whether the
> > > > > > > > > > > > > service instance is started up using the
> > > > > > > > > > > > > “com.sun.jini.start” mechanism, your Commission
> > > > > > > > > > > > > interface, some other IOC container (Rio,
> > > > > > > > > > > > > Harvester, Seven or RiverContainer) or some
> > > > > > > > > > > > > unknown mechanism that starts with a static
> > > > > > > > > > > > > main() method.
> > > > > > > > > > > > >
> > > > > > > > > > > > > JSK2.0 was 2.0 because of the introduction of the
> > > > > > > > > > > > > proxy verification mechanisms, as well as JERI.
> > > > > > > > > > > > > Absent some new client usage mechanism, River
> > > > > > > > > > > > > doesn’t need to go to 3.0.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Greg.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Dec 17, 2013, at 1:58 PM, Peter
> > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > I think changing services to use safe
> > > > > > > > > > > > > > construction techniques is enough to cause the
> > > > > > > > > > > > > > version jump.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > At this point I've allowed services to continue
> > > > > > > > > > > > > > unsafe construction practices, while logging a
> > > > > > > > > > > > > > SEVERE warning when the Commission interface
> > > > > > > > > > > > > > isn't implemented, rather than fail.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This is a fundamental change to the way
> > > > > > > > > > > > > > services are written.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Assuming that there aren’t major
> > > > > > > > > > > > > > > incompatibilities, I think that would be a
> > > > > > > > > > > > > > > “minor” version change according to our
> > > > > > > > > > > > > > > versioning policy, so we’d be looking at the
> > > > > > > > > > > > > > > “2.3” branch rather than a “3.0” release.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I’m still unnerved by the massive amounts of
> > > > > > > > > > > > > > > changes to both code and tests in the
> > > > > > > > > > > > > > > qa_refactor branch, as well as the apparent
> > > > > > > > > > > > > > > instability of the code, although that seems
> > > > > > > > > > > > > > > to be improving. In the next few weeks I’m
> > > > > > > > > > > > > > > going to try and setup a cross-test case, to
> > > > > > > > > > > > > > > see what the “2.2” tests say about the
> > > > > > > > > > > > > > > potential “2.3” release and vice-versa.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I think what I’d really like to see is an
> > > > > > > > > > > > > > > incremental approach where we update limited
> > > > > > > > > > > > > > > components of the “2.2” branch, one at a
> > > > > > > > > > > > > > > time. Is there anything that we could pull
> > > > > > > > > > > > > > > out piecemeal? Maybe it
> > > > > > > > > > > > would
> > > > > > > > > > > > > > > make sense to split out the infrastructure
> > > > > > > > > > > > > > > services, like Reggie, Mahalo, and Outrigger
> > > > > > > > > > > > > > > into different sub-projects that could be
> > > > > > > > > > > > > > > updated separately?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Greg.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Dec 17, 2013, at 5:03 AM, Peter
> > > > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > When the qa_refactor branch stabilises, I
> > > > > > > > > > > > > > > > plan to merge trunk and provide a beta
> > > > > > > > > > > > > > > > release for client compatibility testing.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Changes made have been focused on making
> > > > > > > > > > > > > > > > our code thread safe, there are significant
> > > > > > > > > > > > > > > > changes internally, the public api remains
> > > > > > > > > > > > > > > > focused on backward compatibility, however
> > > > > > > > > > > > > > > > it is advisable that client services adopt
> > > > > > > > > > > > > > > > new safe construction techniques for
> > > > > > > > > > > > > > > > services and implement the new Commission
> > > > > > > > > > > > > > > > interface.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > What's a suitable test period for client
> > > > > > > > > > > > > > > > testing?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
Hopefully Startable will get enough votes to get across the line, we need to make progress, if you care about River you'll vote for it, not because it's the perfect solution, simply because it's an acceptable solution that makes sense and it marks a progress milestone.
The reflective constructor signature is acceptable and is used by the starter kit and Activation.
We've got Groovy Configuration.
The service class can also define a static main method.
If we at least accept these signatures for our services, they could run in any container.
Perhaps you also want a public shutdown method for lifecycle management? Closeable would also be acceptable?
Is there anything else you need?
Regards,
Peter.
----- Original message -----
> "What we need is a discussion on a more developer-friendly service
> startup convention to replace ServiceStarter."
>
> Maybe, that is to say, I'm not anti but I feel there's an additional
> wider aspect to kickaround as well....
>
> Should there be a failure of the type we've discussed so far, where does
> the responsibility for sorting it out lie? The service writer, they
> created the bug in the first place. BUT, has it occurred to anyone that
> export() isn't the only place multi-threading problems can occur in a
> service implementation?
>
> Even if they've got service construction right (with whatever convention
> or pattern you care to adopt, me, I'm in the camp of if it works because
> it accounts for the JMM it's fine) have they got all their
> multi-threading right elsewhere? What about all those service methods
> exported and advertised by JoinManager?
>
> Do we actually think it's possible to address that challenge? (You could
> do it by single-locking the entire service so only one call can be
> dispatched at a time, yeah right, that's palatable). What makes export a
> special case in this context? Why does that matter so when there are
> many other ways to screw up?
>
>
> On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com> wrote:
>
> >
> > +0. Let me explain…
> >
> > Having a separate interface to say start() is all very well and
> > harmless in itself, but:
> >
> > (a) It doesn’t really address the issue of how service instances should
> > be started. We don’t actually have solid recommendations on this
> > issue, although there have been attempts in various containers to
> > systematize it.
> >
> > (b) the proposed warning addresses a case where a service instance is
> > doing “exporter.export(this);”, which I’d call bad practice in any
> > case. It happens that ServiceStarter appears to encourage this use
> > case because it works by calling a constructor. But the called
> > constructor could (and should) construct an instance of some other
> > class, and then export and register that instance rather than “this”.
> > That style doesn’t mandate a Startable interface, and having Startable
> > doesn’t mandate good multi-threaded code. In other words, you could
> > implement Startable and still have poor thread-safety.
> >
> > (c) even where there is an apparent problem, the exposure window is
> > very small. Consider the last bits of the init(…) method in
> > “com.sun.jini.reggie.ServiceRegistrarImpl”
> >
> > myRef = (Registrar) serverExporter.export(this);
> > proxy = RegistrarProxy.getInstance(myRef, myServiceID);
> > myLocator = (proxy instanceof RemoteMethodControl) ?
> > new ConstrainableLookupLocator(
> > unicastDiscoveryHost, unicaster.port, null) :
> > new LookupLocator(unicastDiscoveryHost, unicaster.port);
> > /* register myself */
> > Item item = new Item(new ServiceItem(myServiceID,
> > proxy,
> > lookupAttrs));
> > SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
> > addService(reg);
> > if (log != null) {
> > log.snapshot();
> > }
> >
> > try {
> > DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
> > discoer;
> > String[] groups = dgm.getGroups();
> > if (groups == null || groups.length > 0) {
> > throw new ConfigurationException(
> > "discoveryManager must be initially configured with " +
> > "no groups");
> > }
> > DiscoveryLocatorManagement dlm =
> > (DiscoveryLocatorManagement) discoer;
> > if (dlm.getLocators().length > 0) {
> > throw new ConfigurationException(
> > "discoveryManager must be initially configured with " +
> > "no locators");
> > }
> > dgm.setGroups(lookupGroups);
> > dlm.setLocators(lookupLocators);
> > } catch (ClassCastException e) {
> > throw new ConfigurationException(null, e);
> > }
> > joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
> > discoer, null, config);
> >
> > /* start up all the daemon threads */
> > serviceExpirer.start();
> > eventExpirer.start();
> > unicaster.start();
> > multicaster.start();
> > announcer.start();
> >
> > /* Shutdown hook so reggie sends a final announcement
> > * packet if VM is terminated. If reggie is terminated
> > * through DestroyAdmin.destroy() this hook will have no effect.
> > * A timeout on announcer.join() was considered but not deemed
> > * necessary at this point in time.
> > */
> > Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
> > public void run() {
> > try {
> > announcer.interrupt();
> > announcer.join();
> > } catch (Throwable t) {
> > logThrow(Level.FINEST, getClass().getName(),
> > "run", "exception shutting announcer down",
> > new Object[]{}, t);
> > }
> > }
> > }));
> >
> > snapshotter.start();
> > if (logger.isLoggable(Level.INFO)) {
> > logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
> > new Object[]{ myServiceID,
> > Arrays.asList(memberGroups),
> > myLocator });
> > }
> > ready.ready();
> > }
> >
> >
> > The window from when the join manager is started up (ignore the
> > serverExporter.export(). The only way a client gets to this endpoint
> > is through a proxy downloaded from the registrar) to when the init()
> > method exits is effectively 9 lines of code, during which time a _lot_
> > of network i/o would have to happen in order for the object to be used
> > prior to the constructor exit. As well, the actual remote calls are
> > filtered through a command interface and through the tasker object,
> > and the command classes often go through synchronized() blocks, which
> > (I think) act as happens-before fences.
> >
> > The whole thing seems to me a little like the old servlet days, when we
> > got the SingleThreadModel interface and it only served to give naive
> > developers a false sense of security.
> >
> > What we need is a discussion on a more developer-friendly service
> > startup convention to replace ServiceStarter.
> >
> > Having said that, I’m not overly opposed to having Startable, given
> > that as Dan suggested, any output from the ServiceStarter is in the
> > form of a suggestion rather than an error.
> >
> > Cheers,
> >
> > Greg.
> >
> >
> >
> > On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
> >
> > > +1 Peter.
> > >
> > >
> > > I've been going over the Exporter implementations and Exporter
> > > interface.
> > >
> > > The Exporter interface specifies that export and unexport behaviour
> > > is
> > defined by the implementation.
> > >
> > > That means it's up to the implementation to specify whether a happens
> > before edge occures before export.
> > >
> > > I propose we document in Exporter, that if export is called during
> > object construction then that object's implementation cannot safely use
> > final fields.
> > >
> > > We should also provide a @see reference to Startable to reccommend
> > > its
> > use when final fields are desirable and include Startable in the spec.
> > >
> > > We don't need to rewrite the example of exporting in net.jini.config
> > > as
> > export isn't performed during construction.
> > >
> > > This is a minor update to the Jini Specification.
> > >
> > > The FINE log message is an implementation detail and won't be
> > > included.
> > >
> > > Regards,
> > >
> > > Peter.
> > >
> > > ----- Original message -----
> > > > No, an effectively immutable field is one whose reference doesn't
> > > > change after the service objects safe publication.
> > > >
> > > > An immutable field is final.
> > > >
> > > > It would be Jeri's responsibility to safely publish the service
> > > > during export to ensure a happens before event occurs, ensuring
> > > > all non volatile fields the service contains are safely published
> > > > and visible.
> > > >
> > > > It's possible to have mutable fields guarded by synchronized
> > > > methods, or volatile fields.
> > > >
> > > > But it isn't possible to have final fields when exporting within
> > > > the constructor.
> > > >
> > > > If the service contains effectively immutable field references, it
> > > > must be safely published or if another thread accesses those
> > > > fields through unsynchronized methods, the thread may only see
> > > > their default value.
> > > >
> > > > All our service implementations have final fields. All our
> > > > services
> > had
> > > > some unsynchronized access to non volatile, non final fields.
> > > >
> > > > Trouble is, when you can't have final fields, it's very difficult
> > > > to determine which fields are effectively immutable and the
> > > > original programmers design intent, if there's no comment.
> > > >
> > > > Our service implementations all had unsynchronized init() methods
> > > > called from with the constructor that started threads and exported
> > > > Some
> > fields
> > > > were final.
> > > >
> > > > For me, a service that contains final fields and is exported after
> > > > construction is easier to maintain and understand.
> > > >
> > > > I suspect, like our Jini service imlementations, most people get it
> > > > wrong most of the time.
> > > >
> > > > I get that others won't want to use this method, however it would
> > > > be detrimental if we don't have such a method for those who would.
> > > >
> > > > Regards,
> > > >
> > > > Peter.
> > > >
> > > > ----- Original message -----
> > > > >
> > > > > I feel like we’re going down a rabbit hole here when you start
> > > > > talking about exporting immutable objects. Wouldn’t it be
> > > > > kind of silly to export an immutable service? Isn’t the whole
> > > > > point that you interact with the service (i.e. alter its state)
> > > > > over a remote interface?
> > > > >
> > > > > Perhaps it’s better to say that exported services need to be
> > > > > thread-safe (which should be fairly obvious). Yes, immutable
> > > > > objects are inherently thread-safe, so for sharing information
> > > > > inside a VM, it makes some sense to minimize the number of
> > > > > mutable objects you pass around. But fundamentally, we’re
> > > > > talking about shared-state systems here.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Greg.
> > > > >
> > > > > On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
> > > > >
> > > > > > Pat your comment about non final fields is interesting.
> > > > > >
> > > > > > Isn't it also the case that we need to safely publish an
> > > > > > effectively immutable object to share it among threads?
> > > > > > That usually means copying it to a thread safe collection or
> > > > > > shared via a synchronized method, volatile field, or final
> > > > > > field in another object?
> > > > > >
> > > > > > So we should also make sure that Jeri uses safe publication
> > > > > > during export.
> > > > > >
> > > > > > That would allow a service that has no final fields to start
> > > > > > threads, then export from within a constructor safely,
> > > > > > provided all operations on non final fields happen before
> > > > > > starting threads and exporting.
> > > > > >
> > > > > > All our services have final fields, so Starter is more
> > > > > > appropriate for River's own services.
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > Peter.
> > > > > >
> > > > > > ----- Original message -----
> > > > > > > Hmm, good point, Startable, makes more sense.
> > > > > > >
> > > > > > > An object can be exported using Startable.
> > > > > > >
> > > > > > > I think we should have a policy to strongly discourage
> > > > > > > exporting from constructors.
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > > Peter.
> > > > > > >
> > > > > > > ----- Original message -----
> > > > > > > > As far as I can tell, the special properties of completing
> > > > > > > > a constructor in the JLS memory model are:
> > > > > > > >
> > > > > > > > 1. A happens-before edge from the end of the constructor
> > > > > > > > to the start of a finalizer. (17.4.5)
> > > > > > > >
> > > > > > > > 2. The guarantee that any thread that only sees a
> > > > > > > > reference to an object after the end of the
> > > > > > > > constructor will see the correctly initialized values
> > > > > > > > of all final fields. (17.5)
> > > > > > > >
> > > > > > > > The special issue with final fields is that
> > > > > > > > implementations have freedom to optimize access to
> > > > > > > > final fields in ways that are not permitted for
> > > > > > > > non-final fields. Strategies for thread safety that work
> > > > > > > > for non-final fields do not necessarily work for
> > > > > > > > final fields. The requirement for final field safety
> > > > > > > > is that the constructor end before another thread accesses
> > > > > > > > the newly constructed object.
> > > > > > > >
> > > > > > > > Calling a start() method after construction if the class
> > > > > > > > implements a new interface seems to me to be harmless,
> > > > > > > > backwards compatible, and useful. It enables the
> > > > > > > > simplest and most direct way of preventing access to
> > > > > > > > the new object by another thread during construction.
> > > > > > > >
> > > > > > > > The roadmap issue is whether it should be required, and if
> > > > > > > > so the level of enforcement. For example, there is no
> > > > > > > > reason to require it if the class does not declare
> > > > > > > > any final fields.
> > > > > > > >
> > > > > > > > Incidentally, and as a detail, "Commission" does not
> > > > > > > > immediately make me think of having a start() method
> > > > > > > > that should be called after construction. If you do go
> > > > > > > > this way, the name needs thought. "Startable" would
> > > > > > > > be more obvious, more memorable, more likely to be found
> > > > > > > > on searches, and more compatible with familiar
> > > > > > > > interface names such as "Cloneable" and "Iterable".
> > > > > > > >
> > > > > > > > Patricia
> > > > > > > >
> > > > > > > >
> > > > > > > > On 12/18/2013 2:18 AM, Peter wrote:
> > > > > > > > > Well, now seems like a good time to have the
> > > > > > > > > conversation.
> > > > > > > > >
> > > > > > > > > Yes there are other ways, but I haven't seen one safe
> > > > > > > > > implementation yet, so...
> > > > > > > > >
> > > > > > > > > Does someone have a better way to solve this problem, has
> > > > > > > > > someone already solved this problem I'm unaware of that
> > > > > > > > > we can adopt, or is there a way that's more satisfactory?
> > > > > > > > >
> > > > > > > > > If not, is there something objectionable with the
> > > > > > > > > Commission interface and if so, how can we fix it?
> > > > > > > > >
> > > > > > > > > The SEVERE log message is logged by the River start
> > > > > > > > > package, other containers or frameworks can choose
> > > > > > > > > whether or not to do so, but I'd encourage them to do
> > > > > > > > > something similar, yes we can change it to WARN.
> > > > > > > > >
> > > > > > > > > A much harsher option is to throw an exception during
> > > > > > > > > export which breaks backward compatibility.
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > >
> > > > > > > > > Peter.
> > > > > > > > >
> > > > > > > > > ----- Original message -----
> > > > > > > > > > "org.apache.river.api.util.Commission is an interface
> > > > > > > > > > services should implement"
> > > > > > > > > >
> > > > > > > > > > If it's a SHOULD, not a MUST, chucking out a SEVERE is
> > > > > > > > > > incorrect logger behaviour IMO. You could issue a WARN
> > > > > > > > > > if you like but for even that I'd say you need to
> > > > > > > > > > provide a roadmap explaining why the warning and what
> > > > > > > > > > you intend to do in future and what you expect of
> > > > > > > > > > service writers such as myself.
> > > > > > > > > >
> > > > > > > > > > Commission, at least from my point of view, is your
> > > > > > > > > > means (maybe the River community's - did you ask us?)
> > > > > > > > > > for satisfying your needs in respect of the JMM. As
> > > > > > > > > > we've discussed previously, there are other ways too
> > > > > > > > > > and they work and they are safe if you know what
> > > > > > > > > > you're doing. Your contention was that most don't know
> > > > > > > > > > what they're doing hence, presumably, Commission.
> > > > > > > > > >
> > > > > > > > > > So the thing is, you are seemingly on a road to
> > > > > > > > > > asserting more structure (gosh, a standard?) on the
> > > > > > > > > > way people write their services. If so, you'd best
> > > > > > > > > > start flagging that honestly and openly via a roadmap,
> > > > > > > > > > deprecation and such/whatever rather than sticking out
> > > > > > > > > > logger messages with no clear guidance and at the cost
> > > > > > > > > > of a certain amount of nuisance (no admin I know likes
> > > > > > > > > > SEVERE's being logged for something which isn't
> > > > > > > > > > critical cos it's noise they don't want in log files).
> > > > > > > > > >
> > > > > > > > > > And of course, we all know that when some entity
> > > > > > > > > > asserts a standard or requirement on others for entry,
> > > > > > > > > > they may choose not to enter. Does this help your
> > > > > > > > > > community or hinder it? The answer to that is, it
> > > > > > > > > > depends. On what? Have you asked or tested? How have
> > > > > > > > > > you tested? What would be considered validation or
> > > > > > > > > > lack of support?
> > > > > > > > > >
> > > > > > > > > > I am not out to flame or troll rather I want to see
> > > > > > > > > > this community demonstrating good behaviour and I'm
> > > > > > > > > > not feeling like what's going on around Commission
> > > > > > > > > > (what is that big change in version number really
> > > > > > > > > > saying?) is such.
> > > > > > > > > >
> > > > > > > > > > On 18 December 2013 08:52, Peter <ji...@zeus.net.au>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Just to clarify org.apache.river.api.util.Commission
> > > > > > > > > > > is an interface services should implement, I would
> > > > > > > > > > > encourage all container projects to pick up the
> > > > > > > > > > > interface and make suggestions for improvement if
> > > > > > > > > > > there are any issues.
> > > > > > > > > > >
> > > > > > > > > > > Interface Commission {
> > > > > > > > > > > void start () throws Exception;
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > It's called after JMM safe construction to allow the
> > > > > > > > > > > service to start any threads and be exported.
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > Peter.
> > > > > > > > > > >
> > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > >
> > > > > > > > > > > > The way that services are instantiated and setup
> > > > > > > > > > > > is an implementation detail.
> > > > > > > > > > > > When I think of compatibility I think of the API
> > > > > > > > > > > > and the lookup methods. We think of compatibility
> > > > > > > > > > > > from a client point of view.
> > > > > > > > > > > > From the client point of view, using a service
> > > > > > > > > > > > looks like this:
> > > > > > > > > > > >
> > > > > > > > > > > > - Use multicast of unicast discovery to find one
> > > > > > > > > > > > or more ServiceRegistrar instances
> > > > > > > > > > > > - Call lookup(…) on one or more of
> > > > > > > > > > > > these instances to get a set of service candidates
> > > > > > > > > > > > - Choose a candidate and prepare() it using a
> > > > > > > > > > > > ProxyPreparer, to yield a usable service proxy.
> > > > > > > > > > > > - Make calls on it.
> > > > > > > > > > > > Ideally hang on to this proxy
> > > > > > > > > > > > instance, so you can skip the discovery and lookup
> > > > > > > > > > > > next time you need it.
> > > > > > > > > > > > - If the call fails, repeat the lookup (and
> > > > > > > > > > > > possibly discovery) til you get a proxy that works.
> > > > > > > > > > > >
> > > > > > > > > > > > Nowhere does the client need to know whether the
> > > > > > > > > > > > service instance is started up using the
> > > > > > > > > > > > “com.sun.jini.start” mechanism, your Commission
> > > > > > > > > > > > interface, some other IOC container (Rio,
> > > > > > > > > > > > Harvester, Seven or RiverContainer) or some
> > > > > > > > > > > > unknown mechanism that starts with a static main()
> > > > > > > > > > > > method.
> > > > > > > > > > > >
> > > > > > > > > > > > JSK2.0 was 2.0 because of the introduction of the
> > > > > > > > > > > > proxy verification mechanisms, as well as JERI.
> > > > > > > > > > > > Absent some new client usage mechanism, River
> > > > > > > > > > > > doesn’t need to go to 3.0.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers,
> > > > > > > > > > > >
> > > > > > > > > > > > Greg.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Dec 17, 2013, at 1:58 PM, Peter
> > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > I think changing services to use safe
> > > > > > > > > > > > > construction techniques is enough to cause the
> > > > > > > > > > > > > version jump.
> > > > > > > > > > > > >
> > > > > > > > > > > > > At this point I've allowed services to continue
> > > > > > > > > > > > > unsafe construction practices, while logging a
> > > > > > > > > > > > > SEVERE warning when the Commission interface
> > > > > > > > > > > > > isn't implemented, rather than fail.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This is a fundamental change to the way services
> > > > > > > > > > > > > are written.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Regards,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Peter.
> > > > > > > > > > > > >
> > > > > > > > > > > > > ----- Original message -----
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Assuming that there aren’t major
> > > > > > > > > > > > > > incompatibilities, I think that would be a
> > > > > > > > > > > > > > “minor” version change according to our
> > > > > > > > > > > > > > versioning policy, so we’d be looking at the
> > > > > > > > > > > > > > “2.3” branch rather than a “3.0” release.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I’m still unnerved by the massive amounts of
> > > > > > > > > > > > > > changes to both code and tests in the
> > > > > > > > > > > > > > qa_refactor branch, as well as the apparent
> > > > > > > > > > > > > > instability of the code, although that seems
> > > > > > > > > > > > > > to be improving. In the next few weeks I’m
> > > > > > > > > > > > > > going to try and setup a cross-test case, to
> > > > > > > > > > > > > > see what the “2.2” tests say about the
> > > > > > > > > > > > > > potential “2.3” release and vice-versa.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I think what I’d really like to see is an
> > > > > > > > > > > > > > incremental approach where we update limited
> > > > > > > > > > > > > > components of the “2.2” branch, one at a time.
> > > > > > > > > > > > > > Is there anything that we could pull out
> > > > > > > > > > > > > > piecemeal? Maybe it
> > > > > > > > > > > would
> > > > > > > > > > > > > > make sense to split out the infrastructure
> > > > > > > > > > > > > > services, like Reggie, Mahalo, and Outrigger
> > > > > > > > > > > > > > into different sub-projects that could be
> > > > > > > > > > > > > > updated separately?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Any thoughts?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Greg.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Dec 17, 2013, at 5:03 AM, Peter
> > > > > > > > > > > > > > <ji...@zeus.net.au> wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > When the qa_refactor branch stabilises, I
> > > > > > > > > > > > > > > plan to merge trunk and provide a beta
> > > > > > > > > > > > > > > release for client compatibility testing.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Changes made have been focused on making our
> > > > > > > > > > > > > > > code thread safe, there are significant
> > > > > > > > > > > > > > > changes internally, the public api remains
> > > > > > > > > > > > > > > focused on backward compatibility, however
> > > > > > > > > > > > > > > it is advisable that client services adopt
> > > > > > > > > > > > > > > new safe construction techniques for
> > > > > > > > > > > > > > > services and implement the new Commission
> > > > > > > > > > > > > > > interface.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > What's a suitable test period for client
> > > > > > > > > > > > > > > testing?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Peter.
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
Re: VOTE: add Startable to Jini specification
Posted by Dan Creswell <da...@gmail.com>.
"What we need is a discussion on a more developer-friendly service startup
convention to replace ServiceStarter."
Maybe, that is to say, I'm not anti but I feel there's an additional wider
aspect to kickaround as well....
Should there be a failure of the type we've discussed so far, where does
the responsibility for sorting it out lie? The service writer, they created
the bug in the first place. BUT, has it occurred to anyone that export()
isn't the only place multi-threading problems can occur in a service
implementation?
Even if they've got service construction right (with whatever convention or
pattern you care to adopt, me, I'm in the camp of if it works because it
accounts for the JMM it's fine) have they got all their multi-threading
right elsewhere? What about all those service methods exported and
advertised by JoinManager?
Do we actually think it's possible to address that challenge? (You could do
it by single-locking the entire service so only one call can be dispatched
at a time, yeah right, that's palatable). What makes export a special case
in this context? Why does that matter so when there are many other ways to
screw up?
On 19 December 2013 20:16, Greg Trasuk <tr...@stratuscom.com> wrote:
>
> +0. Let me explain…
>
> Having a separate interface to say start() is all very well and harmless
> in itself, but:
>
> (a) It doesn’t really address the issue of how service instances should
> be started. We don’t actually have solid recommendations on this issue,
> although there have been attempts in various containers to systematize it.
>
> (b) the proposed warning addresses a case where a service instance is
> doing “exporter.export(this);”, which I’d call bad practice in any case.
> It happens that ServiceStarter appears to encourage this use case because
> it works by calling a constructor. But the called constructor could (and
> should) construct an instance of some other class, and then export and
> register that instance rather than “this”. That style doesn’t mandate a
> Startable interface, and having Startable doesn’t mandate good
> multi-threaded code. In other words, you could implement Startable and
> still have poor thread-safety.
>
> (c) even where there is an apparent problem, the exposure window is very
> small. Consider the last bits of the init(…) method in
> “com.sun.jini.reggie.ServiceRegistrarImpl”
>
> myRef = (Registrar) serverExporter.export(this);
> proxy = RegistrarProxy.getInstance(myRef, myServiceID);
> myLocator = (proxy instanceof RemoteMethodControl) ?
> new ConstrainableLookupLocator(
> unicastDiscoveryHost, unicaster.port, null) :
> new LookupLocator(unicastDiscoveryHost, unicaster.port);
> /* register myself */
> Item item = new Item(new ServiceItem(myServiceID,
> proxy,
> lookupAttrs));
> SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
> addService(reg);
> if (log != null) {
> log.snapshot();
> }
>
> try {
> DiscoveryGroupManagement dgm = (DiscoveryGroupManagement)
> discoer;
> String[] groups = dgm.getGroups();
> if (groups == null || groups.length > 0) {
> throw new ConfigurationException(
> "discoveryManager must be initially configured with " +
> "no groups");
> }
> DiscoveryLocatorManagement dlm =
> (DiscoveryLocatorManagement) discoer;
> if (dlm.getLocators().length > 0) {
> throw new ConfigurationException(
> "discoveryManager must be initially configured with " +
> "no locators");
> }
> dgm.setGroups(lookupGroups);
> dlm.setLocators(lookupLocators);
> } catch (ClassCastException e) {
> throw new ConfigurationException(null, e);
> }
> joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
> discoer, null, config);
>
> /* start up all the daemon threads */
> serviceExpirer.start();
> eventExpirer.start();
> unicaster.start();
> multicaster.start();
> announcer.start();
>
> /* Shutdown hook so reggie sends a final announcement
> * packet if VM is terminated. If reggie is terminated
> * through DestroyAdmin.destroy() this hook will have no effect.
> * A timeout on announcer.join() was considered but not deemed
> * necessary at this point in time.
> */
> Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
> public void run() {
> try {
> announcer.interrupt();
> announcer.join();
> } catch (Throwable t) {
> logThrow(Level.FINEST, getClass().getName(),
> "run", "exception shutting announcer down",
> new Object[]{}, t);
> }
> }
> }));
>
> snapshotter.start();
> if (logger.isLoggable(Level.INFO)) {
> logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
> new Object[]{ myServiceID,
> Arrays.asList(memberGroups),
> myLocator });
> }
> ready.ready();
> }
>
>
> The window from when the join manager is started up (ignore the
> serverExporter.export(). The only way a client gets to this endpoint is
> through a proxy downloaded from the registrar) to when the init() method
> exits is effectively 9 lines of code, during which time a _lot_ of network
> i/o would have to happen in order for the object to be used prior to the
> constructor exit. As well, the actual remote calls are filtered through a
> command interface and through the tasker object, and the command classes
> often go through synchronized() blocks, which (I think) act as
> happens-before fences.
>
> The whole thing seems to me a little like the old servlet days, when we
> got the SingleThreadModel interface and it only served to give naive
> developers a false sense of security.
>
> What we need is a discussion on a more developer-friendly service startup
> convention to replace ServiceStarter.
>
> Having said that, I’m not overly opposed to having Startable, given that
> as Dan suggested, any output from the ServiceStarter is in the form of a
> suggestion rather than an error.
>
> Cheers,
>
> Greg.
>
>
>
> On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
>
> > +1 Peter.
> >
> >
> > I've been going over the Exporter implementations and Exporter interface.
> >
> > The Exporter interface specifies that export and unexport behaviour is
> defined by the implementation.
> >
> > That means it's up to the implementation to specify whether a happens
> before edge occures before export.
> >
> > I propose we document in Exporter, that if export is called during
> object construction then that object's implementation cannot safely use
> final fields.
> >
> > We should also provide a @see reference to Startable to reccommend its
> use when final fields are desirable and include Startable in the spec.
> >
> > We don't need to rewrite the example of exporting in net.jini.config as
> export isn't performed during construction.
> >
> > This is a minor update to the Jini Specification.
> >
> > The FINE log message is an implementation detail and won't be included.
> >
> > Regards,
> >
> > Peter.
> >
> > ----- Original message -----
> >> No, an effectively immutable field is one whose reference doesn't change
> >> after the service objects safe publication.
> >>
> >> An immutable field is final.
> >>
> >> It would be Jeri's responsibility to safely publish the service during
> >> export to ensure a happens before event occurs, ensuring all non
> >> volatile fields the service contains are safely published and visible.
> >>
> >> It's possible to have mutable fields guarded by synchronized methods, or
> >> volatile fields.
> >>
> >> But it isn't possible to have final fields when exporting within the
> >> constructor.
> >>
> >> If the service contains effectively immutable field references, it must
> >> be safely published or if another thread accesses those fields through
> >> unsynchronized methods, the thread may only see their default value.
> >>
> >> All our service implementations have final fields. All our services
> had
> >> some unsynchronized access to non volatile, non final fields.
> >>
> >> Trouble is, when you can't have final fields, it's very difficult to
> >> determine which fields are effectively immutable and the original
> >> programmers design intent, if there's no comment.
> >>
> >> Our service implementations all had unsynchronized init() methods called
> >> from with the constructor that started threads and exported Some
> fields
> >> were final.
> >>
> >> For me, a service that contains final fields and is exported after
> >> construction is easier to maintain and understand.
> >>
> >> I suspect, like our Jini service imlementations, most people get it
> >> wrong most of the time.
> >>
> >> I get that others won't want to use this method, however it would be
> >> detrimental if we don't have such a method for those who would.
> >>
> >> Regards,
> >>
> >> Peter.
> >>
> >> ----- Original message -----
> >>>
> >>> I feel like we’re going down a rabbit hole here when you start talking
> >>> about exporting immutable objects. Wouldn’t it be kind of silly to
> >>> export an immutable service? Isn’t the whole point that you interact
> >>> with the service (i.e. alter its state) over a remote interface?
> >>>
> >>> Perhaps it’s better to say that exported services need to be
> >>> thread-safe (which should be fairly obvious). Yes, immutable objects
> >>> are inherently thread-safe, so for sharing information inside a VM, it
> >>> makes some sense to minimize the number of mutable objects you pass
> >>> around. But fundamentally, we’re talking about shared-state systems
> >>> here.
> >>>
> >>> Cheers,
> >>>
> >>> Greg.
> >>>
> >>> On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
> >>>
> >>>> Pat your comment about non final fields is interesting.
> >>>>
> >>>> Isn't it also the case that we need to safely publish an effectively
> >>>> immutable object to share it among threads? That usually means
> >>>> copying it to a thread safe collection or shared via a synchronized
> >>>> method, volatile field, or final field in another object?
> >>>>
> >>>> So we should also make sure that Jeri uses safe publication during
> >>>> export.
> >>>>
> >>>> That would allow a service that has no final fields to start threads,
> >>>> then export from within a constructor safely, provided all operations
> >>>> on non final fields happen before starting threads and exporting.
> >>>>
> >>>> All our services have final fields, so Starter is more appropriate
> >>>> for River's own services.
> >>>>
> >>>> Regards,
> >>>>
> >>>> Peter.
> >>>>
> >>>> ----- Original message -----
> >>>>> Hmm, good point, Startable, makes more sense.
> >>>>>
> >>>>> An object can be exported using Startable.
> >>>>>
> >>>>> I think we should have a policy to strongly discourage exporting
> >>>>> from constructors.
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> Peter.
> >>>>>
> >>>>> ----- Original message -----
> >>>>>> As far as I can tell, the special properties of completing a
> >>>>>> constructor in the JLS memory model are:
> >>>>>>
> >>>>>> 1. A happens-before edge from the end of the constructor to the
> >>>>>> start of a finalizer. (17.4.5)
> >>>>>>
> >>>>>> 2. The guarantee that any thread that only sees a reference to an
> >>>>>> object after the end of the constructor will see the
> >>>>>> correctly initialized values of all final fields. (17.5)
> >>>>>>
> >>>>>> The special issue with final fields is that implementations have
> >>>>>> freedom to optimize access to final fields in ways that are
> >>>>>> not permitted for non-final fields. Strategies for thread
> >>>>>> safety that work for non-final fields do not necessarily
> >>>>>> work for final fields. The requirement for final field
> >>>>>> safety is that the constructor end before another thread
> >>>>>> accesses the newly constructed object.
> >>>>>>
> >>>>>> Calling a start() method after construction if the class
> >>>>>> implements a new interface seems to me to be harmless,
> >>>>>> backwards compatible, and useful. It enables the simplest and
> >>>>>> most direct way of preventing access to the new object by
> >>>>>> another thread during construction.
> >>>>>>
> >>>>>> The roadmap issue is whether it should be required, and if so the
> >>>>>> level of enforcement. For example, there is no reason to
> >>>>>> require it if the class does not declare any final fields.
> >>>>>>
> >>>>>> Incidentally, and as a detail, "Commission" does not immediately
> >>>>>> make me think of having a start() method that should be
> >>>>>> called after construction. If you do go this way, the name
> >>>>>> needs thought. "Startable" would be more obvious, more
> >>>>>> memorable, more likely to be found on searches, and more
> >>>>>> compatible with familiar interface names such as "Cloneable"
> >>>>>> and "Iterable".
> >>>>>>
> >>>>>> Patricia
> >>>>>>
> >>>>>>
> >>>>>> On 12/18/2013 2:18 AM, Peter wrote:
> >>>>>>> Well, now seems like a good time to have the conversation.
> >>>>>>>
> >>>>>>> Yes there are other ways, but I haven't seen one safe
> >>>>>>> implementation yet, so...
> >>>>>>>
> >>>>>>> Does someone have a better way to solve this problem, has
> >>>>>>> someone already solved this problem I'm unaware of that we can
> >>>>>>> adopt, or is there a way that's more satisfactory?
> >>>>>>>
> >>>>>>> If not, is there something objectionable with the Commission
> >>>>>>> interface and if so, how can we fix it?
> >>>>>>>
> >>>>>>> The SEVERE log message is logged by the River start package,
> >>>>>>> other containers or frameworks can choose whether or not to do
> >>>>>>> so, but I'd encourage them to do something similar, yes we can
> >>>>>>> change it to WARN.
> >>>>>>>
> >>>>>>> A much harsher option is to throw an exception during export
> >>>>>>> which breaks backward compatibility.
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>>
> >>>>>>> Peter.
> >>>>>>>
> >>>>>>> ----- Original message -----
> >>>>>>>> "org.apache.river.api.util.Commission is an interface
> >>>>>>>> services should implement"
> >>>>>>>>
> >>>>>>>> If it's a SHOULD, not a MUST, chucking out a SEVERE is
> >>>>>>>> incorrect logger behaviour IMO. You could issue a WARN if you
> >>>>>>>> like but for even that I'd say you need to provide a roadmap
> >>>>>>>> explaining why the warning and what you intend to do in
> >>>>>>>> future and what you expect of service writers such as myself.
> >>>>>>>>
> >>>>>>>> Commission, at least from my point of view, is your means
> >>>>>>>> (maybe the River community's - did you ask us?) for
> >>>>>>>> satisfying your needs in respect of the JMM. As we've
> >>>>>>>> discussed previously, there are other ways too and they work
> >>>>>>>> and they are safe if you know what you're doing. Your
> >>>>>>>> contention was that most don't know what they're doing
> >>>>>>>> hence, presumably, Commission.
> >>>>>>>>
> >>>>>>>> So the thing is, you are seemingly on a road to asserting
> >>>>>>>> more structure (gosh, a standard?) on the way people write
> >>>>>>>> their services. If so, you'd best start flagging that
> >>>>>>>> honestly and openly via a roadmap, deprecation and
> >>>>>>>> such/whatever rather than sticking out logger messages with
> >>>>>>>> no clear guidance and at the cost of a certain amount of
> >>>>>>>> nuisance (no admin I know likes SEVERE's being logged for
> >>>>>>>> something which isn't critical cos it's noise they don't
> >>>>>>>> want in log files).
> >>>>>>>>
> >>>>>>>> And of course, we all know that when some entity asserts a
> >>>>>>>> standard or requirement on others for entry, they may choose
> >>>>>>>> not to enter. Does this help your community or hinder it? The
> >>>>>>>> answer to that is, it depends. On what? Have you asked or
> >>>>>>>> tested? How have you tested? What would be considered
> >>>>>>>> validation or lack of support?
> >>>>>>>>
> >>>>>>>> I am not out to flame or troll rather I want to see this
> >>>>>>>> community demonstrating good behaviour and I'm not feeling
> >>>>>>>> like what's going on around Commission (what is that big
> >>>>>>>> change in version number really saying?) is such.
> >>>>>>>>
> >>>>>>>> On 18 December 2013 08:52, Peter <ji...@zeus.net.au> wrote:
> >>>>>>>>
> >>>>>>>>> Just to clarify org.apache.river.api.util.Commission is an
> >>>>>>>>> interface services should implement, I would encourage all
> >>>>>>>>> container projects to pick up the interface and make
> >>>>>>>>> suggestions for improvement if there are any issues.
> >>>>>>>>>
> >>>>>>>>> Interface Commission {
> >>>>>>>>> void start () throws Exception;
> >>>>>>>>> }
> >>>>>>>>>
> >>>>>>>>> It's called after JMM safe construction to allow the
> >>>>>>>>> service to start any threads and be exported.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>>
> >>>>>>>>> Peter.
> >>>>>>>>>
> >>>>>>>>> ----- Original message -----
> >>>>>>>>>>
> >>>>>>>>>> The way that services are instantiated and setup is an
> >>>>>>>>>> implementation detail. When I think of
> >>>>>>>>>> compatibility I think of the API and the lookup methods.
> >>>>>>>>>> We think of compatibility from a
> >>>>>>>>>> client point of view. From the client
> >>>>>>>>>> point of view, using a service looks like this:
> >>>>>>>>>>
> >>>>>>>>>> - Use multicast of unicast discovery to find one or more
> >>>>>>>>>> ServiceRegistrar instances -
> >>>>>>>>>> Call lookup(…) on one or more of these instances to get
> >>>>>>>>>> a set of service candidates - Choose a candidate and
> >>>>>>>>>> prepare() it using a ProxyPreparer, to yield a usable
> >>>>>>>>>> service proxy. - Make calls on
> >>>>>>>>>> it. Ideally hang on to this proxy
> >>>>>>>>>> instance, so you can skip the discovery and lookup next
> >>>>>>>>>> time you need it. - If the
> >>>>>>>>>> call fails, repeat the lookup (and possibly discovery)
> >>>>>>>>>> til you get a proxy that works.
> >>>>>>>>>>
> >>>>>>>>>> Nowhere does the client need to know whether the service
> >>>>>>>>>> instance is started up using the “com.sun.jini.start”
> >>>>>>>>>> mechanism, your Commission interface, some other IOC
> >>>>>>>>>> container (Rio, Harvester, Seven or RiverContainer) or
> >>>>>>>>>> some unknown mechanism that starts with a static main()
> >>>>>>>>>> method.
> >>>>>>>>>>
> >>>>>>>>>> JSK2.0 was 2.0 because of the introduction of the proxy
> >>>>>>>>>> verification mechanisms, as well as JERI.
> >>>>>>>>>> Absent some new client usage mechanism, River doesn’t
> >>>>>>>>>> need to go to 3.0.
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>>
> >>>>>>>>>> Greg.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Dec 17, 2013, at 1:58 PM, Peter <ji...@zeus.net.au>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> I think changing services to use safe construction
> >>>>>>>>>>> techniques is enough to cause the version jump.
> >>>>>>>>>>>
> >>>>>>>>>>> At this point I've allowed services to continue unsafe
> >>>>>>>>>>> construction practices, while logging a SEVERE warning
> >>>>>>>>>>> when the Commission interface isn't implemented, rather
> >>>>>>>>>>> than fail.
> >>>>>>>>>>>
> >>>>>>>>>>> This is a fundamental change to the way services are
> >>>>>>>>>>> written.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>>
> >>>>>>>>>>> Peter.
> >>>>>>>>>>>
> >>>>>>>>>>> ----- Original message -----
> >>>>>>>>>>>>
> >>>>>>>>>>>> Assuming that there aren’t major incompatibilities, I
> >>>>>>>>>>>> think that would be a “minor” version change
> >>>>>>>>>>>> according to our versioning policy, so we’d be
> >>>>>>>>>>>> looking at the “2.3” branch rather than a “3.0”
> >>>>>>>>>>>> release.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I’m still unnerved by the massive amounts of changes
> >>>>>>>>>>>> to both code and tests in the qa_refactor branch, as
> >>>>>>>>>>>> well as the apparent instability of the code,
> >>>>>>>>>>>> although that seems to be improving.
> >>>>>>>>>>>> In the next few weeks I’m going
> >>>>>>>>>>>> to try and setup a cross-test case, to see what the
> >>>>>>>>>>>> “2.2” tests say about the potential “2.3” release
> >>>>>>>>>>>> and vice-versa.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I think what I’d really like to see is an incremental
> >>>>>>>>>>>> approach where we update limited components of the
> >>>>>>>>>>>> “2.2” branch, one at a time. Is there anything that
> >>>>>>>>>>>> we could pull out piecemeal? Maybe it
> >>>>>>>>> would
> >>>>>>>>>>>> make sense to split out the infrastructure services,
> >>>>>>>>>>>> like Reggie, Mahalo, and Outrigger into different
> >>>>>>>>>>>> sub-projects that could be updated separately?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Any thoughts?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Greg.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Dec 17, 2013, at 5:03 AM, Peter <ji...@zeus.net.au>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> When the qa_refactor branch stabilises, I plan to
> >>>>>>>>>>>>> merge trunk and provide a beta release for client
> >>>>>>>>>>>>> compatibility testing.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Changes made have been focused on making our code
> >>>>>>>>>>>>> thread safe, there are significant changes
> >>>>>>>>>>>>> internally, the public api remains focused on
> >>>>>>>>>>>>> backward compatibility, however it is advisable
> >>>>>>>>>>>>> that client services adopt new safe construction
> >>>>>>>>>>>>> techniques for services and implement the new
> >>>>>>>>>>>>> Commission interface.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What's a suitable test period for client testing?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Peter.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
>
Re: VOTE: add Startable to Jini specification
Posted by Dan Creswell <da...@gmail.com>.
Inline, this time...
On 20 December 2013 06:47, Peter <ji...@zeus.net.au> wrote:
> ----- Original message -----
> > On 12/19/2013 12:16 PM, Greg Trasuk wrote:
> <SNIP>
> > And, using, for example, the strategy you describe of constructing
> > another object, code can be perfectly valid without using Startable. I
> > see value in offering the interface, and a rule that the start method
> > will be called after the constructor for any service implementing it. I
> > don't think River should assume code is not safe just because it does
> > not use that feature - it may be safe for entirely different reasons.
> >
>
> In River's case, every service was constructed unsafely, period, no
> exceptions, in fact the likelihood of not needing it is low, yes there are
> alternate methods including not using final fields, but they are cumbersome
> and unfriendly, there be dragons and only hacker knights would dare tread.
> I offer thee this magic sword with which to slay your dragon. No really,
> it's a fine sword...
>
Constructed unsafely as observed by what or whom?
Simplistically, the point at which this gets multi-threaded on your ass is
when you kick into life an SDM or JoinManager. Although you can use SDM in
a single-threaded way for sure. JoinManager though, that's the dangerous
puppy cos it makes your service reference visible to the world and thus
leaves it open to being invoked concurrently.
> But as Dan suggests, it isn't my job to stop people cutting themselves,
> (Dan, put those scissors down now!) so people are free not to use it and I
> have decided to reduce logging to FINE so you can switch it off if it
> annoys you. Ironically that bug you can't sort will mysterously disappear
> when you turn logging back on as that irritating message about safe
> construction confronts you.
>
There are no scissors, the JMM does not dictate one must use finals to
achieve thread safety. It's one option with caveats, there are others with
different caveats. They all have sharp edges, it's a question of which
tools you're comfortable with. Me, I like final, it has its uses but it is
not the god of fault free multithreaded code.
>
> For other cases where using such an interface would have broken backward
> compatibility, I had to create an abstract superclass with a stateless
> subclass to ensure I had safe construction.
>
> See LookupDiscoveryManager and its abstract superclass in qa_refactor for
> an example. I had to get advice from the concurrency interest list to
> solve that one.
>
> You know JoinManager's private classes were synchronizing on their own
> object locks to access their outer classes state? Oops!
>
>
Those are bugfixes that fall in line with the original specifications/api
docs, thus they are a different beast/class of issue. Go ahead, fix 'em,
it's progress for sure. Make no mistake though, what you're asking to vote
on is a different beast.
> You are a passionate sort with your strengths and weaknesses, like
> everybody, I do enjoy our conversations and could listen while you go blue
> in the face explaining why we should love and preserve our existing code,
> in all its buggy glory. Please stop now and read the code and do some
> research before making any more comments to preserve any credibility that
> remains.
>
> If you're worried about my code, you should be much more worried about the
> existing, can it weather the oncoming muticore cyclone (Australian for
> hurricane)?
>
> How many cores did you have in 2003 son?
>
> One more vote, c'mon at least one for my sense humour? Dan?
>
I don't vote on humour, waste of a vote. I'll give you my vote for this
change but not until you put down the baseball bat (see my other emails).
No, I mean it, I have no problem with your revised proposal but the tone of
your pitch, not liking that so much.
>
> No takers? Oh well...
>
> Merry Christmas,
>
> Peter.
>
> > > (c) even where there is an apparent problem, the exposure window is
> > very small.
> >
> > Remember an exposure window can be small in terms of numbers of
> > statements, but long in terms of elapsed time. Leaving apparently small
> > windows open can result in code that fails mysteriously, for example
> > when a page boundary happens to fall in the window, and the system is
> > under memory pressure.
> >
> > Patricia
>
>
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
Looks like my mobile mail client deleted the wrong paragraph, my reply was intended for Greg's belief we needn't concern ourselves with bugs and refactoring is navel gazing, it was so silly I found myself laughing and probably didn't notice the wrong paragraph was quoted.
I mean, if I didn't find it funny, I probably would have been offended given that he thinks so little of my efforts.
Never mind, he won't think so poorly of it after its release.
Peter.
----- Original message -----
> On 12/19/2013 10:47 PM, Peter wrote:
> > ----- Original message -----
> > > On 12/19/2013 12:16 PM, Greg Trasuk wrote:
> > <SNIP>
> > > And, using, for example, the strategy you describe of constructing
> > > another object, code can be perfectly valid without using Startable.
> > > I see value in offering the interface, and a rule that the start
> > > method will be called after the constructor for any service
> > > implementing it. I don't think River should assume code is not safe
> > > just because it does not use that feature - it may be safe for
> > > entirely different reasons.
> > >
>
> I think the quoting got messed up here. The quoted paragraph is material
> I wrote in reply to one of of Greg's messages.
>
> Patricia
>
Re: VOTE: add Startable to Jini specification
Posted by Patricia Shanahan <pa...@acm.org>.
On 12/19/2013 10:47 PM, Peter wrote:
> ----- Original message -----
>> On 12/19/2013 12:16 PM, Greg Trasuk wrote:
> <SNIP>
>> And, using, for example, the strategy you describe of constructing
>> another object, code can be perfectly valid without using Startable. I
>> see value in offering the interface, and a rule that the start method
>> will be called after the constructor for any service implementing it. I
>> don't think River should assume code is not safe just because it does
>> not use that feature - it may be safe for entirely different reasons.
>>
I think the quoting got messed up here. The quoted paragraph is material
I wrote in reply to one of of Greg's messages.
Patricia
Re: VOTE: add Startable to Jini specification
Posted by Peter <ji...@zeus.net.au>.
----- Original message -----
> On 12/19/2013 12:16 PM, Greg Trasuk wrote:
<SNIP>
> And, using, for example, the strategy you describe of constructing
> another object, code can be perfectly valid without using Startable. I
> see value in offering the interface, and a rule that the start method
> will be called after the constructor for any service implementing it. I
> don't think River should assume code is not safe just because it does
> not use that feature - it may be safe for entirely different reasons.
>
In River's case, every service was constructed unsafely, period, no exceptions, in fact the likelihood of not needing it is low, yes there are alternate methods including not using final fields, but they are cumbersome and unfriendly, there be dragons and only hacker knights would dare tread. I offer thee this magic sword with which to slay your dragon. No really, it's a fine sword...
But as Dan suggests, it isn't my job to stop people cutting themselves, (Dan, put those scissors down now!) so people are free not to use it and I have decided to reduce logging to FINE so you can switch it off if it annoys you. Ironically that bug you can't sort will mysterously disappear when you turn logging back on as that irritating message about safe construction confronts you.
For other cases where using such an interface would have broken backward compatibility, I had to create an abstract superclass with a stateless subclass to ensure I had safe construction.
See LookupDiscoveryManager and its abstract superclass in qa_refactor for an example. I had to get advice from the concurrency interest list to solve that one.
You know JoinManager's private classes were synchronizing on their own object locks to access their outer classes state? Oops!
You are a passionate sort with your strengths and weaknesses, like everybody, I do enjoy our conversations and could listen while you go blue in the face explaining why we should love and preserve our existing code, in all its buggy glory. Please stop now and read the code and do some research before making any more comments to preserve any credibility that remains.
If you're worried about my code, you should be much more worried about the existing, can it weather the oncoming muticore cyclone (Australian for hurricane)?
How many cores did you have in 2003 son?
One more vote, c'mon at least one for my sense humour? Dan?
No takers? Oh well...
Merry Christmas,
Peter.
> > (c) even where there is an apparent problem, the exposure window is
> very small.
>
> Remember an exposure window can be small in terms of numbers of
> statements, but long in terms of elapsed time. Leaving apparently small
> windows open can result in code that fails mysteriously, for example
> when a page boundary happens to fall in the window, and the system is
> under memory pressure.
>
> Patricia
Re: VOTE: add Startable to Jini specification
Posted by Patricia Shanahan <pa...@acm.org>.
On 12/19/2013 12:16 PM, Greg Trasuk wrote:
>
> +0. Let me explain…
>
> Having a separate interface to say start() is all very well and harmless in itself, but:
>
> (a) It doesn’t really address the issue of how service instances
should be started. We don’t actually have solid recommendations on this
issue, although there have been attempts in various containers to
systematize it.
>
> (b) the proposed warning addresses a case where a service instance
> is
doing “exporter.export(this);”, which I’d call bad practice in any case.
It happens that ServiceStarter appears to encourage this use case
because it works by calling a constructor. But the called constructor
could (and should) construct an instance of some other class, and then
export and register that instance rather than “this”. That style doesn’t
mandate a Startable interface, and having Startable doesn’t mandate good
multi-threaded code. In other words, you could implement Startable and
still have poor thread-safety.
And, using, for example, the strategy you describe of constructing
another object, code can be perfectly valid without using Startable. I
see value in offering the interface, and a rule that the start method
will be called after the constructor for any service implementing it. I
don't think River should assume code is not safe just because it does
not use that feature - it may be safe for entirely different reasons.
> (c) even where there is an apparent problem, the exposure window is
very small.
Remember an exposure window can be small in terms of numbers of
statements, but long in terms of elapsed time. Leaving apparently small
windows open can result in code that fails mysteriously, for example
when a page boundary happens to fall in the window, and the system is
under memory pressure.
Patricia
Re: VOTE: add Startable to Jini specification
Posted by Greg Trasuk <tr...@stratuscom.com>.
+0. Let me explain…
Having a separate interface to say start() is all very well and harmless in itself, but:
(a) It doesn’t really address the issue of how service instances should be started. We don’t actually have solid recommendations on this issue, although there have been attempts in various containers to systematize it.
(b) the proposed warning addresses a case where a service instance is doing “exporter.export(this);”, which I’d call bad practice in any case. It happens that ServiceStarter appears to encourage this use case because it works by calling a constructor. But the called constructor could (and should) construct an instance of some other class, and then export and register that instance rather than “this”. That style doesn’t mandate a Startable interface, and having Startable doesn’t mandate good multi-threaded code. In other words, you could implement Startable and still have poor thread-safety.
(c) even where there is an apparent problem, the exposure window is very small. Consider the last bits of the init(…) method in “com.sun.jini.reggie.ServiceRegistrarImpl”
myRef = (Registrar) serverExporter.export(this);
proxy = RegistrarProxy.getInstance(myRef, myServiceID);
myLocator = (proxy instanceof RemoteMethodControl) ?
new ConstrainableLookupLocator(
unicastDiscoveryHost, unicaster.port, null) :
new LookupLocator(unicastDiscoveryHost, unicaster.port);
/* register myself */
Item item = new Item(new ServiceItem(myServiceID,
proxy,
lookupAttrs));
SvcReg reg = new SvcReg(item, myLeaseID, Long.MAX_VALUE);
addService(reg);
if (log != null) {
log.snapshot();
}
try {
DiscoveryGroupManagement dgm = (DiscoveryGroupManagement) discoer;
String[] groups = dgm.getGroups();
if (groups == null || groups.length > 0) {
throw new ConfigurationException(
"discoveryManager must be initially configured with " +
"no groups");
}
DiscoveryLocatorManagement dlm =
(DiscoveryLocatorManagement) discoer;
if (dlm.getLocators().length > 0) {
throw new ConfigurationException(
"discoveryManager must be initially configured with " +
"no locators");
}
dgm.setGroups(lookupGroups);
dlm.setLocators(lookupLocators);
} catch (ClassCastException e) {
throw new ConfigurationException(null, e);
}
joiner = new JoinManager(proxy, lookupAttrs, myServiceID,
discoer, null, config);
/* start up all the daemon threads */
serviceExpirer.start();
eventExpirer.start();
unicaster.start();
multicaster.start();
announcer.start();
/* Shutdown hook so reggie sends a final announcement
* packet if VM is terminated. If reggie is terminated
* through DestroyAdmin.destroy() this hook will have no effect.
* A timeout on announcer.join() was considered but not deemed
* necessary at this point in time.
*/
Runtime.getRuntime().addShutdownHook(new Thread( new Runnable() {
public void run() {
try {
announcer.interrupt();
announcer.join();
} catch (Throwable t) {
logThrow(Level.FINEST, getClass().getName(),
"run", "exception shutting announcer down",
new Object[]{}, t);
}
}
}));
snapshotter.start();
if (logger.isLoggable(Level.INFO)) {
logger.log(Level.INFO, "started Reggie: {0}, {1}, {2}",
new Object[]{ myServiceID,
Arrays.asList(memberGroups),
myLocator });
}
ready.ready();
}
The window from when the join manager is started up (ignore the serverExporter.export(). The only way a client gets to this endpoint is through a proxy downloaded from the registrar) to when the init() method exits is effectively 9 lines of code, during which time a _lot_ of network i/o would have to happen in order for the object to be used prior to the constructor exit. As well, the actual remote calls are filtered through a command interface and through the tasker object, and the command classes often go through synchronized() blocks, which (I think) act as happens-before fences.
The whole thing seems to me a little like the old servlet days, when we got the SingleThreadModel interface and it only served to give naive developers a false sense of security.
What we need is a discussion on a more developer-friendly service startup convention to replace ServiceStarter.
Having said that, I’m not overly opposed to having Startable, given that as Dan suggested, any output from the ServiceStarter is in the form of a suggestion rather than an error.
Cheers,
Greg.
On Dec 18, 2013, at 7:42 PM, Peter <ji...@zeus.net.au> wrote:
> +1 Peter.
>
>
> I've been going over the Exporter implementations and Exporter interface.
>
> The Exporter interface specifies that export and unexport behaviour is defined by the implementation.
>
> That means it's up to the implementation to specify whether a happens before edge occures before export.
>
> I propose we document in Exporter, that if export is called during object construction then that object's implementation cannot safely use final fields.
>
> We should also provide a @see reference to Startable to reccommend its use when final fields are desirable and include Startable in the spec.
>
> We don't need to rewrite the example of exporting in net.jini.config as export isn't performed during construction.
>
> This is a minor update to the Jini Specification.
>
> The FINE log message is an implementation detail and won't be included.
>
> Regards,
>
> Peter.
>
> ----- Original message -----
>> No, an effectively immutable field is one whose reference doesn't change
>> after the service objects safe publication.
>>
>> An immutable field is final.
>>
>> It would be Jeri's responsibility to safely publish the service during
>> export to ensure a happens before event occurs, ensuring all non
>> volatile fields the service contains are safely published and visible.
>>
>> It's possible to have mutable fields guarded by synchronized methods, or
>> volatile fields.
>>
>> But it isn't possible to have final fields when exporting within the
>> constructor.
>>
>> If the service contains effectively immutable field references, it must
>> be safely published or if another thread accesses those fields through
>> unsynchronized methods, the thread may only see their default value.
>>
>> All our service implementations have final fields. All our services had
>> some unsynchronized access to non volatile, non final fields.
>>
>> Trouble is, when you can't have final fields, it's very difficult to
>> determine which fields are effectively immutable and the original
>> programmers design intent, if there's no comment.
>>
>> Our service implementations all had unsynchronized init() methods called
>> from with the constructor that started threads and exported Some fields
>> were final.
>>
>> For me, a service that contains final fields and is exported after
>> construction is easier to maintain and understand.
>>
>> I suspect, like our Jini service imlementations, most people get it
>> wrong most of the time.
>>
>> I get that others won't want to use this method, however it would be
>> detrimental if we don't have such a method for those who would.
>>
>> Regards,
>>
>> Peter.
>>
>> ----- Original message -----
>>>
>>> I feel like we’re going down a rabbit hole here when you start talking
>>> about exporting immutable objects. Wouldn’t it be kind of silly to
>>> export an immutable service? Isn’t the whole point that you interact
>>> with the service (i.e. alter its state) over a remote interface?
>>>
>>> Perhaps it’s better to say that exported services need to be
>>> thread-safe (which should be fairly obvious). Yes, immutable objects
>>> are inherently thread-safe, so for sharing information inside a VM, it
>>> makes some sense to minimize the number of mutable objects you pass
>>> around. But fundamentally, we’re talking about shared-state systems
>>> here.
>>>
>>> Cheers,
>>>
>>> Greg.
>>>
>>> On Dec 18, 2013, at 7:42 AM, Peter <ji...@zeus.net.au> wrote:
>>>
>>>> Pat your comment about non final fields is interesting.
>>>>
>>>> Isn't it also the case that we need to safely publish an effectively
>>>> immutable object to share it among threads? That usually means
>>>> copying it to a thread safe collection or shared via a synchronized
>>>> method, volatile field, or final field in another object?
>>>>
>>>> So we should also make sure that Jeri uses safe publication during
>>>> export.
>>>>
>>>> That would allow a service that has no final fields to start threads,
>>>> then export from within a constructor safely, provided all operations
>>>> on non final fields happen before starting threads and exporting.
>>>>
>>>> All our services have final fields, so Starter is more appropriate
>>>> for River's own services.
>>>>
>>>> Regards,
>>>>
>>>> Peter.
>>>>
>>>> ----- Original message -----
>>>>> Hmm, good point, Startable, makes more sense.
>>>>>
>>>>> An object can be exported using Startable.
>>>>>
>>>>> I think we should have a policy to strongly discourage exporting
>>>>> from constructors.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Peter.
>>>>>
>>>>> ----- Original message -----
>>>>>> As far as I can tell, the special properties of completing a
>>>>>> constructor in the JLS memory model are:
>>>>>>
>>>>>> 1. A happens-before edge from the end of the constructor to the
>>>>>> start of a finalizer. (17.4.5)
>>>>>>
>>>>>> 2. The guarantee that any thread that only sees a reference to an
>>>>>> object after the end of the constructor will see the
>>>>>> correctly initialized values of all final fields. (17.5)
>>>>>>
>>>>>> The special issue with final fields is that implementations have
>>>>>> freedom to optimize access to final fields in ways that are
>>>>>> not permitted for non-final fields. Strategies for thread
>>>>>> safety that work for non-final fields do not necessarily
>>>>>> work for final fields. The requirement for final field
>>>>>> safety is that the constructor end before another thread
>>>>>> accesses the newly constructed object.
>>>>>>
>>>>>> Calling a start() method after construction if the class
>>>>>> implements a new interface seems to me to be harmless,
>>>>>> backwards compatible, and useful. It enables the simplest and
>>>>>> most direct way of preventing access to the new object by
>>>>>> another thread during construction.
>>>>>>
>>>>>> The roadmap issue is whether it should be required, and if so the
>>>>>> level of enforcement. For example, there is no reason to
>>>>>> require it if the class does not declare any final fields.
>>>>>>
>>>>>> Incidentally, and as a detail, "Commission" does not immediately
>>>>>> make me think of having a start() method that should be
>>>>>> called after construction. If you do go this way, the name
>>>>>> needs thought. "Startable" would be more obvious, more
>>>>>> memorable, more likely to be found on searches, and more
>>>>>> compatible with familiar interface names such as "Cloneable"
>>>>>> and "Iterable".
>>>>>>
>>>>>> Patricia
>>>>>>
>>>>>>
>>>>>> On 12/18/2013 2:18 AM, Peter wrote:
>>>>>>> Well, now seems like a good time to have the conversation.
>>>>>>>
>>>>>>> Yes there are other ways, but I haven't seen one safe
>>>>>>> implementation yet, so...
>>>>>>>
>>>>>>> Does someone have a better way to solve this problem, has
>>>>>>> someone already solved this problem I'm unaware of that we can
>>>>>>> adopt, or is there a way that's more satisfactory?
>>>>>>>
>>>>>>> If not, is there something objectionable with the Commission
>>>>>>> interface and if so, how can we fix it?
>>>>>>>
>>>>>>> The SEVERE log message is logged by the River start package,
>>>>>>> other containers or frameworks can choose whether or not to do
>>>>>>> so, but I'd encourage them to do something similar, yes we can
>>>>>>> change it to WARN.
>>>>>>>
>>>>>>> A much harsher option is to throw an exception during export
>>>>>>> which breaks backward compatibility.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Peter.
>>>>>>>
>>>>>>> ----- Original message -----
>>>>>>>> "org.apache.river.api.util.Commission is an interface
>>>>>>>> services should implement"
>>>>>>>>
>>>>>>>> If it's a SHOULD, not a MUST, chucking out a SEVERE is
>>>>>>>> incorrect logger behaviour IMO. You could issue a WARN if you
>>>>>>>> like but for even that I'd say you need to provide a roadmap
>>>>>>>> explaining why the warning and what you intend to do in
>>>>>>>> future and what you expect of service writers such as myself.
>>>>>>>>
>>>>>>>> Commission, at least from my point of view, is your means
>>>>>>>> (maybe the River community's - did you ask us?) for
>>>>>>>> satisfying your needs in respect of the JMM. As we've
>>>>>>>> discussed previously, there are other ways too and they work
>>>>>>>> and they are safe if you know what you're doing. Your
>>>>>>>> contention was that most don't know what they're doing
>>>>>>>> hence, presumably, Commission.
>>>>>>>>
>>>>>>>> So the thing is, you are seemingly on a road to asserting
>>>>>>>> more structure (gosh, a standard?) on the way people write
>>>>>>>> their services. If so, you'd best start flagging that
>>>>>>>> honestly and openly via a roadmap, deprecation and
>>>>>>>> such/whatever rather than sticking out logger messages with
>>>>>>>> no clear guidance and at the cost of a certain amount of
>>>>>>>> nuisance (no admin I know likes SEVERE's being logged for
>>>>>>>> something which isn't critical cos it's noise they don't
>>>>>>>> want in log files).
>>>>>>>>
>>>>>>>> And of course, we all know that when some entity asserts a
>>>>>>>> standard or requirement on others for entry, they may choose
>>>>>>>> not to enter. Does this help your community or hinder it? The
>>>>>>>> answer to that is, it depends. On what? Have you asked or
>>>>>>>> tested? How have you tested? What would be considered
>>>>>>>> validation or lack of support?
>>>>>>>>
>>>>>>>> I am not out to flame or troll rather I want to see this
>>>>>>>> community demonstrating good behaviour and I'm not feeling
>>>>>>>> like what's going on around Commission (what is that big
>>>>>>>> change in version number really saying?) is such.
>>>>>>>>
>>>>>>>> On 18 December 2013 08:52, Peter <ji...@zeus.net.au> wrote:
>>>>>>>>
>>>>>>>>> Just to clarify org.apache.river.api.util.Commission is an
>>>>>>>>> interface services should implement, I would encourage all
>>>>>>>>> container projects to pick up the interface and make
>>>>>>>>> suggestions for improvement if there are any issues.
>>>>>>>>>
>>>>>>>>> Interface Commission {
>>>>>>>>> void start () throws Exception;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> It's called after JMM safe construction to allow the
>>>>>>>>> service to start any threads and be exported.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Peter.
>>>>>>>>>
>>>>>>>>> ----- Original message -----
>>>>>>>>>>
>>>>>>>>>> The way that services are instantiated and setup is an
>>>>>>>>>> implementation detail. When I think of
>>>>>>>>>> compatibility I think of the API and the lookup methods.
>>>>>>>>>> We think of compatibility from a
>>>>>>>>>> client point of view. From the client
>>>>>>>>>> point of view, using a service looks like this:
>>>>>>>>>>
>>>>>>>>>> - Use multicast of unicast discovery to find one or more
>>>>>>>>>> ServiceRegistrar instances -
>>>>>>>>>> Call lookup(…) on one or more of these instances to get
>>>>>>>>>> a set of service candidates - Choose a candidate and
>>>>>>>>>> prepare() it using a ProxyPreparer, to yield a usable
>>>>>>>>>> service proxy. - Make calls on
>>>>>>>>>> it. Ideally hang on to this proxy
>>>>>>>>>> instance, so you can skip the discovery and lookup next
>>>>>>>>>> time you need it. - If the
>>>>>>>>>> call fails, repeat the lookup (and possibly discovery)
>>>>>>>>>> til you get a proxy that works.
>>>>>>>>>>
>>>>>>>>>> Nowhere does the client need to know whether the service
>>>>>>>>>> instance is started up using the “com.sun.jini.start”
>>>>>>>>>> mechanism, your Commission interface, some other IOC
>>>>>>>>>> container (Rio, Harvester, Seven or RiverContainer) or
>>>>>>>>>> some unknown mechanism that starts with a static main()
>>>>>>>>>> method.
>>>>>>>>>>
>>>>>>>>>> JSK2.0 was 2.0 because of the introduction of the proxy
>>>>>>>>>> verification mechanisms, as well as JERI.
>>>>>>>>>> Absent some new client usage mechanism, River doesn’t
>>>>>>>>>> need to go to 3.0.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Greg.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Dec 17, 2013, at 1:58 PM, Peter <ji...@zeus.net.au>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I think changing services to use safe construction
>>>>>>>>>>> techniques is enough to cause the version jump.
>>>>>>>>>>>
>>>>>>>>>>> At this point I've allowed services to continue unsafe
>>>>>>>>>>> construction practices, while logging a SEVERE warning
>>>>>>>>>>> when the Commission interface isn't implemented, rather
>>>>>>>>>>> than fail.
>>>>>>>>>>>
>>>>>>>>>>> This is a fundamental change to the way services are
>>>>>>>>>>> written.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>
>>>>>>>>>>> Peter.
>>>>>>>>>>>
>>>>>>>>>>> ----- Original message -----
>>>>>>>>>>>>
>>>>>>>>>>>> Assuming that there aren’t major incompatibilities, I
>>>>>>>>>>>> think that would be a “minor” version change
>>>>>>>>>>>> according to our versioning policy, so we’d be
>>>>>>>>>>>> looking at the “2.3” branch rather than a “3.0”
>>>>>>>>>>>> release.
>>>>>>>>>>>>
>>>>>>>>>>>> I’m still unnerved by the massive amounts of changes
>>>>>>>>>>>> to both code and tests in the qa_refactor branch, as
>>>>>>>>>>>> well as the apparent instability of the code,
>>>>>>>>>>>> although that seems to be improving.
>>>>>>>>>>>> In the next few weeks I’m going
>>>>>>>>>>>> to try and setup a cross-test case, to see what the
>>>>>>>>>>>> “2.2” tests say about the potential “2.3” release
>>>>>>>>>>>> and vice-versa.
>>>>>>>>>>>>
>>>>>>>>>>>> I think what I’d really like to see is an incremental
>>>>>>>>>>>> approach where we update limited components of the
>>>>>>>>>>>> “2.2” branch, one at a time. Is there anything that
>>>>>>>>>>>> we could pull out piecemeal? Maybe it
>>>>>>>>> would
>>>>>>>>>>>> make sense to split out the infrastructure services,
>>>>>>>>>>>> like Reggie, Mahalo, and Outrigger into different
>>>>>>>>>>>> sub-projects that could be updated separately?
>>>>>>>>>>>>
>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> Greg.
>>>>>>>>>>>>
>>>>>>>>>>>> On Dec 17, 2013, at 5:03 AM, Peter <ji...@zeus.net.au>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> When the qa_refactor branch stabilises, I plan to
>>>>>>>>>>>>> merge trunk and provide a beta release for client
>>>>>>>>>>>>> compatibility testing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Changes made have been focused on making our code
>>>>>>>>>>>>> thread safe, there are significant changes
>>>>>>>>>>>>> internally, the public api remains focused on
>>>>>>>>>>>>> backward compatibility, however it is advisable
>>>>>>>>>>>>> that client services adopt new safe construction
>>>>>>>>>>>>> techniques for services and implement the new
>>>>>>>>>>>>> Commission interface.
>>>>>>>>>>>>>
>>>>>>>>>>>>> What's a suitable test period for client testing?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Peter.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: VOTE: add Startable to Jini specification
Posted by Simon IJskes - QCG <si...@qcg.nl>.
On 19-12-13 01:42, Peter wrote:
> +1 Peter.
+1 Simon
--
QCG, Software voor het MKB, 071-5890970, http://www.qcg.nl
Quality Consultancy Group b.v., Leiderdorp, Kvk Den Haag: 28088397