You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Ken Giusti <kg...@redhat.com> on 2013/05/03 17:51:24 UTC
Re: QMFv2 object update bug [WAS Re: Questions from a novice]
Bill, Fraser,
Thanks for the well thought out replies.
I want to step back a bit.
Before I spend any more time on this problem, I want to know what the QPID project's plan is for QMFv1/v2 support in the C++ broker going forward.
If the consensus is that a 1.x C++ broker is going to support both QMFv1 and QMFv2, then simply reverting all the changes I've made to qmf.console and reverting to the old behavior is probably the least risky move.
On the other hand, if the consensus is to deprecate QMFv1 support prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on QMFv2 - then the final fix for this needs to take that plan into account.
So I'm going to call a discussion among the developers, proposing we vote on what QMF support a 1.x C++ broker will provide.
Once that's decided, we can determine the most appropriate fix.
As an aside - we've talked a lot about preserving behavior and functionality across 0.X releases in this thread. While nobody wants to gratuitously break stuff, this project has reserved the right to remove and change functionality between 0.x releases in order to fix bugs and provide new features. Things may be different once we get to 1.x, but for 0.x, this has been the case.
-K
----- Original Message -----
> From: "Bill Freeman" <ke...@gmail.com>
> To: "Qpid Dev" <de...@qpid.apache.org>
> Sent: Monday, April 29, 2013 2:53:29 PM
> Subject: Fwd: QMFv2 object update bug [WAS Re: Questions from a novice]
>
> I messed up and only sent this tor Fraser originally, but intended it for
> all.
>
> Bill
>
> ---------- Forwarded message ----------
> From: Bill Freeman <ke...@gmail.com>
> Date: Mon, Apr 29, 2013 at 11:18 AM
> Subject: Re: QMFv2 object update bug [WAS Re: Questions from a novice]
> To: Fraser Adams <fr...@blueyonder.co.uk>
>
>
>
>
>
> On Sat, Apr 27, 2013 at 5:45 AM, Fraser Adams <fraser.adams@blueyonder.co.uk
> > wrote:
>
> > On 26/04/13 17:18, Ken Giusti wrote:
> >
> >> Hey Bill,
> >>
> >>
> >>
> >> I first started to implement the additional objectUpdate callback as
> >> originally proposed. Very easy to do. But that additional api required
> >> other tools to be updated - and additional documentation changes, etc.
> >>
> > Hmm I remain a little baffled here. So as I understood it prior to any
> > fixing going on here there existed a bug that actually prevented QMF2
> > updates actually hitting callbacks on the asynchronous python API, that's
> > good IMHO because most clients of that API almost certainly were expecting
> > a single callback call for a given object.
> >
> > I'm kind of unclear about "other tools to be updated" 'cause I'd have
> > assumed given the previous QMF2 bug that no existing tools were actually
> > receiving v2 updates so wouldn't need "fixed". Sure they will need modified
> > to handle V2 if they want to consume both, but I'd argue that it might be
> > better to modify tools that need to handle both than to risk breaking
> > changes which will blat an indeterminate number of third party tools. It's
> > not ideal for sure, but a non-breaking API extension requiring an addition
> > to clients to use it feels better and TBH it's actually clearer IMHO
> > because there's some conscious choice being made.
> >
> >
> >
> >> After implementing this, I discovered a problem with the fix: tools that
> >> need to monitor both QMFv1 and QMFv2 agents starting reporting incorrect
> >> results (qpid-tool, to be specific). These tools need to support both the
> >> old callbacks (for QMFv1 updates) and the new one (for QMFv2 updates), in
> >> order to see both QMFv1 and QMFv2 agents.
> >>
> > To be honest as I suggest above IMHO I actually think it's a good thing
> > for tools that wish to support both to be explicit. I think from the
> > various sections of this thread it's pretty clear that trying to crowbar
> > everything together is a *bad idea*. Sure QMF1 and QMF2 are essentially
> > representing equivalent information but there are sufficient differences
> > and subtleties to flash a great big red light. The reason it's good to have
> > a separate objectUpdate is because the QMF2 API *is* different at that
> > point because the protocol is sending what amounts to incompatible data.
> >
> > At that point there are two choices 1) an additional non-breaking API
> > callback or 2) a shim in the client side that makes things behave
> > identically.
> >
> > The latter may be a bad choice if both v1 and v2 updates arrive because
> > it'd need to apply a filter for the case of v1 plus v2 but it'd probably be
> > a good choice if it could select either v1 or v2 because then existing
> > clients wouldn't need to be modified at all.
> >
> >
> As long as I'm including the whole context, let me say that I think that
> Fraser is completely correct here. To paraphrase from a perhaps different
> point of view: Existing tools should work as they did before any of these
> un-released fixes were made. Having the previously undelivered v2 updates
> arrive at a callback that the existing tools didn't know about, and and,
> thus, for which they can't possibly be providing handlers, seems to fulfill
> this bill of particulars.
>
> One possible addition to make the old and new more similar would be to add
> a keyword argument to enable the first fix, meaning that tools which don't
> know bout that fix won't be consuming additional network bandwidth and
> compute cycles because of it. (I'm not suggesting this, since I don't see
> the additional traffic and processing as onerous, but if there are enough
> queues in someone else's circumstance, it could be relevant.) The
> corresponding change, a keyword to inhibit the reception of v1 updates, is
> at least as useful, but is riskier, or at least would require more thorough
> testing, since I'm not certain that the console whether tool itself
> depends upon information that I only knows how to extract from a v1
> response.
>
> >
> >
> >
> >
> >> And that's when I realized that this fix was just a hack to work around
> >> the actual problem. The real issue at hand is that a QMF agent must not
> >> transmit both QMFv1 and QMFv2 updates for the same object.
> >>
> >> That's really what's broken here, and the C++ broker is doing precisely
> >> that.
> >>
> > Whoa there :-) I think that the problem is that this is *exactly* what it
> > should be doing :-)
> >
> > OK what I mean is how does it know? Bear in mind that the updates causing
> > the fun here are *unsolicited* data pushes! These updates are being pushed
> > to a topic, so unless any client is subscribing to
> > qmf.default.topic/data.ind.# or whatever it isn't going to be getting v2
> > updates, similarly if it's not subscribing to the equivalent on
> > qpid.management/ it's not going to be getting v1 updates. OTOH if it's
> > subscribing to both that's exactly what it's getting.
> >
> > The key thing is there's a fairly large level of decoupling going on here,
> > so I don't believe that there's an especially easy way for the broker
> > ManagementAgent to work out who's asking for what and thus avoid
> > transmitting both v1 and v2.
> >
> >
> > Funnily enough in the v2 protocol and API there's actually no concept of
> > an *unsolicited* data push, so I'd argue that what the broker is currently
> > doing for the v2 updates is an "undocumented feature" :-), what V2 actually
> > talks about is the concept of "query subscriptions". With query
> > subscriptions the asynchronous pushes are actually *solicited* by the
> > client actually requesting some specified subset of the data, so the
> > Management Agent does actually have a mechanism to track what data any
> > given subscribers are actually interested in.
> >
>
> From the practical use case perspective, eliminating updates for queues
> which are not of interest, such as those involved in the implementation of
> QMF itself, sounds desirable (again, bandwidth and processing). On the
> other hand, I would have had to enumerate the queues I care about. Right
> now I'm using the cheap cop out that the names of all of "out" queues begin
> with a particular three character sequence.
>
> >
> > That's clearly a moot point, but it would help with this pickle.
> >
> >
> > I was musing over this problem and I believe that there is an option,
> > albeit a non-trivial one :-(
> >
> > So the *real* issue is that it's difficult to manage the carnage caused
> > when both v1 and v2 updates happen. You've previously suggested disabling
> > v1 by default, I've sort of got sympathy with that :-) However there are
> > two (I suspect significant) use cases that will be impacted by that choice:
> > 1) Any asynchronous console that relies on the existing v1 callbacks will
> > stop working - I suspect that in practice that'll mean every single darned
> > one of them except (at a push) qpid-tool. There may be a lot of third party
> > tools using this API so it may be an issue. At the very least there needs
> > to be some real good communication to the user list.
> > 2) If you disable v1 then any third party Agents will break too. Again
> > perhaps they need to change to v2 and this is a catalyst, but it just may
> > cause non-trivial business impact on users.
> >
> > Ultimately I think that it's very worth pushing out a survey on the user
> > list to gauge the potential impact.
> >
>
> I'd like to offer a caution about expecting this mailing list to inform
> users that a minor update (e.g.; 0.10 to 0.10.1 or even 0.11, as opposed
> to, for example, 0.10 to 1.0) will require using organizations to update
> custom tools. I essentially had my code working (with v1) before ever
> subscribing here (to learn what promises there might be about string
> representations of object IDs uniqueness). When this contract ends, I
> maybe back to doing Django, or *nix kernel internals, or even hardware
> development, and may not be able to justify the time to continue to read
> here. I'm sure that my circumstance isn't that unusual. So the client may
> no longer have someone following the list (let alone having invested the
> time to know what will impact them).
>
>
> > The non-trivial mitigation that I was musing over was whether there was a
> > possibility of constructing a v1->v2 bridge Agent? What I mean is that it
> > might be possible to get the broker ManagementAgent to only send v2 updates
> > if there's a mechanism for the broker to intercept v1 updates from third
> > party Agents and map those into v2 too, thus making every data push
> > consistently V2? I *think* this could work because I believe that v1
> > requires broker involvement in order to work (I think that was one of the
> > reasons for moving to v2).
> >
> > If it were possible to get to a clear point of only sending v2 updates
> > then it should be possible to have the asynchronous Console API shimmed in
> > such a way that it behaves *exactly* the same way with v2 updates as it did
> > with v1 updates so it's possible to mitigate the impact on third party
> > console applications (yay!).
> >
> > It has to be said It's be much better if users didn't have to rewrite
> > applications because of objectUpdate versus propUpdate/statUpdate but
> > clearly a shim would be required to make v2 updates behave correctly and
> > transparently for propUpdate/statUpdate callbacks.
> >
> > The result of the survey question "how many people have custom v1 Agents"
> > would help determine whether it's worth investing in a v1->v2 bridge Agent.
> >
> > A bridge ought to be possible, that's actually exactly what I wrote for
> > the Java Broker in the Qmf2ManagementAgent to get that to talk QMF2 (albeit
> > I was mapping from its internal model and not QMF1, but the principal is
> > the same - just don't ask me to do it in C++ though :-))
> >
> >
> >
> >
> >> Why is that a problem? Because a console receiving these two updates
> >> cannot tell that they are for the same object. To the console
> >> application,
> >> it appears as two separate objects.
> >>
> > As I mention above I don't believe that it's any easier for the broker to
> > do a "diff" given the unsolicited nature of these updates :-)
> >
> > I'm proposing that we should solve this problem by disabling QMFv1
> >> updates coming from the C++ broker. This should be the default for the
> >> next release (0.26). It should still be possible to turn them on manually
> >> if desired, but the C++ broker should only transmit QMFv2 type updates
> >> going forward.
> >>
> > As I say above I've got a fair amount of sympathy with this, and
> > parochially if it means that by doing this you can make statUpdate and
> > propUpdate for v2 behave *exactly* the same as for v1 so clients don't need
> > to be rewritten for v2 then it'd be better for me 'cause I only ever use v2
> > aside from this.
> >
> > However as I said above the only reason that this Console API ever
> > supported both was to cater for the case where a user needed to handle both
> > v1 and v2 updates and that use case is actually third party v1 Agents.
> >
> > I think it's important to at least check for how much impact this is going
> > to cause users.
> >
> >
> >
> >
> >> We can minimize the impact to console applications by having the
> >> objectProps and objectStats callbacks be invoked when a QMFv2 object
> >> arrives instead of introducing a new callback. [Bill - I'll fix the
> >> console
> >> to not call objectProps when only stats are present]. I think this should
> >> eliminate the need to re-code console applications.
> >>
> > So I don't think it's about "minimising the impact" I think that the API
> > needs to behave *exactly* the same for this to be any use. Basically any
> > asynchronous client hit by only v2 updates should behave identically to how
> > they behaved prior to this update or it's still introducing a regression
> > into an indeterminate number of applications. I've no idea how hard it'd be
> > to achieve this nirvana though.
> >
> >
> >
> >> This will cause problems for folks that use an old python console against
> >> the next release of the broker. These folks will have to upgrade the
> >> console as well.
> >>
> >> Does anyone have a better alternative?
> >>
> > Well I'm not necessarily saying that my suggestions above are *better* :-)
> > It's difficult for sure. I guess that my biggest concern is introducing
> > breaking changes against an unknown number of third party Consoles and
> > Agents.
> >
> > One other thing makes me nervous. I guess that there's the scenario where
> > users are working with a mixed economy of brokers at different versions or
> > who (like Bill) may be unable to alter the broker config to only push out a
> > particular version.
> >
> > All brokers from at least 0.8 can push out v2 so it *might* be safe to
> > have the console only subscribe to v2 updates (providing a switch to
> > specify v1). If you can make propUpdate and statUpdate behave consistently
> > then that only leaves the case where there are v1 Agents to support, which
> > is where the bridge Agent may come in.
> >
> > How much do you regret starting on this fix now Ken? :-D
> >
> >
> > It's worth pointing out that this is I suspect only the tip of the iceberg
> > for the sort of problems that might start to be encountered when migration
> > to AMQP 1.0 Management starts to gain traction. My strong hunch is that
> > bridge Agents/ManagementNodes are going to become pretty important friends
> > when it comes to trying to manage any transition. It's something that's
> > going to bite sooner or later and the C++, Java & Python communities are
> > going to have to be pretty joined up. I quite like QMF, but clearly some
> > mistakes were made along the way given the lack of support in the Java
> > community and frankly the fragmentation of APIs and lack of take up on the
> > "official" QMF2 API. We *really* need to try harder next time round and act
> > as an exemplar in the AMQP community.
> >
> > Frase
> >
> > Also, Ken previously mentioned, but Fraser didn't quote, that v1 and v2
> object identifiers are unrelated. I would like to point out that it is
> possible to synthesize the string representation of a v2 objectId using the
> information in a v1 objectProps() call. (After adding the first patch, and
> figuring out how to make things work again I was doing this to conjoin
> information from the v1 and v2 updates for a single queue.) The
> correspondences needed probably fall into the undocumented and subject to
> change category, but I suspect inertial will keep them from changing before
> 1.0.
>
> Bill
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re:
Questions from a novice])
Posted by Bill Freeman <ke...@gmail.com>.
Just a few thoughts:
Since it appears to be the only asynchronous python QMF client, my own
project needs the qmf.console package to continue to work.
I presume that those tools, which Ken mentions as being based on QMFv2 now,
do not use this package.
Now that (the patched version of) it is actually capable of getting QMFv2
updates, I'm happy to use that (and have been). I am concerned, however,
that some of its internals (e.g.; fetching a schema) might depend on
QMFv1. They certainly worked before the patch that enabled v2, so they
must have been using v1. Thus my question is whether, if QMFv1 suddenly
went away, these internal operations would seamlessly switch to using v2,
or would they just break?
What I'd like to see is for someone who knows what they are doing* to patch
qmf/console.py (probably in that same _TryToConnect() method) so that it
does not establish any v1 connections at all, confirm that they think that
it still works, and pass me a copy to test in my project. If all that goes
well, or if the library used by all the tools grows asynchronous capability
(and I learn to use it), then, and only then, would I be comfortable with
the disappearance of any QMFv1 capabilities.
[* That is, not me. As has been pointed out here recently, it uses the man
behind the curtain sections of the Qpid python library, not the giiant face
on the screen "messaging" interface.]
I don't know whether anyone besides me (and the QMF python tutorial I
started with) uses qmf/console.py . I haven't noticed anyone speaking up
during our discussions, so I may be the only one affected. But there could
well be users not reading the list. While they may not have been promised
compatibility across dot releases, it seems a shame to break their code
unless there are real savings to be had. The v1 code exists. Unless other
changes require it to be maintained, or unless it represents a significant
performance drag, why go to the effort of removing it, at least before AMPQ
1.0?
Bill
On Mon, May 6, 2013 at 8:54 AM, Ken Giusti <kg...@redhat.com> wrote:
> Folks,
>
> My overall concern is that this project doesn't have the bandwidth to
> adequately support two different management implementations. That's really
> the crux of the issue for me.
>
> The current problems we've been dealing with come down to the fact that
> there's inadequate test coverage for dual use QMFv1 and QMFv2. The latest
> problem (QMFv2 updates from the broker being dropped by qmf.console) has
> been broken for a long time. We shipped this broken for several releases.
> Why? Inadequate test coverage.
>
> Most - if not all - of the python tests use QMFv1. Yet qpid-tools are now
> (mostly) based on QMFv2. I'm not even sure to what extent the QMF agent
> libraries are tested - perhaps not at all for the cmake builds. This
> results in partial test coverage of both implementations - neither
> implementation gets fully tested.
>
> In the end, we try to support both, but are not able to guarantee the
> level of quality our users deserve for either.
>
> We should be concentrating our efforts on one implementation, and making
> it the best we can.
>
> -K
>
> ----- Original Message -----
> > From: "Gordon Sim" <gs...@redhat.com>
> > To: users@qpid.apache.org
> > Cc: dev@qpid.apache.org
> > Sent: Monday, May 6, 2013 7:11:18 AM
> > Subject: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
> Re: Questions from a novice])
> >
> > On 05/03/2013 04:51 PM, Ken Giusti wrote:
> > > I want to know what the
> > > QPID project's plan is for QMFv1/v2 support in the C++ broker going
> > > forward.
> > >
> > > If the consensus is that a 1.x C++ broker is going to support both
> > > QMFv1 and QMFv2, then simply reverting all the changes I've made to
> > > qmf.console and reverting to the old behavior is probably the least
> > > risky move.
> > >
> > > On the other hand, if the consensus is to deprecate QMFv1 support
> > > prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> > > QMFv2 - then the final fix for this needs to take that plan into
> > > account.
> > >
> > > So I'm going to call a discussion among the developers, proposing we
> > > vote on what QMF support a 1.x C++ broker will provide.
> >
> > I'd like to move the discussion to the user list as there are folks
> > there who may not be on dev list but would be impacted or might
> > otherwise have an opinion. I'm assuming everyone on the dev list is also
> > on the user list (if not you really should be :-)
> >
> > By way of a short introduction, Ken's question comes after a long thread
> > focused on some issues around the dual support of v1 and v2 of the QMF
> > protocol in the console.py python library for applications built to
> > manage/monitor the c++ broker.
> >
> > > Once that's decided, we can determine the most appropriate fix.
> > >
> > > As an aside - we've talked a lot about preserving behavior and
> > > functionality across 0.X releases in this thread. While nobody wants
> > > to gratuitously break stuff, this project has reserved the right to
> > > remove and change functionality between 0.x releases in order to fix
> > > bugs and provide new features. Things may be different once we get
> > > to 1.x, but for 0.x, this has been the case.
> >
> > For my part, I am in favour of removing QMFv1 support, if necessary
> > providing some sort of adapter or plugin to deal with any use cases for
> > which this might prove problematic. I think QMFv2 is now firmly
> > established as the current management mechanism for the c++ broker.
> >
> > Ultimately I also think we should be starting to think about an AMQP 1.0
> > based management standard and how we would transition to that. That is
> > however a larger question that shouldn't hold up the immediate decision
> > regarding QMFv1 though it may inform it a little.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
> >
>
> --
> -K
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
Re: Questions from a novice])
Posted by Ken Giusti <kg...@redhat.com>.
Folks,
My overall concern is that this project doesn't have the bandwidth to adequately support two different management implementations. That's really the crux of the issue for me.
The current problems we've been dealing with come down to the fact that there's inadequate test coverage for dual use QMFv1 and QMFv2. The latest problem (QMFv2 updates from the broker being dropped by qmf.console) has been broken for a long time. We shipped this broken for several releases. Why? Inadequate test coverage.
Most - if not all - of the python tests use QMFv1. Yet qpid-tools are now (mostly) based on QMFv2. I'm not even sure to what extent the QMF agent libraries are tested - perhaps not at all for the cmake builds. This results in partial test coverage of both implementations - neither implementation gets fully tested.
In the end, we try to support both, but are not able to guarantee the level of quality our users deserve for either.
We should be concentrating our efforts on one implementation, and making it the best we can.
-K
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: dev@qpid.apache.org
> Sent: Monday, May 6, 2013 7:11:18 AM
> Subject: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re: Questions from a novice])
>
> On 05/03/2013 04:51 PM, Ken Giusti wrote:
> > I want to know what the
> > QPID project's plan is for QMFv1/v2 support in the C++ broker going
> > forward.
> >
> > If the consensus is that a 1.x C++ broker is going to support both
> > QMFv1 and QMFv2, then simply reverting all the changes I've made to
> > qmf.console and reverting to the old behavior is probably the least
> > risky move.
> >
> > On the other hand, if the consensus is to deprecate QMFv1 support
> > prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> > QMFv2 - then the final fix for this needs to take that plan into
> > account.
> >
> > So I'm going to call a discussion among the developers, proposing we
> > vote on what QMF support a 1.x C++ broker will provide.
>
> I'd like to move the discussion to the user list as there are folks
> there who may not be on dev list but would be impacted or might
> otherwise have an opinion. I'm assuming everyone on the dev list is also
> on the user list (if not you really should be :-)
>
> By way of a short introduction, Ken's question comes after a long thread
> focused on some issues around the dual support of v1 and v2 of the QMF
> protocol in the console.py python library for applications built to
> manage/monitor the c++ broker.
>
> > Once that's decided, we can determine the most appropriate fix.
> >
> > As an aside - we've talked a lot about preserving behavior and
> > functionality across 0.X releases in this thread. While nobody wants
> > to gratuitously break stuff, this project has reserved the right to
> > remove and change functionality between 0.x releases in order to fix
> > bugs and provide new features. Things may be different once we get
> > to 1.x, but for 0.x, this has been the case.
>
> For my part, I am in favour of removing QMFv1 support, if necessary
> providing some sort of adapter or plugin to deal with any use cases for
> which this might prove problematic. I think QMFv2 is now firmly
> established as the current management mechanism for the c++ broker.
>
> Ultimately I also think we should be starting to think about an AMQP 1.0
> based management standard and how we would transition to that. That is
> however a larger question that shouldn't hold up the immediate decision
> regarding QMFv1 though it may inform it a little.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
Re: Questions from a novice])
Posted by Ken Giusti <kg...@redhat.com>.
Folks,
My overall concern is that this project doesn't have the bandwidth to adequately support two different management implementations. That's really the crux of the issue for me.
The current problems we've been dealing with come down to the fact that there's inadequate test coverage for dual use QMFv1 and QMFv2. The latest problem (QMFv2 updates from the broker being dropped by qmf.console) has been broken for a long time. We shipped this broken for several releases. Why? Inadequate test coverage.
Most - if not all - of the python tests use QMFv1. Yet qpid-tools are now (mostly) based on QMFv2. I'm not even sure to what extent the QMF agent libraries are tested - perhaps not at all for the cmake builds. This results in partial test coverage of both implementations - neither implementation gets fully tested.
In the end, we try to support both, but are not able to guarantee the level of quality our users deserve for either.
We should be concentrating our efforts on one implementation, and making it the best we can.
-K
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: dev@qpid.apache.org
> Sent: Monday, May 6, 2013 7:11:18 AM
> Subject: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re: Questions from a novice])
>
> On 05/03/2013 04:51 PM, Ken Giusti wrote:
> > I want to know what the
> > QPID project's plan is for QMFv1/v2 support in the C++ broker going
> > forward.
> >
> > If the consensus is that a 1.x C++ broker is going to support both
> > QMFv1 and QMFv2, then simply reverting all the changes I've made to
> > qmf.console and reverting to the old behavior is probably the least
> > risky move.
> >
> > On the other hand, if the consensus is to deprecate QMFv1 support
> > prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> > QMFv2 - then the final fix for this needs to take that plan into
> > account.
> >
> > So I'm going to call a discussion among the developers, proposing we
> > vote on what QMF support a 1.x C++ broker will provide.
>
> I'd like to move the discussion to the user list as there are folks
> there who may not be on dev list but would be impacted or might
> otherwise have an opinion. I'm assuming everyone on the dev list is also
> on the user list (if not you really should be :-)
>
> By way of a short introduction, Ken's question comes after a long thread
> focused on some issues around the dual support of v1 and v2 of the QMF
> protocol in the console.py python library for applications built to
> manage/monitor the c++ broker.
>
> > Once that's decided, we can determine the most appropriate fix.
> >
> > As an aside - we've talked a lot about preserving behavior and
> > functionality across 0.X releases in this thread. While nobody wants
> > to gratuitously break stuff, this project has reserved the right to
> > remove and change functionality between 0.x releases in order to fix
> > bugs and provide new features. Things may be different once we get
> > to 1.x, but for 0.x, this has been the case.
>
> For my part, I am in favour of removing QMFv1 support, if necessary
> providing some sort of adapter or plugin to deal with any use cases for
> which this might prove problematic. I think QMFv2 is now firmly
> established as the current management mechanism for the c++ broker.
>
> Ultimately I also think we should be starting to think about an AMQP 1.0
> based management standard and how we would transition to that. That is
> however a larger question that shouldn't hold up the immediate decision
> regarding QMFv1 though it may inform it a little.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re:
Questions from a novice])
Posted by Rob Godfrey <ro...@gmail.com>.
So, the initial thinking for AMQP Management has (finally) now been
uploaded so that anyone can see it [1]. As Fraser mentioned, the scope of
the AMQP Management work at OASIS is currently purely about "mechanism" and
not defining specific operations / attributes that are available on a
managed object.
-- Rob
[1] https://lists.oasis-open.org/archives/amqp/201305/msg00005.html
On 7 May 2013 20:18, Fraser Adams <fr...@blueyonder.co.uk> wrote:
> On 06/05/13 12:11, Gordon Sim wrote:
>
>>
>> For my part, I am in favour of removing QMFv1 support, if necessary
>> providing some sort of adapter or plugin to deal with any use cases for
>> which this might prove problematic. I think QMFv2 is now firmly established
>> as the current management mechanism for the c++ broker.
>>
> My take on this is that I'm more than happy to see the back of QMF1 but
> I'd be keen to understand whether or not this results in *breaking changes*.
>
> A couple of things I mentioned on the dev group around this point are
> a) The asynchronous python API - that's where this discussion started the
> problem is that the callbacks that have actually worked to date work for
> QMF1, there was a suggestion of an additional objectUpdate() call for QMF2
> which is OK as it wouldn't break existing clients as long as QMF1 data was
> still pushed, but would if it was disabled. Forcing clients to change from
> statUpdate() and propUpdate() to objectUpdate() (and any changes regression
> testing that this involves) is clearly a breaking change. Parochially I've
> only got one tool that uses the async API but I'd prefer it not to be
> broken (mainly because I wrote it at home for a work use and I'm not
> responsible for supporting it and I'd end up having to explain the
> subtleties we're discussing here to someone with a lot less QMF experience
> than me :'( ). At a push as long as I have decent notice I can rewrite it
> myself, but others may not have that luxury - as Bill has mentioned
> previously he's a contractor so when he finishes up his contract and
> someone else takes over his work they won't have this background and it
> might "silently" stop working properly.
> b) If anyone has been writing Agents using QMF1 then clearly they are
> likely to break. Parochially this doesn't affect me, but I'm not sure quite
> how we can gauge how much it may affect others.
>
> Perhaps rather like the automake to cmake change we start off by giving
> gentle nudges then get a bit more forceful, so disabling the QMF1 broker
> updates as Ken has suggested might be a start (along with some decent
> documentation and a deprecation notice in the qpidd -h text).
>
> I'd personally quite like to see the python async API shim the QMF2 update
> to make it behave like statUpdate()/propUpdate() though I accept this might
> be difficult and would have to bow to people who know more about the
> implementation of this than me.
>
>
>
>> Ultimately I also think we should be starting to think about an AMQP 1.0
>> based management standard and how we would transition to that. That is
>> however a larger question that shouldn't hold up the immediate decision
>> regarding QMFv1 though it may inform it a little.
>>
>> Yeah I totally agree, I hope that it's not too long before Rob is able
> to share the draft AMQP 1.0 Management Spec. It'd be good to have a "lively
> discussion" about this among a wider group. Rob and I had a couple of
> decent chats over Easter but I'm keen to start some proper thinking on how
> we can make this a reality. Unfortunately I suspect that the QMF1/QMF2
> quirks are likely to be minor compared to moving to AMQP 1.0 Management,
> but hopefully not insurmountable, I'm hopeful that it should be possible to
> shim between ManagementAgent and ManagementNode and vice versa which should
> help with migration. As we've discussed a pluggable Management layer for
> the C++ broker would be good, it really helped in the Java broker - I was
> able to get QMF2 support up and working in a couple of weekends with no
> prior knowledge of the Java broker code base.
>
> <rant>
>
> As an aside the AMQP 1.0 Management specification is only covering the
> protocol as I understand it, but I'd be quite keen for us to get our act
> together with respect to APIs. Yes I know it's possible to write QMF2 using
> the protocol directly, but frankly it's not as interoperable as it might be
> (C++ binary strings versus UTF8 strings are a personal "favourite" of mine
> resulting in byte[] or String respectively in Java - ClassCastException
> hell :-)). So I think it's useful to have a decent API, however the four or
> five APIs that we seem to have for QMF is less helpful :-D - can we agree
> on one when we start looking at AMQP 1.0 Management?
>
> </rant> :-)
>
> Cheers,
> Frase
>
>
>
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.**org<de...@qpid.apache.org>
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
Re: Questions from a novice])
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
On 06/05/13 12:11, Gordon Sim wrote:
>
> For my part, I am in favour of removing QMFv1 support, if necessary
> providing some sort of adapter or plugin to deal with any use cases
> for which this might prove problematic. I think QMFv2 is now firmly
> established as the current management mechanism for the c++ broker.
My take on this is that I'm more than happy to see the back of QMF1 but
I'd be keen to understand whether or not this results in *breaking changes*.
A couple of things I mentioned on the dev group around this point are
a) The asynchronous python API - that's where this discussion started
the problem is that the callbacks that have actually worked to date work
for QMF1, there was a suggestion of an additional objectUpdate() call
for QMF2 which is OK as it wouldn't break existing clients as long as
QMF1 data was still pushed, but would if it was disabled. Forcing
clients to change from statUpdate() and propUpdate() to objectUpdate()
(and any changes regression testing that this involves) is clearly a
breaking change. Parochially I've only got one tool that uses the async
API but I'd prefer it not to be broken (mainly because I wrote it at
home for a work use and I'm not responsible for supporting it and I'd
end up having to explain the subtleties we're discussing here to someone
with a lot less QMF experience than me :'( ). At a push as long as I
have decent notice I can rewrite it myself, but others may not have that
luxury - as Bill has mentioned previously he's a contractor so when he
finishes up his contract and someone else takes over his work they won't
have this background and it might "silently" stop working properly.
b) If anyone has been writing Agents using QMF1 then clearly they are
likely to break. Parochially this doesn't affect me, but I'm not sure
quite how we can gauge how much it may affect others.
Perhaps rather like the automake to cmake change we start off by giving
gentle nudges then get a bit more forceful, so disabling the QMF1 broker
updates as Ken has suggested might be a start (along with some decent
documentation and a deprecation notice in the qpidd -h text).
I'd personally quite like to see the python async API shim the QMF2
update to make it behave like statUpdate()/propUpdate() though I accept
this might be difficult and would have to bow to people who know more
about the implementation of this than me.
>
> Ultimately I also think we should be starting to think about an AMQP
> 1.0 based management standard and how we would transition to that.
> That is however a larger question that shouldn't hold up the immediate
> decision regarding QMFv1 though it may inform it a little.
>
Yeah I totally agree, I hope that it's not too long before Rob is able
to share the draft AMQP 1.0 Management Spec. It'd be good to have a
"lively discussion" about this among a wider group. Rob and I had a
couple of decent chats over Easter but I'm keen to start some proper
thinking on how we can make this a reality. Unfortunately I suspect that
the QMF1/QMF2 quirks are likely to be minor compared to moving to AMQP
1.0 Management, but hopefully not insurmountable, I'm hopeful that it
should be possible to shim between ManagementAgent and ManagementNode
and vice versa which should help with migration. As we've discussed a
pluggable Management layer for the C++ broker would be good, it really
helped in the Java broker - I was able to get QMF2 support up and
working in a couple of weekends with no prior knowledge of the Java
broker code base.
<rant>
As an aside the AMQP 1.0 Management specification is only covering the
protocol as I understand it, but I'd be quite keen for us to get our act
together with respect to APIs. Yes I know it's possible to write QMF2
using the protocol directly, but frankly it's not as interoperable as it
might be (C++ binary strings versus UTF8 strings are a personal
"favourite" of mine resulting in byte[] or String respectively in Java -
ClassCastException hell :-)). So I think it's useful to have a decent
API, however the four or five APIs that we seem to have for QMF is less
helpful :-D - can we agree on one when we start looking at AMQP 1.0
Management?
</rant> :-)
Cheers,
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re:
Questions from a novice])
Posted by Gordon Sim <gs...@redhat.com>.
On 05/03/2013 04:51 PM, Ken Giusti wrote:
> I want to know what the
> QPID project's plan is for QMFv1/v2 support in the C++ broker going
> forward.
>
> If the consensus is that a 1.x C++ broker is going to support both
> QMFv1 and QMFv2, then simply reverting all the changes I've made to
> qmf.console and reverting to the old behavior is probably the least
> risky move.
>
> On the other hand, if the consensus is to deprecate QMFv1 support
> prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> QMFv2 - then the final fix for this needs to take that plan into
> account.
>
> So I'm going to call a discussion among the developers, proposing we
> vote on what QMF support a 1.x C++ broker will provide.
I'd like to move the discussion to the user list as there are folks
there who may not be on dev list but would be impacted or might
otherwise have an opinion. I'm assuming everyone on the dev list is also
on the user list (if not you really should be :-)
By way of a short introduction, Ken's question comes after a long thread
focused on some issues around the dual support of v1 and v2 of the QMF
protocol in the console.py python library for applications built to
manage/monitor the c++ broker.
> Once that's decided, we can determine the most appropriate fix.
>
> As an aside - we've talked a lot about preserving behavior and
> functionality across 0.X releases in this thread. While nobody wants
> to gratuitously break stuff, this project has reserved the right to
> remove and change functionality between 0.x releases in order to fix
> bugs and provide new features. Things may be different once we get
> to 1.x, but for 0.x, this has been the case.
For my part, I am in favour of removing QMFv1 support, if necessary
providing some sort of adapter or plugin to deal with any use cases for
which this might prove problematic. I think QMFv2 is now firmly
established as the current management mechanism for the c++ broker.
Ultimately I also think we should be starting to think about an AMQP 1.0
based management standard and how we would transition to that. That is
however a larger question that shouldn't hold up the immediate decision
regarding QMFv1 though it may inform it a little.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re:
Questions from a novice])
Posted by Gordon Sim <gs...@redhat.com>.
On 05/03/2013 04:51 PM, Ken Giusti wrote:
> I want to know what the
> QPID project's plan is for QMFv1/v2 support in the C++ broker going
> forward.
>
> If the consensus is that a 1.x C++ broker is going to support both
> QMFv1 and QMFv2, then simply reverting all the changes I've made to
> qmf.console and reverting to the old behavior is probably the least
> risky move.
>
> On the other hand, if the consensus is to deprecate QMFv1 support
> prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> QMFv2 - then the final fix for this needs to take that plan into
> account.
>
> So I'm going to call a discussion among the developers, proposing we
> vote on what QMF support a 1.x C++ broker will provide.
I'd like to move the discussion to the user list as there are folks
there who may not be on dev list but would be impacted or might
otherwise have an opinion. I'm assuming everyone on the dev list is also
on the user list (if not you really should be :-)
By way of a short introduction, Ken's question comes after a long thread
focused on some issues around the dual support of v1 and v2 of the QMF
protocol in the console.py python library for applications built to
manage/monitor the c++ broker.
> Once that's decided, we can determine the most appropriate fix.
>
> As an aside - we've talked a lot about preserving behavior and
> functionality across 0.X releases in this thread. While nobody wants
> to gratuitously break stuff, this project has reserved the right to
> remove and change functionality between 0.x releases in order to fix
> bugs and provide new features. Things may be different once we get
> to 1.x, but for 0.x, this has been the case.
For my part, I am in favour of removing QMFv1 support, if necessary
providing some sort of adapter or plugin to deal with any use cases for
which this might prove problematic. I think QMFv2 is now firmly
established as the current management mechanism for the c++ broker.
Ultimately I also think we should be starting to think about an AMQP 1.0
based management standard and how we would transition to that. That is
however a larger question that shouldn't hold up the immediate decision
regarding QMFv1 though it may inform it a little.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org