You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Gordon Sim <gs...@redhat.com> on 2013/05/06 13:11:18 UTC
Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re:
Questions from a novice])
On 05/03/2013 04:51 PM, Ken Giusti wrote:
> I want to know what the
> QPID project's plan is for QMFv1/v2 support in the C++ broker going
> forward.
>
> If the consensus is that a 1.x C++ broker is going to support both
> QMFv1 and QMFv2, then simply reverting all the changes I've made to
> qmf.console and reverting to the old behavior is probably the least
> risky move.
>
> On the other hand, if the consensus is to deprecate QMFv1 support
> prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> QMFv2 - then the final fix for this needs to take that plan into
> account.
>
> So I'm going to call a discussion among the developers, proposing we
> vote on what QMF support a 1.x C++ broker will provide.
I'd like to move the discussion to the user list as there are folks
there who may not be on dev list but would be impacted or might
otherwise have an opinion. I'm assuming everyone on the dev list is also
on the user list (if not you really should be :-)
By way of a short introduction, Ken's question comes after a long thread
focused on some issues around the dual support of v1 and v2 of the QMF
protocol in the console.py python library for applications built to
manage/monitor the c++ broker.
> Once that's decided, we can determine the most appropriate fix.
>
> As an aside - we've talked a lot about preserving behavior and
> functionality across 0.X releases in this thread. While nobody wants
> to gratuitously break stuff, this project has reserved the right to
> remove and change functionality between 0.x releases in order to fix
> bugs and provide new features. Things may be different once we get
> to 1.x, but for 0.x, this has been the case.
For my part, I am in favour of removing QMFv1 support, if necessary
providing some sort of adapter or plugin to deal with any use cases for
which this might prove problematic. I think QMFv2 is now firmly
established as the current management mechanism for the c++ broker.
Ultimately I also think we should be starting to think about an AMQP 1.0
based management standard and how we would transition to that. That is
however a larger question that shouldn't hold up the immediate decision
regarding QMFv1 though it may inform it a little.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re:
Questions from a novice])
Posted by Bill Freeman <ke...@gmail.com>.
Just a few thoughts:
Since it appears to be the only asynchronous python QMF client, my own
project needs the qmf.console package to continue to work.
I presume that those tools, which Ken mentions as being based on QMFv2 now,
do not use this package.
Now that (the patched version of) it is actually capable of getting QMFv2
updates, I'm happy to use that (and have been). I am concerned, however,
that some of its internals (e.g.; fetching a schema) might depend on
QMFv1. They certainly worked before the patch that enabled v2, so they
must have been using v1. Thus my question is whether, if QMFv1 suddenly
went away, these internal operations would seamlessly switch to using v2,
or would they just break?
What I'd like to see is for someone who knows what they are doing* to patch
qmf/console.py (probably in that same _TryToConnect() method) so that it
does not establish any v1 connections at all, confirm that they think that
it still works, and pass me a copy to test in my project. If all that goes
well, or if the library used by all the tools grows asynchronous capability
(and I learn to use it), then, and only then, would I be comfortable with
the disappearance of any QMFv1 capabilities.
[* That is, not me. As has been pointed out here recently, it uses the man
behind the curtain sections of the Qpid python library, not the giiant face
on the screen "messaging" interface.]
I don't know whether anyone besides me (and the QMF python tutorial I
started with) uses qmf/console.py . I haven't noticed anyone speaking up
during our discussions, so I may be the only one affected. But there could
well be users not reading the list. While they may not have been promised
compatibility across dot releases, it seems a shame to break their code
unless there are real savings to be had. The v1 code exists. Unless other
changes require it to be maintained, or unless it represents a significant
performance drag, why go to the effort of removing it, at least before AMPQ
1.0?
Bill
On Mon, May 6, 2013 at 8:54 AM, Ken Giusti <kg...@redhat.com> wrote:
> Folks,
>
> My overall concern is that this project doesn't have the bandwidth to
> adequately support two different management implementations. That's really
> the crux of the issue for me.
>
> The current problems we've been dealing with come down to the fact that
> there's inadequate test coverage for dual use QMFv1 and QMFv2. The latest
> problem (QMFv2 updates from the broker being dropped by qmf.console) has
> been broken for a long time. We shipped this broken for several releases.
> Why? Inadequate test coverage.
>
> Most - if not all - of the python tests use QMFv1. Yet qpid-tools are now
> (mostly) based on QMFv2. I'm not even sure to what extent the QMF agent
> libraries are tested - perhaps not at all for the cmake builds. This
> results in partial test coverage of both implementations - neither
> implementation gets fully tested.
>
> In the end, we try to support both, but are not able to guarantee the
> level of quality our users deserve for either.
>
> We should be concentrating our efforts on one implementation, and making
> it the best we can.
>
> -K
>
> ----- Original Message -----
> > From: "Gordon Sim" <gs...@redhat.com>
> > To: users@qpid.apache.org
> > Cc: dev@qpid.apache.org
> > Sent: Monday, May 6, 2013 7:11:18 AM
> > Subject: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
> Re: Questions from a novice])
> >
> > On 05/03/2013 04:51 PM, Ken Giusti wrote:
> > > I want to know what the
> > > QPID project's plan is for QMFv1/v2 support in the C++ broker going
> > > forward.
> > >
> > > If the consensus is that a 1.x C++ broker is going to support both
> > > QMFv1 and QMFv2, then simply reverting all the changes I've made to
> > > qmf.console and reverting to the old behavior is probably the least
> > > risky move.
> > >
> > > On the other hand, if the consensus is to deprecate QMFv1 support
> > > prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> > > QMFv2 - then the final fix for this needs to take that plan into
> > > account.
> > >
> > > So I'm going to call a discussion among the developers, proposing we
> > > vote on what QMF support a 1.x C++ broker will provide.
> >
> > I'd like to move the discussion to the user list as there are folks
> > there who may not be on dev list but would be impacted or might
> > otherwise have an opinion. I'm assuming everyone on the dev list is also
> > on the user list (if not you really should be :-)
> >
> > By way of a short introduction, Ken's question comes after a long thread
> > focused on some issues around the dual support of v1 and v2 of the QMF
> > protocol in the console.py python library for applications built to
> > manage/monitor the c++ broker.
> >
> > > Once that's decided, we can determine the most appropriate fix.
> > >
> > > As an aside - we've talked a lot about preserving behavior and
> > > functionality across 0.X releases in this thread. While nobody wants
> > > to gratuitously break stuff, this project has reserved the right to
> > > remove and change functionality between 0.x releases in order to fix
> > > bugs and provide new features. Things may be different once we get
> > > to 1.x, but for 0.x, this has been the case.
> >
> > For my part, I am in favour of removing QMFv1 support, if necessary
> > providing some sort of adapter or plugin to deal with any use cases for
> > which this might prove problematic. I think QMFv2 is now firmly
> > established as the current management mechanism for the c++ broker.
> >
> > Ultimately I also think we should be starting to think about an AMQP 1.0
> > based management standard and how we would transition to that. That is
> > however a larger question that shouldn't hold up the immediate decision
> > regarding QMFv1 though it may inform it a little.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
> >
>
> --
> -K
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
Re: Questions from a novice])
Posted by Ken Giusti <kg...@redhat.com>.
Folks,
My overall concern is that this project doesn't have the bandwidth to adequately support two different management implementations. That's really the crux of the issue for me.
The current problems we've been dealing with come down to the fact that there's inadequate test coverage for dual use QMFv1 and QMFv2. The latest problem (QMFv2 updates from the broker being dropped by qmf.console) has been broken for a long time. We shipped this broken for several releases. Why? Inadequate test coverage.
Most - if not all - of the python tests use QMFv1. Yet qpid-tools are now (mostly) based on QMFv2. I'm not even sure to what extent the QMF agent libraries are tested - perhaps not at all for the cmake builds. This results in partial test coverage of both implementations - neither implementation gets fully tested.
In the end, we try to support both, but are not able to guarantee the level of quality our users deserve for either.
We should be concentrating our efforts on one implementation, and making it the best we can.
-K
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: dev@qpid.apache.org
> Sent: Monday, May 6, 2013 7:11:18 AM
> Subject: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re: Questions from a novice])
>
> On 05/03/2013 04:51 PM, Ken Giusti wrote:
> > I want to know what the
> > QPID project's plan is for QMFv1/v2 support in the C++ broker going
> > forward.
> >
> > If the consensus is that a 1.x C++ broker is going to support both
> > QMFv1 and QMFv2, then simply reverting all the changes I've made to
> > qmf.console and reverting to the old behavior is probably the least
> > risky move.
> >
> > On the other hand, if the consensus is to deprecate QMFv1 support
> > prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> > QMFv2 - then the final fix for this needs to take that plan into
> > account.
> >
> > So I'm going to call a discussion among the developers, proposing we
> > vote on what QMF support a 1.x C++ broker will provide.
>
> I'd like to move the discussion to the user list as there are folks
> there who may not be on dev list but would be impacted or might
> otherwise have an opinion. I'm assuming everyone on the dev list is also
> on the user list (if not you really should be :-)
>
> By way of a short introduction, Ken's question comes after a long thread
> focused on some issues around the dual support of v1 and v2 of the QMF
> protocol in the console.py python library for applications built to
> manage/monitor the c++ broker.
>
> > Once that's decided, we can determine the most appropriate fix.
> >
> > As an aside - we've talked a lot about preserving behavior and
> > functionality across 0.X releases in this thread. While nobody wants
> > to gratuitously break stuff, this project has reserved the right to
> > remove and change functionality between 0.x releases in order to fix
> > bugs and provide new features. Things may be different once we get
> > to 1.x, but for 0.x, this has been the case.
>
> For my part, I am in favour of removing QMFv1 support, if necessary
> providing some sort of adapter or plugin to deal with any use cases for
> which this might prove problematic. I think QMFv2 is now firmly
> established as the current management mechanism for the c++ broker.
>
> Ultimately I also think we should be starting to think about an AMQP 1.0
> based management standard and how we would transition to that. That is
> however a larger question that shouldn't hold up the immediate decision
> regarding QMFv1 though it may inform it a little.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
Re: Questions from a novice])
Posted by Ken Giusti <kg...@redhat.com>.
Folks,
My overall concern is that this project doesn't have the bandwidth to adequately support two different management implementations. That's really the crux of the issue for me.
The current problems we've been dealing with come down to the fact that there's inadequate test coverage for dual use QMFv1 and QMFv2. The latest problem (QMFv2 updates from the broker being dropped by qmf.console) has been broken for a long time. We shipped this broken for several releases. Why? Inadequate test coverage.
Most - if not all - of the python tests use QMFv1. Yet qpid-tools are now (mostly) based on QMFv2. I'm not even sure to what extent the QMF agent libraries are tested - perhaps not at all for the cmake builds. This results in partial test coverage of both implementations - neither implementation gets fully tested.
In the end, we try to support both, but are not able to guarantee the level of quality our users deserve for either.
We should be concentrating our efforts on one implementation, and making it the best we can.
-K
----- Original Message -----
> From: "Gordon Sim" <gs...@redhat.com>
> To: users@qpid.apache.org
> Cc: dev@qpid.apache.org
> Sent: Monday, May 6, 2013 7:11:18 AM
> Subject: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re: Questions from a novice])
>
> On 05/03/2013 04:51 PM, Ken Giusti wrote:
> > I want to know what the
> > QPID project's plan is for QMFv1/v2 support in the C++ broker going
> > forward.
> >
> > If the consensus is that a 1.x C++ broker is going to support both
> > QMFv1 and QMFv2, then simply reverting all the changes I've made to
> > qmf.console and reverting to the old behavior is probably the least
> > risky move.
> >
> > On the other hand, if the consensus is to deprecate QMFv1 support
> > prior to releasing a 1.0 C++ broker - or perhaps toss in the towel on
> > QMFv2 - then the final fix for this needs to take that plan into
> > account.
> >
> > So I'm going to call a discussion among the developers, proposing we
> > vote on what QMF support a 1.x C++ broker will provide.
>
> I'd like to move the discussion to the user list as there are folks
> there who may not be on dev list but would be impacted or might
> otherwise have an opinion. I'm assuming everyone on the dev list is also
> on the user list (if not you really should be :-)
>
> By way of a short introduction, Ken's question comes after a long thread
> focused on some issues around the dual support of v1 and v2 of the QMF
> protocol in the console.py python library for applications built to
> manage/monitor the c++ broker.
>
> > Once that's decided, we can determine the most appropriate fix.
> >
> > As an aside - we've talked a lot about preserving behavior and
> > functionality across 0.X releases in this thread. While nobody wants
> > to gratuitously break stuff, this project has reserved the right to
> > remove and change functionality between 0.x releases in order to fix
> > bugs and provide new features. Things may be different once we get
> > to 1.x, but for 0.x, this has been the case.
>
> For my part, I am in favour of removing QMFv1 support, if necessary
> providing some sort of adapter or plugin to deal with any use cases for
> which this might prove problematic. I think QMFv2 is now firmly
> established as the current management mechanism for the c++ broker.
>
> Ultimately I also think we should be starting to think about an AMQP 1.0
> based management standard and how we would transition to that. That is
> however a larger question that shouldn't hold up the immediate decision
> regarding QMFv1 though it may inform it a little.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS Re:
Questions from a novice])
Posted by Rob Godfrey <ro...@gmail.com>.
So, the initial thinking for AMQP Management has (finally) now been
uploaded so that anyone can see it [1]. As Fraser mentioned, the scope of
the AMQP Management work at OASIS is currently purely about "mechanism" and
not defining specific operations / attributes that are available on a
managed object.
-- Rob
[1] https://lists.oasis-open.org/archives/amqp/201305/msg00005.html
On 7 May 2013 20:18, Fraser Adams <fr...@blueyonder.co.uk> wrote:
> On 06/05/13 12:11, Gordon Sim wrote:
>
>>
>> For my part, I am in favour of removing QMFv1 support, if necessary
>> providing some sort of adapter or plugin to deal with any use cases for
>> which this might prove problematic. I think QMFv2 is now firmly established
>> as the current management mechanism for the c++ broker.
>>
> My take on this is that I'm more than happy to see the back of QMF1 but
> I'd be keen to understand whether or not this results in *breaking changes*.
>
> A couple of things I mentioned on the dev group around this point are
> a) The asynchronous python API - that's where this discussion started the
> problem is that the callbacks that have actually worked to date work for
> QMF1, there was a suggestion of an additional objectUpdate() call for QMF2
> which is OK as it wouldn't break existing clients as long as QMF1 data was
> still pushed, but would if it was disabled. Forcing clients to change from
> statUpdate() and propUpdate() to objectUpdate() (and any changes regression
> testing that this involves) is clearly a breaking change. Parochially I've
> only got one tool that uses the async API but I'd prefer it not to be
> broken (mainly because I wrote it at home for a work use and I'm not
> responsible for supporting it and I'd end up having to explain the
> subtleties we're discussing here to someone with a lot less QMF experience
> than me :'( ). At a push as long as I have decent notice I can rewrite it
> myself, but others may not have that luxury - as Bill has mentioned
> previously he's a contractor so when he finishes up his contract and
> someone else takes over his work they won't have this background and it
> might "silently" stop working properly.
> b) If anyone has been writing Agents using QMF1 then clearly they are
> likely to break. Parochially this doesn't affect me, but I'm not sure quite
> how we can gauge how much it may affect others.
>
> Perhaps rather like the automake to cmake change we start off by giving
> gentle nudges then get a bit more forceful, so disabling the QMF1 broker
> updates as Ken has suggested might be a start (along with some decent
> documentation and a deprecation notice in the qpidd -h text).
>
> I'd personally quite like to see the python async API shim the QMF2 update
> to make it behave like statUpdate()/propUpdate() though I accept this might
> be difficult and would have to bow to people who know more about the
> implementation of this than me.
>
>
>
>> Ultimately I also think we should be starting to think about an AMQP 1.0
>> based management standard and how we would transition to that. That is
>> however a larger question that shouldn't hold up the immediate decision
>> regarding QMFv1 though it may inform it a little.
>>
>> Yeah I totally agree, I hope that it's not too long before Rob is able
> to share the draft AMQP 1.0 Management Spec. It'd be good to have a "lively
> discussion" about this among a wider group. Rob and I had a couple of
> decent chats over Easter but I'm keen to start some proper thinking on how
> we can make this a reality. Unfortunately I suspect that the QMF1/QMF2
> quirks are likely to be minor compared to moving to AMQP 1.0 Management,
> but hopefully not insurmountable, I'm hopeful that it should be possible to
> shim between ManagementAgent and ManagementNode and vice versa which should
> help with migration. As we've discussed a pluggable Management layer for
> the C++ broker would be good, it really helped in the Java broker - I was
> able to get QMF2 support up and working in a couple of weekends with no
> prior knowledge of the Java broker code base.
>
> <rant>
>
> As an aside the AMQP 1.0 Management specification is only covering the
> protocol as I understand it, but I'd be quite keen for us to get our act
> together with respect to APIs. Yes I know it's possible to write QMF2 using
> the protocol directly, but frankly it's not as interoperable as it might be
> (C++ binary strings versus UTF8 strings are a personal "favourite" of mine
> resulting in byte[] or String respectively in Java - ClassCastException
> hell :-)). So I think it's useful to have a decent API, however the four or
> five APIs that we seem to have for QMF is less helpful :-D - can we agree
> on one when we start looking at AMQP 1.0 Management?
>
> </rant> :-)
>
> Cheers,
> Frase
>
>
>
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.**org<de...@qpid.apache.org>
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>
Re: Plans for QMF v1 and v2 (was Re: QMFv2 object update bug [WAS
Re: Questions from a novice])
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
On 06/05/13 12:11, Gordon Sim wrote:
>
> For my part, I am in favour of removing QMFv1 support, if necessary
> providing some sort of adapter or plugin to deal with any use cases
> for which this might prove problematic. I think QMFv2 is now firmly
> established as the current management mechanism for the c++ broker.
My take on this is that I'm more than happy to see the back of QMF1 but
I'd be keen to understand whether or not this results in *breaking changes*.
A couple of things I mentioned on the dev group around this point are
a) The asynchronous python API - that's where this discussion started
the problem is that the callbacks that have actually worked to date work
for QMF1, there was a suggestion of an additional objectUpdate() call
for QMF2 which is OK as it wouldn't break existing clients as long as
QMF1 data was still pushed, but would if it was disabled. Forcing
clients to change from statUpdate() and propUpdate() to objectUpdate()
(and any changes regression testing that this involves) is clearly a
breaking change. Parochially I've only got one tool that uses the async
API but I'd prefer it not to be broken (mainly because I wrote it at
home for a work use and I'm not responsible for supporting it and I'd
end up having to explain the subtleties we're discussing here to someone
with a lot less QMF experience than me :'( ). At a push as long as I
have decent notice I can rewrite it myself, but others may not have that
luxury - as Bill has mentioned previously he's a contractor so when he
finishes up his contract and someone else takes over his work they won't
have this background and it might "silently" stop working properly.
b) If anyone has been writing Agents using QMF1 then clearly they are
likely to break. Parochially this doesn't affect me, but I'm not sure
quite how we can gauge how much it may affect others.
Perhaps rather like the automake to cmake change we start off by giving
gentle nudges then get a bit more forceful, so disabling the QMF1 broker
updates as Ken has suggested might be a start (along with some decent
documentation and a deprecation notice in the qpidd -h text).
I'd personally quite like to see the python async API shim the QMF2
update to make it behave like statUpdate()/propUpdate() though I accept
this might be difficult and would have to bow to people who know more
about the implementation of this than me.
>
> Ultimately I also think we should be starting to think about an AMQP
> 1.0 based management standard and how we would transition to that.
> That is however a larger question that shouldn't hold up the immediate
> decision regarding QMFv1 though it may inform it a little.
>
Yeah I totally agree, I hope that it's not too long before Rob is able
to share the draft AMQP 1.0 Management Spec. It'd be good to have a
"lively discussion" about this among a wider group. Rob and I had a
couple of decent chats over Easter but I'm keen to start some proper
thinking on how we can make this a reality. Unfortunately I suspect that
the QMF1/QMF2 quirks are likely to be minor compared to moving to AMQP
1.0 Management, but hopefully not insurmountable, I'm hopeful that it
should be possible to shim between ManagementAgent and ManagementNode
and vice versa which should help with migration. As we've discussed a
pluggable Management layer for the C++ broker would be good, it really
helped in the Java broker - I was able to get QMF2 support up and
working in a couple of weekends with no prior knowledge of the Java
broker code base.
<rant>
As an aside the AMQP 1.0 Management specification is only covering the
protocol as I understand it, but I'd be quite keen for us to get our act
together with respect to APIs. Yes I know it's possible to write QMF2
using the protocol directly, but frankly it's not as interoperable as it
might be (C++ binary strings versus UTF8 strings are a personal
"favourite" of mine resulting in byte[] or String respectively in Java -
ClassCastException hell :-)). So I think it's useful to have a decent
API, however the four or five APIs that we seem to have for QMF is less
helpful :-D - can we agree on one when we start looking at AMQP 1.0
Management?
</rant> :-)
Cheers,
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org