You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk> on 2013/01/02 15:54:43 UTC
QMF dead to new requests after invoking some create calls
Hello all,
I've been testing out the add binding functionality on the Qpid UI I've
been writing and came across some "exciting" broker behaviour :-)
In précis I was fairly randomly adding and deleting bindings, which
ultimately result in QMF create/delete calls with type of binding.
Eventually I noticed in my UI that I wasn't seeing new bindings being
added. I checked in another instance on another machine (one was from my
iPad, the other my laptop) and noticed I'd stopped getting QMF updates.
I restarted the REST Server that serives the GUI AJAX calls as this
contains the QMF Console instance and that didn't help so it looks like
it was more an issue with the broker, sure enough when I checked the
broker log I saw.
2013-01-01 19:46:26 error Detected two management objects with the same
identifier:
0-1541-1--3452(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f3)
2013-01-02 12:44:07 error Detected two management objects with the same
identifier:
0-1541-1--3557(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f2)
2013-01-02 12:46:09 error Detected two management objects with the same
identifier:
0-1541-1--3559(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
2013-01-02 13:51:15 error Detected two management objects with the same
identifier:
0-1541-1--3655(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
Restarting the broker fixed things, but this looks like some defensive
logic is missing on the broker QMF create call (I've been testing
against qpid 0.12).
If I try adding queues or exchanges with a name that already exists I
get an exception sent back from QMF along the lines of "object already
exists: amq.match" so there's definitely some defensive logic, so this
appears only to be a problem with create binding.
Unfortunately I only came across this as I was fairly randomly adding
and removing bindings so I can't systematically reproduce this behaviour
yet.
I did add a binding between my test queue and amq.match with a binding
key of f1 and x-match all, when I tried again with x-match any QMF
accepted this without an exception but I got
2013-01-02 14:18:41 error Detected two management objects with the same
identifier:
0-1542-1--45(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
From the broker, but this time I didn't get QMF hanging on me as I'd
had before so it's definitely a bit weird.
For create binding the QMF identifier (name) is of the form
<exchangeName>/<queueName>[/<bindingKey>] so as I was able to add
amq.match/test/f1 twice - once with properties {x-match: all} the other
with {x-match-any} it looks like the test for duplicate names that's in
place for create queue and create exchange isn't present for create binding.
Another slightly odd thing that I noticed in all this is that after
adding and removing a bunch of bindings between amq.match and the test
queue it looks like the bindingCount property of amq.match is being
incorrectly maintained, it's reporting a value of 6 when I've ended up
with none. This looks like a discrepancy between the value reported in
the exchange object's bindingCount property and the actual number of
binding Management Objects that have associations with the exchange in
question.
I expect I'm probably pushing the bounds of QMF with this stuff, though
if I do:
qpid-config bind amq.match test f1 all k1=v1
qpid-config bind amq.match test f1 any k1=v1
I don't get any errors from qpid-config, but the broker says
2013-01-02 14:49:39 error Detected two management objects with the same
identifier:
0-1543-1--138(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
Again though I've not yet been able to reproduce the case where it
really gets its knickers in a twist and becomes dead to other QMF
requests. I'll update this post if and when I manage to break it
systematically, but in the mean time it definitely looks like there's at
least some weirdness going on with adding bindings.
Regards,
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QMF dead to new requests after invoking some create calls
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
On 03/01/13 19:18, Ken Giusti wrote:
> Hi Fraser,
>
> I think the fix for this bug is to prevent the headers exchange from allowing multiple bindings with the same keys.
>
> In that case, when entering the following commands:
>
> qpid-config bind amq.match test f1 all k1=v1
> qpid-config bind amq.match test f1 any k1=v1
>
> the second qpid-config command would fail (binding already in use).
Yes I agree I think that the second command *should* fail as the
<exchangeName>/<queueName>[/<bindingKey>] combination is the same for
both and it's that combination of names that are used internally. At
least for 0.12 duplicates are accepted without QMF sending an exception
(unlike for adding queues or exchanges where an exception "object
already exists <name>" gets reported by by the invokeMethod response).
> I think that - for headers exchanges - the binding key isn't really used beyond identifying the binding (so you can eventually delete it).
I personally encourage the use of a binding key for the headers
exchange, you are correct that it is optional and it looks as though if
you add a single binding without a key you can actually add it and
delete it, but having an explicit key helps add clarity IMHO - I
generally use descriptive keys that relate to what the binding is trying
to achieve.
> So you could get the same behavior doing:
>
> qpid-config bind amq.match test f1 all k1=v1
> qpid-config bind amq.match test f2 any k1=v1
>
> instead. At least that is what I remember - it's been awhile. What do you think?
Yes with different keys it's possible to add an any and an all match,
though the former then becomes redundant.
If you then do
qpid-config bind amq.match test f3 all k1=v1
so basically add a duplicate binding but with a different key it doesn't
get added (silently). I think that not adding it is probably the right
thing to do, but I don't like it silently failing I think an exception
should be sent back.
There's definitely plenty of inconsistency though, I've just tried
adding bindings to an xml exchange, but if I add two bindings with keys
f1 and f2 with the same xquery they get added - this is largely
equivalent to adding headers bindings with the same k/v properties,
which as said above doesn't get added, so that seems inconsistent.
What's even weirder is if I try to add another xml binding with key f1
but with a different xquery that binding doesn't get added (silently)
but I don't get "error Detected two management objects with the same
identifier:" from the broker, which suggests that for the xml exchange
some sort of test is being done on the overall key (unlike the case for
the headers exchange) but an exception isn't being reported back.
>> Hopefully my Qpid GUI will be in a releasable state in the next couple
>> of days, I'm currently going through some final bugs & snags and a
>> little bit of work to get it looking nice on mobile browsers.
> Very cool! Have you tried running it against 0.20 release candidate qpidd? Should work fine, but it's always nice to get extra exposure before we cut the release.
>
> -K
>
It's looking pretty good now, it's quite flashy with iOS style animated
page transitions. It's basically a single page web-app using AJAX to
update the state. What I've done is write a Java Qpid REST API bridging
to my QMF2 implementation then on the JavaScript side I've written a QMF
API wrapper to the REST API, so my core JavaScript GUI code actually
talks the QMF2 API.
I did it that way 'cause it provides much cleaner abstraction. At the
moment the bridge is pure REST/HTTP but I might extend that in the
future to use WebSockets, I've not done that yet as a) I've not got much
Web Socket experience and b) it has poorer cross-browser support. At the
moment by GUI runs on everything from IE6 up to modern mobile browsers.
The main bit of styling I've yet to do is to do a bit of "real-estate
adjustment" for small displays like iPhone - it's currently got a
sidebar a la iPad which needs to be hidden on smaller devices. Nearly
there though :-)
I've not tried it with anything later than 0.12 yet. Unfortunately every
time I've upgraded qpid version I've needed to fight the makefile. I
noticed that Gordon Sim has incorporated a patch I did for the makefile
into 0.20 so hopefully that pain will go away shortly. I'll give it a
whirl with 0.20 before I release.
I *hope* nothing breaks. It *should* be OK unless anyone has messed
around with the QMF2 logic??? I recall between 0.8 and 0.10 there was a
change to QMF Events where they started to get sent as AMQP lists from
0.10 unfortunately amqp lists don't map terribly well to JMS and I ended
up using some of Gordon's code that used the low-level AMQP decodes. So
unless anyone has been messing around with mappings from AMQP list to
JMS it should just work.
I'm off to write some more code now :-)
Frase
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QMF dead to new requests after invoking some create calls
Posted by Ken Giusti <kg...@redhat.com>.
Hi Fraser,
I think the fix for this bug is to prevent the headers exchange from allowing multiple bindings with the same keys.
In that case, when entering the following commands:
qpid-config bind amq.match test f1 all k1=v1
qpid-config bind amq.match test f1 any k1=v1
the second qpid-config command would fail (binding already in use). I think that - for headers exchanges - the binding key isn't really used beyond identifying the binding (so you can eventually delete it). So you could get the same behavior doing:
qpid-config bind amq.match test f1 all k1=v1
qpid-config bind amq.match test f2 any k1=v1
instead. At least that is what I remember - it's been awhile. What do you think?
> Hopefully my Qpid GUI will be in a releasable state in the next couple
> of days, I'm currently going through some final bugs & snags and a
> little bit of work to get it looking nice on mobile browsers.
Very cool! Have you tried running it against 0.20 release candidate qpidd? Should work fine, but it's always nice to get extra exposure before we cut the release.
-K
----- Original Message -----
> Thanks for the update Ken.
>
> "but since the problem is rarely hit" clearly we're not adding enough
> bindings :-) I'll soon change that :-D
>
>
> I'll probably add some defensive code in my GUI. It's a little bit of
> a
> faff to do but I've already got a cache of Binding, Queue and
> Exchange
> QMF objects, so when I go to add a new binding I can look through the
> Binding objects for ones that match my binding key then dereference
> the
> queueRef/exchangeRef IDs to lookup queue/exchange names associated
> with
> said binding to check if my <exchangeName>/<queueName>/<bindingName>
> key
> is unique.
>
> It's not *quite* infallable as the cache only gets updated every 10
> seconds, but it's *probably* good enough. I might look at forcing a
> cache refresh just before the test, but that's getting a lot more
> complex as I'm actually getting my QMF objects back from an AJAX call
> to
> a REST implementation of the QMF2 API :-)
>
> Hopefully my Qpid GUI will be in a releasable state in the next
> couple
> of days, I'm currently going through some final bugs & snags and a
> little bit of work to get it looking nice on mobile browsers.
>
> Regards,
> Frase
>
> On 03/01/13 17:32, Ken Giusti wrote:
> >
> > This appears to be a bug in the way the amq.match (Headers
> > Exchange) is implemented.
> >
> > The exchange+queue+binding-key need to be unique - QMF uses that
> > tuple as the mgmt object index. However, it seems the Headers
> > Exchange also considers the header configuration (k1=v1, etc) when
> > it checks for existing bindings. In the above case, since the
> > header values are different, the exchange creates two bindings
> > even though the exchange+queue+binding key are the same. That's a
> > bug.
> >
> > Gordon hit this bug awhile ago while using federation -
> > https://issues.apache.org/jira/browse/QPID-3356.
> >
> > The other problem - the hang - is probably related: QMF's handling
> > of duplicate object errors has been improved quite a bit since
> > 0.12, but since the problem is rarely hit there are probably still
> > bugs in it.
> >
> >
> > -K
> >
> > -
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QMF dead to new requests after invoking some create calls
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Thanks for the update Ken.
"but since the problem is rarely hit" clearly we're not adding enough
bindings :-) I'll soon change that :-D
I'll probably add some defensive code in my GUI. It's a little bit of a
faff to do but I've already got a cache of Binding, Queue and Exchange
QMF objects, so when I go to add a new binding I can look through the
Binding objects for ones that match my binding key then dereference the
queueRef/exchangeRef IDs to lookup queue/exchange names associated with
said binding to check if my <exchangeName>/<queueName>/<bindingName> key
is unique.
It's not *quite* infallable as the cache only gets updated every 10
seconds, but it's *probably* good enough. I might look at forcing a
cache refresh just before the test, but that's getting a lot more
complex as I'm actually getting my QMF objects back from an AJAX call to
a REST implementation of the QMF2 API :-)
Hopefully my Qpid GUI will be in a releasable state in the next couple
of days, I'm currently going through some final bugs & snags and a
little bit of work to get it looking nice on mobile browsers.
Regards,
Frase
On 03/01/13 17:32, Ken Giusti wrote:
>
> This appears to be a bug in the way the amq.match (Headers Exchange) is implemented.
>
> The exchange+queue+binding-key need to be unique - QMF uses that tuple as the mgmt object index. However, it seems the Headers Exchange also considers the header configuration (k1=v1, etc) when it checks for existing bindings. In the above case, since the header values are different, the exchange creates two bindings even though the exchange+queue+binding key are the same. That's a bug.
>
> Gordon hit this bug awhile ago while using federation - https://issues.apache.org/jira/browse/QPID-3356.
>
> The other problem - the hang - is probably related: QMF's handling of duplicate object errors has been improved quite a bit since 0.12, but since the problem is rarely hit there are probably still bugs in it.
>
>
> -K
>
> -
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: QMF dead to new requests after invoking some create calls
Posted by Ken Giusti <kg...@redhat.com>.
Hi Fraser,
>I expect I'm probably pushing the bounds of QMF with this stuff, though
>if I do:
>
>qpid-config bind amq.match test f1 all k1=v1
>qpid-config bind amq.match test f1 any k1=v1
>
>I don't get any errors from qpid-config, but the broker says
>
>2013-01-02 14:49:39 error Detected two management objects with the same
>dentifier:
>-1543-1--138(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
This appears to be a bug in the way the amq.match (Headers Exchange) is implemented.
The exchange+queue+binding-key need to be unique - QMF uses that tuple as the mgmt object index. However, it seems the Headers Exchange also considers the header configuration (k1=v1, etc) when it checks for existing bindings. In the above case, since the header values are different, the exchange creates two bindings even though the exchange+queue+binding key are the same. That's a bug.
Gordon hit this bug awhile ago while using federation - https://issues.apache.org/jira/browse/QPID-3356.
The other problem - the hang - is probably related: QMF's handling of duplicate object errors has been improved quite a bit since 0.12, but since the problem is rarely hit there are probably still bugs in it.
-K
----- Original Message -----
> Hello all,
> I've been testing out the add binding functionality on the Qpid UI
> I've
> been writing and came across some "exciting" broker behaviour :-)
>
> In précis I was fairly randomly adding and deleting bindings, which
> ultimately result in QMF create/delete calls with type of binding.
> Eventually I noticed in my UI that I wasn't seeing new bindings being
> added. I checked in another instance on another machine (one was from
> my
> iPad, the other my laptop) and noticed I'd stopped getting QMF
> updates.
>
> I restarted the REST Server that serives the GUI AJAX calls as this
> contains the QMF Console instance and that didn't help so it looks
> like
> it was more an issue with the broker, sure enough when I checked the
> broker log I saw.
>
> 2013-01-01 19:46:26 error Detected two management objects with the
> same
> identifier:
> 0-1541-1--3452(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f3)
> 2013-01-02 12:44:07 error Detected two management objects with the
> same
> identifier:
> 0-1541-1--3557(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f2)
> 2013-01-02 12:46:09 error Detected two management objects with the
> same
> identifier:
> 0-1541-1--3559(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
> 2013-01-02 13:51:15 error Detected two management objects with the
> same
> identifier:
> 0-1541-1--3655(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
>
>
> Restarting the broker fixed things, but this looks like some
> defensive
> logic is missing on the broker QMF create call (I've been testing
> against qpid 0.12).
>
> If I try adding queues or exchanges with a name that already exists I
> get an exception sent back from QMF along the lines of "object
> already
> exists: amq.match" so there's definitely some defensive logic, so
> this
> appears only to be a problem with create binding.
>
>
> Unfortunately I only came across this as I was fairly randomly adding
> and removing bindings so I can't systematically reproduce this
> behaviour
> yet.
>
> I did add a binding between my test queue and amq.match with a
> binding
> key of f1 and x-match all, when I tried again with x-match any QMF
> accepted this without an exception but I got
>
> 2013-01-02 14:18:41 error Detected two management objects with the
> same
> identifier:
> 0-1542-1--45(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
>
>
> From the broker, but this time I didn't get QMF hanging on me as I'd
> had before so it's definitely a bit weird.
>
>
> For create binding the QMF identifier (name) is of the form
> <exchangeName>/<queueName>[/<bindingKey>] so as I was able to add
> amq.match/test/f1 twice - once with properties {x-match: all} the
> other
> with {x-match-any} it looks like the test for duplicate names that's
> in
> place for create queue and create exchange isn't present for create
> binding.
>
> Another slightly odd thing that I noticed in all this is that after
> adding and removing a bunch of bindings between amq.match and the
> test
> queue it looks like the bindingCount property of amq.match is being
> incorrectly maintained, it's reporting a value of 6 when I've ended
> up
> with none. This looks like a discrepancy between the value reported
> in
> the exchange object's bindingCount property and the actual number of
> binding Management Objects that have associations with the exchange
> in
> question.
>
>
> I expect I'm probably pushing the bounds of QMF with this stuff,
> though
> if I do:
>
> qpid-config bind amq.match test f1 all k1=v1
> qpid-config bind amq.match test f1 any k1=v1
>
> I don't get any errors from qpid-config, but the broker says
>
> 2013-01-02 14:49:39 error Detected two management objects with the
> same
> identifier:
> 0-1543-1--138(org.apache.qpid.broker:binding:org.apache.qpid.broker:exchange:amq.match,org.apache.qpid.broker:queue:test,f1)
>
>
> Again though I've not yet been able to reproduce the case where it
> really gets its knickers in a twist and becomes dead to other QMF
> requests. I'll update this post if and when I manage to break it
> systematically, but in the mean time it definitely looks like there's
> at
> least some weirdness going on with adding bindings.
>
> Regards,
> Frase
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org