You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Fraser Adams <fr...@blueyonder.co.uk.INVALID> on 2021/07/12 12:31:25 UTC
Does Qpid Java broker close connection upon channel exception as well
as closing the channel?
For context I'm working on a project that is primarily using the
RabbitMQ broker and using the Pika Python client library. As the Qpid
Java broker supports AMQP 0.9.1 too I figured I'd try to connect to that
too to see if I could compare relative performance.
I ran into a couple of issues, the first is that by default the Java
broker doesn't support PLAIN authentication on insecure transports so I
needed to mod the config.json to add
"secureOnlyMechanisms" : [ ],
to "authenticationproviders"
I thought/hoped that'd be enough, but unfortunately things kept failing
with a 504 error complaining about channel :2 not existing
As it happens, what I'm actually doing during the startup of my
application is to check if an exchange with the name of the specified
destination exists.
The way I do that is to do an exchange_declare with passive=True. Now I
know that when an exception/error occurs (e.g. a 404 NOT_FOUND) if the
exchange doesn't exist then the channel is closed by the broker. To
cater for this I create a temporary channel (in this case it would be
channel :2) and call the exchange_declare passive=True using that
channel and I get a pika.exceptions.ChannelClosedByBroker, which is what
I'd expect.
So that's how it is with RabbitMQ broker, but with Qpid Java subsequent
things were failing and I narrowed it down to the connection having been
closed in addition to the channel.
I've tried creating an additional temporary *connection* in addition to
the temporary channel that I was previously creating in order to run the
passive=True exchange_declare and doing that seems to "work" in terms of
the rest of the application now seems to carry on correctly.
So really I'm curious, is the Qpid Java broker closing the connection in
addition to closing the channel the expected behaviour in this scenario?
It is certainly different compared with what RabbitMQ does, but I have
no idea what the "correct" behaviour is in this circumstance.
MTIA
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does Qpid Java broker close connection upon channel exception as
well as closing the channel?
Posted by Fraser Adams <fr...@blueyonder.co.uk.INVALID>.
On "
In particular your note of the eventual error indicating 'channel 2
not existing' would to me as much potentially imply that something was
still trying to use channel 2 at a point it shouldnt
"
That was my initial thought too, so I spent a fair bit of time trying to
see if I'd "done something stupid" before I posted the question. What I
eventually did was to add additional on_close_callback handlers to the
channel and connection and also do a good old hacky asyncio.sleep() in
my code after my call to exchange_declare to hold off reaching any
potentially dodgy bits of my code and what I ended up seeing was a
channel close followed fairly quickly by a connection close, both
referring to channel :2 in the (I think) 504 (ConnectionClosedByBroker?)
response (which was the temp channel used for the exchange_declare). So
it definitely looks like the channel error caused by the exchange not
being there is also closing the connection, 'cause that channel is
definitely not being used in my code after the exchange_declare call.
Thanks for the other responses, I'll do some digging to see if I can get
any connection properties info out of Pika and yeah I realised that
image was using a fairly old version, TBH I just used that one as a
quick and dirty stop-gap to try and get something up and running initially.
I agree it'd be better to use something more recent. Is there a reason
there aren't any official images? TBH in 2021 firing things up in Docker
is pretty much the de facto way most people "Quickstart" trying out
various services, so I'd imagine that not having official images
available just might put people off using Qpid.
On 12/07/2021 16:39, Robbie Gemmell wrote:
> On Mon, 12 Jul 2021 at 15:40, Fraser Adams
> <fr...@blueyonder.co.uk.invalid> wrote:
>> Thanks Rob, so the bottom line then is that the Qpid Java broker *is*
>> closing the connection in addition to the channel, right?
>>
> It may be, though I'm not sure what's been said actually proves that,
> or if it is then precisely why.
>
> In particular your note of the eventual error indicating 'channel 2
> not existing' would to me as much potentially imply that something was
> still trying to use channel 2 at a point it shouldnt, which could seem
> peculiar given it was always expected to be closed immediately. Do you
> recreate it?
>
>> If that's the case at least that's reassuring in that it's doing what
>> it's _supposed_ to be doing (even if that may not be quite "correct"). I
>> was mainly concerned that I may have been misunderstanding something or
>> doing something dumb/wrong myself, it was driving me slightly mad over
>> the weekend . If it's just "a quirk" I can probably live with it.
>>
>> Another couple of questions:
>>
>> 1. I don't suppose you know of any simple way I could "detect" if I'm
>> connecting to Qpid vice RabbitMQ from any of the connection/channel
>> responses. I'm guessing probably not, but it might be nice to avoid
>> creating that extra connection if there's a cheap and simple way to do
>> it, but not worth bothering if I need to go through hoops to work out
>> what I'm connecting to.
>>
> The broker adds its name in the connection properties during
> opening...though that isnt to say you can actually inspect them using
> the client.
>
>> 2. My (very superficial) initial experiments seemed to indicate that the
>> throughput of my application was a little slower with the Qpid Java
>> broker than RabbitMQ. It was a fairly basic producer/consumer example
>> just using the direct exchange and publishing 5K payloads. In all
>> honesty I've not dug deeply as I was initially focussed on "just getting
>> it to work" and for simplicity I just used this docker image
>> abh1sh3k/apache-qpid.
> Seems to be using 6.0.4, which just turned 5 years old. Might be worth
> trying something newer. Setup is essentially extracting the archive,
> optionally set QPID_WORK env variable.
>
>> The config was pretty much the default for that
>> image apart from the "secureOnlyMechanisms" : [ ],. I don't suppose you
>> have any experience of comparing the two brokers and where one might
>> outperform the other? The difference wasn't huge, though weirdly my
>> Python client seemed to be using more CPU with the Java broker.
>>
>> 3. Is there an "official" Docker image for the Java broker? There was
>> some talk of Qpid on Docker some time ago and Irina has some images
>> here: https://hub.docker.com/u/irinabov, but those were last updated 4
>> years ago and relate to the c++ broker and dispatch.
>>
> There is not.
>
>
>> Thanks again for the reply!
>>
>> F.
>>
>> On 12/07/2021 14:48, Rob Godfrey wrote:
>>> So, according to the 0-9-1 definition (
>>> https://www.amqp.org/specification/0-9-1/amqp-org-download), 404 is
>>> supposed to be a channel error - so RabbitMQ is likely more correct than
>>> the Java Broker. IIRC there were a whole boatload of issues with clients
>>> (particularly the old JMS client) trying to handle channel level
>>> exceptions, so the decision was made to just treat everything as a
>>> connection level exception. Others may have a more accurate/detailed
>>> memory.
>>>
>>> -- Rob
>>>
>>> On Mon, 12 Jul 2021 at 14:31, Fraser Adams
>>> <fr...@blueyonder.co.uk.invalid> wrote:
>>>
>>>> For context I'm working on a project that is primarily using the
>>>> RabbitMQ broker and using the Pika Python client library. As the Qpid
>>>> Java broker supports AMQP 0.9.1 too I figured I'd try to connect to that
>>>> too to see if I could compare relative performance.
>>>>
>>>> I ran into a couple of issues, the first is that by default the Java
>>>> broker doesn't support PLAIN authentication on insecure transports so I
>>>> needed to mod the config.json to add
>>>>
>>>> "secureOnlyMechanisms" : [ ],
>>>>
>>>> to "authenticationproviders"
>>>>
>>>> I thought/hoped that'd be enough, but unfortunately things kept failing
>>>> with a 504 error complaining about channel :2 not existing
>>>>
>>>>
>>>> As it happens, what I'm actually doing during the startup of my
>>>> application is to check if an exchange with the name of the specified
>>>> destination exists.
>>>>
>>>> The way I do that is to do an exchange_declare with passive=True. Now I
>>>> know that when an exception/error occurs (e.g. a 404 NOT_FOUND) if the
>>>> exchange doesn't exist then the channel is closed by the broker. To
>>>> cater for this I create a temporary channel (in this case it would be
>>>> channel :2) and call the exchange_declare passive=True using that
>>>> channel and I get a pika.exceptions.ChannelClosedByBroker, which is what
>>>> I'd expect.
>>>>
>>>> So that's how it is with RabbitMQ broker, but with Qpid Java subsequent
>>>> things were failing and I narrowed it down to the connection having been
>>>> closed in addition to the channel.
>>>>
>>>> I've tried creating an additional temporary *connection* in addition to
>>>> the temporary channel that I was previously creating in order to run the
>>>> passive=True exchange_declare and doing that seems to "work" in terms of
>>>> the rest of the application now seems to carry on correctly.
>>>>
>>>>
>>>> So really I'm curious, is the Qpid Java broker closing the connection in
>>>> addition to closing the channel the expected behaviour in this scenario?
>>>> It is certainly different compared with what RabbitMQ does, but I have
>>>> no idea what the "correct" behaviour is in this circumstance.
>>>>
>>>> MTIA
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>>
>>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does Qpid Java broker close connection upon channel exception as
well as closing the channel?
Posted by Robbie Gemmell <ro...@gmail.com>.
On Mon, 12 Jul 2021 at 15:40, Fraser Adams
<fr...@blueyonder.co.uk.invalid> wrote:
>
> Thanks Rob, so the bottom line then is that the Qpid Java broker *is*
> closing the connection in addition to the channel, right?
>
It may be, though I'm not sure what's been said actually proves that,
or if it is then precisely why.
In particular your note of the eventual error indicating 'channel 2
not existing' would to me as much potentially imply that something was
still trying to use channel 2 at a point it shouldnt, which could seem
peculiar given it was always expected to be closed immediately. Do you
recreate it?
> If that's the case at least that's reassuring in that it's doing what
> it's _supposed_ to be doing (even if that may not be quite "correct"). I
> was mainly concerned that I may have been misunderstanding something or
> doing something dumb/wrong myself, it was driving me slightly mad over
> the weekend . If it's just "a quirk" I can probably live with it.
>
> Another couple of questions:
>
> 1. I don't suppose you know of any simple way I could "detect" if I'm
> connecting to Qpid vice RabbitMQ from any of the connection/channel
> responses. I'm guessing probably not, but it might be nice to avoid
> creating that extra connection if there's a cheap and simple way to do
> it, but not worth bothering if I need to go through hoops to work out
> what I'm connecting to.
>
The broker adds its name in the connection properties during
opening...though that isnt to say you can actually inspect them using
the client.
> 2. My (very superficial) initial experiments seemed to indicate that the
> throughput of my application was a little slower with the Qpid Java
> broker than RabbitMQ. It was a fairly basic producer/consumer example
> just using the direct exchange and publishing 5K payloads. In all
> honesty I've not dug deeply as I was initially focussed on "just getting
> it to work" and for simplicity I just used this docker image
> abh1sh3k/apache-qpid.
Seems to be using 6.0.4, which just turned 5 years old. Might be worth
trying something newer. Setup is essentially extracting the archive,
optionally set QPID_WORK env variable.
>The config was pretty much the default for that
> image apart from the "secureOnlyMechanisms" : [ ],. I don't suppose you
> have any experience of comparing the two brokers and where one might
> outperform the other? The difference wasn't huge, though weirdly my
> Python client seemed to be using more CPU with the Java broker.
>
> 3. Is there an "official" Docker image for the Java broker? There was
> some talk of Qpid on Docker some time ago and Irina has some images
> here: https://hub.docker.com/u/irinabov, but those were last updated 4
> years ago and relate to the c++ broker and dispatch.
>
There is not.
> Thanks again for the reply!
>
> F.
>
> On 12/07/2021 14:48, Rob Godfrey wrote:
> > So, according to the 0-9-1 definition (
> > https://www.amqp.org/specification/0-9-1/amqp-org-download), 404 is
> > supposed to be a channel error - so RabbitMQ is likely more correct than
> > the Java Broker. IIRC there were a whole boatload of issues with clients
> > (particularly the old JMS client) trying to handle channel level
> > exceptions, so the decision was made to just treat everything as a
> > connection level exception. Others may have a more accurate/detailed
> > memory.
> >
> > -- Rob
> >
> > On Mon, 12 Jul 2021 at 14:31, Fraser Adams
> > <fr...@blueyonder.co.uk.invalid> wrote:
> >
> >> For context I'm working on a project that is primarily using the
> >> RabbitMQ broker and using the Pika Python client library. As the Qpid
> >> Java broker supports AMQP 0.9.1 too I figured I'd try to connect to that
> >> too to see if I could compare relative performance.
> >>
> >> I ran into a couple of issues, the first is that by default the Java
> >> broker doesn't support PLAIN authentication on insecure transports so I
> >> needed to mod the config.json to add
> >>
> >> "secureOnlyMechanisms" : [ ],
> >>
> >> to "authenticationproviders"
> >>
> >> I thought/hoped that'd be enough, but unfortunately things kept failing
> >> with a 504 error complaining about channel :2 not existing
> >>
> >>
> >> As it happens, what I'm actually doing during the startup of my
> >> application is to check if an exchange with the name of the specified
> >> destination exists.
> >>
> >> The way I do that is to do an exchange_declare with passive=True. Now I
> >> know that when an exception/error occurs (e.g. a 404 NOT_FOUND) if the
> >> exchange doesn't exist then the channel is closed by the broker. To
> >> cater for this I create a temporary channel (in this case it would be
> >> channel :2) and call the exchange_declare passive=True using that
> >> channel and I get a pika.exceptions.ChannelClosedByBroker, which is what
> >> I'd expect.
> >>
> >> So that's how it is with RabbitMQ broker, but with Qpid Java subsequent
> >> things were failing and I narrowed it down to the connection having been
> >> closed in addition to the channel.
> >>
> >> I've tried creating an additional temporary *connection* in addition to
> >> the temporary channel that I was previously creating in order to run the
> >> passive=True exchange_declare and doing that seems to "work" in terms of
> >> the rest of the application now seems to carry on correctly.
> >>
> >>
> >> So really I'm curious, is the Qpid Java broker closing the connection in
> >> addition to closing the channel the expected behaviour in this scenario?
> >> It is certainly different compared with what RabbitMQ does, but I have
> >> no idea what the "correct" behaviour is in this circumstance.
> >>
> >> MTIA
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> >> For additional commands, e-mail: users-help@qpid.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does Qpid Java broker close connection upon channel exception as
well as closing the channel?
Posted by Fraser Adams <fr...@blueyonder.co.uk.INVALID>.
Thanks Rob, so the bottom line then is that the Qpid Java broker *is*
closing the connection in addition to the channel, right?
If that's the case at least that's reassuring in that it's doing what
it's _supposed_ to be doing (even if that may not be quite "correct"). I
was mainly concerned that I may have been misunderstanding something or
doing something dumb/wrong myself, it was driving me slightly mad over
the weekend . If it's just "a quirk" I can probably live with it.
Another couple of questions:
1. I don't suppose you know of any simple way I could "detect" if I'm
connecting to Qpid vice RabbitMQ from any of the connection/channel
responses. I'm guessing probably not, but it might be nice to avoid
creating that extra connection if there's a cheap and simple way to do
it, but not worth bothering if I need to go through hoops to work out
what I'm connecting to.
2. My (very superficial) initial experiments seemed to indicate that the
throughput of my application was a little slower with the Qpid Java
broker than RabbitMQ. It was a fairly basic producer/consumer example
just using the direct exchange and publishing 5K payloads. In all
honesty I've not dug deeply as I was initially focussed on "just getting
it to work" and for simplicity I just used this docker image
abh1sh3k/apache-qpid. The config was pretty much the default for that
image apart from the "secureOnlyMechanisms" : [ ],. I don't suppose you
have any experience of comparing the two brokers and where one might
outperform the other? The difference wasn't huge, though weirdly my
Python client seemed to be using more CPU with the Java broker.
3. Is there an "official" Docker image for the Java broker? There was
some talk of Qpid on Docker some time ago and Irina has some images
here: https://hub.docker.com/u/irinabov, but those were last updated 4
years ago and relate to the c++ broker and dispatch.
Thanks again for the reply!
F.
On 12/07/2021 14:48, Rob Godfrey wrote:
> So, according to the 0-9-1 definition (
> https://www.amqp.org/specification/0-9-1/amqp-org-download), 404 is
> supposed to be a channel error - so RabbitMQ is likely more correct than
> the Java Broker. IIRC there were a whole boatload of issues with clients
> (particularly the old JMS client) trying to handle channel level
> exceptions, so the decision was made to just treat everything as a
> connection level exception. Others may have a more accurate/detailed
> memory.
>
> -- Rob
>
> On Mon, 12 Jul 2021 at 14:31, Fraser Adams
> <fr...@blueyonder.co.uk.invalid> wrote:
>
>> For context I'm working on a project that is primarily using the
>> RabbitMQ broker and using the Pika Python client library. As the Qpid
>> Java broker supports AMQP 0.9.1 too I figured I'd try to connect to that
>> too to see if I could compare relative performance.
>>
>> I ran into a couple of issues, the first is that by default the Java
>> broker doesn't support PLAIN authentication on insecure transports so I
>> needed to mod the config.json to add
>>
>> "secureOnlyMechanisms" : [ ],
>>
>> to "authenticationproviders"
>>
>> I thought/hoped that'd be enough, but unfortunately things kept failing
>> with a 504 error complaining about channel :2 not existing
>>
>>
>> As it happens, what I'm actually doing during the startup of my
>> application is to check if an exchange with the name of the specified
>> destination exists.
>>
>> The way I do that is to do an exchange_declare with passive=True. Now I
>> know that when an exception/error occurs (e.g. a 404 NOT_FOUND) if the
>> exchange doesn't exist then the channel is closed by the broker. To
>> cater for this I create a temporary channel (in this case it would be
>> channel :2) and call the exchange_declare passive=True using that
>> channel and I get a pika.exceptions.ChannelClosedByBroker, which is what
>> I'd expect.
>>
>> So that's how it is with RabbitMQ broker, but with Qpid Java subsequent
>> things were failing and I narrowed it down to the connection having been
>> closed in addition to the channel.
>>
>> I've tried creating an additional temporary *connection* in addition to
>> the temporary channel that I was previously creating in order to run the
>> passive=True exchange_declare and doing that seems to "work" in terms of
>> the rest of the application now seems to carry on correctly.
>>
>>
>> So really I'm curious, is the Qpid Java broker closing the connection in
>> addition to closing the channel the expected behaviour in this scenario?
>> It is certainly different compared with what RabbitMQ does, but I have
>> no idea what the "correct" behaviour is in this circumstance.
>>
>> MTIA
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Does Qpid Java broker close connection upon channel exception as
well as closing the channel?
Posted by Rob Godfrey <ro...@gmail.com>.
So, according to the 0-9-1 definition (
https://www.amqp.org/specification/0-9-1/amqp-org-download), 404 is
supposed to be a channel error - so RabbitMQ is likely more correct than
the Java Broker. IIRC there were a whole boatload of issues with clients
(particularly the old JMS client) trying to handle channel level
exceptions, so the decision was made to just treat everything as a
connection level exception. Others may have a more accurate/detailed
memory.
-- Rob
On Mon, 12 Jul 2021 at 14:31, Fraser Adams
<fr...@blueyonder.co.uk.invalid> wrote:
> For context I'm working on a project that is primarily using the
> RabbitMQ broker and using the Pika Python client library. As the Qpid
> Java broker supports AMQP 0.9.1 too I figured I'd try to connect to that
> too to see if I could compare relative performance.
>
> I ran into a couple of issues, the first is that by default the Java
> broker doesn't support PLAIN authentication on insecure transports so I
> needed to mod the config.json to add
>
> "secureOnlyMechanisms" : [ ],
>
> to "authenticationproviders"
>
> I thought/hoped that'd be enough, but unfortunately things kept failing
> with a 504 error complaining about channel :2 not existing
>
>
> As it happens, what I'm actually doing during the startup of my
> application is to check if an exchange with the name of the specified
> destination exists.
>
> The way I do that is to do an exchange_declare with passive=True. Now I
> know that when an exception/error occurs (e.g. a 404 NOT_FOUND) if the
> exchange doesn't exist then the channel is closed by the broker. To
> cater for this I create a temporary channel (in this case it would be
> channel :2) and call the exchange_declare passive=True using that
> channel and I get a pika.exceptions.ChannelClosedByBroker, which is what
> I'd expect.
>
> So that's how it is with RabbitMQ broker, but with Qpid Java subsequent
> things were failing and I narrowed it down to the connection having been
> closed in addition to the channel.
>
> I've tried creating an additional temporary *connection* in addition to
> the temporary channel that I was previously creating in order to run the
> passive=True exchange_declare and doing that seems to "work" in terms of
> the rest of the application now seems to carry on correctly.
>
>
> So really I'm curious, is the Qpid Java broker closing the connection in
> addition to closing the channel the expected behaviour in this scenario?
> It is certainly different compared with what RabbitMQ does, but I have
> no idea what the "correct" behaviour is in this circumstance.
>
> MTIA
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>