You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Jakub Scholz <ja...@scholz.cz> on 2013/08/30 12:30:57 UTC
Queue level message sequencing doesn't work with 1.0 client
Hi,
The JIRA QPID-4591 added the queue level sequencing. I played with it with
the 0.24RC3 release and it seems it doesn't work properly with AMQP 1.0
producers (C++ / Qpid Messaging API).
My queue is configured with following option:
u'qpid.queue_msg_sequence': u'queue.sequence'
The connection which sends a message into this queue seems to be closed due
to the amqp:internal-error:
[0xda1200:0] <- CLOSE @24 [@29 [:"amqp:internal-error", "Expected
descriptor of type ulong or symbol; found ", null]]
[0xda1200:0] <- EOS
2013-08-30 13:51:18 [Network] debug localhost:20000 decoded 107 bytes from
107
Exception: Connection closed by peer
2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
TcpTransport closing...
2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
Socket closed
2013-08-30 13:51:18 [Messaging] debug Driver stopped
And the broker log seems to complain about the type:
2013-08-30 13:51:18 [Broker] warning Exchange response cannot deliver to
queue response.ABCFR_ABCFRALMMACC1.b3458152-e106-42a6-9c8c-4984c08cff81:
Expected descriptor of type ulong or symbol; found
2013-08-30 13:51:18 [Broker] error qpid.127.0.0.1:20000-127.0.0.1:39092:
Expected descriptor of type ulong or symbol; found
Is this a known error or should I enter a JIRA for that?
Thanks & Regards
Jakub
Re: [c++] message annotations (was Re: Queue level message sequencing
doesn't work with 1.0 client)
Posted by Alan Conway <ac...@redhat.com>.
On 09/04/2013 11:03 AM, Gordon Sim wrote:
> On 09/04/2013 03:34 PM, Gordon Sim wrote:
>> On 09/04/2013 03:14 PM, Alan Conway wrote:
>>> Having a connection option that changes how messages are presented seems
>>> confusing and error prone.
>>
>> I'm not sure I agree. I don't really see a problem with the behaviour
>> being configurable. And if it is, doing so at the granularity of
>> connection seems reasonable to me.
>
> An alternative to being configurable would be to add a distinct way of accessing
> annotations, accessing application properties and accessing the combined set.
>
> E.g. we could add getAnnotations() and getApplicationProperties() alongside the
> existing getProperties() (the existing method would be the combination of
> annotations and application properties).
>
> That way in switching from 0-10 (which doesn't make a distinction) to 1.0 (which
> does), the getProperties() works the same, but if needed there are ways to treat
> them as distinct.
>
> What do you think? I think I prefer that.
>
>
+1
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [c++] message annotations (was Re: Queue level message sequencing
doesn't work with 1.0 client)
Posted by Gordon Sim <gs...@redhat.com>.
On 09/04/2013 06:06 PM, Andrew Stitcher wrote:
> I think the fundamental question here is what is the API model of
> message properties: Until now the API has been loosely modeled (for
> better or worse) on JMS - JMS has no distinction as to different kinds
> of message properties (as far as I know). Adding in different kinds of
> message properties may possibly make this API too focussed on amqp 1.0
> if we ever want to attach different protocols to it. However if we've
> taken a decision that it's only meant to represent amqp 1.0 then I think
> this is fine.
I don't think its only meant to represent AMQP 1.0, but I do think it
should be possible to express the distinction should be in some way when
using a protocol where the distinction is relevant (e.g. AMQP 1.0).
So as it stands, the connection-option is only available on AMQP 1.0,
and if the distinction is important to an application they turn it on
and the annotations are exposed as a sub-map.
With the approach of adding extra accessors, over a protocol that didn't
support annotations as distinct from application properties, the
properties map and application properties map would always be identical
and the annotations map would always be empty.
> I was under the impression that one of the original design goals of the
> qpid messaging API was not to be overly protocol specific in the API and
> to add protocol specific nuances via configuration options/text strings.
>
> Given that, I like the idea that normal applications, when reading
> properties at least, don't have to known what kind of property is being
> read.
Agreed, and in both the current code and the suggested extended API,
that would remain the case.
> However if setting properties on a message this may need to be
> more explicit. Also perhaps there should be a "property property" which
> can tell you the kind of property you read.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [c++] message annotations (was Re: Queue level message
sequencing doesn't work with 1.0 client)
Posted by Andrew Stitcher <as...@redhat.com>.
On Wed, 2013-09-04 at 16:03 +0100, Gordon Sim wrote:
> On 09/04/2013 03:34 PM, Gordon Sim wrote:
> > On 09/04/2013 03:14 PM, Alan Conway wrote:
> >> Having a connection option that changes how messages are presented seems
> >> confusing and error prone.
> >
> > I'm not sure I agree. I don't really see a problem with the behaviour
> > being configurable. And if it is, doing so at the granularity of
> > connection seems reasonable to me.
>
> An alternative to being configurable would be to add a distinct way of
> accessing annotations, accessing application properties and accessing
> the combined set.
>
> E.g. we could add getAnnotations() and getApplicationProperties()
> alongside the existing getProperties() (the existing method would be the
> combination of annotations and application properties).
>
> That way in switching from 0-10 (which doesn't make a distinction) to
> 1.0 (which does), the getProperties() works the same, but if needed
> there are ways to treat them as distinct.
>
> What do you think? I think I prefer that.
I think the fundamental question here is what is the API model of
message properties: Until now the API has been loosely modeled (for
better or worse) on JMS - JMS has no distinction as to different kinds
of message properties (as far as I know). Adding in different kinds of
message properties may possibly make this API too focussed on amqp 1.0
if we ever want to attach different protocols to it. However if we've
taken a decision that it's only meant to represent amqp 1.0 then I think
this is fine.
I was under the impression that one of the original design goals of the
qpid messaging API was not to be overly protocol specific in the API and
to add protocol specific nuances via configuration options/text strings.
Given that, I like the idea that normal applications, when reading
properties at least, don't have to known what kind of property is being
read. However if setting properties on a message this may need to be
more explicit. Also perhaps there should be a "property property" which
can tell you the kind of property you read.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [c++] message annotations (was Re: Queue level message sequencing
doesn't work with 1.0 client)
Posted by Gordon Sim <gs...@redhat.com>.
On 09/04/2013 03:34 PM, Gordon Sim wrote:
> On 09/04/2013 03:14 PM, Alan Conway wrote:
>> Having a connection option that changes how messages are presented seems
>> confusing and error prone.
>
> I'm not sure I agree. I don't really see a problem with the behaviour
> being configurable. And if it is, doing so at the granularity of
> connection seems reasonable to me.
An alternative to being configurable would be to add a distinct way of
accessing annotations, accessing application properties and accessing
the combined set.
E.g. we could add getAnnotations() and getApplicationProperties()
alongside the existing getProperties() (the existing method would be the
combination of annotations and application properties).
That way in switching from 0-10 (which doesn't make a distinction) to
1.0 (which does), the getProperties() works the same, but if needed
there are ways to treat them as distinct.
What do you think? I think I prefer that.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [c++] message annotations (was Re: Queue level message sequencing
doesn't work with 1.0 client)
Posted by Gordon Sim <gs...@redhat.com>.
On 09/04/2013 03:14 PM, Alan Conway wrote:
> Having a connection option that changes how messages are presented seems
> confusing and error prone.
I'm not sure I agree. I don't really see a problem with the behaviour
being configurable. And if it is, doing so at the granularity of
connection seems reasonable to me.
Can you elaborate on your concerns a little?
> Is it possible to present both to the client thus avoiding the need for
> a configuration option? I.e. put annotations in the 1.0 nested maps
> _and_ in message properties,
Yes, it's possible, but the point of having them nested is so that they
aren't intermingled with application properties.
If the application cared about the distinction they would then have to
go through the properties, compare them against the nested annotations
and assume that annotation names don't clash with application property
names.
> and fold the 2 together to avoid duplicates
> when sending messages?
When sending only the nested annotations will be sent as annotations. If
these were also included as top-level properties, the library would then
have to exclude any top level property with the same name as an annotation.
> That way old & new code would both work without a
> config option.
To be clear, up until this point annotations haven't worked over 1.0, so
I'm pretty sure there is no code in existence at present that relies on
them being nested.
The reason for supporting both options is not to handle legacy concerns
but to cater for two different use cases. One where the application
doesn't care about the distinction, one where it does.
To me, the combined option satisfies neither particularly well.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [c++] message annotations (was Re: Queue level message sequencing
doesn't work with 1.0 client)
Posted by Alan Conway <ac...@redhat.com>.
On 09/02/2013 12:26 PM, Gordon Sim wrote:
> On 08/30/2013 11:30 AM, Jakub Scholz wrote:
>> Hi,
>>
>> The JIRA QPID-4591 added the queue level sequencing. I played with it with
>> the 0.24RC3 release and it seems it doesn't work properly with AMQP 1.0
>> producers (C++ / Qpid Messaging API).
>>
>> My queue is configured with following option:
>> u'qpid.queue_msg_sequence': u'queue.sequence'
>>
>> The connection which sends a message into this queue seems to be closed due
>> to the amqp:internal-error:
>> [0xda1200:0] <- CLOSE @24 [@29 [:"amqp:internal-error", "Expected
>> descriptor of type ulong or symbol; found ", null]]
>> [0xda1200:0] <- EOS
>> 2013-08-30 13:51:18 [Network] debug localhost:20000 decoded 107 bytes from
>> 107
>> Exception: Connection closed by peer
>> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
>> TcpTransport closing...
>> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
>> Socket closed
>> 2013-08-30 13:51:18 [Messaging] debug Driver stopped
>>
>> And the broker log seems to complain about the type:
>> 2013-08-30 13:51:18 [Broker] warning Exchange response cannot deliver to
>> queue response.ABCFR_ABCFRALMMACC1.b3458152-e106-42a6-9c8c-4984c08cff81:
>> Expected descriptor of type ulong or symbol; found
>> 2013-08-30 13:51:18 [Broker] error qpid.127.0.0.1:20000-127.0.0.1:39092:
>> Expected descriptor of type ulong or symbol; found
>>
>> Is this a known error or should I enter a JIRA for that?
>
> This was a bug in the broker's handling of annotations. I have now fixed that, I
> have also made a slight change to the way 1.0 messages are presented through the
> qpid::messaging API.
>
> In AMQP 0-10 any attributes the broker wanted to add to a message had to go in
> the application properties. In AMQP 1.0 a distinction is made between
> annotations (which intermediaries can add to) and the application properties
> (which are set by the publishing application and can't be modified by
> intermediaries). (This makes it clear what the original message was, and what
> was added in transit, useful e.g. for message signing).
>
> Previously when using AMQP 1.0 with qpid::messaging any annotations have been
> made available through a nested property x-amqp-message-annotations or
> x-amqp-delivery-annotations respectively.
>
> However in a case like the one you mention above, this would force the
> application logic to be slightly different when using 1.0 or 0-10, which is
> something we are keen to avoid where possible.
>
> So now, this behaviour is configurable through the nest_annotations connection
> option. If true, then the behaviour is as it was previously, i.e. the
> annotations are held in nested maps making it clear how they were encoded when
> received. If false, annotations are just added to the message properties by the
> receiving client (note this doesn't affect how the broker encodes messages with
> annotations, just how the messaging API presents the data).
>
Having a connection option that changes how messages are presented seems
confusing and error prone.
Is it possible to present both to the client thus avoiding the need for a
configuration option? I.e. put annotations in the 1.0 nested maps _and_ in
message properties, and fold the 2 together to avoid duplicates when sending
messages? That way old & new code would both work without a config option.
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
[c++] message annotations (was Re: Queue level message sequencing
doesn't work with 1.0 client)
Posted by Gordon Sim <gs...@redhat.com>.
On 08/30/2013 11:30 AM, Jakub Scholz wrote:
> Hi,
>
> The JIRA QPID-4591 added the queue level sequencing. I played with it with
> the 0.24RC3 release and it seems it doesn't work properly with AMQP 1.0
> producers (C++ / Qpid Messaging API).
>
> My queue is configured with following option:
> u'qpid.queue_msg_sequence': u'queue.sequence'
>
> The connection which sends a message into this queue seems to be closed due
> to the amqp:internal-error:
> [0xda1200:0] <- CLOSE @24 [@29 [:"amqp:internal-error", "Expected
> descriptor of type ulong or symbol; found ", null]]
> [0xda1200:0] <- EOS
> 2013-08-30 13:51:18 [Network] debug localhost:20000 decoded 107 bytes from
> 107
> Exception: Connection closed by peer
> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
> TcpTransport closing...
> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
> Socket closed
> 2013-08-30 13:51:18 [Messaging] debug Driver stopped
>
> And the broker log seems to complain about the type:
> 2013-08-30 13:51:18 [Broker] warning Exchange response cannot deliver to
> queue response.ABCFR_ABCFRALMMACC1.b3458152-e106-42a6-9c8c-4984c08cff81:
> Expected descriptor of type ulong or symbol; found
> 2013-08-30 13:51:18 [Broker] error qpid.127.0.0.1:20000-127.0.0.1:39092:
> Expected descriptor of type ulong or symbol; found
>
> Is this a known error or should I enter a JIRA for that?
This was a bug in the broker's handling of annotations. I have now fixed
that, I have also made a slight change to the way 1.0 messages are
presented through the qpid::messaging API.
In AMQP 0-10 any attributes the broker wanted to add to a message had to
go in the application properties. In AMQP 1.0 a distinction is made
between annotations (which intermediaries can add to) and the
application properties (which are set by the publishing application and
can't be modified by intermediaries). (This makes it clear what the
original message was, and what was added in transit, useful e.g. for
message signing).
Previously when using AMQP 1.0 with qpid::messaging any annotations have
been made available through a nested property
x-amqp-message-annotations or x-amqp-delivery-annotations respectively.
However in a case like the one you mention above, this would force the
application logic to be slightly different when using 1.0 or 0-10, which
is something we are keen to avoid where possible.
So now, this behaviour is configurable through the nest_annotations
connection option. If true, then the behaviour is as it was previously,
i.e. the annotations are held in nested maps making it clear how they
were encoded when received. If false, annotations are just added to the
message properties by the receiving client (note this doesn't affect how
the broker encodes messages with annotations, just how the messaging API
presents the data).
The default is currently false. I.e. by default no distinction will be
made at the API level between application properties and annotations,
meaning that the behaviour, though different from the past couple of
releases as far as 1.0 support goes, is similar to 0-10.
If anyone feels the default is wrong or there is a better way to handle
this I'd be eager as always to hear from you and we can improve on what
is now checked in.
--Gordon.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Queue level message sequencing doesn't work with 1.0 client
Posted by Jakub Scholz <ja...@scholz.cz>.
Hi Gordon,
I raised it as QPID-5106. Thanks for having a look at it.
Thanks & Regards
Jakub
On Fri, Aug 30, 2013 at 12:47 PM, Gordon Sim <gs...@redhat.com> wrote:
> On 08/30/2013 11:30 AM, Jakub Scholz wrote:
>
>> Hi,
>>
>> The JIRA QPID-4591 added the queue level sequencing. I played with it with
>> the 0.24RC3 release and it seems it doesn't work properly with AMQP 1.0
>> producers (C++ / Qpid Messaging API).
>>
>> My queue is configured with following option:
>> u'qpid.queue_msg_sequence': u'queue.sequence'
>>
>> The connection which sends a message into this queue seems to be closed
>> due
>> to the amqp:internal-error:
>> [0xda1200:0] <- CLOSE @24 [@29 [:"amqp:internal-error", "Expected
>> descriptor of type ulong or symbol; found ", null]]
>> [0xda1200:0] <- EOS
>> 2013-08-30 13:51:18 [Network] debug localhost:20000 decoded 107 bytes from
>> 107
>> Exception: Connection closed by peer
>> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:**20000]
>> TcpTransport closing...
>> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:**20000]
>> Socket closed
>> 2013-08-30 13:51:18 [Messaging] debug Driver stopped
>>
>> And the broker log seems to complain about the type:
>> 2013-08-30 13:51:18 [Broker] warning Exchange response cannot deliver to
>> queue response.ABCFR_ABCFRALMMACC1.**b3458152-e106-42a6-9c8c-**
>> 4984c08cff81:
>> Expected descriptor of type ulong or symbol; found
>> 2013-08-30 13:51:18 [Broker] error qpid.127.0.0.1:20000-127.0.0.**
>> 1:39092:
>> Expected descriptor of type ulong or symbol; found
>>
>> Is this a known error or should I enter a JIRA for that?
>>
>
> A JIRA would be great, thanks! I'll have a look into it.
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.**org<us...@qpid.apache.org>
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Queue level message sequencing doesn't work with 1.0 client
Posted by Gordon Sim <gs...@redhat.com>.
On 08/30/2013 11:30 AM, Jakub Scholz wrote:
> Hi,
>
> The JIRA QPID-4591 added the queue level sequencing. I played with it with
> the 0.24RC3 release and it seems it doesn't work properly with AMQP 1.0
> producers (C++ / Qpid Messaging API).
>
> My queue is configured with following option:
> u'qpid.queue_msg_sequence': u'queue.sequence'
>
> The connection which sends a message into this queue seems to be closed due
> to the amqp:internal-error:
> [0xda1200:0] <- CLOSE @24 [@29 [:"amqp:internal-error", "Expected
> descriptor of type ulong or symbol; found ", null]]
> [0xda1200:0] <- EOS
> 2013-08-30 13:51:18 [Network] debug localhost:20000 decoded 107 bytes from
> 107
> Exception: Connection closed by peer
> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
> TcpTransport closing...
> 2013-08-30 13:51:18 [Messaging] debug [127.0.0.1:39092-localhost:20000]
> Socket closed
> 2013-08-30 13:51:18 [Messaging] debug Driver stopped
>
> And the broker log seems to complain about the type:
> 2013-08-30 13:51:18 [Broker] warning Exchange response cannot deliver to
> queue response.ABCFR_ABCFRALMMACC1.b3458152-e106-42a6-9c8c-4984c08cff81:
> Expected descriptor of type ulong or symbol; found
> 2013-08-30 13:51:18 [Broker] error qpid.127.0.0.1:20000-127.0.0.1:39092:
> Expected descriptor of type ulong or symbol; found
>
> Is this a known error or should I enter a JIRA for that?
A JIRA would be great, thanks! I'll have a look into it.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org