You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Ken Giusti <kg...@redhat.com> on 2011/11/15 19:43:07 UTC
Question re: Java broker's message selector implementation...
Hi folks,
I'm working on adding support for server-side message selectors to the C++ Broker.
See https://issues.apache.org/jira/browse/QPID-530 (better late than never...)
It would be preferable to have the C++ broker's use model for message selectors be consistent with the Java Broker's implementation - and I believe the Java Broker already implements Message Selectors (correct?)
I was planning on implementing the feature's behavior so it will be compliant with the description of Message Selections as per the JMS 1.1 spec. Is this the model for message selectors that is supported by the Java Broker? If not, is there documentation available that describes its implementation?
thanks,
-K
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Question re: Java broker's message selector implementation...
Posted by Rob Godfrey <ro...@gmail.com>.
On 17 November 2011 15:28, Ken Giusti <kg...@redhat.com> wrote:
> Hi Rob, thanks for the info - very helpful. I'll write up a more formal
> description of the proposed functionality shortly - give us something a
> little more tangible to review.
>
> One question specific to the Java broker's behavior - an edge-case I'm
> trying to wrap my head around:
>
> With selectors, I'm planning on associating a current position with each
> consumer - just as you described below. This is somewhat new to the C++
> broker, as currently all consumers consume from the queue's head. A
> position attribute adds a new wrinkle to the broker's handling of released
> messages:
>
> Does the Java broker reset/rewind a consumer's position when an acquired
> message is released back to the queue? For example, assume a queue with
> multiple subscribers consuming from it in parallel, with the potential that
> their selector match criteria overlaps. If a consumer acquires a message,
> then at some later point releases it, does the Java broker 'back up' the
> other consumers that may have skipped over that message due to it being
> acquired?
>
Essentially, yes - when a message is released into the queue we iterate
over the subscriptions to ensure that none of them is "ahead" of where the
released message has been returned to. There are a number of wrinkles in
the Java code here due to the way we implement the queues. It gets even
more hairy where your queue ordering is not strictly FIFO (for Priority
Queues, or the Sorted Queue implementation that we've also developed).
-- Rob
Re: Question re: Java broker's message selector implementation...
Posted by Ken Giusti <kg...@redhat.com>.
Hi Rob, thanks for the info - very helpful. I'll write up a more formal description of the proposed functionality shortly - give us something a little more tangible to review.
One question specific to the Java broker's behavior - an edge-case I'm trying to wrap my head around:
With selectors, I'm planning on associating a current position with each consumer - just as you described below. This is somewhat new to the C++ broker, as currently all consumers consume from the queue's head. A position attribute adds a new wrinkle to the broker's handling of released messages:
Does the Java broker reset/rewind a consumer's position when an acquired message is released back to the queue? For example, assume a queue with multiple subscribers consuming from it in parallel, with the potential that their selector match criteria overlaps. If a consumer acquires a message, then at some later point releases it, does the Java broker 'back up' the other consumers that may have skipped over that message due to it being acquired?
thanks,
-K
----- Original Message -----
> Hi Ken,
>
> On 15 November 2011 19:43, Ken Giusti <kg...@redhat.com> wrote:
>
> > Hi folks,
> >
> > I'm working on adding support for server-side message selectors to
> > the C++
> > Broker.
> >
> > See https://issues.apache.org/jira/browse/QPID-530 (better late
> > than
> > never...)
> >
> >
> Excellent... not having to implement selectors on the client side
> will be a
> real win!
>
>
> > It would be preferable to have the C++ broker's use model for
> > message
> > selectors be consistent with the Java Broker's implementation - and
> > I
> > believe the Java Broker already implements Message Selectors
> > (correct?)
> >
> >
> The Java Broker does indeed implement message selectors
>
>
> > I was planning on implementing the feature's behavior so it will be
> > compliant with the description of Message Selections as per the JMS
> > 1.1
> > spec. Is this the model for message selectors that is supported by
> > the
> > Java Broker? If not, is there documentation available that
> > describes its
> > implementation?
> >
> >
> I'm not sure if there is any documentation on message selectors and
> their
> implementation. One thing to be aware of is that selectors probably
> need
> to be implemented both on the queue consumers and (because of the way
> that
> AMQP models topics) on the bindings between exchanges and queues.
>
> The bindings model is easier since you are just implementing an
> additional
> filter on which messages are routed to a given queue through the
> exchange,
> and the JMS Selector is simply another filter to evaluate in
> conjunction
> with the other tests associated with a given binding.
>
> For the queue consumer implementation, this basically depends on the
> queue
> implementation. The way that the Java Broker works internally, each
> consumer keeps a pointer into the queue, the pointer is advanced when
> the
> consumer has capacity to receive a message, and the message at the
> pointer
> is evaluated against the criteria of the consumer (e.g. the message
> selector, whether the consumer is "no-local" or not).
>
> IIRC the selector is passed to the consumer as an argument called
> "x-filter-jms-selector" where the value is a selector in the defined
> JMS
> selector syntax (the same name is used to pass the selector when
> binding a
> queue to an exchange for topic style selectors).
>
> Also note that I believe there has recently been some work done to
> identify
> on connection whether a broker supports selectors or not, so that the
> Java
> client can determine whether it needs to use the client-side
> selectors or
> can utilise the server side selectors (
> https://issues.apache.org/jira/browse/QPID-3518) - if you implement
> selectors in the C++ broker you should advertise this through the
> connection properties in the same way.
>
> Cheers,
> Rob
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: Question re: Java broker's message selector implementation...
Posted by Robert Godfrey <rg...@apache.org>.
Hi Ken,
On 15 November 2011 19:43, Ken Giusti <kg...@redhat.com> wrote:
> Hi folks,
>
> I'm working on adding support for server-side message selectors to the C++
> Broker.
>
> See https://issues.apache.org/jira/browse/QPID-530 (better late than
> never...)
>
>
Excellent... not having to implement selectors on the client side will be a
real win!
> It would be preferable to have the C++ broker's use model for message
> selectors be consistent with the Java Broker's implementation - and I
> believe the Java Broker already implements Message Selectors (correct?)
>
>
The Java Broker does indeed implement message selectors
> I was planning on implementing the feature's behavior so it will be
> compliant with the description of Message Selections as per the JMS 1.1
> spec. Is this the model for message selectors that is supported by the
> Java Broker? If not, is there documentation available that describes its
> implementation?
>
>
I'm not sure if there is any documentation on message selectors and their
implementation. One thing to be aware of is that selectors probably need
to be implemented both on the queue consumers and (because of the way that
AMQP models topics) on the bindings between exchanges and queues.
The bindings model is easier since you are just implementing an additional
filter on which messages are routed to a given queue through the exchange,
and the JMS Selector is simply another filter to evaluate in conjunction
with the other tests associated with a given binding.
For the queue consumer implementation, this basically depends on the queue
implementation. The way that the Java Broker works internally, each
consumer keeps a pointer into the queue, the pointer is advanced when the
consumer has capacity to receive a message, and the message at the pointer
is evaluated against the criteria of the consumer (e.g. the message
selector, whether the consumer is "no-local" or not).
IIRC the selector is passed to the consumer as an argument called
"x-filter-jms-selector" where the value is a selector in the defined JMS
selector syntax (the same name is used to pass the selector when binding a
queue to an exchange for topic style selectors).
Also note that I believe there has recently been some work done to identify
on connection whether a broker supports selectors or not, so that the Java
client can determine whether it needs to use the client-side selectors or
can utilise the server side selectors (
https://issues.apache.org/jira/browse/QPID-3518) - if you implement
selectors in the C++ broker you should advertise this through the
connection properties in the same way.
Cheers,
Rob