You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Pavel Moravec <pm...@redhat.com> on 2012/04/12 15:53:52 UTC
Selective message acknowledgment in Java client (and C++ broker)?
Hello,
having Java client and C++ broker, I want to implement selective consumption of messages from a queue (something like JMS message selectors in Java broker). I.e.:
1) Having CLIENT_ACKNOWLEDGE mode,
2) In a loop through all messages of the queue:
a) receive a message
b) decide if to consume or not
c) if so, acknowledge it, if not so, ignore it
3) Close the session to retrieve the unacknowledged messages back to the queue.
However, I see no way of doing so.
Method Message.acknowledge(); is not suitable as it acknowledges also all previous messages, following JMS specification.
Using method AbstractJMSMessage.acknowledgeThis(), I got the same results. Though the method name would suggest, it shall acknowledge just this message, no else. But checking source code:
public void acknowledgeThis() throws JMSException
{
// the JMS 1.1 spec says in section 3.6 that calls to acknowledge are ignored when client acknowledge
// is not specified. In our case, we only set the session field where client acknowledge mode is specified.
if (_session != null && _session.getAcknowledgeMode() == Session.CLIENT_ACKNOWLEDGE)
{
if (_session.getAMQConnection().isClosed())
{
throw new javax.jms.IllegalStateException("Connection is already closed");
}
// we set multiple to true here since acknowledgment implies acknowledge of all previous messages
// received on the session
_session.acknowledgeMessage(_deliveryTag, true);
}
}
See the latest comment - it acknowledges all previous messages on purpose. Having the last command:
_session.acknowledgeMessage(_deliveryTag, false);
only this message will be acknowledged, no else.
Is there some reason for this behavior? If so, how can I achieve a real single message acknowledgment?
Thanks in advance for your thoughs.
Kind regards,
Pavel
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Selective message acknowledgment in Java client (and C++ broker)?
Posted by Rajith Attapattu <ra...@gmail.com>.
I'm currently on vacation and has very limited access to email.
Thanks Robbie for picking this up.
Rajith
On Sat, Apr 14, 2012 at 11:44 AM, Robbie Gemmell
<ro...@gmail.com>wrote:
> You probably want to try the 0.16 RCs or trunk if you are interested
> in using the client-side selector support (available when connecting
> to AMQP 0-10 brokers) of the Java client, as several related changes
> were made to that area since Qpid 0.14 (in order to improve
> efficiency/correctness when connecting to the Java broker).
>
> Although the Java broker performs server-side selection, the clients
> AMQP 0-10 support was originally only used with the C++ broker which
> currently does not. As a result the client was previously still
> enabling its own selection support when connected to the Java broker
> using AMQP 0-10, which made it incurr unecessary overheads as a
> result. When addressing this by making the client able to disable the
> client-side selection when connected to the Java broker, the
> implementation in this area was cleaned up in general and multiple
> defects with the client-side selection support were addressed in the
> process. These likely play right into what you have observed
> previously, so its worth testing with an up to date client version if
> you havent already. If you still have issues when you do, raise a JIRA
> and attach your reproducer.
>
> 0.16 RC2:
> http://people.apache.org/~jross/qpid-0.16-rc2/
>
> Trunk nightly client build:
>
> https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Artefact-Release/lastSuccessfulBuild/artifact/trunk/qpid/java/client/release/
>
> Robbie
>
> On 14 April 2012 14:41, Fraser Adams <fr...@blueyonder.co.uk>
> wrote:
> > Hi Pavel/Jakub/Rajith
> > I posted ages back (March 2011) that I didn't believe that JMS Message
> > Selectors were behaving correctly:
> >
> http://qpid.2158936.n2.nabble.com/JMS-Message-Selectors-Behaving-Strangely-td6158445.html
> >
> > My observations concur with the observations of Pavel and Jakob namely
> I'd
> > expect any unselected messages to remain on the queue, but not the
> messages
> > that matched the selector.
> >
> > For info guys one approach that just may be useful is that the "internal"
> > filter class is fairly easily accessibly so I had code that looks like.
> >
> > ...
> > import org.apache.qpid.filter.JMSSelectorFilter;
> > import org.apache.qpid.client.message.AbstractJMSMessage;
> > ...
> >
> > private JMSSelectorFilter filter;
> > ....
> > // Then roundabouts where you'd create a consumer using a message
> selector
> > do..
> > // Clearly just an example but this selects a header called amqp-delivery
> > with the value fadams
> > // any selector could be used.
> > filter = new JMSSelectorFilter("amqp-delivery = 'fadams'");
> > .....
> > // Later in say onMessage(Message message)
> > ....
> > if (filter.matches((AbstractJMSMessage)message)) {
> > // Logic for matching message.
> >
> > }
> > ...
> >
> > Clearly the code above isn't ideal as it isn't pure JMS and also it uses
> > internal classes that could change, but it's the closest approximation
> that
> > I could get to using message selectors that gave me some finer control.
> >
> >
> > Hope this helps you guys. I'd really like to see selectors behaving in a
> way
> > that acknowledges selected messages and leaves unselected messages on the
> > queue, this is what I'd intuitively expect to happen.
> >
> > If that's not possible at the least it would be nice to have the
> > JMSSelectorFilter exposed as an "official" Qpid API so it could be used
> > without fear of it suddenly disappearing.
> >
> > I'd be interested in thoughts on this, I've been meaning to bring it up
> > again but have been distracted on other things.
> >
> > Frase
> >
> >
> >
> > On 13/04/12 08:55, Jakub Scholz wrote:
> >>
> >> Hi Rajith,
> >>
> >> I was playing with the JMS selectors last week. While they do work and
> >> really select the messages based on the filter, it seemed to me that
> >> it switched off the acknowledgments completely. The Java application
> >> was getting only the selected messages, but I was unable to
> >> acknowledge them and they stayed in the queues. This may be OK in some
> >> situations, but in my case I wanted to get the messages really
> >> acknowledged and removed from the queues on the broker.
> >>
> >> Then I considered to try the same trick as Pavel did - read all
> >> messages and filter them manually. But unlike in the C++ API, I didn't
> >> found any methods for acknowledging specific message as well as for
> >> releasing or rejecting a message.
> >>
> >> Regards
> >> Jakub
> >>
> >> On Fri, Apr 13, 2012 at 07:43, Rajith Attapattu<ra...@gmail.com>
> >> wrote:
> >>>
> >>> (Note for C++ broker we do client side selectors, which does exactly
> what
> >>> you have described).
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> >> For additional commands, e-mail: users-help@qpid.apache.org
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Selective message acknowledgment in Java client (and C++ broker)?
Posted by Robbie Gemmell <ro...@gmail.com>.
You probably want to try the 0.16 RCs or trunk if you are interested
in using the client-side selector support (available when connecting
to AMQP 0-10 brokers) of the Java client, as several related changes
were made to that area since Qpid 0.14 (in order to improve
efficiency/correctness when connecting to the Java broker).
Although the Java broker performs server-side selection, the clients
AMQP 0-10 support was originally only used with the C++ broker which
currently does not. As a result the client was previously still
enabling its own selection support when connected to the Java broker
using AMQP 0-10, which made it incurr unecessary overheads as a
result. When addressing this by making the client able to disable the
client-side selection when connected to the Java broker, the
implementation in this area was cleaned up in general and multiple
defects with the client-side selection support were addressed in the
process. These likely play right into what you have observed
previously, so its worth testing with an up to date client version if
you havent already. If you still have issues when you do, raise a JIRA
and attach your reproducer.
0.16 RC2:
http://people.apache.org/~jross/qpid-0.16-rc2/
Trunk nightly client build:
https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Artefact-Release/lastSuccessfulBuild/artifact/trunk/qpid/java/client/release/
Robbie
On 14 April 2012 14:41, Fraser Adams <fr...@blueyonder.co.uk> wrote:
> Hi Pavel/Jakub/Rajith
> I posted ages back (March 2011) that I didn't believe that JMS Message
> Selectors were behaving correctly:
> http://qpid.2158936.n2.nabble.com/JMS-Message-Selectors-Behaving-Strangely-td6158445.html
>
> My observations concur with the observations of Pavel and Jakob namely I'd
> expect any unselected messages to remain on the queue, but not the messages
> that matched the selector.
>
> For info guys one approach that just may be useful is that the "internal"
> filter class is fairly easily accessibly so I had code that looks like.
>
> ...
> import org.apache.qpid.filter.JMSSelectorFilter;
> import org.apache.qpid.client.message.AbstractJMSMessage;
> ...
>
> private JMSSelectorFilter filter;
> ....
> // Then roundabouts where you'd create a consumer using a message selector
> do..
> // Clearly just an example but this selects a header called amqp-delivery
> with the value fadams
> // any selector could be used.
> filter = new JMSSelectorFilter("amqp-delivery = 'fadams'");
> .....
> // Later in say onMessage(Message message)
> ....
> if (filter.matches((AbstractJMSMessage)message)) {
> // Logic for matching message.
>
> }
> ...
>
> Clearly the code above isn't ideal as it isn't pure JMS and also it uses
> internal classes that could change, but it's the closest approximation that
> I could get to using message selectors that gave me some finer control.
>
>
> Hope this helps you guys. I'd really like to see selectors behaving in a way
> that acknowledges selected messages and leaves unselected messages on the
> queue, this is what I'd intuitively expect to happen.
>
> If that's not possible at the least it would be nice to have the
> JMSSelectorFilter exposed as an "official" Qpid API so it could be used
> without fear of it suddenly disappearing.
>
> I'd be interested in thoughts on this, I've been meaning to bring it up
> again but have been distracted on other things.
>
> Frase
>
>
>
> On 13/04/12 08:55, Jakub Scholz wrote:
>>
>> Hi Rajith,
>>
>> I was playing with the JMS selectors last week. While they do work and
>> really select the messages based on the filter, it seemed to me that
>> it switched off the acknowledgments completely. The Java application
>> was getting only the selected messages, but I was unable to
>> acknowledge them and they stayed in the queues. This may be OK in some
>> situations, but in my case I wanted to get the messages really
>> acknowledged and removed from the queues on the broker.
>>
>> Then I considered to try the same trick as Pavel did - read all
>> messages and filter them manually. But unlike in the C++ API, I didn't
>> found any methods for acknowledging specific message as well as for
>> releasing or rejecting a message.
>>
>> Regards
>> Jakub
>>
>> On Fri, Apr 13, 2012 at 07:43, Rajith Attapattu<ra...@gmail.com>
>> wrote:
>>>
>>> (Note for C++ broker we do client side selectors, which does exactly what
>>> you have described).
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Selective message acknowledgment in Java client (and C++ broker)?
Posted by Fraser Adams <fr...@blueyonder.co.uk>.
Hi Pavel/Jakub/Rajith
I posted ages back (March 2011) that I didn't believe that JMS Message
Selectors were behaving correctly:
http://qpid.2158936.n2.nabble.com/JMS-Message-Selectors-Behaving-Strangely-td6158445.html
My observations concur with the observations of Pavel and Jakob namely
I'd expect any unselected messages to remain on the queue, but not the
messages that matched the selector.
For info guys one approach that just may be useful is that the
"internal" filter class is fairly easily accessibly so I had code that
looks like.
...
import org.apache.qpid.filter.JMSSelectorFilter;
import org.apache.qpid.client.message.AbstractJMSMessage;
...
private JMSSelectorFilter filter;
....
// Then roundabouts where you'd create a consumer using a message
selector do..
// Clearly just an example but this selects a header called
amqp-delivery with the value fadams
// any selector could be used.
filter = new JMSSelectorFilter("amqp-delivery = 'fadams'");
.....
// Later in say onMessage(Message message)
....
if (filter.matches((AbstractJMSMessage)message)) {
// Logic for matching message.
}
...
Clearly the code above isn't ideal as it isn't pure JMS and also it uses
internal classes that could change, but it's the closest approximation
that I could get to using message selectors that gave me some finer control.
Hope this helps you guys. I'd really like to see selectors behaving in a
way that acknowledges selected messages and leaves unselected messages
on the queue, this is what I'd intuitively expect to happen.
If that's not possible at the least it would be nice to have the
JMSSelectorFilter exposed as an "official" Qpid API so it could be used
without fear of it suddenly disappearing.
I'd be interested in thoughts on this, I've been meaning to bring it up
again but have been distracted on other things.
Frase
On 13/04/12 08:55, Jakub Scholz wrote:
> Hi Rajith,
>
> I was playing with the JMS selectors last week. While they do work and
> really select the messages based on the filter, it seemed to me that
> it switched off the acknowledgments completely. The Java application
> was getting only the selected messages, but I was unable to
> acknowledge them and they stayed in the queues. This may be OK in some
> situations, but in my case I wanted to get the messages really
> acknowledged and removed from the queues on the broker.
>
> Then I considered to try the same trick as Pavel did - read all
> messages and filter them manually. But unlike in the C++ API, I didn't
> found any methods for acknowledging specific message as well as for
> releasing or rejecting a message.
>
> Regards
> Jakub
>
> On Fri, Apr 13, 2012 at 07:43, Rajith Attapattu<ra...@gmail.com> wrote:
>> (Note for C++ broker we do client side selectors, which does exactly what
>> you have described).
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Selective message acknowledgment in Java client (and C++ broker)?
Posted by Jakub Scholz <ja...@scholz.cz>.
Hi Rajith,
I was playing with the JMS selectors last week. While they do work and
really select the messages based on the filter, it seemed to me that
it switched off the acknowledgments completely. The Java application
was getting only the selected messages, but I was unable to
acknowledge them and they stayed in the queues. This may be OK in some
situations, but in my case I wanted to get the messages really
acknowledged and removed from the queues on the broker.
Then I considered to try the same trick as Pavel did - read all
messages and filter them manually. But unlike in the C++ API, I didn't
found any methods for acknowledging specific message as well as for
releasing or rejecting a message.
Regards
Jakub
On Fri, Apr 13, 2012 at 07:43, Rajith Attapattu <ra...@gmail.com> wrote:
> (Note for C++ broker we do client side selectors, which does exactly what
> you have described).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Selective message acknowledgment in Java client (and C++ broker)?
Posted by Rajith Attapattu <ra...@gmail.com>.
Reading your query again, it seems you want selectors to work not really to
do selective message acking.
So I wondering why you can't use a jms selector ? The JMS client supports
message selectors.
I'm wondering why you need to use a workaround here ?
(Note for C++ broker we do client side selectors, which does exactly what
you have described).
Regards,
Rajith
On Thu, Apr 12, 2012 at 9:53 AM, Pavel Moravec <pm...@redhat.com> wrote:
> Hello,
> having Java client and C++ broker, I want to implement selective
> consumption of messages from a queue (something like JMS message selectors
> in Java broker). I.e.:
>
> 1) Having CLIENT_ACKNOWLEDGE mode,
> 2) In a loop through all messages of the queue:
> a) receive a message
> b) decide if to consume or not
> c) if so, acknowledge it, if not so, ignore it
> 3) Close the session to retrieve the unacknowledged messages back to the
> queue.
>
> However, I see no way of doing so.
>
> Method Message.acknowledge(); is not suitable as it acknowledges also all
> previous messages, following JMS specification.
>
> Using method AbstractJMSMessage.acknowledgeThis(), I got the same results.
> Though the method name would suggest, it shall acknowledge just this
> message, no else. But checking source code:
>
> public void acknowledgeThis() throws JMSException
> {
> // the JMS 1.1 spec says in section 3.6 that calls to acknowledge
> are ignored when client acknowledge
> // is not specified. In our case, we only set the session field
> where client acknowledge mode is specified.
> if (_session != null && _session.getAcknowledgeMode() ==
> Session.CLIENT_ACKNOWLEDGE)
> {
> if (_session.getAMQConnection().isClosed())
> {
> throw new javax.jms.IllegalStateException("Connection is
> already closed");
> }
>
> // we set multiple to true here since acknowledgment implies
> acknowledge of all previous messages
> // received on the session
> _session.acknowledgeMessage(_deliveryTag, true);
> }
> }
>
> See the latest comment - it acknowledges all previous messages on purpose.
> Having the last command:
> _session.acknowledgeMessage(_deliveryTag, false);
>
> only this message will be acknowledged, no else.
>
> Is there some reason for this behavior? If so, how can I achieve a real
> single message acknowledgment?
>
> Thanks in advance for your thoughs.
>
>
> Kind regards,
> Pavel
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Selective message acknowledgment in Java client (and C++ broker)?
Posted by Rajith Attapattu <ra...@gmail.com>.
On Thu, Apr 12, 2012 at 9:53 AM, Pavel Moravec <pm...@redhat.com> wrote:
> Hello,
> having Java client and C++ broker, I want to implement selective
> consumption of messages from a queue (something like JMS message selectors
> in Java broker). I.e.:
>
> 1) Having CLIENT_ACKNOWLEDGE mode,
> 2) In a loop through all messages of the queue:
> a) receive a message
> b) decide if to consume or not
> c) if so, acknowledge it, if not so, ignore it
> 3) Close the session to retrieve the unacknowledged messages back to the
> queue.
>
> However, I see no way of doing so.
>
> Method Message.acknowledge(); is not suitable as it acknowledges also all
> previous messages, following JMS specification.
>
> Using method AbstractJMSMessage.acknowledgeThis(), I got the same results.
> Though the method name would suggest, it shall acknowledge just this
> message, no else. But checking source code:
>
> public void acknowledgeThis() throws JMSException
> {
> // the JMS 1.1 spec says in section 3.6 that calls to acknowledge
> are ignored when client acknowledge
> // is not specified. In our case, we only set the session field
> where client acknowledge mode is specified.
> if (_session != null && _session.getAcknowledgeMode() ==
> Session.CLIENT_ACKNOWLEDGE)
> {
> if (_session.getAMQConnection().isClosed())
> {
> throw new javax.jms.IllegalStateException("Connection is
> already closed");
> }
>
> // we set multiple to true here since acknowledgment implies
> acknowledge of all previous messages
> // received on the session
> _session.acknowledgeMessage(_deliveryTag, true);
> }
> }
>
> See the latest comment - it acknowledges all previous messages on purpose.
> Having the last command:
> _session.acknowledgeMessage(_deliveryTag, false);
>
> only this message will be acknowledged, no else.
>
> Is there some reason for this behavior? If so, how can I achieve a real
> single message acknowledgment?
>
This method is used by the client ack mechanism, hence the need for
acknowling all messages up to that point. As you pointed out there is no
way to selectively ack messages in JMS.
It's possible to add a method that would just do
_session.acknowledgeMessage(_deliveryTag, false);
However, I would not recommend using any of the inner classes and adding
hacks as it will prevent the user from easily upgrading to a newer version
if these classes change.
>
> Thanks in advance for your thoughs.
>
>
> Kind regards,
> Pavel
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>