You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by "zhaoyi0113@gmail.com" <zh...@gmail.com> on 2010/10/26 11:17:13 UTC
Publish-Subscribe queue question
I created a pub-sub application two subscribers and one publisher. I found
that the queue management is different between c++ client and Java client.
When I use Java client, the two subscribers can share one queue. When I use
c++ client, the two subscribers can also share one queue but the messages
published to this queue can only be delivered to one of the subscribers.
Does anyone have such problem when using c++ and Java?
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5673932.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 11/03/2010 01:29 AM, zhaoyi0113@gmail.com wrote:
> I have read the example for pub-sub pattern. But it doesn't use Topic, it
> uses queues. What I want is to use a Topic for pub-sub. For example, if two
> subscribers subscribe on one queue, one message will only be delivered to
> one subscriber not both of them. How can I set up a Topic in old API?
The concept of a 'topic' as JMS defines it does not directly exist in
AMQP 0-10. Rather it is constructed by creating an distinct exclusive
subscription queue for each subscriber, binding that to an exchange and
then subscribing to that queue. Messages published to this exchange then
get routed to each subscription queue and thus each subscriber sees each
message. The subscribers are not sharing a queue, they each have their
own queue.
When you use the Qpid JMS client, and you use a Topic, it is doing this
work for you. The broker doesn't see a topic being used, it sees a queue
created for each subscriber and then being bound to an exchange.
The old c++ API did not have that same functionality built in to it, so
the application developer needed to go through the steps manually as in
the pub-sub example. In the pub-sub example you can start multiple
lilsteners and then when you publish messages, each listener will
receive every published message it is interested in. This is the same
pattern of message flow as generally implied by a JMS Topic, and the
brokers view is the same in each case.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
Hi,
I have read the example for pub-sub pattern. But it doesn't use Topic, it
uses queues. What I want is to use a Topic for pub-sub. For example, if two
subscribers subscribe on one queue, one message will only be delivered to
one subscriber not both of them. How can I set up a Topic in old API?
Thanks,
Zhao Yi
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5699801.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Jonathan Robie <jo...@redhat.com>.
On 11/01/2010 09:04 PM, zhaoyi0113@gmail.com wrote:
>
> In this thread
>
> http://apache-qpid-users.2158936.n2.nabble.com/What-is-the-different-between-qpid-client-Session-h-and-qpid-messaging-Session-h-tt5569296.html#a5569296
>
> you told me that the new API is in active development. what I think is that
> the new API is not finished and not safe to use. So I will use old API until
> the new API is ready.
Your call. Out of curiosity, what is it that you think is not ready or
may be unsafe? Personally, I would feel perfectly safe using the new API.
> Could you let me know how I can create a topic in old API? The pub-sub
> example uses queue to handle the pub-sub pattern, how can I use topic to
> handle this?
Sure, here's a very simple example using the AMQP direct exchange, where
the publishers and subscribers match based on a simple routing key:
Declare queues:
http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/examples/direct/declare_queues.cpp?revision=726140&view=markup
Publisher:
http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/examples/direct/direct_producer.cpp?view=markup
Listener:
http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/examples/direct/listener.cpp?view=markup
Here's the code for a topic publisher that uses hierarchical keys and
the AMQP topic exchange:
http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/examples/pub-sub/topic_publisher.cpp?view=markup
And a topic listener:
http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/examples/pub-sub/topic_listener.cpp?view=co
These examples are included with older versions of Qpid. You might want,
for example, to download the examples that come with Qpid release 0.5.
Jonathan
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
In this thread
http://apache-qpid-users.2158936.n2.nabble.com/What-is-the-different-between-qpid-client-Session-h-and-qpid-messaging-Session-h-tt5569296.html#a5569296
you told me that the new API is in active development. what I think is that
the new API is not finished and not safe to use. So I will use old API until
the new API is ready.
Could you let me know how I can create a topic in old API? The pub-sub
example uses queue to handle the pub-sub pattern, how can I use topic to
handle this?
Thanks,
Zhao Yi
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5695668.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Jonathan Robie <jo...@redhat.com>.
On 10/31/2010 09:40 PM, zhaoyi0113@gmail.com wrote:
>
> Let me confirm one thing. Old C++ API doesn't have Topic class to implement
> pub-sub pattern. New C++ API have implement this patter. Do I understand
> correctly?
Hi Zhao - Both APIs support both patterns. It's simpler in the new API.
Is there a reason you can't use the new API? The new API is better
documented on the Apache site, and it's the API we're using going forward.
Red Hat contributed a tutorial on the old API, but it required work to
make it generic for the Apache site, and it doesn't make sense to do
that work now, since we're moving forward on the basis of the new API. I
can send you a copy if you want it, but I really encourage you to move
to the new API and use Programming in Apache Qpid instead.
You can find Programming in Apache Qpid here:
http://qpid.apache.org/books/0.7/Programming-In-Apache-Qpid/html/ch02.html
Jonathan
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
Let me confirm one thing. Old C++ API doesn't have Topic class to implement
pub-sub pattern. New C++ API have implement this patter. Do I understand
correctly?
Thanks,
Zhao Yi
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5692661.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/29/2010 01:59 AM, zhaoyi0113@gmail.com wrote:
> The C++ pub-sub example doesn't use Topic. It uses different queues for
> different subscribers. How can I use Topic in C++ client side (qpid::client
> namespace)?
In AMQP (pre 1.0), 'topics' i.e. the publish-subscribe pattern, is
implemented as an exchange where each subscriber binds their own private
subscription queue. Any message sent to the exchange will then be
received by all interested subscribers.
The JMS topic concept is implemented in this way by the client library.
The old c++ API exposed AMQP 0-10 directly, leaving the user to
implement these sorts of patterns themselves directly. The new c++ API
operates at a higher level of abstraction and thus can encapsulate and
automate some of this work for you.
Does that make sense?
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
The C++ pub-sub example doesn't use Topic. It uses different queues for
different subscribers. How can I use Topic in C++ client side (qpid::client
namespace)?
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5684875.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 04:53 PM, zhaoyi0113@gmail.com wrote:
>
>
> If I am using the old API. Does this mean I can't use Topic? I didn't find
> topic related class in qpid::client namespace. The pub-sub example uses
> different queues to handle the topic subscription.
You can use the exact same pattern. You just need to do it explicitly as
shown by the pub-sub example you adapted. This is what the JMS client
does under the covers for a topic.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
If I am using the old API. Does this mean I can't use Topic? I didn't find
topic related class in qpid::client namespace. The pub-sub example uses
different queues to handle the topic subscription.
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5683157.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 04:39 PM, zhaoyi0113@gmail.com wrote:
> Does C++ have a topic class? I didn't find the topic class in C++ doc. I
> want to do exact the same thing in C++, how can I do this?
In the old API (qpid::client namespace) you do it in the same way the
original pub-sub example shows, by explicitly creating an exclusive
subscription queue and binding that to an exchange.
There is now a new API for c++ (old one still supported, but new one is
where the new features will be generally added) in the qpid::messaging
namespace. This API is more similar to JMS and will handle the details
described above automatically.
You can read more about this API online and it will be part of the
upcoming release:
http://qpid.apache.org/books/0.7/Programming-In-Apache-Qpid/html/
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
Ho yes.
Does C++ have a topic class? I didn't find the topic class in C++ doc. I
want to do exact the same thing in C++, how can I do this?
thanks,
Zhao Yi
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5683098.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 04:08 PM, zhaoyi0113@gmail.com wrote:
>
> Hi,
>
> I have uploaded my Java code. Please try this code and you will get the
> subscribers share one queue and the messages will be delivered to both of
> them.
>
> http://apache-qpid-users.2158936.n2.nabble.com/file/n5682939/Subscriber.java
> Subscriber.java
That subscriber is using a topic, not a queue.
> http://apache-qpid-users.2158936.n2.nabble.com/file/n5682939/Publisher.java
> Publisher.java
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
Hi,
I have uploaded my Java code. Please try this code and you will get the
subscribers share one queue and the messages will be delivered to both of
them.
http://apache-qpid-users.2158936.n2.nabble.com/file/n5682939/Subscriber.java
Subscriber.java
http://apache-qpid-users.2158936.n2.nabble.com/file/n5682939/Publisher.java
Publisher.java
Zhao Yi
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5682939.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 11:53 AM, zhaoyi0113@gmail.com wrote:
>
> why does Java client have different behavior? If you set up a listener in
> Java client, then you will see that one message will come to all listeners.
What is the code and configuration for your java client? My guess is
that the consumers there are not using the same queue (or if they are
they are perhaps browsing it).
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
why does Java client have different behavior? If you set up a listener in
Java client, then you will see that one message will come to all listeners.
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5682144.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 11:42 AM, zhaoyi0113@gmail.com wrote:
>
> Hi,
>
> I uploaded my code and the producer is topic_publisher.cpp I didn't change.
>
> http://apache-qpid-users.2158936.n2.nabble.com/file/n5682097/topic_listener.cpp
> topic_listener.cpp
>
> If you run this listener in two processes and run the topic_publisher, you
> will see that one message will come to only one listener.
That is expected though. Each message will only be allocated to one
consumer (this is the standard queueing semantic). However some messages
should get to each consumer (just distinct sets).
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
Hi,
I uploaded my code and the producer is topic_publisher.cpp I didn't change.
http://apache-qpid-users.2158936.n2.nabble.com/file/n5682097/topic_listener.cpp
topic_listener.cpp
If you run this listener in two processes and run the topic_publisher, you
will see that one message will come to only one listener.
Zhao Yi
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5682097.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 01:37 PM, Bill Whiting wrote:
> If you have multiple listeners with separate subscriptions, shouldn't
> each listener get a copy of each message published?
Not if the subscriptions are on the same queue (as opposed to topic in
the JMS sense).
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Bill Whiting <te...@bellsouth.net>.
If you have multiple listeners with separate subscriptions, shouldn't
each listener get a copy of each message published?
//Bill
On 10/28/2010 05:55 AM, Gordon Sim wrote:
> On 10/28/2010 10:36 AM, zhaoyi0113@gmail.com wrote:
>> I have tried your method and it works as before. It doesn't solve the
>> issue.
>
> Can you post your code and the expected and actual behaviour? I
> modifed the topic example to use the same queue and found that
> messages are shared out between multiple listeners as expected. I'm
> not sure what your issue is.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 10:36 AM, zhaoyi0113@gmail.com wrote:
> I have tried your method and it works as before. It doesn't solve the issue.
Can you post your code and the expected and actual behaviour? I modifed
the topic example to use the same queue and found that messages are
shared out between multiple listeners as expected. I'm not sure what
your issue is.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
Hi,
I have tried your method and it works as before. It doesn't solve the issue.
Regards,
Zhao Yi
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5681912.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 09:44 AM, zhaoyi0113@gmail.com wrote:
> I am using the example code examples/pub-sub/topic_listener.cpp. In this
> code, it declares queue by adding a session id so different sessions will
> have different queue names. If you change the code to use the same queue
> name for different sessions, you will get the problem. The code looks
> basically like below:
>
>
> session.queueDeclare(arg::queue = queue_name, arg::exclusive = false,
> arg::autoDelete = true, arg::durable = false);
> session.exchangeBind(arg::exchange = exchange,
> arg::queue = queue_name, arg::bindingKey = binding);
> session.exchangeBind(arg::exchange = exchange,
> arg::queue = queue_name, arg::bindingKey =
> "control");
> subscriptionsMgr = new SubscriptionManager(
> session);
try changing:
> subscriptionsMgr->subscribe(listener, queue_name);
to:
subscriptionsMgr->subscribe(listener, queue_name,
SubscriptionSettings(FlowControl::messageWindow(1)));
Does that address the issue you are seeing?
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
I am using the example code examples/pub-sub/topic_listener.cpp. In this
code, it declares queue by adding a session id so different sessions will
have different queue names. If you change the code to use the same queue
name for different sessions, you will get the problem. The code looks
basically like below:
session.queueDeclare(arg::queue = queue_name, arg::exclusive = false,
arg::autoDelete = true, arg::durable = false);
session.exchangeBind(arg::exchange = exchange,
arg::queue = queue_name, arg::bindingKey = binding);
session.exchangeBind(arg::exchange = exchange,
arg::queue = queue_name, arg::bindingKey =
"control");
subscriptionsMgr = new SubscriptionManager(
session);
subscriptionsMgr->subscribe(listener, queue_name);
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5681763.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2010 08:01 AM, zhaoyi0113@gmail.com wrote:
>
> Does anyone have meet such issue? I found that the problem is C++ client side
> code. When two C++ subscribers subscribe to one queue, why only one of the
> subscribers can get message when publish to this queue? When I use Java
> client side, it works fine.
What does your c++ client look like? It may be that the prefetch
specified is resulting in the first client taking all the messages - you
can reduce the prefetch to address that if needed.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Publish-Subscribe queue question
Posted by "zhaoyi0113@gmail.com" <zh...@gmail.com>.
Does anyone have meet such issue? I found that the problem is C++ client side
code. When two C++ subscribers subscribe to one queue, why only one of the
subscribers can get message when publish to this queue? When I use Java
client side, it works fine.
--
View this message in context: http://apache-qpid-users.2158936.n2.nabble.com/Publish-Subscribe-queue-question-tp5673932p5681507.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org