You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by "GS.Chandra N" <gs...@gmail.com> on 2009/02/16 04:27:53 UTC
SQL-92 subscription with Headers exchange on C++ broker
Hi,
I'm trying to try out different ways of subscribing from header exchange.
Can i use SQL like syntax, as described in the standards draft?
I am trying to subscribe something like "h1=val1 AND h2 = val2 AND h3 IN
(val5, val6, val7, val8.....valn)", where h3 can match any of val5 to valn.
How do i achieve this ?
Right now iam creating multiple session.exchange_bind's using the python api
of this format -
1. h1 = val1 AND h2 = val2 and h3 = val5
2. h1= val1 AND h2 = val2 and h3 = val6
.....
n.h1= val1 AND h2 = val2 and h3 = valn
Surely this is not a very efficient way of doing things as the h1 and h2
field matches *might* have to inspected multiple times in this format rather
than once, unless of course the broker understands this to be the case and
recollates them into what is actually intended.
Would someone please tell me if there are more efficient ways to do create
complex subscriptions (like using SQL-92 syntax) on the cpp broker, rather
than break them down into multiple simplified any/all subscriptions ?
Thanks in advance
gs
Re: XML Exchange and Headers (was Re: SQL-92 subscription with
Headers exchange on C++ broker)
Posted by "GS.Chandra N" <gs...@gmail.com>.
Thats great news Jonathan - cant wait to check the difference
Thanks
gs
On Sat, Feb 28, 2009 at 3:07 AM, Jonathan Robie
<jo...@redhat.com>wrote:
> Gordon Sim wrote:
>
>> The (non-standard) XML exchange allows XQuery selection based on headers,
>> but currently I believe it requires the body to be in XML even if your query
>> doesn't actually access the body.
>>
>
> I just checked in a version of the XML Exchange that does not attempt to
> parse the message content if it is not used in the query, so you can use it
> to route based on message headers even if the message content is not XML.
>
> Jonathan
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
Re: XML Exchange and Headers (was Re: SQL-92 subscription with
Headers exchange on C++ broker)
Posted by "GS.Chandra N" <gs...@gmail.com>.
Thats great news Jonathan - cant wait to check the difference
Thanks
gs
On Sat, Feb 28, 2009 at 3:07 AM, Jonathan Robie
<jo...@redhat.com>wrote:
> Gordon Sim wrote:
>
>> The (non-standard) XML exchange allows XQuery selection based on headers,
>> but currently I believe it requires the body to be in XML even if your query
>> doesn't actually access the body.
>>
>
> I just checked in a version of the XML Exchange that does not attempt to
> parse the message content if it is not used in the query, so you can use it
> to route based on message headers even if the message content is not XML.
>
> Jonathan
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
XML Exchange and Headers (was Re: SQL-92 subscription with Headers
exchange on C++ broker)
Posted by Jonathan Robie <jo...@redhat.com>.
Gordon Sim wrote:
> The (non-standard) XML exchange allows XQuery selection based on
> headers, but currently I believe it requires the body to be in XML
> even if your query doesn't actually access the body.
I just checked in a version of the XML Exchange that does not attempt to
parse the message content if it is not used in the query, so you can use
it to route based on message headers even if the message content is not XML.
Jonathan
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by Jonathan Robie <jo...@redhat.com>.
GS.Chandra N wrote:
>> On Tue, Feb 17, 2009 at 8:20 PM, Jonathan Robie <jo...@redhat.com>wrote:
>> I honestly do not know which approach would perform better - we'll see! How
>>
> many conditions are you likely to be testing in a single query?
>
> upto 100 conditions in 30% of the cases, upto 500 in 50% of the cases and
> upto 1200 in the remaining cases.
>
> The messages will be typically of 500 byte sizes with 5 headers on average.
> The headers and values can be small, 4-15 bytes in size. The conditions will
> involve headers and values of the same sizes. 99% of the conditions will
> involve the same header in the format of the condition i have shown below.
>
> A *minimum* of 500 clients is what will be required to be connected to a
> single server.
>
> Let me know if you need any other information. I will run some tests too and
> post the results with many small subscriptions vs a huge few case scenario
> in a few days time.
>
Well, that's enough conditions and clients to be worth the experiment!
Jonathan
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by "GS.Chandra N" <gs...@gmail.com>.
>On Tue, Feb 17, 2009 at 8:20 PM, Jonathan Robie <jo...@redhat.com>wrote:
>I honestly do not know which approach would perform better - we'll see! How
many conditions are you likely to be testing in a single query?
upto 100 conditions in 30% of the cases, upto 500 in 50% of the cases and
upto 1200 in the remaining cases.
The messages will be typically of 500 byte sizes with 5 headers on average.
The headers and values can be small, 4-15 bytes in size. The conditions will
involve headers and values of the same sizes. 99% of the conditions will
involve the same header in the format of the condition i have shown below.
A *minimum* of 500 clients is what will be required to be connected to a
single server.
Let me know if you need any other information. I will run some tests too and
post the results with many small subscriptions vs a huge few case scenario
in a few days time.
Thanks
gs
GS.Chandra N wrote:
>
>> That would be really cool Jonathan.
>>
>> The major reason i fear to take the multiple subscriptions route is due to
>> fears of performance degradation. If i can make do with a single
>> subscription, which performs better than equivalent multiple
>> subscriptions,
>> i could connect more clients to one server (plausibly - still need to do
>> tests to figure out how many subscriptions can be supported by cpp broker)
>>
>>
>
> I honestly do not know which approach would perform better - we'll see! How
> many conditions are you likely to be testing in a single query?
>
>
> Jonathan
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by Jonathan Robie <jo...@redhat.com>.
GS.Chandra N wrote:
> That would be really cool Jonathan.
>
> The major reason i fear to take the multiple subscriptions route is due to
> fears of performance degradation. If i can make do with a single
> subscription, which performs better than equivalent multiple subscriptions,
> i could connect more clients to one server (plausibly - still need to do
> tests to figure out how many subscriptions can be supported by cpp broker)
>
I honestly do not know which approach would perform better - we'll see!
How many conditions are you likely to be testing in a single query?
Jonathan
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by "GS.Chandra N" <gs...@gmail.com>.
That would be really cool Jonathan.
The major reason i fear to take the multiple subscriptions route is due to
fears of performance degradation. If i can make do with a single
subscription, which performs better than equivalent multiple subscriptions,
i could connect more clients to one server (plausibly - still need to do
tests to figure out how many subscriptions can be supported by cpp broker)
This would reduce the no of boxes required in a bigger deployment and
directly help to reduce costs in large deployments, though few deployments
(banks etc) ever reach such massive scales.
Thanks
gs
ps : I am running some tests to try out the different scenarios and will
post some results.
On Mon, Feb 16, 2009 at 8:10 PM, Jonathan Robie
<jo...@redhat.com>wrote:
> Gordon Sim wrote:
>
>> I think the simplest fix/change to address your need would be a
>> modification to the xml exchange to allow xqueries on headers only, where
>> the body is not itself xml.
>>
>
> I will do that - it means I have to detect statically whether the message
> body is used in the query, but I think that's possible. I'll post back when
> I have a version that works.
>
> Jonathan
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by Jonathan Robie <jo...@redhat.com>.
Gordon Sim wrote:
> I think the simplest fix/change to address your need would be a
> modification to the xml exchange to allow xqueries on headers only,
> where the body is not itself xml.
I will do that - it means I have to detect statically whether the
message body is used in the query, but I think that's possible. I'll
post back when I have a version that works.
Jonathan
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by Gordon Sim <gs...@redhat.com>.
GS.Chandra N wrote:
>> On Mon, Feb 16, 2009 at 2:57 PM, Gordon Sim <gs...@redhat.com> wrote:
>> The headers exchange only supports the limited mechanisms documented in
> the 0-10 spec.
>
> In that case, are there any ways to create Links between exchanges at the
> server such that i can populate certain well known exchanges with messages
> corresponding to some known conditions ?
>
> Required condition = h1 = val1 and h2 = val2 and h3 in (val4, val5....valn)
>
> So i create one exchange h1h2 such that all messages that match h1 = val1
> and h2 = val2 gets send to this queue (via a link between two queues?) and
> clients can subscribe using h3 = val4 or h3 = val5 .... syntax from this
> exchange.
Unfortunately not.
> If there is some rich functionality available around this, i could use this
> to simplify the client subscriptions and probably reduce the load on the
> broker. I saw some broker administration examples on the amqp standard pdf
> (section 9) that showed some similar scenarios. Are they available ?
No, I suspect you are referring to the draft 1.0 spec which is still a
work in progress.
I think the simplest fix/change to address your need would be a
modification to the xml exchange to allow xqueries on headers only,
where the body is not itself xml.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by "GS.Chandra N" <gs...@gmail.com>.
>On Mon, Feb 16, 2009 at 2:57 PM, Gordon Sim <gs...@redhat.com> wrote:
> The headers exchange only supports the limited mechanisms documented in
the 0-10 spec.
In that case, are there any ways to create Links between exchanges at the
server such that i can populate certain well known exchanges with messages
corresponding to some known conditions ?
Required condition = h1 = val1 and h2 = val2 and h3 in (val4, val5....valn)
So i create one exchange h1h2 such that all messages that match h1 = val1
and h2 = val2 gets send to this queue (via a link between two queues?) and
clients can subscribe using h3 = val4 or h3 = val5 .... syntax from this
exchange.
If there is some rich functionality available around this, i could use this
to simplify the client subscriptions and probably reduce the load on the
broker. I saw some broker administration examples on the amqp standard pdf
(section 9) that showed some similar scenarios. Are they available ?
Thanks
gs
On Mon, Feb 16, 2009 at 2:57 PM, Gordon Sim <gs...@redhat.com> wrote:
> GS.Chandra N wrote:
>
>> I'm trying to try out different ways of subscribing from header exchange.
>> Can i use SQL like syntax, as described in the standards draft?
>>
>
> The headers exchange only supports the limited mechanisms documented in the
> 0-10 spec. You specify the key-value pairs you want to match on and can then
> either choose to match them all (AND) or any of them (OR).
>
> The (non-standard) XML exchange allows Xquery selection based on headers,
> but currently I believe it requires the body to be in XML even if your query
> doesn't actually access the body.
>
> Selectors on subscribe is on the roadmap[1] but I think now unlikely to
> make it for the M5 feature freeze (hopefully it will be in the subsequent
> release however).
>
> [1] https://issues.apache.org/jira/browse/QPID-530
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
XML Exchange and Headers (was Re: SQL-92 subscription with Headers
exchange on C++ broker)
Posted by Jonathan Robie <jo...@redhat.com>.
Gordon Sim wrote:
> The (non-standard) XML exchange allows XQuery selection based on
> headers, but currently I believe it requires the body to be in XML
> even if your query doesn't actually access the body.
I just checked in a version of the XML Exchange that does not attempt to
parse the message content if it is not used in the query, so you can use
it to route based on message headers even if the message content is not XML.
Jonathan
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: SQL-92 subscription with Headers exchange on C++ broker
Posted by Gordon Sim <gs...@redhat.com>.
GS.Chandra N wrote:
> I'm trying to try out different ways of subscribing from header exchange.
> Can i use SQL like syntax, as described in the standards draft?
The headers exchange only supports the limited mechanisms documented in
the 0-10 spec. You specify the key-value pairs you want to match on and
can then either choose to match them all (AND) or any of them (OR).
The (non-standard) XML exchange allows Xquery selection based on
headers, but currently I believe it requires the body to be in XML even
if your query doesn't actually access the body.
Selectors on subscribe is on the roadmap[1] but I think now unlikely to
make it for the M5 feature freeze (hopefully it will be in the
subsequent release however).
[1] https://issues.apache.org/jira/browse/QPID-530
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org