You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Sivananda Reddys Thummala Abbigari <st...@salesforce.com> on 2015/05/05 06:31:15 UTC
[NEED HELP] Qpid benchmark Issues/Questions
Hi,
*I am trying to benchmark Qpid with the following use case*:
# Default Qpid configs are used(ex: 2GB is the max memory set), broker and
client are on the same machine
# I have 1 connection and 256 sessions per connection, each sessions has a
producer and consumer. So, there are 256 producers and 256 consumers
# All the producers/consumers are created before they start
producing/consuming messages. Each producer/consumer is a thread and they
run parallely
# Consumers start consuming(they wait with .receive())
# producers start producing messages, each producer produces only 1
message, so there are 256 messages produced in total
# A fanout exchange is used(topic.fanout=fanout://amq.fanout//fanOutTopic),
and there are 256 consumers, each consumer receives 256 messages and so
there are 256*256 messages received in total
# *Following are the response times(RT's) for the messages*:
- Response time is defined as the difference in the time when the
message is sent to the broker and the time at which the message is received
at the client
- min: 144.0 ms
- max: 350454.0 ms
- average: 151933.02 ms
- stddev: 113347.89 ms
- 95th percentile: 330559.0 ms
Is there any thing that I am doing wrong fundamentally?. I am worried about
the avg response times of "*152 secs*". Is this expected from qpid?. I see
a pattern here, as the test is running the RT's are increasing linearly
over time.
Thank you,
Siva.
Re: [NEED HELP] Qpid benchmark Issues/Questions
Posted by Gordon Sim <gs...@redhat.com>.
On 05/05/2015 05:31 AM, Sivananda Reddys Thummala Abbigari wrote:
> Hi,
>
> *I am trying to benchmark Qpid with the following use case*:
> # Default Qpid configs are used(ex: 2GB is the max memory set), broker and
> client are on the same machine
> # I have 1 connection and 256 sessions per connection, each sessions has a
> producer and consumer. So, there are 256 producers and 256 consumers
> # All the producers/consumers are created before they start
> producing/consuming messages. Each producer/consumer is a thread and they
> run parallely
You don't mention which client/broker you are using. However I think its
true in most cases that there will be no performance benefit in running
the producers and consumers in parallel if they share a connection.
> # Consumers start consuming(they wait with .receive())
> # producers start producing messages, each producer produces only 1
> message, so there are 256 messages produced in total
> # A fanout exchange is used(topic.fanout=fanout://amq.fanout//fanOutTopic),
> and there are 256 consumers, each consumer receives 256 messages and so
> there are 256*256 messages received in total
> # *Following are the response times(RT's) for the messages*:
>
> - Response time is defined as the difference in the time when the
> message is sent to the broker and the time at which the message is received
> at the client
> - min: 144.0 ms
> - max: 350454.0 ms
> - average: 151933.02 ms
> - stddev: 113347.89 ms
> - 95th percentile: 330559.0 ms
>
> Is there any thing that I am doing wrong fundamentally?. I am worried about
> the avg response times of "*152 secs*". Is this expected from qpid?. I see
> a pattern here, as the test is running the RT's are increasing linearly
> over time.
The messages will need to be serialised over the wire. Therefore if they
all started at the same time, the time taken for each would be expected
to be a little more for each received message.
The times do sounds excessively long, so there could be something
unexpected going on. Details of the client and broker used may help.
However the nature of the test seems a little odd to me. What is it
exactly you want to test here? Is having lots of
sessions/producers/consumers on the same connection an important aspect?
At the very least I would try to arrange the test to run several times
in a loop (in the same process) to make sure you are not getting
initialisation effects in the timings.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [NEED HELP] Qpid benchmark Issues/Questions
Posted by Sivananda Reddys Thummala Abbigari <st...@salesforce.com>.
Hi,
Thanks a lot for all the replies. The problem was a single connection, once
I moved to 1 session/connection the problem resolved.
Regards,
Siva.
On Thu, May 14, 2015 at 1:24 PM, Alan Conway <ac...@redhat.com> wrote:
> On Wed, 2015-05-06 at 22:38 -0700, Sivananda Reddys Thummala Abbigari
> > Based on the above comments I have a questions here to both of you guys:
> > "How many sessions do you normal use per connection, in production?.
> Also,
> > if we have one sessions per connection we would be bound by number of
> > TCP/IP connections to the broker at some point(OS would need to maintain
> > more file handlers as the number of connections increase)". Could you
> > please suggest how many connections should I be using?. Please advice.
> >
>
> The number of sessions is not really significant. Sessions are cheap but
> you don't get much for your money ;) They are significant if you need
> multiple parallel transactions over a connection, otherwise not really.
>
> The performance issue is connections. The connection is a point of
> serialization (it's just a TCP connection after all) Even if you feed
> many sessions or links concurrently from threads, the data all has to
> line up to leave the process if it's on one connection.
>
> On the broker side, the connection is the unit of concurrency so a
> broker running on a multi-core host will be able to do more work in
> parallel if there are more connections.
>
> It's no different from other protocols. A httpd on a multi-core machine
> can parallelize more of the work if it gets 1 request each on N
> connections than if it gets N requests sequentially on 1 connection.
>
> So if an application will have a lot of clients, a throughput or scale
> benchmark should have a lot of connections (ideally from separate
> clients on separate hosts on an unloaded network, but it is never
> ideal ;) Stuffing all the test data down one connection will give you an
> idea of througput for a single client, but will not give an accurate
> idea of how the broker scales with more clients.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: [NEED HELP] Qpid benchmark Issues/Questions
Posted by Alan Conway <ac...@redhat.com>.
On Wed, 2015-05-06 at 22:38 -0700, Sivananda Reddys Thummala Abbigari
> Based on the above comments I have a questions here to both of you guys:
> "How many sessions do you normal use per connection, in production?. Also,
> if we have one sessions per connection we would be bound by number of
> TCP/IP connections to the broker at some point(OS would need to maintain
> more file handlers as the number of connections increase)". Could you
> please suggest how many connections should I be using?. Please advice.
>
The number of sessions is not really significant. Sessions are cheap but
you don't get much for your money ;) They are significant if you need
multiple parallel transactions over a connection, otherwise not really.
The performance issue is connections. The connection is a point of
serialization (it's just a TCP connection after all) Even if you feed
many sessions or links concurrently from threads, the data all has to
line up to leave the process if it's on one connection.
On the broker side, the connection is the unit of concurrency so a
broker running on a multi-core host will be able to do more work in
parallel if there are more connections.
It's no different from other protocols. A httpd on a multi-core machine
can parallelize more of the work if it gets 1 request each on N
connections than if it gets N requests sequentially on 1 connection.
So if an application will have a lot of clients, a throughput or scale
benchmark should have a lot of connections (ideally from separate
clients on separate hosts on an unloaded network, but it is never
ideal ;) Stuffing all the test data down one connection will give you an
idea of througput for a single client, but will not give an accurate
idea of how the broker scales with more clients.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [NEED HELP] Qpid benchmark Issues/Questions
Posted by Rob Godfrey <ro...@gmail.com>.
Hi Siva,
I'm having difficulty reproducing your results... I wrote a test
program similar to the one you described - 1 connection, 256 sessions
each session sends 1 message to a fanout exchange and consumers 256.
The total runtime for me was 5 seconds. Switching to use one session
per connection and 256 connections it took 3 seconds. Can you share
your test code by creating a JIRA and attaching it.
Thanks,
Rob
On 7 May 2015 at 07:38, Sivananda Reddys Thummala Abbigari
<st...@salesforce.com> wrote:
> Hi Gordon/Rob,
>
> Sincere apologizes for the delayed response.
> Please find my responses:
>
> @Gordon:
> Q) You don't mention which client/broker you are using. However I think its
> true in most cases that there will be no performance benefit in running the
> producers and consumers in parallel if they share a connection.
> A) I am using Java client/broker. Right, I agree with you, I saw that by
> decreasing number of sessions per connection I saw the increase in response
> times.
>
> Q) What is it exactly you want to test here? Is having lots of
> sessions/producers/consumers on the same connection an important aspect?
> A) My test case is simple, I create a topic and there are multiple
> publishers/durable subscribers to that topic. My test case doesn't need
> many sessions per connection(I just experimented with many sessions per
> connection).
>
> @Rob:
> Q) In particular, how big are the messages you are sending? Are they transient
> or persistent (if the latter then which store are you usin - Derby or BDB
> or a JDBC store)? Are you using transactions, if not which acknowledgement
> mode are you using?
> A) I am sending a serialized object(which contains a string(< 500 chars),
> two timeStamp variables) to the receiver. I am running a single border and
> so didn't use any storage for HA purposes. I am not using transactions. I
> am suing AUTO_ACKNOWLEDGE mode. My test case is very simple.
>
> Based on the above comments I have a questions here to both of you guys:
> "How many sessions do you normal use per connection, in production?. Also,
> if we have one sessions per connection we would be bound by number of
> TCP/IP connections to the broker at some point(OS would need to maintain
> more file handlers as the number of connections increase)". Could you
> please suggest how many connections should I be using?. Please advice.
>
> Thank you,
> Siva.
>
>
>
> On Tue, May 5, 2015 at 3:43 AM, Rob Godfrey <ro...@gmail.com> wrote:
>
>> Hi Siva,
>>
>> Firstly are you testing the Java or the C++ broker? Which client?
>>
>> Secondly, it would probably help to know a little more about the set
>> up of your test (or you could create a JIRA and attach your test code
>> there if you like)...
>>
>> In particular, how big are the messages you are sending? Are they
>> transient or persistent (if the latter then which store are you using
>> - Derby or BDB or a JDBC store)?
>> Are you using transactions, if not which acknowledgement mode are you
>> using?
>>
>> The numbers you are seeing are certainly unreasonable (unless you're
>> using multi gigabyte messages :-) )...
>>
>> -- Rob
>>
>> On 5 May 2015 at 06:31, Sivananda Reddys Thummala Abbigari
>> <st...@salesforce.com> wrote:
>> > Hi,
>> >
>> > *I am trying to benchmark Qpid with the following use case*:
>> > # Default Qpid configs are used(ex: 2GB is the max memory set), broker
>> and
>> > client are on the same machine
>> > # I have 1 connection and 256 sessions per connection, each sessions
>> has a
>> > producer and consumer. So, there are 256 producers and 256 consumers
>> > # All the producers/consumers are created before they start
>> > producing/consuming messages. Each producer/consumer is a thread and they
>> > run parallely
>> > # Consumers start consuming(they wait with .receive())
>> > # producers start producing messages, each producer produces only 1
>> > message, so there are 256 messages produced in total
>> > # A fanout exchange is
>> used(topic.fanout=fanout://amq.fanout//fanOutTopic),
>> > and there are 256 consumers, each consumer receives 256 messages and so
>> > there are 256*256 messages received in total
>> > # *Following are the response times(RT's) for the messages*:
>> >
>> > - Response time is defined as the difference in the time when the
>> > message is sent to the broker and the time at which the message is
>> received
>> > at the client
>> > - min: 144.0 ms
>> > - max: 350454.0 ms
>> > - average: 151933.02 ms
>> > - stddev: 113347.89 ms
>> > - 95th percentile: 330559.0 ms
>> >
>> > Is there any thing that I am doing wrong fundamentally?. I am worried
>> about
>> > the avg response times of "*152 secs*". Is this expected from qpid?. I
>> see
>> > a pattern here, as the test is running the RT's are increasing linearly
>> > over time.
>> >
>> > Thank you,
>> > Siva.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [NEED HELP] Qpid benchmark Issues/Questions
Posted by Gordon Sim <gs...@redhat.com>.
On 05/07/2015 06:38 AM, Sivananda Reddys Thummala Abbigari wrote:
> Hi Gordon/Rob,
>
> Sincere apologizes for the delayed response.
> Please find my responses:
>
> @Gordon:
> Q) You don't mention which client/broker you are using. However I think its
> true in most cases that there will be no performance benefit in running the
> producers and consumers in parallel if they share a connection.
> A) I am using Java client/broker. Right, I agree with you, I saw that by
> decreasing number of sessions per connection I saw the increase in response
> times.
>
> Q) What is it exactly you want to test here? Is having lots of
> sessions/producers/consumers on the same connection an important aspect?
> A) My test case is simple, I create a topic and there are multiple
> publishers/durable subscribers to that topic. My test case doesn't need
> many sessions per connection(I just experimented with many sessions per
> connection).
>
> @Rob:
> Q) In particular, how big are the messages you are sending? Are they transient
> or persistent (if the latter then which store are you usin - Derby or BDB
> or a JDBC store)? Are you using transactions, if not which acknowledgement
> mode are you using?
> A) I am sending a serialized object(which contains a string(< 500 chars),
> two timeStamp variables) to the receiver. I am running a single border and
> so didn't use any storage for HA purposes. I am not using transactions. I
> am suing AUTO_ACKNOWLEDGE mode. My test case is very simple.
>
> Based on the above comments I have a questions here to both of you guys:
> "How many sessions do you normal use per connection, in production?. Also,
> if we have one sessions per connection we would be bound by number of
> TCP/IP connections to the broker at some point(OS would need to maintain
> more file handlers as the number of connections increase)". Could you
> please suggest how many connections should I be using?. Please advice.
How many subscribers do you expect to have? I would start off with a
connection per subscriber, and only revisit that if there were some real
need.
If the subscribers are on the same topic, there isn't really any need to
have more than one in the same process anyway. Why would you need to
pull the exact same set of messages off the broker more than once.
If what you want to do is *distribute* the messages between competing
consumers, then you want a shared subscription, or a queue bound to the
exchange with multiple consumers on the queue (to put in pre 1.0 model
terms). That way each subscription is pulling a different set of
messages that are processed in parallel for greater overall throughput.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: [NEED HELP] Qpid benchmark Issues/Questions
Posted by Sivananda Reddys Thummala Abbigari <st...@salesforce.com>.
Hi Gordon/Rob,
Sincere apologizes for the delayed response.
Please find my responses:
@Gordon:
Q) You don't mention which client/broker you are using. However I think its
true in most cases that there will be no performance benefit in running the
producers and consumers in parallel if they share a connection.
A) I am using Java client/broker. Right, I agree with you, I saw that by
decreasing number of sessions per connection I saw the increase in response
times.
Q) What is it exactly you want to test here? Is having lots of
sessions/producers/consumers on the same connection an important aspect?
A) My test case is simple, I create a topic and there are multiple
publishers/durable subscribers to that topic. My test case doesn't need
many sessions per connection(I just experimented with many sessions per
connection).
@Rob:
Q) In particular, how big are the messages you are sending? Are they transient
or persistent (if the latter then which store are you usin - Derby or BDB
or a JDBC store)? Are you using transactions, if not which acknowledgement
mode are you using?
A) I am sending a serialized object(which contains a string(< 500 chars),
two timeStamp variables) to the receiver. I am running a single border and
so didn't use any storage for HA purposes. I am not using transactions. I
am suing AUTO_ACKNOWLEDGE mode. My test case is very simple.
Based on the above comments I have a questions here to both of you guys:
"How many sessions do you normal use per connection, in production?. Also,
if we have one sessions per connection we would be bound by number of
TCP/IP connections to the broker at some point(OS would need to maintain
more file handlers as the number of connections increase)". Could you
please suggest how many connections should I be using?. Please advice.
Thank you,
Siva.
On Tue, May 5, 2015 at 3:43 AM, Rob Godfrey <ro...@gmail.com> wrote:
> Hi Siva,
>
> Firstly are you testing the Java or the C++ broker? Which client?
>
> Secondly, it would probably help to know a little more about the set
> up of your test (or you could create a JIRA and attach your test code
> there if you like)...
>
> In particular, how big are the messages you are sending? Are they
> transient or persistent (if the latter then which store are you using
> - Derby or BDB or a JDBC store)?
> Are you using transactions, if not which acknowledgement mode are you
> using?
>
> The numbers you are seeing are certainly unreasonable (unless you're
> using multi gigabyte messages :-) )...
>
> -- Rob
>
> On 5 May 2015 at 06:31, Sivananda Reddys Thummala Abbigari
> <st...@salesforce.com> wrote:
> > Hi,
> >
> > *I am trying to benchmark Qpid with the following use case*:
> > # Default Qpid configs are used(ex: 2GB is the max memory set), broker
> and
> > client are on the same machine
> > # I have 1 connection and 256 sessions per connection, each sessions
> has a
> > producer and consumer. So, there are 256 producers and 256 consumers
> > # All the producers/consumers are created before they start
> > producing/consuming messages. Each producer/consumer is a thread and they
> > run parallely
> > # Consumers start consuming(they wait with .receive())
> > # producers start producing messages, each producer produces only 1
> > message, so there are 256 messages produced in total
> > # A fanout exchange is
> used(topic.fanout=fanout://amq.fanout//fanOutTopic),
> > and there are 256 consumers, each consumer receives 256 messages and so
> > there are 256*256 messages received in total
> > # *Following are the response times(RT's) for the messages*:
> >
> > - Response time is defined as the difference in the time when the
> > message is sent to the broker and the time at which the message is
> received
> > at the client
> > - min: 144.0 ms
> > - max: 350454.0 ms
> > - average: 151933.02 ms
> > - stddev: 113347.89 ms
> > - 95th percentile: 330559.0 ms
> >
> > Is there any thing that I am doing wrong fundamentally?. I am worried
> about
> > the avg response times of "*152 secs*". Is this expected from qpid?. I
> see
> > a pattern here, as the test is running the RT's are increasing linearly
> > over time.
> >
> > Thank you,
> > Siva.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: [NEED HELP] Qpid benchmark Issues/Questions
Posted by Rob Godfrey <ro...@gmail.com>.
Hi Siva,
Firstly are you testing the Java or the C++ broker? Which client?
Secondly, it would probably help to know a little more about the set
up of your test (or you could create a JIRA and attach your test code
there if you like)...
In particular, how big are the messages you are sending? Are they
transient or persistent (if the latter then which store are you using
- Derby or BDB or a JDBC store)?
Are you using transactions, if not which acknowledgement mode are you using?
The numbers you are seeing are certainly unreasonable (unless you're
using multi gigabyte messages :-) )...
-- Rob
On 5 May 2015 at 06:31, Sivananda Reddys Thummala Abbigari
<st...@salesforce.com> wrote:
> Hi,
>
> *I am trying to benchmark Qpid with the following use case*:
> # Default Qpid configs are used(ex: 2GB is the max memory set), broker and
> client are on the same machine
> # I have 1 connection and 256 sessions per connection, each sessions has a
> producer and consumer. So, there are 256 producers and 256 consumers
> # All the producers/consumers are created before they start
> producing/consuming messages. Each producer/consumer is a thread and they
> run parallely
> # Consumers start consuming(they wait with .receive())
> # producers start producing messages, each producer produces only 1
> message, so there are 256 messages produced in total
> # A fanout exchange is used(topic.fanout=fanout://amq.fanout//fanOutTopic),
> and there are 256 consumers, each consumer receives 256 messages and so
> there are 256*256 messages received in total
> # *Following are the response times(RT's) for the messages*:
>
> - Response time is defined as the difference in the time when the
> message is sent to the broker and the time at which the message is received
> at the client
> - min: 144.0 ms
> - max: 350454.0 ms
> - average: 151933.02 ms
> - stddev: 113347.89 ms
> - 95th percentile: 330559.0 ms
>
> Is there any thing that I am doing wrong fundamentally?. I am worried about
> the avg response times of "*152 secs*". Is this expected from qpid?. I see
> a pattern here, as the test is running the RT's are increasing linearly
> over time.
>
> Thank you,
> Siva.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org