You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Praveen M <le...@gmail.com> on 2012/01/10 01:27:47 UTC
JMS Client usage - best practices?
Hi,
I'm writing a JMS Client and have a few best practices questions.
1) I understand that there are different ways ways to create a reference to
a destination to enqueue a message (using JMS createQueue, using JNDI
lookup).
What would be the recommended way to use?
2) I create a connection to the broker on my application startup, and
create a session pool. on every enqueue, i checkout a session from my pool
use it and return to pool on enqueue.
Does this sound reasonable? (code sample below)
What would be the pattern that is typically used/recommended for
session management/connection management in the client side?
Or is this an overkill to do? Is CreateSession() and CreateProducer()
really cheap that it is just done on each enqueue?
In my use case, I'd expect quite some concurrency (assume n (maybe
5-10) threads doing enqueues at the same time) and I'd prefer it to not
lock/block and go through asap.
I do remember from reading the qpid client code that there is a lock
held on the connection object whenever a new session is created on that
connection. I'm assuming it is one of the places where my enqueue can slow
down
when going in parallel if it has to create a session for every enqueue
of a message. There might be more such blocks, which I'm not aware of.
Taking performance into mind, what would be the best design for
session/connection management?
A snippet of what I have currently is below,
// a thread safe queue with the available session producer pairs which
can be used to create a message and enqueue.
ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
availableSessionProducerPairs = new ConcurrentLinkedQueue<Pair<Session,
MessageProducer>> ();
// initialize the sessionProducerPairs, create upto n pairs
initializePool(Connection qpidConnection) {
for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ ) {
Session session = qpidConnection.createSession(true,
Session.SESSION_TRANSACTED);
MessageProducer producer = session.createProducer(null);
availableSessionProducerPairs.add(Pair.newPair(session,
producer));
}
}
// on enqueue we checkout a session/producer pair use it for the enqueue
and then return it back to the session/produer pair pool.
enqueueMessage(String queueName, byte[] message) {
// remove the session producer pair
Pair<Session, MessageProducer> sessionProducerPair =
availableSessionProducerPairs.poll();
if(sessionProducerPair == null) {
// gack here and return null.
}
Session session = sessionProducerPair.getFirst();
MessageProducer producer = sessionProducerPair.getSecond();
BytesMessage jmsMessage = session.createBytesMessage();
jmsMessage.writeBytes(message);
Queue queueRef = queueNameToReferenceMap.get(queueName);
producer.send(queueRef, jmsMessage, DeliveryMode.PERSISTENT, 0, 0);
session.commit();
availableSessionProducersPair.add()
}
Thank you,
-Praveen
Re: JMS Client usage - best practices?
Posted by suman <pr...@gmail.com>.
Hello Robbie
Thanks for your response.Some of my sessions has message consuming rate of
around 1000 message per minute during peak time.It seems to go well so far
with no memory leaks and stuff.Thanks for the advise and will keep you
posted
Regards
Suman
--
View this message in context: http://qpid.2158936.n2.nabble.com/JMS-Client-usage-best-practices-tp7170254p7273491.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by Robbie Gemmell <ro...@gmail.com>.
Hi Suman,
I'm afraid i have little experience with Spring and none at all with
Camel, so I won't be much help on that side of things. If your
throughput figures are that low ( < 1msg / hr per session) then having
so many sessions wont be a performance bottleneck, although it is
still a bit odd.
I can however tell you how to alter the configuration so the Java
broker will let you have more than 256 Sessions per connection. For
example, if you add the element
<maximumChannelCount>1024</maximumChannelCount> inside the <broker>
tag within the <install dir>\etc\config.xml file that would allow you
1024 Sessions per connection. Legal values are up to 2^16.
Regards,
Robbie
On 9 February 2012 10:44, suman <pr...@gmail.com> wrote:
> Thanks guys for your answers.I had a look at JCA/Jencks/JMSRA in the past but
> QPID seems to suggest JCA is only available for c++ message brokers
> https://svn.apache.org/repos/asf/qpid/trunk/qpid/java/jca/README.txt
>
> So i had to look at alternative options.It would be good if QPID also
> provides something similar to activemq pool project
>
> Now coming to my usecase.
> Iam connecting to an external QPID message broker using java JMS client of
> QPID(Well i use apache camel amqp component which inturn uses spring
> jmstemplate)
> Iam connecting to this DIRECT exchange and listen for various message
> types.And in our domain we have various events.Each event will have its own
> unique eventid.So publisher publishes to the QPID exchange with this eventid
> as routing key and than i bind to this DIRECT exchange using eventid as
> routing key(Which in turn creates a queue and binds to it using eventid as
> routing key).I use the concept of Dynamic Destinations where i setup and
> teardown these destinations when event starts/completes.
> Now this is the configuration iam using
>
> <bean id="amqp" class="org.apache.camel.component.amqp.AMQPComponent">
> <property name="connectionFactory" ref="cachingConnectionFactory"/>
> <property name="cacheLevel" value="2"/>
> </bean>
>
>
> <bean id="cachingConnectionFactory"
>
> class="org.springframework.jms.connection.CachingConnectionFactory">
> <property name="targetConnectionFactory"
> ref="amqConnectionFactory"/>
> </bean>
>
>
> <bean id="amqConnectionFactory"
> class="org.apache.qpid.client.AMQConnectionFactory">
> <constructor-arg
>
> value="amqp://guest:guest@clientid/push?brokerlist='${broker.url}'"/>
> </bean>
>
> So as you can see iam caching upto SESSION level.So by doing this way each
> event is bound to single session which inturn is bound to one QUEUE(With
> eventid as binding key).
> In our system we will have more than 256 events happening at a single time
> and each event will have timeframe of 1 to 6 hrs.When event
> completes,sessions are closed and dynamic queues get deleted.
>
> So is there anything wrong in the way i have configured this solution?Many
> thanks for any advise
>
> Regards
> Suman
>
>
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/JMS-Client-usage-best-practices-tp7170254p7268873.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by suman <pr...@gmail.com>.
Thanks guys for your answers.I had a look at JCA/Jencks/JMSRA in the past but
QPID seems to suggest JCA is only available for c++ message brokers
https://svn.apache.org/repos/asf/qpid/trunk/qpid/java/jca/README.txt
So i had to look at alternative options.It would be good if QPID also
provides something similar to activemq pool project
Now coming to my usecase.
Iam connecting to an external QPID message broker using java JMS client of
QPID(Well i use apache camel amqp component which inturn uses spring
jmstemplate)
Iam connecting to this DIRECT exchange and listen for various message
types.And in our domain we have various events.Each event will have its own
unique eventid.So publisher publishes to the QPID exchange with this eventid
as routing key and than i bind to this DIRECT exchange using eventid as
routing key(Which in turn creates a queue and binds to it using eventid as
routing key).I use the concept of Dynamic Destinations where i setup and
teardown these destinations when event starts/completes.
Now this is the configuration iam using
<bean id="amqp" class="org.apache.camel.component.amqp.AMQPComponent">
<property name="connectionFactory" ref="cachingConnectionFactory"/>
<property name="cacheLevel" value="2"/>
</bean>
<bean id="cachingConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory"
ref="amqConnectionFactory"/>
</bean>
<bean id="amqConnectionFactory"
class="org.apache.qpid.client.AMQConnectionFactory">
<constructor-arg
value="amqp://guest:guest@clientid/push?brokerlist='${broker.url}'"/>
</bean>
So as you can see iam caching upto SESSION level.So by doing this way each
event is bound to single session which inturn is bound to one QUEUE(With
eventid as binding key).
In our system we will have more than 256 events happening at a single time
and each event will have timeframe of 1 to 6 hrs.When event
completes,sessions are closed and dynamic queues get deleted.
So is there anything wrong in the way i have configured this solution?Many
thanks for any advise
Regards
Suman
--
View this message in context: http://qpid.2158936.n2.nabble.com/JMS-Client-usage-best-practices-tp7170254p7268873.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by Robbie Gemmell <ro...@gmail.com>.
I noticed the mention of a 256 Session restriction. The client doesnt
actually have any limit except that of the underlying protocol (which
allows for up to 2^16 Sessions), but the default configuration of the
Java broker happens to set the negotiated channel limit at 256 so I
would guess that means it is being used here? The 256 value is
configurable on the broker if need be, although attention isnt really
drawn to that as there are few use cases in which it would seem to be
expected or desirable to have that many sessions on a single
connection (congratulations Suman, you are actually the first person
to have mentioned running into that since it was put in place :) ) so
as Weston wonders, what is the use case?
Robbie
On 8 February 2012 20:28, Weston M. Price <wp...@redhat.com> wrote:
> Hi Suman,
> It's a good question. Currently the JMS client does not provide connection pooling outside of an application server. This is something we are looking at for future releases. Currently you have the option of using the Qpid JCA adapter that provides for JEE integration that allows for both Connection and Session pooling. Out of curiosity, why do you need 256+ sessions simultaneously?
>
> Regards,
>
> Weston
> On Feb 7, 2012, at 11:07 AM, suman wrote:
>
>> Hello All,
>> Iam quite new to QPID.Iam using QpidClient of Java and using JMS.
>> Iam just wondering how to do connection pooling using AMQConnectionFactory.
>> I tried using SingleConnectionFactory of spring but one connection in QPid
>> will let me create only 256 sessions at same time and i need more than that
>>
>> Many Thanks
>>
>> --
>> View this message in context: http://qpid.2158936.n2.nabble.com/JMS-Client-usage-best-practices-tp7170254p7262798.html
>> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by "Weston M. Price" <wp...@redhat.com>.
Hi Suman,
It's a good question. Currently the JMS client does not provide connection pooling outside of an application server. This is something we are looking at for future releases. Currently you have the option of using the Qpid JCA adapter that provides for JEE integration that allows for both Connection and Session pooling. Out of curiosity, why do you need 256+ sessions simultaneously?
Regards,
Weston
On Feb 7, 2012, at 11:07 AM, suman wrote:
> Hello All,
> Iam quite new to QPID.Iam using QpidClient of Java and using JMS.
> Iam just wondering how to do connection pooling using AMQConnectionFactory.
> I tried using SingleConnectionFactory of spring but one connection in QPid
> will let me create only 256 sessions at same time and i need more than that
>
> Many Thanks
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/JMS-Client-usage-best-practices-tp7170254p7262798.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by suman <pr...@gmail.com>.
Hello All,
Iam quite new to QPID.Iam using QpidClient of Java and using JMS.
Iam just wondering how to do connection pooling using AMQConnectionFactory.
I tried using SingleConnectionFactory of spring but one connection in QPid
will let me create only 256 sessions at same time and i need more than that
Many Thanks
--
View this message in context: http://qpid.2158936.n2.nabble.com/JMS-Client-usage-best-practices-tp7170254p7262798.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by Praveen M <le...@gmail.com>.
Thanks for writing Robbie. That was a lot of useful info.
I sure didn't think of the multiple consumer - single session case. Thanks
for the tip.
I'll try spreading by consumers across 2 connections and see if that helps.
Thank you,
Praveen
On Fri, Jan 13, 2012 at 10:10 AM, Robbie Gemmell
<ro...@gmail.com>wrote:
> Hi Praveen,
>
> 180 connections should also be entirely fine, I have used several
> hundred. The only limit is going to be memory based.The key issue with
> using large amounts of connections currently is that when we moved
> away from Mina as the IO layer for 0.14 we moved to a model that uses
> 2 threads per connection, so there are a large number of Threads
> created which in turn will require memory for their stacks and heap
> for the objects they contain. Additionally, there are currently some
> large buffers placed in the heap on a per-connection basis that can
> quickly mount up. We are going to look at the impact those have on
> performance at particular buffer sizes and see if we can reduce the
> default, but it can certainly be reconfigured to meet a particular
> users requirements.
>
> With such a low throughput of <=1msg/s per consumer, there is vastly
> reduced scope for write-combining the store updates and so the use of
> a single connection becomes far less of issue. There will be scope to
> write-combine with other consumer instances and also the various
> producers, but it may still be worth looking at something inbetween
> the extremes of 20 Consumers per Connection and 1 Consumer per
> Connection (eg 10 Consumers per Connection), to see if it is possible
> to extract some scalability.
>
> With your slow onMessage handler the thing you should be most worried
> about is avoiding using multiple Consumers on the same Session,
> because each Session only has a single thread performing the
> deliveries. You will want to make sure you use a Session per Consumer.
>
> Robbie
>
> On 11 January 2012 19:02, Praveen M <le...@gmail.com> wrote:
> > ah, my bad... I expect to have 20 consumer threads per instance. Sorry,
> so
> > that'd be 600 consumer threads in the environment.
> >
> > On Wed, Jan 11, 2012 at 10:14 AM, Praveen M <le...@gmail.com>
> wrote:
> >
> >> Great, that helps.
> >>
> >> Would you have a similar recommendation for the consumer side?
> >>
> >> In my environment, I expect about 200 consumer threads per instance.
> Total
> >> (30 instance * 200 threads/per instance) - 6000 consumer threads
> >>
> >> I was thinking I'd just use one connection per instance and then spin
> off
> >> 200 sessions/consumers.
> >>
> >> my onMessage() callback is expected to take atleast 1 second to process
> (I
> >> don't see a reason why a message should be submitted for async
> processing
> >> if it takes lesser than that to process)
> >> , so I assumed I must be ok with one connection.
> >>
> >> A latency is totally acceptable before a message gets picked up.
> >> Is there any recommendations towards connection management per
> >> instance for the consumer side of things? I'd like to hear what you
> think
> >> about it. If I'm right to assume 1 connection per
> >> instance should be good?
> >>
> >> Also,
> >> Since the consumer side would add more parallel connections to the
> broker,
> >> I'm curious upon the limits I could push the broker.
> >> Is there any tested limit in terms of the number of connections that the
> >> broker could support?
> >> We are looking at 180 concurrent connections to the broker if I say 1
> >> connection per instance for all the consumers and 5 connections per
> >> instance for enqueues Should that be ok?
> >>
> >> Also, are there any cons for having a lot of connections to the broker
> >> which I should be aware of?
> >>
> >> Thanks a lot,
> >> Praveen
> >>
> >>
> >> On Wed, Jan 11, 2012 at 9:52 AM, Robbie Gemmell <
> robbie.gemmell@gmail.com>wrote:
> >>
> >>> 150 connections should be no problem.
> >>>
> >>> I would expect any locking on the client to be of little significance
> >>> to overall performance when using persistent messages, hitting the
> >>> disk is going to dominate overall performance. The issue with using
> >>> multiple Sessions on a Connection with persistent messages is that it
> >>> the Sessions are ultimately multiplexed onto the same TCP connection
> >>> and their data processed sequentially on arrival at the broker. If for
> >>> example you were to do 5 concurrent sends on 5 Sessions on the same
> >>> Connection, they will be processed onto the connection sequentially
> >>> and thus into the store sequentially upon arrival at the broker,
> >>> whereas if you do 5 concurrent sends for Sessions on individual
> >>> Connections there is scope for them to be processed into the store
> >>> concurrently and enable some/all of them to be write-combined and
> >>> synced to disk at the same time, reducing the required synchronous
> >>> disk IO to perform the operations and thus providing increased overall
> >>> performance/scalability by effectively 'doing more with less' so to
> >>> speak.
> >>>
> >>> Robbie
> >>>
> >>> On 11 January 2012 16:24, Praveen M <le...@gmail.com> wrote:
> >>> > Hi Robbie,
> >>> >
> >>> > Thanks for writing. I just had a few followup questions. Please see
> >>> in-line.
> >>> >
> >>> > Thanks a lot for helping out with this,
> >>> > Praveen
> >>> >
> >>> > On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <
> >>> robbie.gemmell@gmail.com>wrote:
> >>> >
> >>> >> Hi Praveen,
> >>> >>
> >>> >> If you are predominantly going to use persistent messages, then I
> >>> >> would actually look at your publishers using a single single session
> >>> >> per connection for the most part.
> >>> >>
> >>> >> The broker is currently able to best extract performance scaling
> from
> >>> >> the persistent stores (more so the BDBstore as per Robs recent
> >>> >> testing...though the results of which are again slightly out of date
> >>> >> now due to the upgrade to BDB5 earlier today) by performing actions
> on
> >>> >> a per-connection basis, so it can extract parrallism over commits to
> >>> >> the store. Using a lower number of connections with 5-10 sessions
> each
> >>> >> could be noticably slower at peak when using 150-300 publishers.
> >>> >>
> >>> >
> >>> > that totally makes sense. But now that I'd have approximately (30
> >>> instance
> >>> > * 5 connections) = 150
> >>> > connections to the broker, I wouldn't be overwhelming the broker
> right?
> >>> I'm
> >>> > guessing it shouldn't be a problem, But just want to run it by you?
> >>> >
> >>> > I understand that having separate connections is significantly
> faster as
> >>> > each
> >>> > connection would operate on it's own socket.
> >>> > Just out of curiosity I'd like to know, what are the performance
> >>> > degradation
> >>> > causes when using sessions sharing the same connection? I know for
> one
> >>> that
> >>> > the session has to hold on creation locks (for creation) and
> connection
> >>> > failover locks on send.
> >>> > Do you think this will be the cause of the major bottle necks or is
> >>> there
> >>> > something else too?
> >>> > (I'm sure you know the code back to front to answer this better) :)
> >>> >
> >>> >
> >>> >
> >>> >> If you are sending to a huge variety of Destinations then the way
> you
> >>> >> are using your Producers seems appropriate; creating producers isnt
> >>> >> necessarily that heavyweight an operation, but not doing so is
> >>> >> undeniably far more efficient.
> >>> >>
> >>> >
> >>> > ah, yep. that makes sense. I think I will stick to creating
> producers on
> >>> > connection pooling
> >>> > and creating the session
> >>> >
> >>> >
> >>> >>
> >>> >> I have actually never personally used sync_publish, but given that
> >>> >> both the underlying sync() call and session.commit() hit the broker
> >>> >> synchronously I would guess their performance should be pretty
> similar
> >>> >> for persistent messages with a batch size of 1, in which case I
> would
> >>> >> probably just go with transactions for 'pure JMS' purposes. If you
> >>> >> ever were using batches of >1 message that would seem to favor use
> of
> >>> >> transactions due to lowering the number of synchronous round trips
> to
> >>> >> the broker which would be required, increasing performance.
> >>> >>
> >>> >> yep, I totally agree with you on using transactions over the
> >>> sync_publish
> >>> > option
> >>> > for pure JMS purposes.
> >>> >
> >>> >
> >>> >> Robbie
> >>> >>
> >>> >> On 10 January 2012 17:48, Praveen M <le...@gmail.com>
> wrote:
> >>> >> > Hi Robbie,
> >>> >> >
> >>> >> > Thanks for writing.
> >>> >> >
> >>> >> > Here is some more context for my 2nd question.
> >>> >> >
> >>> >> > I'm expected to have about 30 client instances connecting to one
> >>> broker.
> >>> >> > About 90% of the messages to the broker are expected to be
> persistent
> >>> >> > messages.
> >>> >> >
> >>> >> > I haven't decided whether to use transactions or use the option
> >>> >> > sync_publish. The
> >>> >> > guarantee that we'd expect is that the broker receives the message
> >>> before
> >>> >> > proceeding to
> >>> >> > send the next message. I'd be happy to hear your recommendations
> >>> around
> >>> >> > this.
> >>> >> >
> >>> >> > From the code I read, it looked like sync_publish was working to
> that
> >>> >> > effect. But one thing I
> >>> >> > saw was the sync() call in the client held to a connection
> failover
> >>> lock,
> >>> >> > and I wasn't sure if the sync() call was blocking.
> >>> >> > Anyways, that said do you have a recommendation between
> transacted vs
> >>> >> > sync_publish in a multi client environment (things going in
> >>> parallel).
> >>> >> I'd
> >>> >> > really like to hear what you think about this.
> >>> >> >
> >>> >> > There will not be any batch enqueues (a top of my head). Almost
> all
> >>> our
> >>> >> > traffic will be having messages
> >>> >> > enqueued one at a time.
> >>> >> >
> >>> >> > The number of destinations we have could be about 4000-5000. But
> yes,
> >>> >> there
> >>> >> > is a strong likelihood of one
> >>> >> > queue being more popular and all the traffic getting routed
> there. We
> >>> >> > sometimes receive bursty enqueues of a certain type
> >>> >> > and is fair to assume that 5 or more enqueues of the same type to
> the
> >>> >> same
> >>> >> > queue is possible to happen concurrently.
> >>> >> >
> >>> >> > We plan to have own framework on top of Qpid which will have the
> >>> ability
> >>> >> to
> >>> >> > load balance between the destinations
> >>> >> > if one queue goes over a certain threshold (some hand-picked
> >>> number..not
> >>> >> > decided on yet) and then route traffic to another destination.
> >>> >> >
> >>> >> > And the overall traffic expected is somewhere between 10 million
> - 20
> >>> >> > million messages a day maybe.
> >>> >> >
> >>> >> > Please do let me know if you'd like to hear any other information.
> >>> >> >
> >>> >> > Thank you,
> >>> >> > Praveen
> >>> >> >
> >>> >> >
> >>> >> > But yes, we'd be using one of the two (whichever would perform
> best
> >>> with
> >>> >> > multiple concurrent clients).
> >>> >> >
> >>> >> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
> >>> >> robbie.gemmell@gmail.com>wrote:
> >>> >> >
> >>> >> >> Hi Praveen,
> >>> >> >>
> >>> >> >> Using either JNDI or Session.createQueue should work fine, but
> the
> >>> >> >> obvious limitation of the latter is that the (possibly
> >>> >> >> provider-specific) configuration is then part of your code
> (unless
> >>> you
> >>> >> >> look the information up somewhere before using it) and cant be
> >>> changed
> >>> >> >> without modifying the code.
> >>> >> >>
> >>> >> >> To better answer the other question, it would be good to first
> know
> >>> >> >> more specifics about the application and its messaging model, eg:
> >>> >> >> How many instances of the application are you likely to be
> running?
> >>> >> >> Will all your messages be persistent?
> >>> >> >> Will you always use transactions (and if so will it always of
> batch
> >>> >> >> size 1 like below)?
> >>> >> >> You mention having maybe 5-10 concurrent publishers, but how many
> >>> >> >> different Destinations are you likely to be sending to in total?
> >>> >> >> Are any of the above Destinations likely to be used a lot more
> than
> >>> the
> >>> >> >> others?
> >>> >> >>
> >>> >> >> Robbie
> >>> >> >>
> >>> >> >> On 10 January 2012 00:27, Praveen M <le...@gmail.com>
> >>> wrote:
> >>> >> >> > Hi,
> >>> >> >> >
> >>> >> >> > I'm writing a JMS Client and have a few best practices
> >>> questions.
> >>> >> >> >
> >>> >> >> > 1) I understand that there are different ways ways to create a
> >>> >> reference
> >>> >> >> to
> >>> >> >> > a destination to enqueue a message (using JMS createQueue,
> using
> >>> JNDI
> >>> >> >> > lookup).
> >>> >> >> > What would be the recommended way to use?
> >>> >> >> >
> >>> >> >> > 2) I create a connection to the broker on my application
> startup,
> >>> and
> >>> >> >> > create a session pool. on every enqueue, i checkout a session
> >>> from my
> >>> >> >> pool
> >>> >> >> > use it and return to pool on enqueue.
> >>> >> >> > Does this sound reasonable? (code sample below)
> >>> >> >> >
> >>> >> >> > What would be the pattern that is typically used/recommended
> >>> for
> >>> >> >> > session management/connection management in the client side?
> >>> >> >> >
> >>> >> >> > Or is this an overkill to do? Is CreateSession() and
> >>> >> CreateProducer()
> >>> >> >> > really cheap that it is just done on each enqueue?
> >>> >> >> > In my use case, I'd expect quite some concurrency (assume n
> >>> (maybe
> >>> >> >> > 5-10) threads doing enqueues at the same time) and I'd prefer
> it
> >>> to
> >>> >> not
> >>> >> >> > lock/block and go through asap.
> >>> >> >> >
> >>> >> >> > I do remember from reading the qpid client code that there
> is a
> >>> >> lock
> >>> >> >> > held on the connection object whenever a new session is
> created on
> >>> >> that
> >>> >> >> > connection. I'm assuming it is one of the places where my
> enqueue
> >>> can
> >>> >> >> slow
> >>> >> >> > down
> >>> >> >> > when going in parallel if it has to create a session for
> every
> >>> >> enqueue
> >>> >> >> > of a message. There might be more such blocks, which I'm not
> >>> aware of.
> >>> >> >> >
> >>> >> >> > Taking performance into mind, what would be the best design
> for
> >>> >> >> > session/connection management?
> >>> >> >> >
> >>> >> >> > A snippet of what I have currently is below,
> >>> >> >> >
> >>> >> >> > // a thread safe queue with the available session producer
> >>> pairs
> >>> >> which
> >>> >> >> > can be used to create a message and enqueue.
> >>> >> >> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
> >>> >> >> > availableSessionProducerPairs = new
> >>> >> ConcurrentLinkedQueue<Pair<Session,
> >>> >> >> > MessageProducer>> ();
> >>> >> >> >
> >>> >> >> > // initialize the sessionProducerPairs, create upto n pairs
> >>> >> >> > initializePool(Connection qpidConnection) {
> >>> >> >> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i
> ++
> >>> ) {
> >>> >> >> > Session session = qpidConnection.createSession(true,
> >>> >> >> > Session.SESSION_TRANSACTED);
> >>> >> >> > MessageProducer producer =
> >>> session.createProducer(null);
> >>> >> >> >
> availableSessionProducerPairs.add(Pair.newPair(session,
> >>> >> >> > producer));
> >>> >> >> > }
> >>> >> >> > }
> >>> >> >> >
> >>> >> >> > // on enqueue we checkout a session/producer pair use it for
> the
> >>> >> enqueue
> >>> >> >> > and then return it back to the session/produer pair pool.
> >>> >> >> > enqueueMessage(String queueName, byte[] message) {
> >>> >> >> > // remove the session producer pair
> >>> >> >> > Pair<Session, MessageProducer> sessionProducerPair
> =
> >>> >> >> > availableSessionProducerPairs.poll();
> >>> >> >> > if(sessionProducerPair == null) {
> >>> >> >> > // gack here and return null.
> >>> >> >> > }
> >>> >> >> > Session session = sessionProducerPair.getFirst();
> >>> >> >> > MessageProducer producer =
> sessionProducerPair.getSecond();
> >>> >> >> > BytesMessage jmsMessage = session.createBytesMessage();
> >>> >> >> > jmsMessage.writeBytes(message);
> >>> >> >> > Queue queueRef = queueNameToReferenceMap.get(queueName);
> >>> >> >> > producer.send(queueRef, jmsMessage,
> >>> DeliveryMode.PERSISTENT, 0,
> >>> >> >> 0);
> >>> >> >> > session.commit();
> >>> >> >> > availableSessionProducersPair.add()
> >>> >> >> > }
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >
> >>> >> >> > Thank you,
> >>> >> >> > -Praveen
> >>> >> >>
> >>> >> >>
> >>> ---------------------------------------------------------------------
> >>> >> >> Apache Qpid - AMQP Messaging Implementation
> >>> >> >> Project: http://qpid.apache.org
> >>> >> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >>> >> >>
> >>> >> >>
> >>> >> >
> >>> >> >
> >>> >> > --
> >>> >> > -Praveen
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> Apache Qpid - AMQP Messaging Implementation
> >>> >> Project: http://qpid.apache.org
> >>> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>> > --
> >>> > -Praveen
> >>>
> >>> ---------------------------------------------------------------------
> >>> Apache Qpid - AMQP Messaging Implementation
> >>> Project: http://qpid.apache.org
> >>> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >>>
> >>>
> >>
> >>
> >> --
> >> -Praveen
> >>
> >
> >
> >
> > --
> > -Praveen
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
--
-Praveen
Re: JMS Client usage - best practices?
Posted by Robbie Gemmell <ro...@gmail.com>.
Hi Praveen,
180 connections should also be entirely fine, I have used several
hundred. The only limit is going to be memory based.The key issue with
using large amounts of connections currently is that when we moved
away from Mina as the IO layer for 0.14 we moved to a model that uses
2 threads per connection, so there are a large number of Threads
created which in turn will require memory for their stacks and heap
for the objects they contain. Additionally, there are currently some
large buffers placed in the heap on a per-connection basis that can
quickly mount up. We are going to look at the impact those have on
performance at particular buffer sizes and see if we can reduce the
default, but it can certainly be reconfigured to meet a particular
users requirements.
With such a low throughput of <=1msg/s per consumer, there is vastly
reduced scope for write-combining the store updates and so the use of
a single connection becomes far less of issue. There will be scope to
write-combine with other consumer instances and also the various
producers, but it may still be worth looking at something inbetween
the extremes of 20 Consumers per Connection and 1 Consumer per
Connection (eg 10 Consumers per Connection), to see if it is possible
to extract some scalability.
With your slow onMessage handler the thing you should be most worried
about is avoiding using multiple Consumers on the same Session,
because each Session only has a single thread performing the
deliveries. You will want to make sure you use a Session per Consumer.
Robbie
On 11 January 2012 19:02, Praveen M <le...@gmail.com> wrote:
> ah, my bad... I expect to have 20 consumer threads per instance. Sorry, so
> that'd be 600 consumer threads in the environment.
>
> On Wed, Jan 11, 2012 at 10:14 AM, Praveen M <le...@gmail.com> wrote:
>
>> Great, that helps.
>>
>> Would you have a similar recommendation for the consumer side?
>>
>> In my environment, I expect about 200 consumer threads per instance. Total
>> (30 instance * 200 threads/per instance) - 6000 consumer threads
>>
>> I was thinking I'd just use one connection per instance and then spin off
>> 200 sessions/consumers.
>>
>> my onMessage() callback is expected to take atleast 1 second to process (I
>> don't see a reason why a message should be submitted for async processing
>> if it takes lesser than that to process)
>> , so I assumed I must be ok with one connection.
>>
>> A latency is totally acceptable before a message gets picked up.
>> Is there any recommendations towards connection management per
>> instance for the consumer side of things? I'd like to hear what you think
>> about it. If I'm right to assume 1 connection per
>> instance should be good?
>>
>> Also,
>> Since the consumer side would add more parallel connections to the broker,
>> I'm curious upon the limits I could push the broker.
>> Is there any tested limit in terms of the number of connections that the
>> broker could support?
>> We are looking at 180 concurrent connections to the broker if I say 1
>> connection per instance for all the consumers and 5 connections per
>> instance for enqueues Should that be ok?
>>
>> Also, are there any cons for having a lot of connections to the broker
>> which I should be aware of?
>>
>> Thanks a lot,
>> Praveen
>>
>>
>> On Wed, Jan 11, 2012 at 9:52 AM, Robbie Gemmell <ro...@gmail.com>wrote:
>>
>>> 150 connections should be no problem.
>>>
>>> I would expect any locking on the client to be of little significance
>>> to overall performance when using persistent messages, hitting the
>>> disk is going to dominate overall performance. The issue with using
>>> multiple Sessions on a Connection with persistent messages is that it
>>> the Sessions are ultimately multiplexed onto the same TCP connection
>>> and their data processed sequentially on arrival at the broker. If for
>>> example you were to do 5 concurrent sends on 5 Sessions on the same
>>> Connection, they will be processed onto the connection sequentially
>>> and thus into the store sequentially upon arrival at the broker,
>>> whereas if you do 5 concurrent sends for Sessions on individual
>>> Connections there is scope for them to be processed into the store
>>> concurrently and enable some/all of them to be write-combined and
>>> synced to disk at the same time, reducing the required synchronous
>>> disk IO to perform the operations and thus providing increased overall
>>> performance/scalability by effectively 'doing more with less' so to
>>> speak.
>>>
>>> Robbie
>>>
>>> On 11 January 2012 16:24, Praveen M <le...@gmail.com> wrote:
>>> > Hi Robbie,
>>> >
>>> > Thanks for writing. I just had a few followup questions. Please see
>>> in-line.
>>> >
>>> > Thanks a lot for helping out with this,
>>> > Praveen
>>> >
>>> > On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <
>>> robbie.gemmell@gmail.com>wrote:
>>> >
>>> >> Hi Praveen,
>>> >>
>>> >> If you are predominantly going to use persistent messages, then I
>>> >> would actually look at your publishers using a single single session
>>> >> per connection for the most part.
>>> >>
>>> >> The broker is currently able to best extract performance scaling from
>>> >> the persistent stores (more so the BDBstore as per Robs recent
>>> >> testing...though the results of which are again slightly out of date
>>> >> now due to the upgrade to BDB5 earlier today) by performing actions on
>>> >> a per-connection basis, so it can extract parrallism over commits to
>>> >> the store. Using a lower number of connections with 5-10 sessions each
>>> >> could be noticably slower at peak when using 150-300 publishers.
>>> >>
>>> >
>>> > that totally makes sense. But now that I'd have approximately (30
>>> instance
>>> > * 5 connections) = 150
>>> > connections to the broker, I wouldn't be overwhelming the broker right?
>>> I'm
>>> > guessing it shouldn't be a problem, But just want to run it by you?
>>> >
>>> > I understand that having separate connections is significantly faster as
>>> > each
>>> > connection would operate on it's own socket.
>>> > Just out of curiosity I'd like to know, what are the performance
>>> > degradation
>>> > causes when using sessions sharing the same connection? I know for one
>>> that
>>> > the session has to hold on creation locks (for creation) and connection
>>> > failover locks on send.
>>> > Do you think this will be the cause of the major bottle necks or is
>>> there
>>> > something else too?
>>> > (I'm sure you know the code back to front to answer this better) :)
>>> >
>>> >
>>> >
>>> >> If you are sending to a huge variety of Destinations then the way you
>>> >> are using your Producers seems appropriate; creating producers isnt
>>> >> necessarily that heavyweight an operation, but not doing so is
>>> >> undeniably far more efficient.
>>> >>
>>> >
>>> > ah, yep. that makes sense. I think I will stick to creating producers on
>>> > connection pooling
>>> > and creating the session
>>> >
>>> >
>>> >>
>>> >> I have actually never personally used sync_publish, but given that
>>> >> both the underlying sync() call and session.commit() hit the broker
>>> >> synchronously I would guess their performance should be pretty similar
>>> >> for persistent messages with a batch size of 1, in which case I would
>>> >> probably just go with transactions for 'pure JMS' purposes. If you
>>> >> ever were using batches of >1 message that would seem to favor use of
>>> >> transactions due to lowering the number of synchronous round trips to
>>> >> the broker which would be required, increasing performance.
>>> >>
>>> >> yep, I totally agree with you on using transactions over the
>>> sync_publish
>>> > option
>>> > for pure JMS purposes.
>>> >
>>> >
>>> >> Robbie
>>> >>
>>> >> On 10 January 2012 17:48, Praveen M <le...@gmail.com> wrote:
>>> >> > Hi Robbie,
>>> >> >
>>> >> > Thanks for writing.
>>> >> >
>>> >> > Here is some more context for my 2nd question.
>>> >> >
>>> >> > I'm expected to have about 30 client instances connecting to one
>>> broker.
>>> >> > About 90% of the messages to the broker are expected to be persistent
>>> >> > messages.
>>> >> >
>>> >> > I haven't decided whether to use transactions or use the option
>>> >> > sync_publish. The
>>> >> > guarantee that we'd expect is that the broker receives the message
>>> before
>>> >> > proceeding to
>>> >> > send the next message. I'd be happy to hear your recommendations
>>> around
>>> >> > this.
>>> >> >
>>> >> > From the code I read, it looked like sync_publish was working to that
>>> >> > effect. But one thing I
>>> >> > saw was the sync() call in the client held to a connection failover
>>> lock,
>>> >> > and I wasn't sure if the sync() call was blocking.
>>> >> > Anyways, that said do you have a recommendation between transacted vs
>>> >> > sync_publish in a multi client environment (things going in
>>> parallel).
>>> >> I'd
>>> >> > really like to hear what you think about this.
>>> >> >
>>> >> > There will not be any batch enqueues (a top of my head). Almost all
>>> our
>>> >> > traffic will be having messages
>>> >> > enqueued one at a time.
>>> >> >
>>> >> > The number of destinations we have could be about 4000-5000. But yes,
>>> >> there
>>> >> > is a strong likelihood of one
>>> >> > queue being more popular and all the traffic getting routed there. We
>>> >> > sometimes receive bursty enqueues of a certain type
>>> >> > and is fair to assume that 5 or more enqueues of the same type to the
>>> >> same
>>> >> > queue is possible to happen concurrently.
>>> >> >
>>> >> > We plan to have own framework on top of Qpid which will have the
>>> ability
>>> >> to
>>> >> > load balance between the destinations
>>> >> > if one queue goes over a certain threshold (some hand-picked
>>> number..not
>>> >> > decided on yet) and then route traffic to another destination.
>>> >> >
>>> >> > And the overall traffic expected is somewhere between 10 million - 20
>>> >> > million messages a day maybe.
>>> >> >
>>> >> > Please do let me know if you'd like to hear any other information.
>>> >> >
>>> >> > Thank you,
>>> >> > Praveen
>>> >> >
>>> >> >
>>> >> > But yes, we'd be using one of the two (whichever would perform best
>>> with
>>> >> > multiple concurrent clients).
>>> >> >
>>> >> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
>>> >> robbie.gemmell@gmail.com>wrote:
>>> >> >
>>> >> >> Hi Praveen,
>>> >> >>
>>> >> >> Using either JNDI or Session.createQueue should work fine, but the
>>> >> >> obvious limitation of the latter is that the (possibly
>>> >> >> provider-specific) configuration is then part of your code (unless
>>> you
>>> >> >> look the information up somewhere before using it) and cant be
>>> changed
>>> >> >> without modifying the code.
>>> >> >>
>>> >> >> To better answer the other question, it would be good to first know
>>> >> >> more specifics about the application and its messaging model, eg:
>>> >> >> How many instances of the application are you likely to be running?
>>> >> >> Will all your messages be persistent?
>>> >> >> Will you always use transactions (and if so will it always of batch
>>> >> >> size 1 like below)?
>>> >> >> You mention having maybe 5-10 concurrent publishers, but how many
>>> >> >> different Destinations are you likely to be sending to in total?
>>> >> >> Are any of the above Destinations likely to be used a lot more than
>>> the
>>> >> >> others?
>>> >> >>
>>> >> >> Robbie
>>> >> >>
>>> >> >> On 10 January 2012 00:27, Praveen M <le...@gmail.com>
>>> wrote:
>>> >> >> > Hi,
>>> >> >> >
>>> >> >> > I'm writing a JMS Client and have a few best practices
>>> questions.
>>> >> >> >
>>> >> >> > 1) I understand that there are different ways ways to create a
>>> >> reference
>>> >> >> to
>>> >> >> > a destination to enqueue a message (using JMS createQueue, using
>>> JNDI
>>> >> >> > lookup).
>>> >> >> > What would be the recommended way to use?
>>> >> >> >
>>> >> >> > 2) I create a connection to the broker on my application startup,
>>> and
>>> >> >> > create a session pool. on every enqueue, i checkout a session
>>> from my
>>> >> >> pool
>>> >> >> > use it and return to pool on enqueue.
>>> >> >> > Does this sound reasonable? (code sample below)
>>> >> >> >
>>> >> >> > What would be the pattern that is typically used/recommended
>>> for
>>> >> >> > session management/connection management in the client side?
>>> >> >> >
>>> >> >> > Or is this an overkill to do? Is CreateSession() and
>>> >> CreateProducer()
>>> >> >> > really cheap that it is just done on each enqueue?
>>> >> >> > In my use case, I'd expect quite some concurrency (assume n
>>> (maybe
>>> >> >> > 5-10) threads doing enqueues at the same time) and I'd prefer it
>>> to
>>> >> not
>>> >> >> > lock/block and go through asap.
>>> >> >> >
>>> >> >> > I do remember from reading the qpid client code that there is a
>>> >> lock
>>> >> >> > held on the connection object whenever a new session is created on
>>> >> that
>>> >> >> > connection. I'm assuming it is one of the places where my enqueue
>>> can
>>> >> >> slow
>>> >> >> > down
>>> >> >> > when going in parallel if it has to create a session for every
>>> >> enqueue
>>> >> >> > of a message. There might be more such blocks, which I'm not
>>> aware of.
>>> >> >> >
>>> >> >> > Taking performance into mind, what would be the best design for
>>> >> >> > session/connection management?
>>> >> >> >
>>> >> >> > A snippet of what I have currently is below,
>>> >> >> >
>>> >> >> > // a thread safe queue with the available session producer
>>> pairs
>>> >> which
>>> >> >> > can be used to create a message and enqueue.
>>> >> >> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
>>> >> >> > availableSessionProducerPairs = new
>>> >> ConcurrentLinkedQueue<Pair<Session,
>>> >> >> > MessageProducer>> ();
>>> >> >> >
>>> >> >> > // initialize the sessionProducerPairs, create upto n pairs
>>> >> >> > initializePool(Connection qpidConnection) {
>>> >> >> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++
>>> ) {
>>> >> >> > Session session = qpidConnection.createSession(true,
>>> >> >> > Session.SESSION_TRANSACTED);
>>> >> >> > MessageProducer producer =
>>> session.createProducer(null);
>>> >> >> > availableSessionProducerPairs.add(Pair.newPair(session,
>>> >> >> > producer));
>>> >> >> > }
>>> >> >> > }
>>> >> >> >
>>> >> >> > // on enqueue we checkout a session/producer pair use it for the
>>> >> enqueue
>>> >> >> > and then return it back to the session/produer pair pool.
>>> >> >> > enqueueMessage(String queueName, byte[] message) {
>>> >> >> > // remove the session producer pair
>>> >> >> > Pair<Session, MessageProducer> sessionProducerPair =
>>> >> >> > availableSessionProducerPairs.poll();
>>> >> >> > if(sessionProducerPair == null) {
>>> >> >> > // gack here and return null.
>>> >> >> > }
>>> >> >> > Session session = sessionProducerPair.getFirst();
>>> >> >> > MessageProducer producer = sessionProducerPair.getSecond();
>>> >> >> > BytesMessage jmsMessage = session.createBytesMessage();
>>> >> >> > jmsMessage.writeBytes(message);
>>> >> >> > Queue queueRef = queueNameToReferenceMap.get(queueName);
>>> >> >> > producer.send(queueRef, jmsMessage,
>>> DeliveryMode.PERSISTENT, 0,
>>> >> >> 0);
>>> >> >> > session.commit();
>>> >> >> > availableSessionProducersPair.add()
>>> >> >> > }
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > Thank you,
>>> >> >> > -Praveen
>>> >> >>
>>> >> >>
>>> ---------------------------------------------------------------------
>>> >> >> Apache Qpid - AMQP Messaging Implementation
>>> >> >> Project: http://qpid.apache.org
>>> >> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>> >> >>
>>> >> >>
>>> >> >
>>> >> >
>>> >> > --
>>> >> > -Praveen
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> Apache Qpid - AMQP Messaging Implementation
>>> >> Project: http://qpid.apache.org
>>> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>> >>
>>> >>
>>> >
>>> >
>>> > --
>>> > -Praveen
>>>
>>> ---------------------------------------------------------------------
>>> Apache Qpid - AMQP Messaging Implementation
>>> Project: http://qpid.apache.org
>>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>>
>>>
>>
>>
>> --
>> -Praveen
>>
>
>
>
> --
> -Praveen
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by Praveen M <le...@gmail.com>.
ah, my bad... I expect to have 20 consumer threads per instance. Sorry, so
that'd be 600 consumer threads in the environment.
On Wed, Jan 11, 2012 at 10:14 AM, Praveen M <le...@gmail.com> wrote:
> Great, that helps.
>
> Would you have a similar recommendation for the consumer side?
>
> In my environment, I expect about 200 consumer threads per instance. Total
> (30 instance * 200 threads/per instance) - 6000 consumer threads
>
> I was thinking I'd just use one connection per instance and then spin off
> 200 sessions/consumers.
>
> my onMessage() callback is expected to take atleast 1 second to process (I
> don't see a reason why a message should be submitted for async processing
> if it takes lesser than that to process)
> , so I assumed I must be ok with one connection.
>
> A latency is totally acceptable before a message gets picked up.
> Is there any recommendations towards connection management per
> instance for the consumer side of things? I'd like to hear what you think
> about it. If I'm right to assume 1 connection per
> instance should be good?
>
> Also,
> Since the consumer side would add more parallel connections to the broker,
> I'm curious upon the limits I could push the broker.
> Is there any tested limit in terms of the number of connections that the
> broker could support?
> We are looking at 180 concurrent connections to the broker if I say 1
> connection per instance for all the consumers and 5 connections per
> instance for enqueues Should that be ok?
>
> Also, are there any cons for having a lot of connections to the broker
> which I should be aware of?
>
> Thanks a lot,
> Praveen
>
>
> On Wed, Jan 11, 2012 at 9:52 AM, Robbie Gemmell <ro...@gmail.com>wrote:
>
>> 150 connections should be no problem.
>>
>> I would expect any locking on the client to be of little significance
>> to overall performance when using persistent messages, hitting the
>> disk is going to dominate overall performance. The issue with using
>> multiple Sessions on a Connection with persistent messages is that it
>> the Sessions are ultimately multiplexed onto the same TCP connection
>> and their data processed sequentially on arrival at the broker. If for
>> example you were to do 5 concurrent sends on 5 Sessions on the same
>> Connection, they will be processed onto the connection sequentially
>> and thus into the store sequentially upon arrival at the broker,
>> whereas if you do 5 concurrent sends for Sessions on individual
>> Connections there is scope for them to be processed into the store
>> concurrently and enable some/all of them to be write-combined and
>> synced to disk at the same time, reducing the required synchronous
>> disk IO to perform the operations and thus providing increased overall
>> performance/scalability by effectively 'doing more with less' so to
>> speak.
>>
>> Robbie
>>
>> On 11 January 2012 16:24, Praveen M <le...@gmail.com> wrote:
>> > Hi Robbie,
>> >
>> > Thanks for writing. I just had a few followup questions. Please see
>> in-line.
>> >
>> > Thanks a lot for helping out with this,
>> > Praveen
>> >
>> > On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <
>> robbie.gemmell@gmail.com>wrote:
>> >
>> >> Hi Praveen,
>> >>
>> >> If you are predominantly going to use persistent messages, then I
>> >> would actually look at your publishers using a single single session
>> >> per connection for the most part.
>> >>
>> >> The broker is currently able to best extract performance scaling from
>> >> the persistent stores (more so the BDBstore as per Robs recent
>> >> testing...though the results of which are again slightly out of date
>> >> now due to the upgrade to BDB5 earlier today) by performing actions on
>> >> a per-connection basis, so it can extract parrallism over commits to
>> >> the store. Using a lower number of connections with 5-10 sessions each
>> >> could be noticably slower at peak when using 150-300 publishers.
>> >>
>> >
>> > that totally makes sense. But now that I'd have approximately (30
>> instance
>> > * 5 connections) = 150
>> > connections to the broker, I wouldn't be overwhelming the broker right?
>> I'm
>> > guessing it shouldn't be a problem, But just want to run it by you?
>> >
>> > I understand that having separate connections is significantly faster as
>> > each
>> > connection would operate on it's own socket.
>> > Just out of curiosity I'd like to know, what are the performance
>> > degradation
>> > causes when using sessions sharing the same connection? I know for one
>> that
>> > the session has to hold on creation locks (for creation) and connection
>> > failover locks on send.
>> > Do you think this will be the cause of the major bottle necks or is
>> there
>> > something else too?
>> > (I'm sure you know the code back to front to answer this better) :)
>> >
>> >
>> >
>> >> If you are sending to a huge variety of Destinations then the way you
>> >> are using your Producers seems appropriate; creating producers isnt
>> >> necessarily that heavyweight an operation, but not doing so is
>> >> undeniably far more efficient.
>> >>
>> >
>> > ah, yep. that makes sense. I think I will stick to creating producers on
>> > connection pooling
>> > and creating the session
>> >
>> >
>> >>
>> >> I have actually never personally used sync_publish, but given that
>> >> both the underlying sync() call and session.commit() hit the broker
>> >> synchronously I would guess their performance should be pretty similar
>> >> for persistent messages with a batch size of 1, in which case I would
>> >> probably just go with transactions for 'pure JMS' purposes. If you
>> >> ever were using batches of >1 message that would seem to favor use of
>> >> transactions due to lowering the number of synchronous round trips to
>> >> the broker which would be required, increasing performance.
>> >>
>> >> yep, I totally agree with you on using transactions over the
>> sync_publish
>> > option
>> > for pure JMS purposes.
>> >
>> >
>> >> Robbie
>> >>
>> >> On 10 January 2012 17:48, Praveen M <le...@gmail.com> wrote:
>> >> > Hi Robbie,
>> >> >
>> >> > Thanks for writing.
>> >> >
>> >> > Here is some more context for my 2nd question.
>> >> >
>> >> > I'm expected to have about 30 client instances connecting to one
>> broker.
>> >> > About 90% of the messages to the broker are expected to be persistent
>> >> > messages.
>> >> >
>> >> > I haven't decided whether to use transactions or use the option
>> >> > sync_publish. The
>> >> > guarantee that we'd expect is that the broker receives the message
>> before
>> >> > proceeding to
>> >> > send the next message. I'd be happy to hear your recommendations
>> around
>> >> > this.
>> >> >
>> >> > From the code I read, it looked like sync_publish was working to that
>> >> > effect. But one thing I
>> >> > saw was the sync() call in the client held to a connection failover
>> lock,
>> >> > and I wasn't sure if the sync() call was blocking.
>> >> > Anyways, that said do you have a recommendation between transacted vs
>> >> > sync_publish in a multi client environment (things going in
>> parallel).
>> >> I'd
>> >> > really like to hear what you think about this.
>> >> >
>> >> > There will not be any batch enqueues (a top of my head). Almost all
>> our
>> >> > traffic will be having messages
>> >> > enqueued one at a time.
>> >> >
>> >> > The number of destinations we have could be about 4000-5000. But yes,
>> >> there
>> >> > is a strong likelihood of one
>> >> > queue being more popular and all the traffic getting routed there. We
>> >> > sometimes receive bursty enqueues of a certain type
>> >> > and is fair to assume that 5 or more enqueues of the same type to the
>> >> same
>> >> > queue is possible to happen concurrently.
>> >> >
>> >> > We plan to have own framework on top of Qpid which will have the
>> ability
>> >> to
>> >> > load balance between the destinations
>> >> > if one queue goes over a certain threshold (some hand-picked
>> number..not
>> >> > decided on yet) and then route traffic to another destination.
>> >> >
>> >> > And the overall traffic expected is somewhere between 10 million - 20
>> >> > million messages a day maybe.
>> >> >
>> >> > Please do let me know if you'd like to hear any other information.
>> >> >
>> >> > Thank you,
>> >> > Praveen
>> >> >
>> >> >
>> >> > But yes, we'd be using one of the two (whichever would perform best
>> with
>> >> > multiple concurrent clients).
>> >> >
>> >> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
>> >> robbie.gemmell@gmail.com>wrote:
>> >> >
>> >> >> Hi Praveen,
>> >> >>
>> >> >> Using either JNDI or Session.createQueue should work fine, but the
>> >> >> obvious limitation of the latter is that the (possibly
>> >> >> provider-specific) configuration is then part of your code (unless
>> you
>> >> >> look the information up somewhere before using it) and cant be
>> changed
>> >> >> without modifying the code.
>> >> >>
>> >> >> To better answer the other question, it would be good to first know
>> >> >> more specifics about the application and its messaging model, eg:
>> >> >> How many instances of the application are you likely to be running?
>> >> >> Will all your messages be persistent?
>> >> >> Will you always use transactions (and if so will it always of batch
>> >> >> size 1 like below)?
>> >> >> You mention having maybe 5-10 concurrent publishers, but how many
>> >> >> different Destinations are you likely to be sending to in total?
>> >> >> Are any of the above Destinations likely to be used a lot more than
>> the
>> >> >> others?
>> >> >>
>> >> >> Robbie
>> >> >>
>> >> >> On 10 January 2012 00:27, Praveen M <le...@gmail.com>
>> wrote:
>> >> >> > Hi,
>> >> >> >
>> >> >> > I'm writing a JMS Client and have a few best practices
>> questions.
>> >> >> >
>> >> >> > 1) I understand that there are different ways ways to create a
>> >> reference
>> >> >> to
>> >> >> > a destination to enqueue a message (using JMS createQueue, using
>> JNDI
>> >> >> > lookup).
>> >> >> > What would be the recommended way to use?
>> >> >> >
>> >> >> > 2) I create a connection to the broker on my application startup,
>> and
>> >> >> > create a session pool. on every enqueue, i checkout a session
>> from my
>> >> >> pool
>> >> >> > use it and return to pool on enqueue.
>> >> >> > Does this sound reasonable? (code sample below)
>> >> >> >
>> >> >> > What would be the pattern that is typically used/recommended
>> for
>> >> >> > session management/connection management in the client side?
>> >> >> >
>> >> >> > Or is this an overkill to do? Is CreateSession() and
>> >> CreateProducer()
>> >> >> > really cheap that it is just done on each enqueue?
>> >> >> > In my use case, I'd expect quite some concurrency (assume n
>> (maybe
>> >> >> > 5-10) threads doing enqueues at the same time) and I'd prefer it
>> to
>> >> not
>> >> >> > lock/block and go through asap.
>> >> >> >
>> >> >> > I do remember from reading the qpid client code that there is a
>> >> lock
>> >> >> > held on the connection object whenever a new session is created on
>> >> that
>> >> >> > connection. I'm assuming it is one of the places where my enqueue
>> can
>> >> >> slow
>> >> >> > down
>> >> >> > when going in parallel if it has to create a session for every
>> >> enqueue
>> >> >> > of a message. There might be more such blocks, which I'm not
>> aware of.
>> >> >> >
>> >> >> > Taking performance into mind, what would be the best design for
>> >> >> > session/connection management?
>> >> >> >
>> >> >> > A snippet of what I have currently is below,
>> >> >> >
>> >> >> > // a thread safe queue with the available session producer
>> pairs
>> >> which
>> >> >> > can be used to create a message and enqueue.
>> >> >> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
>> >> >> > availableSessionProducerPairs = new
>> >> ConcurrentLinkedQueue<Pair<Session,
>> >> >> > MessageProducer>> ();
>> >> >> >
>> >> >> > // initialize the sessionProducerPairs, create upto n pairs
>> >> >> > initializePool(Connection qpidConnection) {
>> >> >> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++
>> ) {
>> >> >> > Session session = qpidConnection.createSession(true,
>> >> >> > Session.SESSION_TRANSACTED);
>> >> >> > MessageProducer producer =
>> session.createProducer(null);
>> >> >> > availableSessionProducerPairs.add(Pair.newPair(session,
>> >> >> > producer));
>> >> >> > }
>> >> >> > }
>> >> >> >
>> >> >> > // on enqueue we checkout a session/producer pair use it for the
>> >> enqueue
>> >> >> > and then return it back to the session/produer pair pool.
>> >> >> > enqueueMessage(String queueName, byte[] message) {
>> >> >> > // remove the session producer pair
>> >> >> > Pair<Session, MessageProducer> sessionProducerPair =
>> >> >> > availableSessionProducerPairs.poll();
>> >> >> > if(sessionProducerPair == null) {
>> >> >> > // gack here and return null.
>> >> >> > }
>> >> >> > Session session = sessionProducerPair.getFirst();
>> >> >> > MessageProducer producer = sessionProducerPair.getSecond();
>> >> >> > BytesMessage jmsMessage = session.createBytesMessage();
>> >> >> > jmsMessage.writeBytes(message);
>> >> >> > Queue queueRef = queueNameToReferenceMap.get(queueName);
>> >> >> > producer.send(queueRef, jmsMessage,
>> DeliveryMode.PERSISTENT, 0,
>> >> >> 0);
>> >> >> > session.commit();
>> >> >> > availableSessionProducersPair.add()
>> >> >> > }
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > Thank you,
>> >> >> > -Praveen
>> >> >>
>> >> >>
>> ---------------------------------------------------------------------
>> >> >> Apache Qpid - AMQP Messaging Implementation
>> >> >> Project: http://qpid.apache.org
>> >> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > -Praveen
>> >>
>> >> ---------------------------------------------------------------------
>> >> Apache Qpid - AMQP Messaging Implementation
>> >> Project: http://qpid.apache.org
>> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
>> >>
>> >>
>> >
>> >
>> > --
>> > -Praveen
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>>
>
>
> --
> -Praveen
>
--
-Praveen
Re: JMS Client usage - best practices?
Posted by Praveen M <le...@gmail.com>.
Great, that helps.
Would you have a similar recommendation for the consumer side?
In my environment, I expect about 200 consumer threads per instance. Total
(30 instance * 200 threads/per instance) - 6000 consumer threads
I was thinking I'd just use one connection per instance and then spin off
200 sessions/consumers.
my onMessage() callback is expected to take atleast 1 second to process (I
don't see a reason why a message should be submitted for async processing
if it takes lesser than that to process)
, so I assumed I must be ok with one connection.
A latency is totally acceptable before a message gets picked up.
Is there any recommendations towards connection management per instance for
the consumer side of things? I'd like to hear what you think about it. If
I'm right to assume 1 connection per
instance should be good?
Also,
Since the consumer side would add more parallel connections to the broker,
I'm curious upon the limits I could push the broker.
Is there any tested limit in terms of the number of connections that the
broker could support?
We are looking at 180 concurrent connections to the broker if I say 1
connection per instance for all the consumers and 5 connections per
instance for enqueues Should that be ok?
Also, are there any cons for having a lot of connections to the broker
which I should be aware of?
Thanks a lot,
Praveen
On Wed, Jan 11, 2012 at 9:52 AM, Robbie Gemmell <ro...@gmail.com>wrote:
> 150 connections should be no problem.
>
> I would expect any locking on the client to be of little significance
> to overall performance when using persistent messages, hitting the
> disk is going to dominate overall performance. The issue with using
> multiple Sessions on a Connection with persistent messages is that it
> the Sessions are ultimately multiplexed onto the same TCP connection
> and their data processed sequentially on arrival at the broker. If for
> example you were to do 5 concurrent sends on 5 Sessions on the same
> Connection, they will be processed onto the connection sequentially
> and thus into the store sequentially upon arrival at the broker,
> whereas if you do 5 concurrent sends for Sessions on individual
> Connections there is scope for them to be processed into the store
> concurrently and enable some/all of them to be write-combined and
> synced to disk at the same time, reducing the required synchronous
> disk IO to perform the operations and thus providing increased overall
> performance/scalability by effectively 'doing more with less' so to
> speak.
>
> Robbie
>
> On 11 January 2012 16:24, Praveen M <le...@gmail.com> wrote:
> > Hi Robbie,
> >
> > Thanks for writing. I just had a few followup questions. Please see
> in-line.
> >
> > Thanks a lot for helping out with this,
> > Praveen
> >
> > On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <
> robbie.gemmell@gmail.com>wrote:
> >
> >> Hi Praveen,
> >>
> >> If you are predominantly going to use persistent messages, then I
> >> would actually look at your publishers using a single single session
> >> per connection for the most part.
> >>
> >> The broker is currently able to best extract performance scaling from
> >> the persistent stores (more so the BDBstore as per Robs recent
> >> testing...though the results of which are again slightly out of date
> >> now due to the upgrade to BDB5 earlier today) by performing actions on
> >> a per-connection basis, so it can extract parrallism over commits to
> >> the store. Using a lower number of connections with 5-10 sessions each
> >> could be noticably slower at peak when using 150-300 publishers.
> >>
> >
> > that totally makes sense. But now that I'd have approximately (30
> instance
> > * 5 connections) = 150
> > connections to the broker, I wouldn't be overwhelming the broker right?
> I'm
> > guessing it shouldn't be a problem, But just want to run it by you?
> >
> > I understand that having separate connections is significantly faster as
> > each
> > connection would operate on it's own socket.
> > Just out of curiosity I'd like to know, what are the performance
> > degradation
> > causes when using sessions sharing the same connection? I know for one
> that
> > the session has to hold on creation locks (for creation) and connection
> > failover locks on send.
> > Do you think this will be the cause of the major bottle necks or is there
> > something else too?
> > (I'm sure you know the code back to front to answer this better) :)
> >
> >
> >
> >> If you are sending to a huge variety of Destinations then the way you
> >> are using your Producers seems appropriate; creating producers isnt
> >> necessarily that heavyweight an operation, but not doing so is
> >> undeniably far more efficient.
> >>
> >
> > ah, yep. that makes sense. I think I will stick to creating producers on
> > connection pooling
> > and creating the session
> >
> >
> >>
> >> I have actually never personally used sync_publish, but given that
> >> both the underlying sync() call and session.commit() hit the broker
> >> synchronously I would guess their performance should be pretty similar
> >> for persistent messages with a batch size of 1, in which case I would
> >> probably just go with transactions for 'pure JMS' purposes. If you
> >> ever were using batches of >1 message that would seem to favor use of
> >> transactions due to lowering the number of synchronous round trips to
> >> the broker which would be required, increasing performance.
> >>
> >> yep, I totally agree with you on using transactions over the
> sync_publish
> > option
> > for pure JMS purposes.
> >
> >
> >> Robbie
> >>
> >> On 10 January 2012 17:48, Praveen M <le...@gmail.com> wrote:
> >> > Hi Robbie,
> >> >
> >> > Thanks for writing.
> >> >
> >> > Here is some more context for my 2nd question.
> >> >
> >> > I'm expected to have about 30 client instances connecting to one
> broker.
> >> > About 90% of the messages to the broker are expected to be persistent
> >> > messages.
> >> >
> >> > I haven't decided whether to use transactions or use the option
> >> > sync_publish. The
> >> > guarantee that we'd expect is that the broker receives the message
> before
> >> > proceeding to
> >> > send the next message. I'd be happy to hear your recommendations
> around
> >> > this.
> >> >
> >> > From the code I read, it looked like sync_publish was working to that
> >> > effect. But one thing I
> >> > saw was the sync() call in the client held to a connection failover
> lock,
> >> > and I wasn't sure if the sync() call was blocking.
> >> > Anyways, that said do you have a recommendation between transacted vs
> >> > sync_publish in a multi client environment (things going in parallel).
> >> I'd
> >> > really like to hear what you think about this.
> >> >
> >> > There will not be any batch enqueues (a top of my head). Almost all
> our
> >> > traffic will be having messages
> >> > enqueued one at a time.
> >> >
> >> > The number of destinations we have could be about 4000-5000. But yes,
> >> there
> >> > is a strong likelihood of one
> >> > queue being more popular and all the traffic getting routed there. We
> >> > sometimes receive bursty enqueues of a certain type
> >> > and is fair to assume that 5 or more enqueues of the same type to the
> >> same
> >> > queue is possible to happen concurrently.
> >> >
> >> > We plan to have own framework on top of Qpid which will have the
> ability
> >> to
> >> > load balance between the destinations
> >> > if one queue goes over a certain threshold (some hand-picked
> number..not
> >> > decided on yet) and then route traffic to another destination.
> >> >
> >> > And the overall traffic expected is somewhere between 10 million - 20
> >> > million messages a day maybe.
> >> >
> >> > Please do let me know if you'd like to hear any other information.
> >> >
> >> > Thank you,
> >> > Praveen
> >> >
> >> >
> >> > But yes, we'd be using one of the two (whichever would perform best
> with
> >> > multiple concurrent clients).
> >> >
> >> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
> >> robbie.gemmell@gmail.com>wrote:
> >> >
> >> >> Hi Praveen,
> >> >>
> >> >> Using either JNDI or Session.createQueue should work fine, but the
> >> >> obvious limitation of the latter is that the (possibly
> >> >> provider-specific) configuration is then part of your code (unless
> you
> >> >> look the information up somewhere before using it) and cant be
> changed
> >> >> without modifying the code.
> >> >>
> >> >> To better answer the other question, it would be good to first know
> >> >> more specifics about the application and its messaging model, eg:
> >> >> How many instances of the application are you likely to be running?
> >> >> Will all your messages be persistent?
> >> >> Will you always use transactions (and if so will it always of batch
> >> >> size 1 like below)?
> >> >> You mention having maybe 5-10 concurrent publishers, but how many
> >> >> different Destinations are you likely to be sending to in total?
> >> >> Are any of the above Destinations likely to be used a lot more than
> the
> >> >> others?
> >> >>
> >> >> Robbie
> >> >>
> >> >> On 10 January 2012 00:27, Praveen M <le...@gmail.com> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I'm writing a JMS Client and have a few best practices questions.
> >> >> >
> >> >> > 1) I understand that there are different ways ways to create a
> >> reference
> >> >> to
> >> >> > a destination to enqueue a message (using JMS createQueue, using
> JNDI
> >> >> > lookup).
> >> >> > What would be the recommended way to use?
> >> >> >
> >> >> > 2) I create a connection to the broker on my application startup,
> and
> >> >> > create a session pool. on every enqueue, i checkout a session from
> my
> >> >> pool
> >> >> > use it and return to pool on enqueue.
> >> >> > Does this sound reasonable? (code sample below)
> >> >> >
> >> >> > What would be the pattern that is typically used/recommended for
> >> >> > session management/connection management in the client side?
> >> >> >
> >> >> > Or is this an overkill to do? Is CreateSession() and
> >> CreateProducer()
> >> >> > really cheap that it is just done on each enqueue?
> >> >> > In my use case, I'd expect quite some concurrency (assume n
> (maybe
> >> >> > 5-10) threads doing enqueues at the same time) and I'd prefer it to
> >> not
> >> >> > lock/block and go through asap.
> >> >> >
> >> >> > I do remember from reading the qpid client code that there is a
> >> lock
> >> >> > held on the connection object whenever a new session is created on
> >> that
> >> >> > connection. I'm assuming it is one of the places where my enqueue
> can
> >> >> slow
> >> >> > down
> >> >> > when going in parallel if it has to create a session for every
> >> enqueue
> >> >> > of a message. There might be more such blocks, which I'm not aware
> of.
> >> >> >
> >> >> > Taking performance into mind, what would be the best design for
> >> >> > session/connection management?
> >> >> >
> >> >> > A snippet of what I have currently is below,
> >> >> >
> >> >> > // a thread safe queue with the available session producer pairs
> >> which
> >> >> > can be used to create a message and enqueue.
> >> >> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
> >> >> > availableSessionProducerPairs = new
> >> ConcurrentLinkedQueue<Pair<Session,
> >> >> > MessageProducer>> ();
> >> >> >
> >> >> > // initialize the sessionProducerPairs, create upto n pairs
> >> >> > initializePool(Connection qpidConnection) {
> >> >> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ )
> {
> >> >> > Session session = qpidConnection.createSession(true,
> >> >> > Session.SESSION_TRANSACTED);
> >> >> > MessageProducer producer = session.createProducer(null);
> >> >> > availableSessionProducerPairs.add(Pair.newPair(session,
> >> >> > producer));
> >> >> > }
> >> >> > }
> >> >> >
> >> >> > // on enqueue we checkout a session/producer pair use it for the
> >> enqueue
> >> >> > and then return it back to the session/produer pair pool.
> >> >> > enqueueMessage(String queueName, byte[] message) {
> >> >> > // remove the session producer pair
> >> >> > Pair<Session, MessageProducer> sessionProducerPair =
> >> >> > availableSessionProducerPairs.poll();
> >> >> > if(sessionProducerPair == null) {
> >> >> > // gack here and return null.
> >> >> > }
> >> >> > Session session = sessionProducerPair.getFirst();
> >> >> > MessageProducer producer = sessionProducerPair.getSecond();
> >> >> > BytesMessage jmsMessage = session.createBytesMessage();
> >> >> > jmsMessage.writeBytes(message);
> >> >> > Queue queueRef = queueNameToReferenceMap.get(queueName);
> >> >> > producer.send(queueRef, jmsMessage,
> DeliveryMode.PERSISTENT, 0,
> >> >> 0);
> >> >> > session.commit();
> >> >> > availableSessionProducersPair.add()
> >> >> > }
> >> >> >
> >> >> >
> >> >> >
> >> >> > Thank you,
> >> >> > -Praveen
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> Apache Qpid - AMQP Messaging Implementation
> >> >> Project: http://qpid.apache.org
> >> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > -Praveen
> >>
> >> ---------------------------------------------------------------------
> >> Apache Qpid - AMQP Messaging Implementation
> >> Project: http://qpid.apache.org
> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >>
> >>
> >
> >
> > --
> > -Praveen
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
--
-Praveen
Re: JMS Client usage - best practices?
Posted by Robbie Gemmell <ro...@gmail.com>.
150 connections should be no problem.
I would expect any locking on the client to be of little significance
to overall performance when using persistent messages, hitting the
disk is going to dominate overall performance. The issue with using
multiple Sessions on a Connection with persistent messages is that it
the Sessions are ultimately multiplexed onto the same TCP connection
and their data processed sequentially on arrival at the broker. If for
example you were to do 5 concurrent sends on 5 Sessions on the same
Connection, they will be processed onto the connection sequentially
and thus into the store sequentially upon arrival at the broker,
whereas if you do 5 concurrent sends for Sessions on individual
Connections there is scope for them to be processed into the store
concurrently and enable some/all of them to be write-combined and
synced to disk at the same time, reducing the required synchronous
disk IO to perform the operations and thus providing increased overall
performance/scalability by effectively 'doing more with less' so to
speak.
Robbie
On 11 January 2012 16:24, Praveen M <le...@gmail.com> wrote:
> Hi Robbie,
>
> Thanks for writing. I just had a few followup questions. Please see in-line.
>
> Thanks a lot for helping out with this,
> Praveen
>
> On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <ro...@gmail.com>wrote:
>
>> Hi Praveen,
>>
>> If you are predominantly going to use persistent messages, then I
>> would actually look at your publishers using a single single session
>> per connection for the most part.
>>
>> The broker is currently able to best extract performance scaling from
>> the persistent stores (more so the BDBstore as per Robs recent
>> testing...though the results of which are again slightly out of date
>> now due to the upgrade to BDB5 earlier today) by performing actions on
>> a per-connection basis, so it can extract parrallism over commits to
>> the store. Using a lower number of connections with 5-10 sessions each
>> could be noticably slower at peak when using 150-300 publishers.
>>
>
> that totally makes sense. But now that I'd have approximately (30 instance
> * 5 connections) = 150
> connections to the broker, I wouldn't be overwhelming the broker right? I'm
> guessing it shouldn't be a problem, But just want to run it by you?
>
> I understand that having separate connections is significantly faster as
> each
> connection would operate on it's own socket.
> Just out of curiosity I'd like to know, what are the performance
> degradation
> causes when using sessions sharing the same connection? I know for one that
> the session has to hold on creation locks (for creation) and connection
> failover locks on send.
> Do you think this will be the cause of the major bottle necks or is there
> something else too?
> (I'm sure you know the code back to front to answer this better) :)
>
>
>
>> If you are sending to a huge variety of Destinations then the way you
>> are using your Producers seems appropriate; creating producers isnt
>> necessarily that heavyweight an operation, but not doing so is
>> undeniably far more efficient.
>>
>
> ah, yep. that makes sense. I think I will stick to creating producers on
> connection pooling
> and creating the session
>
>
>>
>> I have actually never personally used sync_publish, but given that
>> both the underlying sync() call and session.commit() hit the broker
>> synchronously I would guess their performance should be pretty similar
>> for persistent messages with a batch size of 1, in which case I would
>> probably just go with transactions for 'pure JMS' purposes. If you
>> ever were using batches of >1 message that would seem to favor use of
>> transactions due to lowering the number of synchronous round trips to
>> the broker which would be required, increasing performance.
>>
>> yep, I totally agree with you on using transactions over the sync_publish
> option
> for pure JMS purposes.
>
>
>> Robbie
>>
>> On 10 January 2012 17:48, Praveen M <le...@gmail.com> wrote:
>> > Hi Robbie,
>> >
>> > Thanks for writing.
>> >
>> > Here is some more context for my 2nd question.
>> >
>> > I'm expected to have about 30 client instances connecting to one broker.
>> > About 90% of the messages to the broker are expected to be persistent
>> > messages.
>> >
>> > I haven't decided whether to use transactions or use the option
>> > sync_publish. The
>> > guarantee that we'd expect is that the broker receives the message before
>> > proceeding to
>> > send the next message. I'd be happy to hear your recommendations around
>> > this.
>> >
>> > From the code I read, it looked like sync_publish was working to that
>> > effect. But one thing I
>> > saw was the sync() call in the client held to a connection failover lock,
>> > and I wasn't sure if the sync() call was blocking.
>> > Anyways, that said do you have a recommendation between transacted vs
>> > sync_publish in a multi client environment (things going in parallel).
>> I'd
>> > really like to hear what you think about this.
>> >
>> > There will not be any batch enqueues (a top of my head). Almost all our
>> > traffic will be having messages
>> > enqueued one at a time.
>> >
>> > The number of destinations we have could be about 4000-5000. But yes,
>> there
>> > is a strong likelihood of one
>> > queue being more popular and all the traffic getting routed there. We
>> > sometimes receive bursty enqueues of a certain type
>> > and is fair to assume that 5 or more enqueues of the same type to the
>> same
>> > queue is possible to happen concurrently.
>> >
>> > We plan to have own framework on top of Qpid which will have the ability
>> to
>> > load balance between the destinations
>> > if one queue goes over a certain threshold (some hand-picked number..not
>> > decided on yet) and then route traffic to another destination.
>> >
>> > And the overall traffic expected is somewhere between 10 million - 20
>> > million messages a day maybe.
>> >
>> > Please do let me know if you'd like to hear any other information.
>> >
>> > Thank you,
>> > Praveen
>> >
>> >
>> > But yes, we'd be using one of the two (whichever would perform best with
>> > multiple concurrent clients).
>> >
>> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
>> robbie.gemmell@gmail.com>wrote:
>> >
>> >> Hi Praveen,
>> >>
>> >> Using either JNDI or Session.createQueue should work fine, but the
>> >> obvious limitation of the latter is that the (possibly
>> >> provider-specific) configuration is then part of your code (unless you
>> >> look the information up somewhere before using it) and cant be changed
>> >> without modifying the code.
>> >>
>> >> To better answer the other question, it would be good to first know
>> >> more specifics about the application and its messaging model, eg:
>> >> How many instances of the application are you likely to be running?
>> >> Will all your messages be persistent?
>> >> Will you always use transactions (and if so will it always of batch
>> >> size 1 like below)?
>> >> You mention having maybe 5-10 concurrent publishers, but how many
>> >> different Destinations are you likely to be sending to in total?
>> >> Are any of the above Destinations likely to be used a lot more than the
>> >> others?
>> >>
>> >> Robbie
>> >>
>> >> On 10 January 2012 00:27, Praveen M <le...@gmail.com> wrote:
>> >> > Hi,
>> >> >
>> >> > I'm writing a JMS Client and have a few best practices questions.
>> >> >
>> >> > 1) I understand that there are different ways ways to create a
>> reference
>> >> to
>> >> > a destination to enqueue a message (using JMS createQueue, using JNDI
>> >> > lookup).
>> >> > What would be the recommended way to use?
>> >> >
>> >> > 2) I create a connection to the broker on my application startup, and
>> >> > create a session pool. on every enqueue, i checkout a session from my
>> >> pool
>> >> > use it and return to pool on enqueue.
>> >> > Does this sound reasonable? (code sample below)
>> >> >
>> >> > What would be the pattern that is typically used/recommended for
>> >> > session management/connection management in the client side?
>> >> >
>> >> > Or is this an overkill to do? Is CreateSession() and
>> CreateProducer()
>> >> > really cheap that it is just done on each enqueue?
>> >> > In my use case, I'd expect quite some concurrency (assume n (maybe
>> >> > 5-10) threads doing enqueues at the same time) and I'd prefer it to
>> not
>> >> > lock/block and go through asap.
>> >> >
>> >> > I do remember from reading the qpid client code that there is a
>> lock
>> >> > held on the connection object whenever a new session is created on
>> that
>> >> > connection. I'm assuming it is one of the places where my enqueue can
>> >> slow
>> >> > down
>> >> > when going in parallel if it has to create a session for every
>> enqueue
>> >> > of a message. There might be more such blocks, which I'm not aware of.
>> >> >
>> >> > Taking performance into mind, what would be the best design for
>> >> > session/connection management?
>> >> >
>> >> > A snippet of what I have currently is below,
>> >> >
>> >> > // a thread safe queue with the available session producer pairs
>> which
>> >> > can be used to create a message and enqueue.
>> >> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
>> >> > availableSessionProducerPairs = new
>> ConcurrentLinkedQueue<Pair<Session,
>> >> > MessageProducer>> ();
>> >> >
>> >> > // initialize the sessionProducerPairs, create upto n pairs
>> >> > initializePool(Connection qpidConnection) {
>> >> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ ) {
>> >> > Session session = qpidConnection.createSession(true,
>> >> > Session.SESSION_TRANSACTED);
>> >> > MessageProducer producer = session.createProducer(null);
>> >> > availableSessionProducerPairs.add(Pair.newPair(session,
>> >> > producer));
>> >> > }
>> >> > }
>> >> >
>> >> > // on enqueue we checkout a session/producer pair use it for the
>> enqueue
>> >> > and then return it back to the session/produer pair pool.
>> >> > enqueueMessage(String queueName, byte[] message) {
>> >> > // remove the session producer pair
>> >> > Pair<Session, MessageProducer> sessionProducerPair =
>> >> > availableSessionProducerPairs.poll();
>> >> > if(sessionProducerPair == null) {
>> >> > // gack here and return null.
>> >> > }
>> >> > Session session = sessionProducerPair.getFirst();
>> >> > MessageProducer producer = sessionProducerPair.getSecond();
>> >> > BytesMessage jmsMessage = session.createBytesMessage();
>> >> > jmsMessage.writeBytes(message);
>> >> > Queue queueRef = queueNameToReferenceMap.get(queueName);
>> >> > producer.send(queueRef, jmsMessage, DeliveryMode.PERSISTENT, 0,
>> >> 0);
>> >> > session.commit();
>> >> > availableSessionProducersPair.add()
>> >> > }
>> >> >
>> >> >
>> >> >
>> >> > Thank you,
>> >> > -Praveen
>> >>
>> >> ---------------------------------------------------------------------
>> >> Apache Qpid - AMQP Messaging Implementation
>> >> Project: http://qpid.apache.org
>> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
>> >>
>> >>
>> >
>> >
>> > --
>> > -Praveen
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>>
>
>
> --
> -Praveen
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by Praveen M <le...@gmail.com>.
Hi Robbie,
Thanks for writing. I just had a few followup questions. Please see in-line.
Thanks a lot for helping out with this,
Praveen
On Wed, Jan 11, 2012 at 6:58 AM, Robbie Gemmell <ro...@gmail.com>wrote:
> Hi Praveen,
>
> If you are predominantly going to use persistent messages, then I
> would actually look at your publishers using a single single session
> per connection for the most part.
>
> The broker is currently able to best extract performance scaling from
> the persistent stores (more so the BDBstore as per Robs recent
> testing...though the results of which are again slightly out of date
> now due to the upgrade to BDB5 earlier today) by performing actions on
> a per-connection basis, so it can extract parrallism over commits to
> the store. Using a lower number of connections with 5-10 sessions each
> could be noticably slower at peak when using 150-300 publishers.
>
that totally makes sense. But now that I'd have approximately (30 instance
* 5 connections) = 150
connections to the broker, I wouldn't be overwhelming the broker right? I'm
guessing it shouldn't be a problem, But just want to run it by you?
I understand that having separate connections is significantly faster as
each
connection would operate on it's own socket.
Just out of curiosity I'd like to know, what are the performance
degradation
causes when using sessions sharing the same connection? I know for one that
the session has to hold on creation locks (for creation) and connection
failover locks on send.
Do you think this will be the cause of the major bottle necks or is there
something else too?
(I'm sure you know the code back to front to answer this better) :)
> If you are sending to a huge variety of Destinations then the way you
> are using your Producers seems appropriate; creating producers isnt
> necessarily that heavyweight an operation, but not doing so is
> undeniably far more efficient.
>
ah, yep. that makes sense. I think I will stick to creating producers on
connection pooling
and creating the session
>
> I have actually never personally used sync_publish, but given that
> both the underlying sync() call and session.commit() hit the broker
> synchronously I would guess their performance should be pretty similar
> for persistent messages with a batch size of 1, in which case I would
> probably just go with transactions for 'pure JMS' purposes. If you
> ever were using batches of >1 message that would seem to favor use of
> transactions due to lowering the number of synchronous round trips to
> the broker which would be required, increasing performance.
>
> yep, I totally agree with you on using transactions over the sync_publish
option
for pure JMS purposes.
> Robbie
>
> On 10 January 2012 17:48, Praveen M <le...@gmail.com> wrote:
> > Hi Robbie,
> >
> > Thanks for writing.
> >
> > Here is some more context for my 2nd question.
> >
> > I'm expected to have about 30 client instances connecting to one broker.
> > About 90% of the messages to the broker are expected to be persistent
> > messages.
> >
> > I haven't decided whether to use transactions or use the option
> > sync_publish. The
> > guarantee that we'd expect is that the broker receives the message before
> > proceeding to
> > send the next message. I'd be happy to hear your recommendations around
> > this.
> >
> > From the code I read, it looked like sync_publish was working to that
> > effect. But one thing I
> > saw was the sync() call in the client held to a connection failover lock,
> > and I wasn't sure if the sync() call was blocking.
> > Anyways, that said do you have a recommendation between transacted vs
> > sync_publish in a multi client environment (things going in parallel).
> I'd
> > really like to hear what you think about this.
> >
> > There will not be any batch enqueues (a top of my head). Almost all our
> > traffic will be having messages
> > enqueued one at a time.
> >
> > The number of destinations we have could be about 4000-5000. But yes,
> there
> > is a strong likelihood of one
> > queue being more popular and all the traffic getting routed there. We
> > sometimes receive bursty enqueues of a certain type
> > and is fair to assume that 5 or more enqueues of the same type to the
> same
> > queue is possible to happen concurrently.
> >
> > We plan to have own framework on top of Qpid which will have the ability
> to
> > load balance between the destinations
> > if one queue goes over a certain threshold (some hand-picked number..not
> > decided on yet) and then route traffic to another destination.
> >
> > And the overall traffic expected is somewhere between 10 million - 20
> > million messages a day maybe.
> >
> > Please do let me know if you'd like to hear any other information.
> >
> > Thank you,
> > Praveen
> >
> >
> > But yes, we'd be using one of the two (whichever would perform best with
> > multiple concurrent clients).
> >
> > On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <
> robbie.gemmell@gmail.com>wrote:
> >
> >> Hi Praveen,
> >>
> >> Using either JNDI or Session.createQueue should work fine, but the
> >> obvious limitation of the latter is that the (possibly
> >> provider-specific) configuration is then part of your code (unless you
> >> look the information up somewhere before using it) and cant be changed
> >> without modifying the code.
> >>
> >> To better answer the other question, it would be good to first know
> >> more specifics about the application and its messaging model, eg:
> >> How many instances of the application are you likely to be running?
> >> Will all your messages be persistent?
> >> Will you always use transactions (and if so will it always of batch
> >> size 1 like below)?
> >> You mention having maybe 5-10 concurrent publishers, but how many
> >> different Destinations are you likely to be sending to in total?
> >> Are any of the above Destinations likely to be used a lot more than the
> >> others?
> >>
> >> Robbie
> >>
> >> On 10 January 2012 00:27, Praveen M <le...@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I'm writing a JMS Client and have a few best practices questions.
> >> >
> >> > 1) I understand that there are different ways ways to create a
> reference
> >> to
> >> > a destination to enqueue a message (using JMS createQueue, using JNDI
> >> > lookup).
> >> > What would be the recommended way to use?
> >> >
> >> > 2) I create a connection to the broker on my application startup, and
> >> > create a session pool. on every enqueue, i checkout a session from my
> >> pool
> >> > use it and return to pool on enqueue.
> >> > Does this sound reasonable? (code sample below)
> >> >
> >> > What would be the pattern that is typically used/recommended for
> >> > session management/connection management in the client side?
> >> >
> >> > Or is this an overkill to do? Is CreateSession() and
> CreateProducer()
> >> > really cheap that it is just done on each enqueue?
> >> > In my use case, I'd expect quite some concurrency (assume n (maybe
> >> > 5-10) threads doing enqueues at the same time) and I'd prefer it to
> not
> >> > lock/block and go through asap.
> >> >
> >> > I do remember from reading the qpid client code that there is a
> lock
> >> > held on the connection object whenever a new session is created on
> that
> >> > connection. I'm assuming it is one of the places where my enqueue can
> >> slow
> >> > down
> >> > when going in parallel if it has to create a session for every
> enqueue
> >> > of a message. There might be more such blocks, which I'm not aware of.
> >> >
> >> > Taking performance into mind, what would be the best design for
> >> > session/connection management?
> >> >
> >> > A snippet of what I have currently is below,
> >> >
> >> > // a thread safe queue with the available session producer pairs
> which
> >> > can be used to create a message and enqueue.
> >> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
> >> > availableSessionProducerPairs = new
> ConcurrentLinkedQueue<Pair<Session,
> >> > MessageProducer>> ();
> >> >
> >> > // initialize the sessionProducerPairs, create upto n pairs
> >> > initializePool(Connection qpidConnection) {
> >> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ ) {
> >> > Session session = qpidConnection.createSession(true,
> >> > Session.SESSION_TRANSACTED);
> >> > MessageProducer producer = session.createProducer(null);
> >> > availableSessionProducerPairs.add(Pair.newPair(session,
> >> > producer));
> >> > }
> >> > }
> >> >
> >> > // on enqueue we checkout a session/producer pair use it for the
> enqueue
> >> > and then return it back to the session/produer pair pool.
> >> > enqueueMessage(String queueName, byte[] message) {
> >> > // remove the session producer pair
> >> > Pair<Session, MessageProducer> sessionProducerPair =
> >> > availableSessionProducerPairs.poll();
> >> > if(sessionProducerPair == null) {
> >> > // gack here and return null.
> >> > }
> >> > Session session = sessionProducerPair.getFirst();
> >> > MessageProducer producer = sessionProducerPair.getSecond();
> >> > BytesMessage jmsMessage = session.createBytesMessage();
> >> > jmsMessage.writeBytes(message);
> >> > Queue queueRef = queueNameToReferenceMap.get(queueName);
> >> > producer.send(queueRef, jmsMessage, DeliveryMode.PERSISTENT, 0,
> >> 0);
> >> > session.commit();
> >> > availableSessionProducersPair.add()
> >> > }
> >> >
> >> >
> >> >
> >> > Thank you,
> >> > -Praveen
> >>
> >> ---------------------------------------------------------------------
> >> Apache Qpid - AMQP Messaging Implementation
> >> Project: http://qpid.apache.org
> >> Use/Interact: mailto:users-subscribe@qpid.apache.org
> >>
> >>
> >
> >
> > --
> > -Praveen
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
--
-Praveen
Re: JMS Client usage - best practices?
Posted by Robbie Gemmell <ro...@gmail.com>.
Hi Praveen,
If you are predominantly going to use persistent messages, then I
would actually look at your publishers using a single single session
per connection for the most part.
The broker is currently able to best extract performance scaling from
the persistent stores (more so the BDBstore as per Robs recent
testing...though the results of which are again slightly out of date
now due to the upgrade to BDB5 earlier today) by performing actions on
a per-connection basis, so it can extract parrallism over commits to
the store. Using a lower number of connections with 5-10 sessions each
could be noticably slower at peak when using 150-300 publishers.
If you are sending to a huge variety of Destinations then the way you
are using your Producers seems appropriate; creating producers isnt
necessarily that heavyweight an operation, but not doing so is
undeniably far more efficient.
I have actually never personally used sync_publish, but given that
both the underlying sync() call and session.commit() hit the broker
synchronously I would guess their performance should be pretty similar
for persistent messages with a batch size of 1, in which case I would
probably just go with transactions for 'pure JMS' purposes. If you
ever were using batches of >1 message that would seem to favor use of
transactions due to lowering the number of synchronous round trips to
the broker which would be required, increasing performance.
Robbie
On 10 January 2012 17:48, Praveen M <le...@gmail.com> wrote:
> Hi Robbie,
>
> Thanks for writing.
>
> Here is some more context for my 2nd question.
>
> I'm expected to have about 30 client instances connecting to one broker.
> About 90% of the messages to the broker are expected to be persistent
> messages.
>
> I haven't decided whether to use transactions or use the option
> sync_publish. The
> guarantee that we'd expect is that the broker receives the message before
> proceeding to
> send the next message. I'd be happy to hear your recommendations around
> this.
>
> From the code I read, it looked like sync_publish was working to that
> effect. But one thing I
> saw was the sync() call in the client held to a connection failover lock,
> and I wasn't sure if the sync() call was blocking.
> Anyways, that said do you have a recommendation between transacted vs
> sync_publish in a multi client environment (things going in parallel). I'd
> really like to hear what you think about this.
>
> There will not be any batch enqueues (a top of my head). Almost all our
> traffic will be having messages
> enqueued one at a time.
>
> The number of destinations we have could be about 4000-5000. But yes, there
> is a strong likelihood of one
> queue being more popular and all the traffic getting routed there. We
> sometimes receive bursty enqueues of a certain type
> and is fair to assume that 5 or more enqueues of the same type to the same
> queue is possible to happen concurrently.
>
> We plan to have own framework on top of Qpid which will have the ability to
> load balance between the destinations
> if one queue goes over a certain threshold (some hand-picked number..not
> decided on yet) and then route traffic to another destination.
>
> And the overall traffic expected is somewhere between 10 million - 20
> million messages a day maybe.
>
> Please do let me know if you'd like to hear any other information.
>
> Thank you,
> Praveen
>
>
> But yes, we'd be using one of the two (whichever would perform best with
> multiple concurrent clients).
>
> On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <ro...@gmail.com>wrote:
>
>> Hi Praveen,
>>
>> Using either JNDI or Session.createQueue should work fine, but the
>> obvious limitation of the latter is that the (possibly
>> provider-specific) configuration is then part of your code (unless you
>> look the information up somewhere before using it) and cant be changed
>> without modifying the code.
>>
>> To better answer the other question, it would be good to first know
>> more specifics about the application and its messaging model, eg:
>> How many instances of the application are you likely to be running?
>> Will all your messages be persistent?
>> Will you always use transactions (and if so will it always of batch
>> size 1 like below)?
>> You mention having maybe 5-10 concurrent publishers, but how many
>> different Destinations are you likely to be sending to in total?
>> Are any of the above Destinations likely to be used a lot more than the
>> others?
>>
>> Robbie
>>
>> On 10 January 2012 00:27, Praveen M <le...@gmail.com> wrote:
>> > Hi,
>> >
>> > I'm writing a JMS Client and have a few best practices questions.
>> >
>> > 1) I understand that there are different ways ways to create a reference
>> to
>> > a destination to enqueue a message (using JMS createQueue, using JNDI
>> > lookup).
>> > What would be the recommended way to use?
>> >
>> > 2) I create a connection to the broker on my application startup, and
>> > create a session pool. on every enqueue, i checkout a session from my
>> pool
>> > use it and return to pool on enqueue.
>> > Does this sound reasonable? (code sample below)
>> >
>> > What would be the pattern that is typically used/recommended for
>> > session management/connection management in the client side?
>> >
>> > Or is this an overkill to do? Is CreateSession() and CreateProducer()
>> > really cheap that it is just done on each enqueue?
>> > In my use case, I'd expect quite some concurrency (assume n (maybe
>> > 5-10) threads doing enqueues at the same time) and I'd prefer it to not
>> > lock/block and go through asap.
>> >
>> > I do remember from reading the qpid client code that there is a lock
>> > held on the connection object whenever a new session is created on that
>> > connection. I'm assuming it is one of the places where my enqueue can
>> slow
>> > down
>> > when going in parallel if it has to create a session for every enqueue
>> > of a message. There might be more such blocks, which I'm not aware of.
>> >
>> > Taking performance into mind, what would be the best design for
>> > session/connection management?
>> >
>> > A snippet of what I have currently is below,
>> >
>> > // a thread safe queue with the available session producer pairs which
>> > can be used to create a message and enqueue.
>> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
>> > availableSessionProducerPairs = new ConcurrentLinkedQueue<Pair<Session,
>> > MessageProducer>> ();
>> >
>> > // initialize the sessionProducerPairs, create upto n pairs
>> > initializePool(Connection qpidConnection) {
>> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ ) {
>> > Session session = qpidConnection.createSession(true,
>> > Session.SESSION_TRANSACTED);
>> > MessageProducer producer = session.createProducer(null);
>> > availableSessionProducerPairs.add(Pair.newPair(session,
>> > producer));
>> > }
>> > }
>> >
>> > // on enqueue we checkout a session/producer pair use it for the enqueue
>> > and then return it back to the session/produer pair pool.
>> > enqueueMessage(String queueName, byte[] message) {
>> > // remove the session producer pair
>> > Pair<Session, MessageProducer> sessionProducerPair =
>> > availableSessionProducerPairs.poll();
>> > if(sessionProducerPair == null) {
>> > // gack here and return null.
>> > }
>> > Session session = sessionProducerPair.getFirst();
>> > MessageProducer producer = sessionProducerPair.getSecond();
>> > BytesMessage jmsMessage = session.createBytesMessage();
>> > jmsMessage.writeBytes(message);
>> > Queue queueRef = queueNameToReferenceMap.get(queueName);
>> > producer.send(queueRef, jmsMessage, DeliveryMode.PERSISTENT, 0,
>> 0);
>> > session.commit();
>> > availableSessionProducersPair.add()
>> > }
>> >
>> >
>> >
>> > Thank you,
>> > -Praveen
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project: http://qpid.apache.org
>> Use/Interact: mailto:users-subscribe@qpid.apache.org
>>
>>
>
>
> --
> -Praveen
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: JMS Client usage - best practices?
Posted by Praveen M <le...@gmail.com>.
Hi Robbie,
Thanks for writing.
Here is some more context for my 2nd question.
I'm expected to have about 30 client instances connecting to one broker.
About 90% of the messages to the broker are expected to be persistent
messages.
I haven't decided whether to use transactions or use the option
sync_publish. The
guarantee that we'd expect is that the broker receives the message before
proceeding to
send the next message. I'd be happy to hear your recommendations around
this.
>From the code I read, it looked like sync_publish was working to that
effect. But one thing I
saw was the sync() call in the client held to a connection failover lock,
and I wasn't sure if the sync() call was blocking.
Anyways, that said do you have a recommendation between transacted vs
sync_publish in a multi client environment (things going in parallel). I'd
really like to hear what you think about this.
There will not be any batch enqueues (a top of my head). Almost all our
traffic will be having messages
enqueued one at a time.
The number of destinations we have could be about 4000-5000. But yes, there
is a strong likelihood of one
queue being more popular and all the traffic getting routed there. We
sometimes receive bursty enqueues of a certain type
and is fair to assume that 5 or more enqueues of the same type to the same
queue is possible to happen concurrently.
We plan to have own framework on top of Qpid which will have the ability to
load balance between the destinations
if one queue goes over a certain threshold (some hand-picked number..not
decided on yet) and then route traffic to another destination.
And the overall traffic expected is somewhere between 10 million - 20
million messages a day maybe.
Please do let me know if you'd like to hear any other information.
Thank you,
Praveen
But yes, we'd be using one of the two (whichever would perform best with
multiple concurrent clients).
On Tue, Jan 10, 2012 at 6:41 AM, Robbie Gemmell <ro...@gmail.com>wrote:
> Hi Praveen,
>
> Using either JNDI or Session.createQueue should work fine, but the
> obvious limitation of the latter is that the (possibly
> provider-specific) configuration is then part of your code (unless you
> look the information up somewhere before using it) and cant be changed
> without modifying the code.
>
> To better answer the other question, it would be good to first know
> more specifics about the application and its messaging model, eg:
> How many instances of the application are you likely to be running?
> Will all your messages be persistent?
> Will you always use transactions (and if so will it always of batch
> size 1 like below)?
> You mention having maybe 5-10 concurrent publishers, but how many
> different Destinations are you likely to be sending to in total?
> Are any of the above Destinations likely to be used a lot more than the
> others?
>
> Robbie
>
> On 10 January 2012 00:27, Praveen M <le...@gmail.com> wrote:
> > Hi,
> >
> > I'm writing a JMS Client and have a few best practices questions.
> >
> > 1) I understand that there are different ways ways to create a reference
> to
> > a destination to enqueue a message (using JMS createQueue, using JNDI
> > lookup).
> > What would be the recommended way to use?
> >
> > 2) I create a connection to the broker on my application startup, and
> > create a session pool. on every enqueue, i checkout a session from my
> pool
> > use it and return to pool on enqueue.
> > Does this sound reasonable? (code sample below)
> >
> > What would be the pattern that is typically used/recommended for
> > session management/connection management in the client side?
> >
> > Or is this an overkill to do? Is CreateSession() and CreateProducer()
> > really cheap that it is just done on each enqueue?
> > In my use case, I'd expect quite some concurrency (assume n (maybe
> > 5-10) threads doing enqueues at the same time) and I'd prefer it to not
> > lock/block and go through asap.
> >
> > I do remember from reading the qpid client code that there is a lock
> > held on the connection object whenever a new session is created on that
> > connection. I'm assuming it is one of the places where my enqueue can
> slow
> > down
> > when going in parallel if it has to create a session for every enqueue
> > of a message. There might be more such blocks, which I'm not aware of.
> >
> > Taking performance into mind, what would be the best design for
> > session/connection management?
> >
> > A snippet of what I have currently is below,
> >
> > // a thread safe queue with the available session producer pairs which
> > can be used to create a message and enqueue.
> > ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
> > availableSessionProducerPairs = new ConcurrentLinkedQueue<Pair<Session,
> > MessageProducer>> ();
> >
> > // initialize the sessionProducerPairs, create upto n pairs
> > initializePool(Connection qpidConnection) {
> > for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ ) {
> > Session session = qpidConnection.createSession(true,
> > Session.SESSION_TRANSACTED);
> > MessageProducer producer = session.createProducer(null);
> > availableSessionProducerPairs.add(Pair.newPair(session,
> > producer));
> > }
> > }
> >
> > // on enqueue we checkout a session/producer pair use it for the enqueue
> > and then return it back to the session/produer pair pool.
> > enqueueMessage(String queueName, byte[] message) {
> > // remove the session producer pair
> > Pair<Session, MessageProducer> sessionProducerPair =
> > availableSessionProducerPairs.poll();
> > if(sessionProducerPair == null) {
> > // gack here and return null.
> > }
> > Session session = sessionProducerPair.getFirst();
> > MessageProducer producer = sessionProducerPair.getSecond();
> > BytesMessage jmsMessage = session.createBytesMessage();
> > jmsMessage.writeBytes(message);
> > Queue queueRef = queueNameToReferenceMap.get(queueName);
> > producer.send(queueRef, jmsMessage, DeliveryMode.PERSISTENT, 0,
> 0);
> > session.commit();
> > availableSessionProducersPair.add()
> > }
> >
> >
> >
> > Thank you,
> > -Praveen
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
--
-Praveen
Re: JMS Client usage - best practices?
Posted by Robbie Gemmell <ro...@gmail.com>.
Hi Praveen,
Using either JNDI or Session.createQueue should work fine, but the
obvious limitation of the latter is that the (possibly
provider-specific) configuration is then part of your code (unless you
look the information up somewhere before using it) and cant be changed
without modifying the code.
To better answer the other question, it would be good to first know
more specifics about the application and its messaging model, eg:
How many instances of the application are you likely to be running?
Will all your messages be persistent?
Will you always use transactions (and if so will it always of batch
size 1 like below)?
You mention having maybe 5-10 concurrent publishers, but how many
different Destinations are you likely to be sending to in total?
Are any of the above Destinations likely to be used a lot more than the others?
Robbie
On 10 January 2012 00:27, Praveen M <le...@gmail.com> wrote:
> Hi,
>
> I'm writing a JMS Client and have a few best practices questions.
>
> 1) I understand that there are different ways ways to create a reference to
> a destination to enqueue a message (using JMS createQueue, using JNDI
> lookup).
> What would be the recommended way to use?
>
> 2) I create a connection to the broker on my application startup, and
> create a session pool. on every enqueue, i checkout a session from my pool
> use it and return to pool on enqueue.
> Does this sound reasonable? (code sample below)
>
> What would be the pattern that is typically used/recommended for
> session management/connection management in the client side?
>
> Or is this an overkill to do? Is CreateSession() and CreateProducer()
> really cheap that it is just done on each enqueue?
> In my use case, I'd expect quite some concurrency (assume n (maybe
> 5-10) threads doing enqueues at the same time) and I'd prefer it to not
> lock/block and go through asap.
>
> I do remember from reading the qpid client code that there is a lock
> held on the connection object whenever a new session is created on that
> connection. I'm assuming it is one of the places where my enqueue can slow
> down
> when going in parallel if it has to create a session for every enqueue
> of a message. There might be more such blocks, which I'm not aware of.
>
> Taking performance into mind, what would be the best design for
> session/connection management?
>
> A snippet of what I have currently is below,
>
> // a thread safe queue with the available session producer pairs which
> can be used to create a message and enqueue.
> ConcurrentLinkedQueue<Pair<Session, MessageProducer>>
> availableSessionProducerPairs = new ConcurrentLinkedQueue<Pair<Session,
> MessageProducer>> ();
>
> // initialize the sessionProducerPairs, create upto n pairs
> initializePool(Connection qpidConnection) {
> for (int i = 1 ; i <= SESSION_PRODUCER_PAIRS_COUNT; i ++ ) {
> Session session = qpidConnection.createSession(true,
> Session.SESSION_TRANSACTED);
> MessageProducer producer = session.createProducer(null);
> availableSessionProducerPairs.add(Pair.newPair(session,
> producer));
> }
> }
>
> // on enqueue we checkout a session/producer pair use it for the enqueue
> and then return it back to the session/produer pair pool.
> enqueueMessage(String queueName, byte[] message) {
> // remove the session producer pair
> Pair<Session, MessageProducer> sessionProducerPair =
> availableSessionProducerPairs.poll();
> if(sessionProducerPair == null) {
> // gack here and return null.
> }
> Session session = sessionProducerPair.getFirst();
> MessageProducer producer = sessionProducerPair.getSecond();
> BytesMessage jmsMessage = session.createBytesMessage();
> jmsMessage.writeBytes(message);
> Queue queueRef = queueNameToReferenceMap.get(queueName);
> producer.send(queueRef, jmsMessage, DeliveryMode.PERSISTENT, 0, 0);
> session.commit();
> availableSessionProducersPair.add()
> }
>
>
>
> Thank you,
> -Praveen
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org