You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Jeremy Levy <je...@gmail.com> on 2012/07/20 01:32:24 UTC
Connection to Broker within an Application Server
I suspect that the way I am currently producing messages within my
application server is wrong.
We using JBoss 5.1 with a standalone ActiveMQ 5.6.0 broker. In regards to
producing messages I am caching a static instance of the
ActiveMQConnectionFactory, everything else
Connection,Session,Destination,Producer is created with each message
created/sent.
I've noticed that under very high message creation it appears as though
many connections to the broker are being created (as observed using
netstat) on the machine producing.
It feel like there is no connection pooling being done.
I could also not use ActiveMQConnectionFactory at all and create a JBoss /
ActiveMQ datasource and do the lookup via InitalContext etc..
As I'm currently doing this is there connection pooling, should I be doing
this differently?
Jeremy
Re: Connection to Broker within an Application Server
Posted by Jeremy Levy <je...@gmail.com>.
Gaurav-
We have redundant master/slave brokers, pretty beefy machines, 3120MB
allocated on the JVM.
<policyEntry queue=">" producerFlowControl="false" memoryLimit="2mb"
prioritizedMessages="true">
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
</policyEntry>
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"
cleanupInterval="300000" checkpointInterval="50000"
journalMaxWriteBatchSize="62k"
journalMaxFileLength="5g"
indexCacheSize="100000" indexWriteBatchSize="100000"/>
</persistenceAdapter>
Using in the producer side using PooledConnectionFactory (with a pool size
of 100, not sure of utilization, but thats held up so far)
If there was something else you were interested in let me know. I've been
thinking about a clustered solution as well. As for message persistance, I
have it turned off, haven't been able to get a combination working nicely...
Jeremy
On Fri, Jul 20, 2012 at 6:11 PM, Gaurav Sharma
<ga...@gmail.com>wrote:
> That's pretty good volume without pooling the connexn factory. I don't
> have numbers just yet but it will be within an order of magnitude of that.
> Mind sharing what message store you are using for persistence and the
> broker topology.
>
> Sent from my iPhone
>
> On Jul 20, 2012, at 12:13, Jeremy Levy <je...@gmail.com> wrote:
>
> > Thanks again for the quick response. That's good info. What volume of
> > messages are you processing? I'm doing about 25 million a day.
> >
> >
> > Jeremy
> >
> > On Fri, Jul 20, 2012 at 2:54 PM, Gaurav Sharma
> > <ga...@gmail.com>wrote:
> >
> >> Connection factory is thread-safe, so, a single instance should be fine.
> >> Pooled connection factory should help greatly. One other optimization I
> >> added was to keep a concurrent LRU cache of DestinationViewMBean keyed
> off
> >> of destination name because I am heavily relying on JMX for destination
> >> lookups. That, in conjunction with a single instance of BrokerViewMBean,
> >> has so far worked well for me. I am still working on productionizing the
> >> config and will be doing some perf-testing to benchmark the config. My
> >> service users have very high throughput requirements (along with message
> >> durability), so, I have had to carefully plan the broker topology
> (current
> >> setup: multi-master/slave brokers with db message store). I have still
> not
> >> figured out if it there is an out of the box config to shard the
> >> activemq_msgs schema along with keeping a single activemq_locks table
> per
> >> every broker master. Hopefully, someone else with a high performance
> >> persistent configuration can share their experience and config.
> >>
> >> On Fri, Jul 20, 2012 at 11:06 AM, Jeremy Levy <je...@gmail.com> wrote:
> >>
> >>> Thanks Gaurav-
> >>>
> >>> I'll give the Pooled Factory a try. What do you think about keeping
> once
> >>> instance of the connection factory around statically? I've played with
> >>> async sending with mixed results.
> >>>
> >>> Jeremy
> >>>
> >>> On Thu, Jul 19, 2012 at 7:53 PM, Gaurav Sharma
> >>> <ga...@gmail.com>wrote:
> >>>
> >>>> You might wanna use the PooledConnectionFactory wrapper with the
> >>>> ActiveMQConnectionFactory. That's what I switched to too. If you have
> a
> >>>> listener, you will have to leave the session open as well until msg
> >>>> delivery. I am using mysql as the msg-store and nio connector
> transport
> >>> to
> >>>> the broker. Also, dep on your use case, try to turn on async sending
> if
> >>>> possible. There are many perf knobs that can be turned.
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>> On Jul 19, 2012, at 16:32, Jeremy Levy <je...@gmail.com> wrote:
> >>>>
> >>>>> I suspect that the way I am currently producing messages within my
> >>>>> application server is wrong.
> >>>>>
> >>>>> We using JBoss 5.1 with a standalone ActiveMQ 5.6.0 broker. In
> >> regards
> >>>> to
> >>>>> producing messages I am caching a static instance of the
> >>>>> ActiveMQConnectionFactory, everything else
> >>>>> Connection,Session,Destination,Producer is created with each message
> >>>>> created/sent.
> >>>>>
> >>>>> I've noticed that under very high message creation it appears as
> >> though
> >>>>> many connections to the broker are being created (as observed using
> >>>>> netstat) on the machine producing.
> >>>>>
> >>>>> It feel like there is no connection pooling being done.
> >>>>>
> >>>>> I could also not use ActiveMQConnectionFactory at all and create a
> >>> JBoss
> >>>> /
> >>>>> ActiveMQ datasource and do the lookup via InitalContext etc..
> >>>>>
> >>>>> As I'm currently doing this is there connection pooling, should I be
> >>>> doing
> >>>>> this differently?
> >>>>>
> >>>>> Jeremy
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Jeremy Levy
> >>>
> >>
> >
> >
> >
> > --
> > Jeremy Levy
>
--
Jeremy Levy
Re: Connection to Broker within an Application Server
Posted by Gaurav Sharma <ga...@gmail.com>.
That's pretty good volume without pooling the connexn factory. I don't have numbers just yet but it will be within an order of magnitude of that. Mind sharing what message store you are using for persistence and the broker topology.
Sent from my iPhone
On Jul 20, 2012, at 12:13, Jeremy Levy <je...@gmail.com> wrote:
> Thanks again for the quick response. That's good info. What volume of
> messages are you processing? I'm doing about 25 million a day.
>
>
> Jeremy
>
> On Fri, Jul 20, 2012 at 2:54 PM, Gaurav Sharma
> <ga...@gmail.com>wrote:
>
>> Connection factory is thread-safe, so, a single instance should be fine.
>> Pooled connection factory should help greatly. One other optimization I
>> added was to keep a concurrent LRU cache of DestinationViewMBean keyed off
>> of destination name because I am heavily relying on JMX for destination
>> lookups. That, in conjunction with a single instance of BrokerViewMBean,
>> has so far worked well for me. I am still working on productionizing the
>> config and will be doing some perf-testing to benchmark the config. My
>> service users have very high throughput requirements (along with message
>> durability), so, I have had to carefully plan the broker topology (current
>> setup: multi-master/slave brokers with db message store). I have still not
>> figured out if it there is an out of the box config to shard the
>> activemq_msgs schema along with keeping a single activemq_locks table per
>> every broker master. Hopefully, someone else with a high performance
>> persistent configuration can share their experience and config.
>>
>> On Fri, Jul 20, 2012 at 11:06 AM, Jeremy Levy <je...@gmail.com> wrote:
>>
>>> Thanks Gaurav-
>>>
>>> I'll give the Pooled Factory a try. What do you think about keeping once
>>> instance of the connection factory around statically? I've played with
>>> async sending with mixed results.
>>>
>>> Jeremy
>>>
>>> On Thu, Jul 19, 2012 at 7:53 PM, Gaurav Sharma
>>> <ga...@gmail.com>wrote:
>>>
>>>> You might wanna use the PooledConnectionFactory wrapper with the
>>>> ActiveMQConnectionFactory. That's what I switched to too. If you have a
>>>> listener, you will have to leave the session open as well until msg
>>>> delivery. I am using mysql as the msg-store and nio connector transport
>>> to
>>>> the broker. Also, dep on your use case, try to turn on async sending if
>>>> possible. There are many perf knobs that can be turned.
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Jul 19, 2012, at 16:32, Jeremy Levy <je...@gmail.com> wrote:
>>>>
>>>>> I suspect that the way I am currently producing messages within my
>>>>> application server is wrong.
>>>>>
>>>>> We using JBoss 5.1 with a standalone ActiveMQ 5.6.0 broker. In
>> regards
>>>> to
>>>>> producing messages I am caching a static instance of the
>>>>> ActiveMQConnectionFactory, everything else
>>>>> Connection,Session,Destination,Producer is created with each message
>>>>> created/sent.
>>>>>
>>>>> I've noticed that under very high message creation it appears as
>> though
>>>>> many connections to the broker are being created (as observed using
>>>>> netstat) on the machine producing.
>>>>>
>>>>> It feel like there is no connection pooling being done.
>>>>>
>>>>> I could also not use ActiveMQConnectionFactory at all and create a
>>> JBoss
>>>> /
>>>>> ActiveMQ datasource and do the lookup via InitalContext etc..
>>>>>
>>>>> As I'm currently doing this is there connection pooling, should I be
>>>> doing
>>>>> this differently?
>>>>>
>>>>> Jeremy
>>>>
>>>
>>>
>>>
>>> --
>>> Jeremy Levy
>>>
>>
>
>
>
> --
> Jeremy Levy
Re: Connection to Broker within an Application Server
Posted by Jeremy Levy <je...@gmail.com>.
Thanks again for the quick response. That's good info. What volume of
messages are you processing? I'm doing about 25 million a day.
Jeremy
On Fri, Jul 20, 2012 at 2:54 PM, Gaurav Sharma
<ga...@gmail.com>wrote:
> Connection factory is thread-safe, so, a single instance should be fine.
> Pooled connection factory should help greatly. One other optimization I
> added was to keep a concurrent LRU cache of DestinationViewMBean keyed off
> of destination name because I am heavily relying on JMX for destination
> lookups. That, in conjunction with a single instance of BrokerViewMBean,
> has so far worked well for me. I am still working on productionizing the
> config and will be doing some perf-testing to benchmark the config. My
> service users have very high throughput requirements (along with message
> durability), so, I have had to carefully plan the broker topology (current
> setup: multi-master/slave brokers with db message store). I have still not
> figured out if it there is an out of the box config to shard the
> activemq_msgs schema along with keeping a single activemq_locks table per
> every broker master. Hopefully, someone else with a high performance
> persistent configuration can share their experience and config.
>
> On Fri, Jul 20, 2012 at 11:06 AM, Jeremy Levy <je...@gmail.com> wrote:
>
> > Thanks Gaurav-
> >
> > I'll give the Pooled Factory a try. What do you think about keeping once
> > instance of the connection factory around statically? I've played with
> > async sending with mixed results.
> >
> > Jeremy
> >
> > On Thu, Jul 19, 2012 at 7:53 PM, Gaurav Sharma
> > <ga...@gmail.com>wrote:
> >
> > > You might wanna use the PooledConnectionFactory wrapper with the
> > > ActiveMQConnectionFactory. That's what I switched to too. If you have a
> > > listener, you will have to leave the session open as well until msg
> > > delivery. I am using mysql as the msg-store and nio connector transport
> > to
> > > the broker. Also, dep on your use case, try to turn on async sending if
> > > possible. There are many perf knobs that can be turned.
> > >
> > > Sent from my iPhone
> > >
> > > On Jul 19, 2012, at 16:32, Jeremy Levy <je...@gmail.com> wrote:
> > >
> > > > I suspect that the way I am currently producing messages within my
> > > > application server is wrong.
> > > >
> > > > We using JBoss 5.1 with a standalone ActiveMQ 5.6.0 broker. In
> regards
> > > to
> > > > producing messages I am caching a static instance of the
> > > > ActiveMQConnectionFactory, everything else
> > > > Connection,Session,Destination,Producer is created with each message
> > > > created/sent.
> > > >
> > > > I've noticed that under very high message creation it appears as
> though
> > > > many connections to the broker are being created (as observed using
> > > > netstat) on the machine producing.
> > > >
> > > > It feel like there is no connection pooling being done.
> > > >
> > > > I could also not use ActiveMQConnectionFactory at all and create a
> > JBoss
> > > /
> > > > ActiveMQ datasource and do the lookup via InitalContext etc..
> > > >
> > > > As I'm currently doing this is there connection pooling, should I be
> > > doing
> > > > this differently?
> > > >
> > > > Jeremy
> > >
> >
> >
> >
> > --
> > Jeremy Levy
> >
>
--
Jeremy Levy
Re: Connection to Broker within an Application Server
Posted by Gaurav Sharma <ga...@gmail.com>.
Connection factory is thread-safe, so, a single instance should be fine.
Pooled connection factory should help greatly. One other optimization I
added was to keep a concurrent LRU cache of DestinationViewMBean keyed off
of destination name because I am heavily relying on JMX for destination
lookups. That, in conjunction with a single instance of BrokerViewMBean,
has so far worked well for me. I am still working on productionizing the
config and will be doing some perf-testing to benchmark the config. My
service users have very high throughput requirements (along with message
durability), so, I have had to carefully plan the broker topology (current
setup: multi-master/slave brokers with db message store). I have still not
figured out if it there is an out of the box config to shard the
activemq_msgs schema along with keeping a single activemq_locks table per
every broker master. Hopefully, someone else with a high performance
persistent configuration can share their experience and config.
On Fri, Jul 20, 2012 at 11:06 AM, Jeremy Levy <je...@gmail.com> wrote:
> Thanks Gaurav-
>
> I'll give the Pooled Factory a try. What do you think about keeping once
> instance of the connection factory around statically? I've played with
> async sending with mixed results.
>
> Jeremy
>
> On Thu, Jul 19, 2012 at 7:53 PM, Gaurav Sharma
> <ga...@gmail.com>wrote:
>
> > You might wanna use the PooledConnectionFactory wrapper with the
> > ActiveMQConnectionFactory. That's what I switched to too. If you have a
> > listener, you will have to leave the session open as well until msg
> > delivery. I am using mysql as the msg-store and nio connector transport
> to
> > the broker. Also, dep on your use case, try to turn on async sending if
> > possible. There are many perf knobs that can be turned.
> >
> > Sent from my iPhone
> >
> > On Jul 19, 2012, at 16:32, Jeremy Levy <je...@gmail.com> wrote:
> >
> > > I suspect that the way I am currently producing messages within my
> > > application server is wrong.
> > >
> > > We using JBoss 5.1 with a standalone ActiveMQ 5.6.0 broker. In regards
> > to
> > > producing messages I am caching a static instance of the
> > > ActiveMQConnectionFactory, everything else
> > > Connection,Session,Destination,Producer is created with each message
> > > created/sent.
> > >
> > > I've noticed that under very high message creation it appears as though
> > > many connections to the broker are being created (as observed using
> > > netstat) on the machine producing.
> > >
> > > It feel like there is no connection pooling being done.
> > >
> > > I could also not use ActiveMQConnectionFactory at all and create a
> JBoss
> > /
> > > ActiveMQ datasource and do the lookup via InitalContext etc..
> > >
> > > As I'm currently doing this is there connection pooling, should I be
> > doing
> > > this differently?
> > >
> > > Jeremy
> >
>
>
>
> --
> Jeremy Levy
>
Re: Connection to Broker within an Application Server
Posted by Jeremy Levy <je...@gmail.com>.
Thanks Gaurav-
I'll give the Pooled Factory a try. What do you think about keeping once
instance of the connection factory around statically? I've played with
async sending with mixed results.
Jeremy
On Thu, Jul 19, 2012 at 7:53 PM, Gaurav Sharma
<ga...@gmail.com>wrote:
> You might wanna use the PooledConnectionFactory wrapper with the
> ActiveMQConnectionFactory. That's what I switched to too. If you have a
> listener, you will have to leave the session open as well until msg
> delivery. I am using mysql as the msg-store and nio connector transport to
> the broker. Also, dep on your use case, try to turn on async sending if
> possible. There are many perf knobs that can be turned.
>
> Sent from my iPhone
>
> On Jul 19, 2012, at 16:32, Jeremy Levy <je...@gmail.com> wrote:
>
> > I suspect that the way I am currently producing messages within my
> > application server is wrong.
> >
> > We using JBoss 5.1 with a standalone ActiveMQ 5.6.0 broker. In regards
> to
> > producing messages I am caching a static instance of the
> > ActiveMQConnectionFactory, everything else
> > Connection,Session,Destination,Producer is created with each message
> > created/sent.
> >
> > I've noticed that under very high message creation it appears as though
> > many connections to the broker are being created (as observed using
> > netstat) on the machine producing.
> >
> > It feel like there is no connection pooling being done.
> >
> > I could also not use ActiveMQConnectionFactory at all and create a JBoss
> /
> > ActiveMQ datasource and do the lookup via InitalContext etc..
> >
> > As I'm currently doing this is there connection pooling, should I be
> doing
> > this differently?
> >
> > Jeremy
>
--
Jeremy Levy
Re: Connection to Broker within an Application Server
Posted by Gaurav Sharma <ga...@gmail.com>.
You might wanna use the PooledConnectionFactory wrapper with the ActiveMQConnectionFactory. That's what I switched to too. If you have a listener, you will have to leave the session open as well until msg delivery. I am using mysql as the msg-store and nio connector transport to the broker. Also, dep on your use case, try to turn on async sending if possible. There are many perf knobs that can be turned.
Sent from my iPhone
On Jul 19, 2012, at 16:32, Jeremy Levy <je...@gmail.com> wrote:
> I suspect that the way I am currently producing messages within my
> application server is wrong.
>
> We using JBoss 5.1 with a standalone ActiveMQ 5.6.0 broker. In regards to
> producing messages I am caching a static instance of the
> ActiveMQConnectionFactory, everything else
> Connection,Session,Destination,Producer is created with each message
> created/sent.
>
> I've noticed that under very high message creation it appears as though
> many connections to the broker are being created (as observed using
> netstat) on the machine producing.
>
> It feel like there is no connection pooling being done.
>
> I could also not use ActiveMQConnectionFactory at all and create a JBoss /
> ActiveMQ datasource and do the lookup via InitalContext etc..
>
> As I'm currently doing this is there connection pooling, should I be doing
> this differently?
>
> Jeremy