You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by Colin MacNaughton <co...@gmail.com> on 2009/09/18 21:48:53 UTC
RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0 releases
Hey Dejan,
FYI, I'm running the RC though the Progress internal performance test suite
over the weekend. Will advise of the results, but it should be interesting
to see how the new default config performs, and we can see if we need to
tweak it.
Colin
-----Original Message-----
From: chubrilo@gmail.com [mailto:chubrilo@gmail.com] On Behalf Of Dejan
Bosanac
Sent: Friday, September 18, 2009 11:14 AM
To: dev@activemq.apache.org
Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0
releases
Ok, I'll modify tomorrow how we create source release and include protobuf
code in it. I guess I'll need to tweak assembly-plugin and
apache-source-release-assembly-descriptor, but have to research it more on
how to do it. If anybody has any experience with this and would provide any
pointers it would be very helpful.
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Fri, Sep 18, 2009 at 7:02 PM, Hiram Chirino <ch...@gmail.com> wrote:
> Yeah that that does not have the source tar ball for the protobuf release.
>
> On Fri, Sep 18, 2009 at 12:52 PM, Bruce Snyder <bruce.snyder@gmail.com
> >wrote:
>
> > On Fri, Sep 18, 2009 at 7:10 AM, Hiram Chirino <ch...@gmail.com>
> wrote:
> > > Could you also post links to the source tarballs? Thanks!
> >
> > He already did:
> >
> >
> >
>
https://repository.apache.org/content/repositories/activemq-staging-030/org/
apache/activemq/activemq-parent/5.3.0/
> >
> > Bruce
> > --
> > perl -e 'print
> > unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> > );'
> >
> > ActiveMQ in Action: http://bit.ly/2je6cQ
> > Blog: http://bruceblog.org/
> > Twitter: http://twitter.com/brucesnyder
> >
>
>
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
> Open Source SOA
> http://fusesource.com/
>
Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0
releases
Posted by Bruce Snyder <br...@gmail.com>.
On Fri, Sep 25, 2009 at 3:42 AM, Dejan Bosanac <de...@nighttale.net> wrote:
> FYI. Removed old release candidate from nexus and svn, and put a warning
> message on the release notes page. A lot of people seems to be thinking it's
> released.
This is exactly why I suggested changing the artifact names from
activemq-5.3.0-bin.tar.gz to activemq-5.3.0-RCX-bin.tar.gz until it's
fully released. It would eliminate any such confusion for users.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0
releases
Posted by Dejan Bosanac <de...@nighttale.net>.
FYI. Removed old release candidate from nexus and svn, and put a warning
message on the release notes page. A lot of people seems to be thinking it's
released.
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Tue, Sep 22, 2009 at 1:59 PM, Rob Davies <ra...@gmail.com> wrote:
> Thats the problem - persistent Queue with no subscribers shouldn't block
> until store limit is reached - this is why flow control has been disabled by
> default now
>
> On 21 Sep 2009, at 23:29, Colin MacNaughton wrote:
>
> Hi Rob,
>>
>> I didn't run such a test, but I'd expect that the queue would pretty
>> quickly
>> fill up and block the senders using the config snippet below since it
>> limits
>> the queue size to 1Mb.
>>
>> Colin
>>
>> -----Original Message-----
>> From: Rob Davies [mailto:rajdavies@gmail.com]
>> Sent: Monday, September 21, 2009 3:25 PM
>> To: dev@activemq.apache.org
>> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0
>> releases
>>
>> Hey Colin - what results do you see with flow control on and no
>> consumers for persistent queues ?
>>
>> On 21 Sep 2009, at 21:01, Colin MacNaughton wrote:
>>
>> So ran into 2 issues running performance tests:
>>>
>>> 1. I ended up tweaking the default config to limit destination sizes
>>> and
>>> enable flow control as follows:
>>>
>>> <destinationPolicy>
>>> <policyMap>
>>> <policyEntries>
>>> <policyEntry topic=">" producerFlowControl="true"
>>> memoryLimit="1mb">
>>>
>>> <pendingSubscriberPolicy>
>>> <vmCursor/>
>>> </pendingSubscriberPolicy>
>>> </policyEntry>
>>> <policyEntry queue=">" producerFlowControl="true"
>>> memoryLimit="1mb"/>
>>> </policyEntries>
>>> </policyMap>
>>> </destinationPolicy>
>>>
>>> The current default config was resulting in really high latencies in
>>> non
>>> persistent pub sub tests (> 2 minutes!). With the new settings
>>> throughput
>>> doubled and average latency dropped to 3 seconds.
>>>
>>> However, it seems like there is some resistance to enabling flow
>>> control by
>>> default: http://issues.apache.org/activemq/browse/AMQ-2318, as naïve
>>> users
>>> might erroneously interpret this as a hang.
>>>
>>> So there is a tradeoff here against guarding again naïve users and
>>> good out
>>> of box performance benchmarking.
>>>
>>> A possible compromise appropriate for the 5.3.0 release time frame
>>> would be
>>> to log a warning the first time flow control is triggered for a
>>> destination,
>>> to assist naive users in troubleshooting producer pauses.
>>>
>>> More long term, it might be worth introucing a more sophisticated
>>> mechanism
>>> for when we page to disk like only do so when there are no consumers
>>> connected. A policy similar to this is already being pursued in the
>>> amq 6.0
>>> prototype.
>>>
>>> I logged this as http://issues.apache.org/activemq/browse/AMQ-2400
>>>
>>> 2. Fan-in to dups_ok queue receivers:
>>> While running performance tests I I was seeing hangs in several tests
>>> involving dups ok queue receivers. My suspicion is that this is
>>> related to
>>> "too lazy" dups_ok acknowledgements. Changing the queue
>>> prefetchLimit to 100
>>> caused this problem to go away. This needs more investigation, but
>>> it seems
>>> like we can get ourselves in to trouble if the queue size is smaller
>>> than
>>> the receiver's prefetchLimit, and this should be avoid. It is also
>>> possible
>>> that there is something more complicated happening in my tests. I
>>> haven't
>>> yet been able to reproduce this outside my performance test
>>> environment.
>>>
>>> Logged as http://issues.apache.org/activemq/browse/AMQ-2401
>>>
>>> -----Original Message-----
>>> From: Colin MacNaughton [mailto:colin.macnaughton@gmail.com]
>>> Sent: Friday, September 18, 2009 12:49 PM
>>> To: dev@activemq.apache.org
>>> Subject: RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
>>> 5.3.0
>>> releases
>>>
>>> Hey Dejan,
>>>
>>> FYI, I'm running the RC though the Progress internal performance
>>> test suite
>>> over the weekend. Will advise of the results, but it should be
>>> interesting
>>> to see how the new default config performs, and we can see if we
>>> need to
>>> tweak it.
>>>
>>> Colin
>>>
>>> -----Original Message-----
>>> From: chubrilo@gmail.com [mailto:chubrilo@gmail.com] On Behalf Of
>>> Dejan
>>> Bosanac
>>> Sent: Friday, September 18, 2009 11:14 AM
>>> To: dev@activemq.apache.org
>>> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
>>> 5.3.0
>>> releases
>>>
>>> Ok, I'll modify tomorrow how we create source release and include
>>> protobuf
>>> code in it. I guess I'll need to tweak assembly-plugin and
>>> apache-source-release-assembly-descriptor, but have to research it
>>> more on
>>> how to do it. If anybody has any experience with this and would
>>> provide any
>>> pointers it would be very helpful.
>>>
>>> Cheers
>>> --
>>> Dejan Bosanac
>>>
>>> Open Source Integration - http://fusesource.com/
>>> ActiveMQ in Action - http://www.manning.com/snyder/
>>> Blog - http://www.nighttale.net
>>>
>>>
>>> On Fri, Sep 18, 2009 at 7:02 PM, Hiram Chirino <ch...@gmail.com>
>>> wrote:
>>>
>>> Yeah that that does not have the source tar ball for the protobuf
>>>> release.
>>>>
>>>> On Fri, Sep 18, 2009 at 12:52 PM, Bruce Snyder
>>>> <bruce.snyder@gmail.com
>>>>
>>>>> wrote:
>>>>>
>>>>
>>>> On Fri, Sep 18, 2009 at 7:10 AM, Hiram Chirino <ch...@gmail.com>
>>>>>
>>>> wrote:
>>>>
>>>>> Could you also post links to the source tarballs? Thanks!
>>>>>>
>>>>>
>>>>> He already did:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>> https://repository.apache.org/content/repositories/activemq-staging-030/org/
>>
>>> apache/activemq/activemq-parent/5.3.0/
>>>
>>>>
>>>>> Bruce
>>>>> --
>>>>> perl -e 'print
>>>>> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
>>>>> );'
>>>>>
>>>>> ActiveMQ in Action: http://bit.ly/2je6cQ
>>>>> Blog: http://bruceblog.org/
>>>>> Twitter: http://twitter.com/brucesnyder
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Hiram
>>>>
>>>> Blog: http://hiramchirino.com
>>>>
>>>> Open Source SOA
>>>> http://fusesource.com/
>>>>
>>>>
>>>
>>>
>> Rob Davies
>> http://twitter.com/rajdavies
>> I work here: http://fusesource.com
>> My Blog: http://rajdavies.blogspot.com/
>> I'm writing this: http://www.manning.com/snyder/
>>
>>
>>
>>
>>
>>
>>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0 releases
Posted by Rob Davies <ra...@gmail.com>.
Thats the problem - persistent Queue with no subscribers shouldn't
block until store limit is reached - this is why flow control has been
disabled by default now
On 21 Sep 2009, at 23:29, Colin MacNaughton wrote:
> Hi Rob,
>
> I didn't run such a test, but I'd expect that the queue would pretty
> quickly
> fill up and block the senders using the config snippet below since
> it limits
> the queue size to 1Mb.
>
> Colin
>
> -----Original Message-----
> From: Rob Davies [mailto:rajdavies@gmail.com]
> Sent: Monday, September 21, 2009 3:25 PM
> To: dev@activemq.apache.org
> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
> 5.3.0
> releases
>
> Hey Colin - what results do you see with flow control on and no
> consumers for persistent queues ?
>
> On 21 Sep 2009, at 21:01, Colin MacNaughton wrote:
>
>> So ran into 2 issues running performance tests:
>>
>> 1. I ended up tweaking the default config to limit destination sizes
>> and
>> enable flow control as follows:
>>
>> <destinationPolicy>
>> <policyMap>
>> <policyEntries>
>> <policyEntry topic=">" producerFlowControl="true"
>> memoryLimit="1mb">
>>
>> <pendingSubscriberPolicy>
>> <vmCursor/>
>> </pendingSubscriberPolicy>
>> </policyEntry>
>> <policyEntry queue=">" producerFlowControl="true"
>> memoryLimit="1mb"/>
>> </policyEntries>
>> </policyMap>
>> </destinationPolicy>
>>
>> The current default config was resulting in really high latencies in
>> non
>> persistent pub sub tests (> 2 minutes!). With the new settings
>> throughput
>> doubled and average latency dropped to 3 seconds.
>>
>> However, it seems like there is some resistance to enabling flow
>> control by
>> default: http://issues.apache.org/activemq/browse/AMQ-2318, as naïve
>> users
>> might erroneously interpret this as a hang.
>>
>> So there is a tradeoff here against guarding again naïve users and
>> good out
>> of box performance benchmarking.
>>
>> A possible compromise appropriate for the 5.3.0 release time frame
>> would be
>> to log a warning the first time flow control is triggered for a
>> destination,
>> to assist naive users in troubleshooting producer pauses.
>>
>> More long term, it might be worth introucing a more sophisticated
>> mechanism
>> for when we page to disk like only do so when there are no consumers
>> connected. A policy similar to this is already being pursued in the
>> amq 6.0
>> prototype.
>>
>> I logged this as http://issues.apache.org/activemq/browse/AMQ-2400
>>
>> 2. Fan-in to dups_ok queue receivers:
>> While running performance tests I I was seeing hangs in several tests
>> involving dups ok queue receivers. My suspicion is that this is
>> related to
>> "too lazy" dups_ok acknowledgements. Changing the queue
>> prefetchLimit to 100
>> caused this problem to go away. This needs more investigation, but
>> it seems
>> like we can get ourselves in to trouble if the queue size is smaller
>> than
>> the receiver's prefetchLimit, and this should be avoid. It is also
>> possible
>> that there is something more complicated happening in my tests. I
>> haven't
>> yet been able to reproduce this outside my performance test
>> environment.
>>
>> Logged as http://issues.apache.org/activemq/browse/AMQ-2401
>>
>> -----Original Message-----
>> From: Colin MacNaughton [mailto:colin.macnaughton@gmail.com]
>> Sent: Friday, September 18, 2009 12:49 PM
>> To: dev@activemq.apache.org
>> Subject: RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
>> 5.3.0
>> releases
>>
>> Hey Dejan,
>>
>> FYI, I'm running the RC though the Progress internal performance
>> test suite
>> over the weekend. Will advise of the results, but it should be
>> interesting
>> to see how the new default config performs, and we can see if we
>> need to
>> tweak it.
>>
>> Colin
>>
>> -----Original Message-----
>> From: chubrilo@gmail.com [mailto:chubrilo@gmail.com] On Behalf Of
>> Dejan
>> Bosanac
>> Sent: Friday, September 18, 2009 11:14 AM
>> To: dev@activemq.apache.org
>> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
>> 5.3.0
>> releases
>>
>> Ok, I'll modify tomorrow how we create source release and include
>> protobuf
>> code in it. I guess I'll need to tweak assembly-plugin and
>> apache-source-release-assembly-descriptor, but have to research it
>> more on
>> how to do it. If anybody has any experience with this and would
>> provide any
>> pointers it would be very helpful.
>>
>> Cheers
>> --
>> Dejan Bosanac
>>
>> Open Source Integration - http://fusesource.com/
>> ActiveMQ in Action - http://www.manning.com/snyder/
>> Blog - http://www.nighttale.net
>>
>>
>> On Fri, Sep 18, 2009 at 7:02 PM, Hiram Chirino <ch...@gmail.com>
>> wrote:
>>
>>> Yeah that that does not have the source tar ball for the protobuf
>>> release.
>>>
>>> On Fri, Sep 18, 2009 at 12:52 PM, Bruce Snyder
>>> <bruce.snyder@gmail.com
>>>> wrote:
>>>
>>>> On Fri, Sep 18, 2009 at 7:10 AM, Hiram Chirino <ch...@gmail.com>
>>> wrote:
>>>>> Could you also post links to the source tarballs? Thanks!
>>>>
>>>> He already did:
>>>>
>>>>
>>>>
>>>
>>
> https://repository.apache.org/content/repositories/activemq-staging-030/org/
>> apache/activemq/activemq-parent/5.3.0/
>>>>
>>>> Bruce
>>>> --
>>>> perl -e 'print
>>>> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;
>>>> \"YC;VT*"
>>>> );'
>>>>
>>>> ActiveMQ in Action: http://bit.ly/2je6cQ
>>>> Blog: http://bruceblog.org/
>>>> Twitter: http://twitter.com/brucesnyder
>>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Hiram
>>>
>>> Blog: http://hiramchirino.com
>>>
>>> Open Source SOA
>>> http://fusesource.com/
>>>
>>
>>
>
> Rob Davies
> http://twitter.com/rajdavies
> I work here: http://fusesource.com
> My Blog: http://rajdavies.blogspot.com/
> I'm writing this: http://www.manning.com/snyder/
>
>
>
>
>
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0 releases
Posted by Colin MacNaughton <co...@gmail.com>.
Hi Rob,
I didn't run such a test, but I'd expect that the queue would pretty quickly
fill up and block the senders using the config snippet below since it limits
the queue size to 1Mb.
Colin
-----Original Message-----
From: Rob Davies [mailto:rajdavies@gmail.com]
Sent: Monday, September 21, 2009 3:25 PM
To: dev@activemq.apache.org
Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0
releases
Hey Colin - what results do you see with flow control on and no
consumers for persistent queues ?
On 21 Sep 2009, at 21:01, Colin MacNaughton wrote:
> So ran into 2 issues running performance tests:
>
> 1. I ended up tweaking the default config to limit destination sizes
> and
> enable flow control as follows:
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true"
> memoryLimit="1mb">
>
> <pendingSubscriberPolicy>
> <vmCursor/>
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true"
> memoryLimit="1mb"/>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> The current default config was resulting in really high latencies in
> non
> persistent pub sub tests (> 2 minutes!). With the new settings
> throughput
> doubled and average latency dropped to 3 seconds.
>
> However, it seems like there is some resistance to enabling flow
> control by
> default: http://issues.apache.org/activemq/browse/AMQ-2318, as naïve
> users
> might erroneously interpret this as a hang.
>
> So there is a tradeoff here against guarding again naïve users and
> good out
> of box performance benchmarking.
>
> A possible compromise appropriate for the 5.3.0 release time frame
> would be
> to log a warning the first time flow control is triggered for a
> destination,
> to assist naive users in troubleshooting producer pauses.
>
> More long term, it might be worth introucing a more sophisticated
> mechanism
> for when we page to disk like only do so when there are no consumers
> connected. A policy similar to this is already being pursued in the
> amq 6.0
> prototype.
>
> I logged this as http://issues.apache.org/activemq/browse/AMQ-2400
>
> 2. Fan-in to dups_ok queue receivers:
> While running performance tests I I was seeing hangs in several tests
> involving dups ok queue receivers. My suspicion is that this is
> related to
> "too lazy" dups_ok acknowledgements. Changing the queue
> prefetchLimit to 100
> caused this problem to go away. This needs more investigation, but
> it seems
> like we can get ourselves in to trouble if the queue size is smaller
> than
> the receiver's prefetchLimit, and this should be avoid. It is also
> possible
> that there is something more complicated happening in my tests. I
> haven't
> yet been able to reproduce this outside my performance test
> environment.
>
> Logged as http://issues.apache.org/activemq/browse/AMQ-2401
>
> -----Original Message-----
> From: Colin MacNaughton [mailto:colin.macnaughton@gmail.com]
> Sent: Friday, September 18, 2009 12:49 PM
> To: dev@activemq.apache.org
> Subject: RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
> 5.3.0
> releases
>
> Hey Dejan,
>
> FYI, I'm running the RC though the Progress internal performance
> test suite
> over the weekend. Will advise of the results, but it should be
> interesting
> to see how the new default config performs, and we can see if we
> need to
> tweak it.
>
> Colin
>
> -----Original Message-----
> From: chubrilo@gmail.com [mailto:chubrilo@gmail.com] On Behalf Of
> Dejan
> Bosanac
> Sent: Friday, September 18, 2009 11:14 AM
> To: dev@activemq.apache.org
> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
> 5.3.0
> releases
>
> Ok, I'll modify tomorrow how we create source release and include
> protobuf
> code in it. I guess I'll need to tweak assembly-plugin and
> apache-source-release-assembly-descriptor, but have to research it
> more on
> how to do it. If anybody has any experience with this and would
> provide any
> pointers it would be very helpful.
>
> Cheers
> --
> Dejan Bosanac
>
> Open Source Integration - http://fusesource.com/
> ActiveMQ in Action - http://www.manning.com/snyder/
> Blog - http://www.nighttale.net
>
>
> On Fri, Sep 18, 2009 at 7:02 PM, Hiram Chirino <ch...@gmail.com>
> wrote:
>
>> Yeah that that does not have the source tar ball for the protobuf
>> release.
>>
>> On Fri, Sep 18, 2009 at 12:52 PM, Bruce Snyder
>> <bruce.snyder@gmail.com
>>> wrote:
>>
>>> On Fri, Sep 18, 2009 at 7:10 AM, Hiram Chirino <ch...@gmail.com>
>> wrote:
>>>> Could you also post links to the source tarballs? Thanks!
>>>
>>> He already did:
>>>
>>>
>>>
>>
>
https://repository.apache.org/content/repositories/activemq-staging-030/org/
> apache/activemq/activemq-parent/5.3.0/
>>>
>>> Bruce
>>> --
>>> perl -e 'print
>>> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
>>> );'
>>>
>>> ActiveMQ in Action: http://bit.ly/2je6cQ
>>> Blog: http://bruceblog.org/
>>> Twitter: http://twitter.com/brucesnyder
>>>
>>
>>
>>
>> --
>> Regards,
>> Hiram
>>
>> Blog: http://hiramchirino.com
>>
>> Open Source SOA
>> http://fusesource.com/
>>
>
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0 releases
Posted by Rob Davies <ra...@gmail.com>.
Hey Colin - what results do you see with flow control on and no
consumers for persistent queues ?
On 21 Sep 2009, at 21:01, Colin MacNaughton wrote:
> So ran into 2 issues running performance tests:
>
> 1. I ended up tweaking the default config to limit destination sizes
> and
> enable flow control as follows:
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true"
> memoryLimit="1mb">
>
> <pendingSubscriberPolicy>
> <vmCursor/>
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="true"
> memoryLimit="1mb"/>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> The current default config was resulting in really high latencies in
> non
> persistent pub sub tests (> 2 minutes!). With the new settings
> throughput
> doubled and average latency dropped to 3 seconds.
>
> However, it seems like there is some resistance to enabling flow
> control by
> default: http://issues.apache.org/activemq/browse/AMQ-2318, as naïve
> users
> might erroneously interpret this as a hang.
>
> So there is a tradeoff here against guarding again naïve users and
> good out
> of box performance benchmarking.
>
> A possible compromise appropriate for the 5.3.0 release time frame
> would be
> to log a warning the first time flow control is triggered for a
> destination,
> to assist naive users in troubleshooting producer pauses.
>
> More long term, it might be worth introucing a more sophisticated
> mechanism
> for when we page to disk like only do so when there are no consumers
> connected. A policy similar to this is already being pursued in the
> amq 6.0
> prototype.
>
> I logged this as http://issues.apache.org/activemq/browse/AMQ-2400
>
> 2. Fan-in to dups_ok queue receivers:
> While running performance tests I I was seeing hangs in several tests
> involving dups ok queue receivers. My suspicion is that this is
> related to
> "too lazy" dups_ok acknowledgements. Changing the queue
> prefetchLimit to 100
> caused this problem to go away. This needs more investigation, but
> it seems
> like we can get ourselves in to trouble if the queue size is smaller
> than
> the receiver's prefetchLimit, and this should be avoid. It is also
> possible
> that there is something more complicated happening in my tests. I
> haven't
> yet been able to reproduce this outside my performance test
> environment.
>
> Logged as http://issues.apache.org/activemq/browse/AMQ-2401
>
> -----Original Message-----
> From: Colin MacNaughton [mailto:colin.macnaughton@gmail.com]
> Sent: Friday, September 18, 2009 12:49 PM
> To: dev@activemq.apache.org
> Subject: RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
> 5.3.0
> releases
>
> Hey Dejan,
>
> FYI, I'm running the RC though the Progress internal performance
> test suite
> over the weekend. Will advise of the results, but it should be
> interesting
> to see how the new default config performs, and we can see if we
> need to
> tweak it.
>
> Colin
>
> -----Original Message-----
> From: chubrilo@gmail.com [mailto:chubrilo@gmail.com] On Behalf Of
> Dejan
> Bosanac
> Sent: Friday, September 18, 2009 11:14 AM
> To: dev@activemq.apache.org
> Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ
> 5.3.0
> releases
>
> Ok, I'll modify tomorrow how we create source release and include
> protobuf
> code in it. I guess I'll need to tweak assembly-plugin and
> apache-source-release-assembly-descriptor, but have to research it
> more on
> how to do it. If anybody has any experience with this and would
> provide any
> pointers it would be very helpful.
>
> Cheers
> --
> Dejan Bosanac
>
> Open Source Integration - http://fusesource.com/
> ActiveMQ in Action - http://www.manning.com/snyder/
> Blog - http://www.nighttale.net
>
>
> On Fri, Sep 18, 2009 at 7:02 PM, Hiram Chirino <ch...@gmail.com>
> wrote:
>
>> Yeah that that does not have the source tar ball for the protobuf
>> release.
>>
>> On Fri, Sep 18, 2009 at 12:52 PM, Bruce Snyder
>> <bruce.snyder@gmail.com
>>> wrote:
>>
>>> On Fri, Sep 18, 2009 at 7:10 AM, Hiram Chirino <ch...@gmail.com>
>> wrote:
>>>> Could you also post links to the source tarballs? Thanks!
>>>
>>> He already did:
>>>
>>>
>>>
>>
> https://repository.apache.org/content/repositories/activemq-staging-030/org/
> apache/activemq/activemq-parent/5.3.0/
>>>
>>> Bruce
>>> --
>>> perl -e 'print
>>> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
>>> );'
>>>
>>> ActiveMQ in Action: http://bit.ly/2je6cQ
>>> Blog: http://bruceblog.org/
>>> Twitter: http://twitter.com/brucesnyder
>>>
>>
>>
>>
>> --
>> Regards,
>> Hiram
>>
>> Blog: http://hiramchirino.com
>>
>> Open Source SOA
>> http://fusesource.com/
>>
>
>
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0 releases
Posted by Colin MacNaughton <co...@gmail.com>.
So ran into 2 issues running performance tests:
1. I ended up tweaking the default config to limit destination sizes and
enable flow control as follows:
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb">
<pendingSubscriberPolicy>
<vmCursor/>
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb"/>
</policyEntries>
</policyMap>
</destinationPolicy>
The current default config was resulting in really high latencies in non
persistent pub sub tests (> 2 minutes!). With the new settings throughput
doubled and average latency dropped to 3 seconds.
However, it seems like there is some resistance to enabling flow control by
default: http://issues.apache.org/activemq/browse/AMQ-2318, as naïve users
might erroneously interpret this as a hang.
So there is a tradeoff here against guarding again naïve users and good out
of box performance benchmarking.
A possible compromise appropriate for the 5.3.0 release time frame would be
to log a warning the first time flow control is triggered for a destination,
to assist naive users in troubleshooting producer pauses.
More long term, it might be worth introucing a more sophisticated mechanism
for when we page to disk like only do so when there are no consumers
connected. A policy similar to this is already being pursued in the amq 6.0
prototype.
I logged this as http://issues.apache.org/activemq/browse/AMQ-2400
2. Fan-in to dups_ok queue receivers:
While running performance tests I I was seeing hangs in several tests
involving dups ok queue receivers. My suspicion is that this is related to
"too lazy" dups_ok acknowledgements. Changing the queue prefetchLimit to 100
caused this problem to go away. This needs more investigation, but it seems
like we can get ourselves in to trouble if the queue size is smaller than
the receiver's prefetchLimit, and this should be avoid. It is also possible
that there is something more complicated happening in my tests. I haven't
yet been able to reproduce this outside my performance test environment.
Logged as http://issues.apache.org/activemq/browse/AMQ-2401
-----Original Message-----
From: Colin MacNaughton [mailto:colin.macnaughton@gmail.com]
Sent: Friday, September 18, 2009 12:49 PM
To: dev@activemq.apache.org
Subject: RE: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0
releases
Hey Dejan,
FYI, I'm running the RC though the Progress internal performance test suite
over the weekend. Will advise of the results, but it should be interesting
to see how the new default config performs, and we can see if we need to
tweak it.
Colin
-----Original Message-----
From: chubrilo@gmail.com [mailto:chubrilo@gmail.com] On Behalf Of Dejan
Bosanac
Sent: Friday, September 18, 2009 11:14 AM
To: dev@activemq.apache.org
Subject: Re: [VOTE] AciveMQ Protocol Buffers 1.0 and Apache ActiveMQ 5.3.0
releases
Ok, I'll modify tomorrow how we create source release and include protobuf
code in it. I guess I'll need to tweak assembly-plugin and
apache-source-release-assembly-descriptor, but have to research it more on
how to do it. If anybody has any experience with this and would provide any
pointers it would be very helpful.
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Fri, Sep 18, 2009 at 7:02 PM, Hiram Chirino <ch...@gmail.com> wrote:
> Yeah that that does not have the source tar ball for the protobuf release.
>
> On Fri, Sep 18, 2009 at 12:52 PM, Bruce Snyder <bruce.snyder@gmail.com
> >wrote:
>
> > On Fri, Sep 18, 2009 at 7:10 AM, Hiram Chirino <ch...@gmail.com>
> wrote:
> > > Could you also post links to the source tarballs? Thanks!
> >
> > He already did:
> >
> >
> >
>
https://repository.apache.org/content/repositories/activemq-staging-030/org/
apache/activemq/activemq-parent/5.3.0/
> >
> > Bruce
> > --
> > perl -e 'print
> > unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> > );'
> >
> > ActiveMQ in Action: http://bit.ly/2je6cQ
> > Blog: http://bruceblog.org/
> > Twitter: http://twitter.com/brucesnyder
> >
>
>
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
> Open Source SOA
> http://fusesource.com/
>