High-speed topic publishing & consumption

[David Crisp <da...@yahoo.com>]

Hi,

     I'm using ActiveMQ 5.1 to publish from a single fast producer to four
consumers.
All messages are non-durable and contain ephemeral data so I have the
message 
time to live set to one second.  My goal is to be able to publish as fast as 
possible and have any messages that don't make it to consumers in the one
second interval be silently dropped by the message system.

     When I run the producerTool example at full-blast and send 2M messages
to a consumer over a network of brokers to a set of consumers, the producer
eventually stops sending.  I have set the prefetch size on the consumers to
a
large number (10K) but I'm still being burned either by flow control or by
the
processing to send an expired message to the dead letter queue.  Given the
use
case described above, what is the right way to accomplish my goal?  Can I
disable
the DLQ and producer flow control?  Are there any other tweaks I should
consider?

Thanks in advance,
D.

-- 
View this message in context: http://www.nabble.com/High-speed-topic-publishing---consumption-tp19113739p19113739.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: High-speed topic publishing & consumption

[Rob Davies <ra...@gmail.com>]

Disable flow control - e.g.:

<destinationPolicy>
       <policyMap>
         <policyEntries>

           <policyEntry topic="FOO.>" producerFlowControl="false"  
memoryLimit="1mb">
             <dispatchPolicy>
               <strictOrderDispatchPolicy/>
             </dispatchPolicy>
             <subscriptionRecoveryPolicy>
               <lastImageSubscriptionRecoveryPolicy/>
             </subscriptionRecoveryPolicy>
           </policyEntry>

         </policyEntries>
       </policyMap>
</destinationPolicy>

thats from - http://activemq.apache.org/producer-flow-control.html -  
that together with ignoring expired messages on the dead letter strategy
should make it fly.

For your consumers - set the optimizeAcknowledge property on the  
ActiveMQConnectionFactory too


cheers,

Rob

Rob Davies
http://rajdavies.blogspot.com/

On 22 Aug 2008, at 20:06, David Crisp wrote:

>
> Hi,
>
>     I'm using ActiveMQ 5.1 to publish from a single fast producer to  
> four
> consumers.
> All messages are non-durable and contain ephemeral data so I have the
> message
> time to live set to one second.  My goal is to be able to publish as  
> fast as
> possible and have any messages that don't make it to consumers in  
> the one
> second interval be silently dropped by the message system.
>
>     When I run the producerTool example at full-blast and send 2M  
> messages
> to a consumer over a network of brokers to a set of consumers, the  
> producer
> eventually stops sending.  I have set the prefetch size on the  
> consumers to
> a
> large number (10K) but I'm still being burned either by flow control  
> or by
> the
> processing to send an expired message to the dead letter queue.   
> Given the
> use
> case described above, what is the right way to accomplish my goal?   
> Can I
> disable
> the DLQ and producer flow control?  Are there any other tweaks I  
> should
> consider?
>
> Thanks in advance,
> D.
>
> -- 
> View this message in context: http://www.nabble.com/High-speed-topic-publishing---consumption-tp19113739p19113739.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>