Re: Trouble with prefetch buffer.

[James Strachan <ja...@gmail.com>]

A combination of setting a prefetch of 1 and using a round robin
dispatch policy should do the trick I think.

http://incubator.apache.org/activemq/dispatch-policies.html

We could also add support for a prefetch of zero - or this issue might
also resolve your issue...

http://issues.apache.org/activemq/browse/AMQ-850

allowing the setting a timeout by which messages are returned to the
broker for timed out consumers...




On 7/31/06, Vadim Pesochinsky <Va...@mscibarra.com> wrote:
>
> Hi!
>
> I have a test case to consider. 2 servers are processing 3 submitted jobs
> with following processing times 10 min, 1 min, 1 min. This sequence should
> finish in 10 minutes as one service will pick up the 10 minutes job,
> meanwhile the other one should manage the two 1 minute jobs. Since I cannot
> set prefetchSize=0, one of the 1 minute jobs is sitting in prefetch buffer
> and the jobs are processed in 11 minutes instead of 10.
>
> This is simplification of the real scenario where I have about 30 consumers
> submitting jobs to 20 consumers through AMQ 4.0.1. I have following
> problems:
>
> •       Messages are sitting in prefetch buffer are not available to processors,
> which results in a lot of idle time.
> •       Order of processing is random. For some reason Job # 20 is processed after
> Job # 1500. Since senders are synchronously blocked this can result in
> time-outs.
>
> What can I do to work around these problems? Any configuration changes? How
> to set prefetchSize=0; should submit a JIRA for this?
>
> Environment & Configuration
> AMQ 4.0.1
> QUEUE_NAME?consumer.prefetchSize=1;consumer.dispatchAsync=true
>
>
> <beans xmlns="http://activemq.org/config/1.0">
>     <!-- Allows us to use system properties as variables in this
> configuration file -->
>     <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
>     <broker useJmx="true" deleteAllMessagesOnStartup="true"
> persistent="true" >
>         <!-- Use the following to configure how ActiveMQ is exposed in JMX
> -->
>
>         <managementContext>
>             <managementContext connectorPort="1099"
> jmxDomainName="org.apache.activemq"/>
>         </managementContext>
>
>         <destinationPolicy>
>             <policyMap>
>                 <policyEntries>
>                     <policyEntry topic="FIN.>">
>                         <dispatchPolicy>
>                             <strictOrderDispatchPolicy />
>                         </dispatchPolicy>
>                         <pendingMessageLimitStrategy>
>                             <constantPendingMessageLimitStrategy limit="0"/>
>                         </pendingMessageLimitStrategy>
>                         <messageEvictionStrategy>
>                             <oldestMessageEvictionStrategy />
>                         </messageEvictionStrategy>
>                     </policyEntry>
>                 </policyEntries>
>             </policyMap>
>         </destinationPolicy>
>
>         <persistenceAdapter>
>             <kahaPersistentAdaptor dir="activemq-kaha"/>
>         </persistenceAdapter>
>
>         <transportConnectors>
>             <transportConnector name="default" uri="tcp://localhost:61616"
> discoveryUri="multicast://default"/>
>         </transportConnectors>
>
>         <networkConnectors>
>             <networkConnector name="default"
> uri="static://(tcp://localhost:61616)"/>
>         </networkConnectors>
>     </broker>
> </beans>
>
> --
> View this message in context: http://www.nabble.com/Trouble-with-prefetch-buffer.-tf2029800.html#a5583273
> Sent from the ActiveMQ - User forum at Nabble.com.
>
>


-- 

James
-------
http://radio.weblogs.com/0112098/

Re: Trouble with prefetch buffer.

[Vadim Pesochinsky <Va...@mscibarra.com>]

I do not think AMQ-850 will help, because in addition to the 2 described
issues I have real-time requests from users going to the same queue and I
cannot wait for the prefetch buffer to time-out. Round robin will not help
me, because it does not garantee that server that picked a long job will not
have anything in the prefetch buffer.

I created a JIRA for this: https://issues.apache.org/activemq/browse/AMQ-855

James, maybe you can give me a few tips on fixing this. It is a party pooper
for me. The company is not very excited about open source and with upset
like this they will be even less excited.

Thanks.
-- 
View this message in context: http://www.nabble.com/Trouble-with-prefetch-buffer.-tf2029800.html#a5599978
Sent from the ActiveMQ - User forum at Nabble.com.