NetworkConnectors and prefetch

[RickSnz <ri...@altran.es>]

Hello,

We are using a network of brokers but it's giving us some problems with load
balancing. Most of the messages are being delivered to the remote broker
instead of being shared between the network connector and the broker's
consumer. If a broker has 5 consumers and 1 network connector, and receives
50 messages, it processes 5-10 in its consumers and the rest outside

We are using prefetch = 1 for the consumers, so we set prefetchSize=1 for
the network connector. We have tried both with and without
conduitSubscriptions. The only configuration that have worked was setting a
high prefetch in consumers and network connector, but that was only for
testing purpose, as we cant modify the consumer's prefetch size.

So we need to know if there's any way to make it work with prefetch = 1 in
the consumers. The expected behaviour in the case explained before would be
processing 25 messages in the broker's consumers and send 25 to the network
connector.

Thanks

--
View this message in context: http://activemq.2283324.n4.nabble.com/NetworkConnectors-and-prefetch-tp4652677.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: NetworkConnectors and prefetch

[Gary Tully <ga...@gmail.com>]

the problem here is that the network bridge 'proxy' consumer is faster
than the local consumers.
you can use decreaseNetworkConsumerPriority=true to slow down the
network bridge but even this will not give you the desired effect with
prefetch=1, but it will help.

The other alternative is to build your own dispatch filter of a
network consumer and have it rate limit what messages are consumed.
There is an example in
org.apache.activemq.network.ConditionalNetworkBridgeFilterFactory
which is like what you need except that it only limits messages that
are being redirected back to their origin.

If  decreaseNetworkConsumerPriority does not work, peek at:
org.apache.activemq.usecases.TwoBrokerQueueClientsReconnectTest#applyRateLimitNetworkFilter

On 24 May 2012 12:38, RickSnz <ri...@altran.es> wrote:
> Hello,
>
> We are using a network of brokers but it's giving us some problems with load
> balancing. Most of the messages are being delivered to the remote broker
> instead of being shared between the network connector and the broker's
> consumer. If a broker has 5 consumers and 1 network connector, and receives
> 50 messages, it processes 5-10 in its consumers and the rest outside
>
> We are using prefetch = 1 for the consumers, so we set prefetchSize=1 for
> the network connector. We have tried both with and without
> conduitSubscriptions. The only configuration that have worked was setting a
> high prefetch in consumers and network connector, but that was only for
> testing purpose, as we cant modify the consumer's prefetch size.
>
> So we need to know if there's any way to make it work with prefetch = 1 in
> the consumers. The expected behaviour in the case explained before would be
> processing 25 messages in the broker's consumers and send 25 to the network
> connector.
>
> Thanks
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/NetworkConnectors-and-prefetch-tp4652677.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.



-- 
http://fusesource.com
http://blog.garytully.com