You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Eric-AWL <er...@atosorigin.com> on 2008/01/30 12:20:14 UTC
Prefetch Limit of 1000 .... without acknowledges
Hi
Context
A "client" process is on "ytsvmx01s" server, Server process is on
"ytsvmx02s" server (an other server). A duplex network of brokers is made by
the client configuration.
<networkConnector name="pool" uri="static:(tcp://ytsvmx02s:61617)"
conduitSubscriptions="false"
dynamicOnly="true"
networkTTL="3"
duplex="true"/>
An embedded VM client process, with 100 threads.
- each thread create a temporary queue
- each thread sends 10 messages, with ReplyTo the temporary queue
70 threads send message on queue1, 20 on queue2, 10 on queue3.
An embedded VM server process which has 3 listeners (one on queue1, one on
queue2, one on queue3)
-each listener listens messages and send with a specific session an answer
on the temporary queue defined by the "ReplyTo" field.
All sessions are in AUTO_ACKNOWLEDGMENT mode.
When the specific session created by the listener which listens to queue1
(the most active queue) send exactly 1000 messages (so, on 70 differents
temporary queues), I have a "prefetch limit" in client logfile (log4j debug)
and no other messages can be sent to queue1.
Other threads continue to send on queue2 and queue3.
You will find the logfile in
http://www.nabble.com/file/p15180095/client100.log client100.log
and the last message sent by ytsvmx02s "queue1" session is
DEBUG ActiveMQMessageConsumer -
ID:ytsvmx01s-44878-1201690176105-3:1:115:1 received message: MessageDispatch
{commandId = 4292, responseRequired = false, consumerId =
ID:ytsvmx01s-44878-1201690176105-3:1:115:1, destination =
temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, message =
ActiveMQTextMessage {commandId = 2440, responseRequired = false, messageId =
ID:ytsvmx02s-52597-1201689535145-2:1:2:1000:1, originalDestination = null,
originalTransactionId = null, producerId =
ID:ytsvmx02s-52597-1201689535145-5:0:1:1, destination =
temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, transactionId = null,
expiration = 0, timestamp = 1201689604289, arrival = 0, brokerInTime =
1201690235463, brokerOutTime = 1201690235463, correlationId = null, replyTo
= null, persistent = true, type = null, priority = 4, groupID = null,
groupSequence = 0, targetConsumerId = null, compressed = false, userID =
null, content = org.apache.activemq.util.ByteSequence@1410a85,
marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
size = 8222, properties = null, readOnlyProperties = true, readOnlyBody =
true, droppable = false, text = null}, redeliveryCounter = 0}
and there isn't any message "ID:ytsvmx02s-52597-1201689535145-2:1:2:1001:1"
after this one and you will see that there is
DEBUG PrefetchSubscription - Prefetch limit.
just after in the logfile.
Eric-AWL
--
View this message in context: http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15180095.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Prefetch Limit of 1000 .... without acknowledges
Posted by Eric-AWL <er...@atosorigin.com>.
ttmdev said me that this "duplex" problem is the same as AMQ-1521 and is
resolved in 5.1.1 .
I will try it.
Thank you.
Eric-AWL
--
View this message in context: http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15223382.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Prefetch Limit of 1000 .... without acknowledges
Posted by Eric-AWL <er...@atosorigin.com>.
Hi James
ttmdev help me to resolve the problem with two half-duplex connections. it
was a problem of synchronization. Resources were freed to quickly.
For the problem of duplex, ttmdev said me that this problem was fixed in 5.1
and was the same as problem AMQ-1521.
Eric-AWL
--
View this message in context: http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15230734.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Prefetch Limit of 1000 .... without acknowledges
Posted by Eric-AWL <er...@atosorigin.com>.
Hi !
I just finished a test with one fixed queue for the answer from the server,
instead of using temporary queues.
On the client, for every message the client want to send, I associate a
JMSCorrelationID, and I create one consumer on this fixed queue with a
selector on this CorrelationID. The server receives the message on one of
the three queues, and send an answer with the CorrelationID wait by the
client consumer.
First, It's 10 times slower than using a temporary queue for each thread.
It's the same problem. The server session which receives 70% of the trafic
is the first session to stop sending answer on the fixed queue. And it is
just after sending 1000 answers with the same session.
Eric
--
View this message in context: http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15206165.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Prefetch Limit of 1000 .... without acknowledges
Posted by Eric-AWL <er...@atosorigin.com>.
Hi!
I'm french so excuse me for my english mistakes.
Now I am in 5.0.0, the release version.
My two servers are Linux servers.
You will find a lot of informations about the problem and tar files of my
configuration in the thread named
"Embedded brokers" "network of brokers" "temporary queues".
In fact, when I use "two half duplex network connections", I have a "Cannot
publish message". When I use "full duplex network connection", I have this
"1000 messages limit".
I think the xml configuration files in the tar files are with the "two harf
duplex connections" configuration.
For the "full duplex", I put
<networkConnector name="pool" uri="static:(tcp://ytsvmx02s:61617)"
conduitSubscriptions="false"
dynamicOnly="true"
networkTTL="3"
duplex="true"/>
in activemq_clt.xml and delete the "network connector" paragraph in the
"activemq_srv.xml".
If you don't succeed in installing, running these programs, I will try to
send you an other package in an other form.
I work in a big IT european company and we really want to fully understand
ActiveMQ and deploy it in a lot of big projects.
Thank you for your help.
As a workaround, I will try to use only one fixed queue for the answer
instead of 100 temporary queues, and use JMScorrelationId and selectors in
each client threads.
James.Strachan wrote:
>
> which version of ActiveMQ?
>
> On 30/01/2008, Eric-AWL <er...@atosorigin.com> wrote:
>>
>> Hi
>>
>> Context
>>
>> A "client" process is on "ytsvmx01s" server, Server process is on
>> "ytsvmx02s" server (an other server). A duplex network of brokers is made
>> by
>> the client configuration.
>>
>> <networkConnector name="pool" uri="static:(tcp://ytsvmx02s:61617)"
>> conduitSubscriptions="false"
>> dynamicOnly="true"
>> networkTTL="3"
>> duplex="true"/>
>>
>>
>> An embedded VM client process, with 100 threads.
>> - each thread create a temporary queue
>> - each thread sends 10 messages, with ReplyTo the temporary queue
>>
>> 70 threads send message on queue1, 20 on queue2, 10 on queue3.
>>
>> An embedded VM server process which has 3 listeners (one on queue1, one
>> on
>> queue2, one on queue3)
>> -each listener listens messages and send with a specific session an
>> answer
>> on the temporary queue defined by the "ReplyTo" field.
>>
>>
>> All sessions are in AUTO_ACKNOWLEDGMENT mode.
>>
>> When the specific session created by the listener which listens to queue1
>> (the most active queue) send exactly 1000 messages (so, on 70 differents
>> temporary queues), I have a "prefetch limit" in client logfile (log4j
>> debug)
>> and no other messages can be sent to queue1.
>>
>> Other threads continue to send on queue2 and queue3.
>>
>> You will find the logfile in
>> http://www.nabble.com/file/p15180095/client100.log client100.log
>>
>>
>> and the last message sent by ytsvmx02s "queue1" session is
>>
>> DEBUG ActiveMQMessageConsumer -
>> ID:ytsvmx01s-44878-1201690176105-3:1:115:1 received message:
>> MessageDispatch
>> {commandId = 4292, responseRequired = false, consumerId =
>> ID:ytsvmx01s-44878-1201690176105-3:1:115:1, destination =
>> temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, message =
>> ActiveMQTextMessage {commandId = 2440, responseRequired = false,
>> messageId =
>> ID:ytsvmx02s-52597-1201689535145-2:1:2:1000:1, originalDestination =
>> null,
>> originalTransactionId = null, producerId =
>> ID:ytsvmx02s-52597-1201689535145-5:0:1:1, destination =
>> temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, transactionId =
>> null,
>> expiration = 0, timestamp = 1201689604289, arrival = 0, brokerInTime =
>> 1201690235463, brokerOutTime = 1201690235463, correlationId = null,
>> replyTo
>> = null, persistent = true, type = null, priority = 4, groupID = null,
>> groupSequence = 0, targetConsumerId = null, compressed = false, userID =
>> null, content = org.apache.activemq.util.ByteSequence@1410a85,
>> marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
>> size = 8222, properties = null, readOnlyProperties = true, readOnlyBody =
>> true, droppable = false, text = null}, redeliveryCounter = 0}
>>
>> and there isn't any message
>> "ID:ytsvmx02s-52597-1201689535145-2:1:2:1001:1"
>> after this one and you will see that there is
>> DEBUG PrefetchSubscription - Prefetch limit.
>> just after in the logfile.
>>
>> Eric-AWL
>> --
>> View this message in context:
>> http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15180095.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>
> Open Source Integration
> http://open.iona.com
>
>
--
View this message in context: http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15200410.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Prefetch Limit of 1000 .... without acknowledges
Posted by ja...@gmail.com.
which version of ActiveMQ?
On 30/01/2008, Eric-AWL <er...@atosorigin.com> wrote:
>
> Hi
>
> Context
>
> A "client" process is on "ytsvmx01s" server, Server process is on
> "ytsvmx02s" server (an other server). A duplex network of brokers is made by
> the client configuration.
>
> <networkConnector name="pool" uri="static:(tcp://ytsvmx02s:61617)"
> conduitSubscriptions="false"
> dynamicOnly="true"
> networkTTL="3"
> duplex="true"/>
>
>
> An embedded VM client process, with 100 threads.
> - each thread create a temporary queue
> - each thread sends 10 messages, with ReplyTo the temporary queue
>
> 70 threads send message on queue1, 20 on queue2, 10 on queue3.
>
> An embedded VM server process which has 3 listeners (one on queue1, one on
> queue2, one on queue3)
> -each listener listens messages and send with a specific session an answer
> on the temporary queue defined by the "ReplyTo" field.
>
>
> All sessions are in AUTO_ACKNOWLEDGMENT mode.
>
> When the specific session created by the listener which listens to queue1
> (the most active queue) send exactly 1000 messages (so, on 70 differents
> temporary queues), I have a "prefetch limit" in client logfile (log4j debug)
> and no other messages can be sent to queue1.
>
> Other threads continue to send on queue2 and queue3.
>
> You will find the logfile in
> http://www.nabble.com/file/p15180095/client100.log client100.log
>
>
> and the last message sent by ytsvmx02s "queue1" session is
>
> DEBUG ActiveMQMessageConsumer -
> ID:ytsvmx01s-44878-1201690176105-3:1:115:1 received message: MessageDispatch
> {commandId = 4292, responseRequired = false, consumerId =
> ID:ytsvmx01s-44878-1201690176105-3:1:115:1, destination =
> temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, message =
> ActiveMQTextMessage {commandId = 2440, responseRequired = false, messageId =
> ID:ytsvmx02s-52597-1201689535145-2:1:2:1000:1, originalDestination = null,
> originalTransactionId = null, producerId =
> ID:ytsvmx02s-52597-1201689535145-5:0:1:1, destination =
> temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, transactionId = null,
> expiration = 0, timestamp = 1201689604289, arrival = 0, brokerInTime =
> 1201690235463, brokerOutTime = 1201690235463, correlationId = null, replyTo
> = null, persistent = true, type = null, priority = 4, groupID = null,
> groupSequence = 0, targetConsumerId = null, compressed = false, userID =
> null, content = org.apache.activemq.util.ByteSequence@1410a85,
> marshalledProperties = null, dataStructure = null, redeliveryCounter = 0,
> size = 8222, properties = null, readOnlyProperties = true, readOnlyBody =
> true, droppable = false, text = null}, redeliveryCounter = 0}
>
> and there isn't any message "ID:ytsvmx02s-52597-1201689535145-2:1:2:1001:1"
> after this one and you will see that there is
> DEBUG PrefetchSubscription - Prefetch limit.
> just after in the logfile.
>
> Eric-AWL
> --
> View this message in context:
> http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15180095.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
James
-------
http://macstrac.blogspot.com/
Open Source Integration
http://open.iona.com
Re: Prefetch Limit of 1000 .... without acknowledges
Posted by Eric-AWL <er...@atosorigin.com>.
Hi !
ttmdev helps me to resolve the problem with two half-duplex connections. it
was a problem of synchronization. Resources were freed to quickly.
For the problem of duplex, ttmdev said me that this problem was fixed in 5.1
and was the same as problem AMQ-1521.
Eric-AWL
--
View this message in context: http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15230764.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Prefetch Limit of 1000 .... without acknowledges
Posted by Rob Davies <ra...@gmail.com>.
On Jan 30, 2008, at 11:20 AM, Eric-AWL wrote:
>
> Hi
>
> Context
>
> A "client" process is on "ytsvmx01s" server, Server process is on
> "ytsvmx02s" server (an other server). A duplex network of brokers is
> made by
> the client configuration.
>
> <networkConnector name="pool" uri="static:(tcp://ytsvmx02s:61617)"
> conduitSubscriptions="false"
> dynamicOnly="true"
> networkTTL="3"
> duplex="true"/>
>
>
> An embedded VM client process, with 100 threads.
> - each thread create a temporary queue
> - each thread sends 10 messages, with ReplyTo the temporary queue
>
> 70 threads send message on queue1, 20 on queue2, 10 on queue3.
>
> An embedded VM server process which has 3 listeners (one on queue1,
> one on
> queue2, one on queue3)
> -each listener listens messages and send with a specific session an
> answer
> on the temporary queue defined by the "ReplyTo" field.
>
>
> All sessions are in AUTO_ACKNOWLEDGMENT mode.
>
> When the specific session created by the listener which listens to
> queue1
> (the most active queue) send exactly 1000 messages (so, on 70
> differents
> temporary queues), I have a "prefetch limit" in client logfile
> (log4j debug)
> and no other messages can be sent to queue1.
>
> Other threads continue to send on queue2 and queue3.
>
> You will find the logfile in
> http://www.nabble.com/file/p15180095/client100.log client100.log
>
>
> and the last message sent by ytsvmx02s "queue1" session is
>
> DEBUG ActiveMQMessageConsumer -
> ID:ytsvmx01s-44878-1201690176105-3:1:115:1 received message:
> MessageDispatch
> {commandId = 4292, responseRequired = false, consumerId =
> ID:ytsvmx01s-44878-1201690176105-3:1:115:1, destination =
> temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, message =
> ActiveMQTextMessage {commandId = 2440, responseRequired = false,
> messageId =
> ID:ytsvmx02s-52597-1201689535145-2:1:2:1000:1, originalDestination =
> null,
> originalTransactionId = null, producerId =
> ID:ytsvmx02s-52597-1201689535145-5:0:1:1, destination =
> temp-queue://ID:ytsvmx01s-44878-1201690176105-3:1:114, transactionId
> = null,
> expiration = 0, timestamp = 1201689604289, arrival = 0, brokerInTime =
> 1201690235463, brokerOutTime = 1201690235463, correlationId = null,
> replyTo
> = null, persistent = true, type = null, priority = 4, groupID = null,
> groupSequence = 0, targetConsumerId = null, compressed = false,
> userID =
> null, content = org.apache.activemq.util.ByteSequence@1410a85,
> marshalledProperties = null, dataStructure = null, redeliveryCounter
> = 0,
> size = 8222, properties = null, readOnlyProperties = true,
> readOnlyBody =
> true, droppable = false, text = null}, redeliveryCounter = 0}
>
> and there isn't any message
> "ID:ytsvmx02s-52597-1201689535145-2:1:2:1001:1"
> after this one and you will see that there is
> DEBUG PrefetchSubscription - Prefetch limit.
> just after in the logfile.
>
> Eric-AWL
> --
> View this message in context: http://www.nabble.com/Prefetch-Limit-of-1000-....-without-acknowledges-tp15180095s2354p15180095.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
Hi eric - can you replicate this in a single junit test case - and
attach it to a jira issue - we will be able to track and resolve it
much quicker if you do
Rob Davies
'Go further faster with Apache Camel!'
http://rajdavies.blogspot.com/