You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Rudolf Janz (JIRA)" <ji...@apache.org> on 2010/01/14 15:26:43 UTC
[jira] Created: (AMQ-2567) Zero Prefetch not working
Zero Prefetch not working
-------------------------
Key: AMQ-2567
URL: https://issues.apache.org/activemq/browse/AMQ-2567
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.3.0
Environment: Unix/Windows, Java 1.6
Reporter: Rudolf Janz
I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
Two consumers (C1 and C2) are instantiated. These messages are sent:
1,2,3,4,5,6,7,8,9
C1 reads 4 times, receives 1,2,3,4 -> this is correct
now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Christoph Kutzinski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=57204#action_57204 ]
Christoph Kutzinski commented on AMQ-2567:
------------------------------------------
Seems like the MD5 and SHA signatures are redeployed every day, but the artifacts themselves (JARs, POMs) are still from January 15th
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Fix For: 5.3.1, 5.4.0
>
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Rudolf Janz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56908#action_56908 ]
Rudolf Janz commented on AMQ-2567:
----------------------------------
Gary Tully wrote:
PrefetchSubscription and prefetchExtension on a deliveryAck in a transaction
are areas that you need to look into to resolve this. Great that you have a
simple junit test case.
---
You are right, prefetchExtension is the problem. I do not understand the reason for it. If I want prefetch 0, it should stay that way. Right now it is increased on each pull.
In acknowledge there is the following comment
if (context.isInTransaction()) {
// extend prefetch window only if not a pulling consumer
if (getPrefetchSize() != 0) {
prefetchExtension = Math.max(
prefetchExtension, index );
In PrefetchSubscription::pullMessage it checks for prefetchSize 0
if (getPrefetchSize() == 0 && !isSlave()) {
final long dispatchCounterBeforePull;
synchronized(this) {
prefetchExtension++;
dispatchCounterBeforePull = dispatchCounter;
}
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Attachments: ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Christoph Kutzinski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=57208#action_57208 ]
Christoph Kutzinski commented on AMQ-2567:
------------------------------------------
Thanks.
Seems like new artifacts were only deployed for ActiveMQ 5.4.
Can you also do it for 5.3.1-SNAPSHOT, please?
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Fix For: 5.3.1, 5.4.0
>
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56909#action_56909 ]
Gary Tully commented on AMQ-2567:
---------------------------------
so the prefetchExtension is the mechanism by which a message is dispatched to a consumer with prefetch=0. Each receive() sends a pull if there are no messages which increments the prefetchExtension so another message is dispatched. With a prefetch of 0, you should only be able to get one message at a time.
I think a deliveredAck may be the problem, a delivered ack uses the prefetchextension to ensure that within a transaction it is possible to read past the prefetch limit. If prefetch=0, a pulling consumer, it should not do that. Check to see if you are getting an ack with ackType=0.
if so, either the logic with a standard ack needs to be duplicated with a deliveredAck or the delivery ack needs to be suppressed on the consumer.
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Attachments: ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=57177#action_57177 ]
Gary Tully commented on AMQ-2567:
---------------------------------
most of the artifacts are there but the release full tar or zip files still have not been updated for the 5.4-SNAPSHOT - need to check why hudson has not done it, it may be pending a full clean build. Some test failures were resolved today so it should be real soon.
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Fix For: 5.3.1, 5.4.0
>
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (AMQ-2567) Zero Prefetch not working
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully reassigned AMQ-2567:
-------------------------------
Assignee: Gary Tully
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-2567) Zero Prefetch not working
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully resolved AMQ-2567.
-----------------------------
Resolution: Fixed
Fix Version/s: 5.4.0
5.3.1
Thanks for the test case, debugging and fix.
Fix with small modification applied to trunk in r901269 and 5.3.1 in r901271
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Fix For: 5.3.1, 5.4.0
>
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2567) Zero Prefetch not working
Posted by "Rudolf Janz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rudolf Janz updated AMQ-2567:
-----------------------------
Attachment: PrefetchSubscription.java
ZeroPrefetchConsumerTest.java
Here is the extended TestCase and the Subscribtion class including the printouts
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=57207#action_57207 ]
Gary Tully commented on AMQ-2567:
---------------------------------
I kicked hudson into doing a redeploy of the artifacts and the full install tar files are now uploaded containing the build artifacts from the 27/01 which should have the fix for this issue.
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Fix For: 5.3.1, 5.4.0
>
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Christoph Kutzinski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=57176#action_57176 ]
Christoph Kutzinski commented on AMQ-2567:
------------------------------------------
Was the fix already released as part of a snapshot release available at https://repository.apache.org/content/repositories/snapshots/org/apache/activemq/ ?
Judging by the JAR file dates I would say no.
If not, when can I expect a snapshot release which includes the fix?
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Fix For: 5.3.1, 5.4.0
>
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Rudolf Janz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56925#action_56925 ]
Rudolf Janz commented on AMQ-2567:
----------------------------------
So I added a printout in the client (Expecting/Read) and in the PrefetchSubscribtion for all Acks,
C1/2 are the consumer n is the number of acked messaged. This is how it happens
Expecting Msg1
Read Msg1
C1: n:1 isDeliveredAck(Before): 1
C1: n:1 isDeliveredAck(After): 1
C1: n:1 isStandardAckXA(Before): 1
C1: n:1 isStandardAckXA(After): 1
Expecting Msg2
C1: n:1 isDeliveredAck(Before): *2* This is wrong it should be in the same state as before the message 1, should have been
C1: n:1 isDeliveredAck(After): 2
Read Msg2
C1: n:1 isStandardAckXA(Before): 2
C1: n:1 isStandardAckXA(After): 2
Expecting Msg3
C1: n:1 isDeliveredAck(Before): 2
C1: n:1 isDeliveredAck(After): 2
Read Msg3
C1: n:1 isStandardAckXA(Before): 2
C1: n:1 isStandardAckXA(After): 2
Expecting Msg4
Read Msg4
C1: n:1 isDeliveredAck(Before): 3
C1: n:1 isDeliveredAck(After): 3
C1: n:1 isStandardAckXA(Before): 3
C1: n:1 isStandardAckXA(After): 3
Expecting Msg5
Read Msg8
C2: n:1 isDeliveredAck(Before): 1
C2: n:1 isDeliveredAck(After): 1
C2: n:1 isStandardAckXA(Before): 1
C2: n:1 isStandardAckXA(After): 1
I think the problem is in the standardAck, this should decrement prefetchIndex. But I guess I am forgetting something (I extended receiving two messages in the transaction in the test case)
{code}
if (ack.isStandardAck()) {
...
if (context.isInTransaction()) {
// extend prefetch window only if not a pulling
// consumer
System.out.println(prefix+"isStandardAckXA(Before): " +prefetchExtension);
//RJ if (getPrefetchSize() != 0) {
//RJ prefetchExtension = Math.max(prefetchExtension, index );
//RJ }
//RJ do the same as in the non transacted case
prefetchExtension = Math.max(0, prefetchExtension - index);
System.out.println(prefix+"isStandardAckXA(After): " +prefetchExtension);
} else {
System.out.println(prefix+"isStandardAck(Before): " +prefetchExtension);
prefetchExtension = Math.max(0,
}}
prefetchExtension - index);
System.out.println(prefix+"isStandardAck(After): " +prefetchExtension);
}
{code}
Expecting Msg1
Read Msg1
C1: n:1 isDeliveredAck(Before): 1
C1: n:1 isDeliveredAck(After): 1
C1: n:1 isStandardAckXA(Before): 1
C1: n:1 isStandardAckXA(After): 0
Expecting Msg2
Read Msg2
C1: n:1 isDeliveredAck(Before): 1
C1: n:1 isDeliveredAck(After): 1
C1: n:1 isStandardAckXA(Before): 1
C1: n:1 isStandardAckXA(After): 0
Expecting Msg3
Read Msg3
C1: n:1 isDeliveredAck(Before): 1
C1: n:1 isDeliveredAck(After): 1
C1: n:1 isStandardAckXA(Before): 1
C1: n:1 isStandardAckXA(After): 0
Expecting Msg4
C1: n:1 isDeliveredAck(Before): 1
C1: n:1 isDeliveredAck(After): 1
Read Msg4
C1: n:1 isStandardAckXA(Before): 1
C1: n:1 isStandardAckXA(After): 0
Expecting Msg5
C2: n:1 isDeliveredAck(Before): 1
C2: n:1 isDeliveredAck(After): 1
Read Msg5
C2: n:1 isStandardAckXA(Before): 1
C2: n:1 isStandardAckXA(After): 0
Expecting Msg6
Read Msg6
Expecting Msg7
C1: n:1 isDeliveredAck(Before): 1
C1: n:1 isDeliveredAck(After): 1
Read Msg7
C1: n:1 isDeliveredAck(Before): 2
C1: n:1 isDeliveredAck(After): 2
C1: n:2 isStandardAckXA(Before): 2
C1: n:2 isStandardAckXA(After): 0
Expecting Msg8
Read Msg8
C2: n:1 isDeliveredAck(Before): 1
C2: n:1 isDeliveredAck(After): 1
C2: n:1 isStandardAckXA(Before): 1
C2: n:1 isStandardAckXA(After): 0
THis look ok to me now
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=57209#action_57209 ]
Gary Tully commented on AMQ-2567:
---------------------------------
done for 5.3.1-SNAPSHOT
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Assignee: Gary Tully
> Fix For: 5.3.1, 5.4.0
>
> Attachments: PrefetchSubscription.java, ZeroPrefetchConsumerTest.java, ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2567) Zero Prefetch not working
Posted by "Rudolf Janz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rudolf Janz updated AMQ-2567:
-----------------------------
Attachment: ZeroPrefetchConsumerTest.java
Extended the test case to send more than one messages and listen and receiving on two consumers.
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Attachments: ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2567) Zero Prefetch not working
Posted by "Rudolf Janz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=56910#action_56910 ]
Rudolf Janz commented on AMQ-2567:
----------------------------------
Ok, I unterstand now, what it is for.
Then I guess prefetchExtension should be reset to 0 again after the transaction? But it kept increasing.
I will have a look at the Ack logic tomorrow.
> Zero Prefetch not working
> -------------------------
>
> Key: AMQ-2567
> URL: https://issues.apache.org/activemq/browse/AMQ-2567
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: Unix/Windows, Java 1.6
> Reporter: Rudolf Janz
> Attachments: ZeroPrefetchConsumerTest.java
>
>
> I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
>
> Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
>
> I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
>
> Two consumers (C1 and C2) are instantiated. These messages are sent:
> 1,2,3,4,5,6,7,8,9
>
> C1 reads 4 times, receives 1,2,3,4 -> this is correct
>
> now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
>
> The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
>
> The last version that we used was 4.1.1, that worked.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.