You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "James Furness (Created) (JIRA)" <ji...@apache.org> on 2012/01/23 00:08:40 UTC

[jira] [Created] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
------------------------------------------------------------------------------------------------------------------------

                 Key: AMQ-3676
                 URL: https://issues.apache.org/jira/browse/AMQ-3676
             Project: ActiveMQ
          Issue Type: Bug
    Affects Versions: 5.5.1
         Environment: ActiveMQ: 5.5.1-fuse-01-13
JVM: 1.6.0_26-b03-383-11A511
            Reporter: James Furness


Hi,

Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.

Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.

We therefore have the following configuration:
 - Low prefetch size (100)
 - Low ConstantPendingMessageLimitStrategy (100)
 - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)

Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.

Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 

>From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.

Have created a test case to reproduce this (attached):

{code:title=testWithoutOptimiseAcknowledge}
Publisher: Send 0
FastConsumer: Receive 0
Publisher: Send 50
FastConsumer: Receive 50
Publisher: Send 100
FastConsumer: Receive 100
Publisher: Send 150
FastConsumer: Receive 150
testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
{code}

{code:title=testWithOptimiseAcknowledge}
Publisher: Send 0
FastConsumer: Receive 0
Publisher: Send 50
Publisher: Send 100
Publisher: Send 150
testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

java.lang.AssertionError: 
Expected :200
Actual   :50
{code}

Cheers,
James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "James Furness (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191550#comment-13191550 ] 

James Furness commented on AMQ-3676:
------------------------------------

Ahh great, so there was something missing from my configuration - thanks for clarifying that. Wasn't aware that the ConnectionFactory prefetch size on the consumer also needs to be set.

Would it be useful/possible to detect this situation (PolicyEntry prefetch < ConnectionFactory prefetch) and alert? In the normal situation of individual acks I guess the mismatch wouldn't have a significant effect, but for OptimiseAcknowledge it obviously causes the consumers to grind to a halt.
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "Timothy Bish (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191447#comment-13191447 ] 

Timothy Bish commented on AMQ-3676:
-----------------------------------

You can see the test pass if you change the buildSession method to the following, and pass in the desired limit:

{noformat}
    private static Session buildSession(String clientId, String url, boolean optimizeAcknowledge, int prefetchLimit) throws JMSException {
        ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);

        connectionFactory.setCopyMessageOnSend(false);
        connectionFactory.setOptimizeAcknowledge(optimizeAcknowledge);
        if (optimizeAcknowledge) {
            connectionFactory.setOptimizeAcknowledgeTimeOut(0);
            connectionFactory.getPrefetchPolicy().setAll(prefetchLimit);
        }

        Connection connection = connectionFactory.createConnection();
        connection.setClientID(clientId);

        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

        connection.start();

        return session;
    }
{noformat}
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "James Furness (Updated) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Furness updated AMQ-3676:
-------------------------------

    Attachment: ActiveMQOptimiseAcknowledgeManualTest.java
    
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "James Furness (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191189#comment-13191189 ] 

James Furness commented on AMQ-3676:
------------------------------------

Yes that does work on 5.6-SNAPSHOT, however I presume this is an effect of Gary's patch from AMQ-3607 causing {{setOptimizeAcknowledgeTimeOut}} to have an effect (previously it wasn't wired through).

If I change to {{setOptimizeAcknowledgeTimeOut(0)}}, simulating the effect of things without a timer the tests fail again. Does this not mean that the acks are not being sent at 65% of the prefetch limit as expected and the timer is necessary to clean them up? 

This would mean the throughput would be limited to {{prefetch size * OptimizeAcknowledgeTimeOut}}, which probably isn't ideal?
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "Timothy Bish (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191158#comment-13191158 ] 

Timothy Bish commented on AMQ-3676:
-----------------------------------

When run against trunk the attached test cases passes without error.  Recommend you try a SNAPSHOT build of 5.6 and see if you issue is resolved.

                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "Timothy Bish (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191442#comment-13191442 ] 

Timothy Bish commented on AMQ-3676:
-----------------------------------

Your test case doesn't set the prefetch limit on the MessageConsumer so the consumer never reaches the 65% of prefetch limit since the default is 32767 so the messages aren't ack'd in your test.
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "James Furness (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191567#comment-13191567 ] 

James Furness commented on AMQ-3676:
------------------------------------

Alright, will chalk that one up to experience then - thanks for your help.
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "Timothy Bish (Commented) (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13191554#comment-13191554 ] 

Timothy Bish commented on AMQ-3676:
-----------------------------------

The client doesn't really know anything about the broker configuration so detecting this isn't trivial.  
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Closed] (AMQ-3676) Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged

Posted by "Timothy Bish (Closed) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/AMQ-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timothy Bish closed AMQ-3676.
-----------------------------

    Resolution: Not A Problem

Working as designed, test case lacked the settings for consumer prefetch and the cyclic barrier wait in the onMessage then caused the test to fail.
                
> Setting OptimizeAcknowledge on a consumer with a small prefetch size appears to prevent messages from being acknowledged
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-3676
>                 URL: https://issues.apache.org/jira/browse/AMQ-3676
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.5.1
>         Environment: ActiveMQ: 5.5.1-fuse-01-13
> JVM: 1.6.0_26-b03-383-11A511
>            Reporter: James Furness
>         Attachments: ActiveMQOptimiseAcknowledgeManualTest.java
>
>
> Hi,
> Am seeing some strange behaviour with OptimizeAcknowledge, may well be something i've missed in terms of the configuration but would be good to get it clarified.
> Our use case is a topic with very fast flowing data, we want all consumers to get messages with a minimum of latency and would prefer that messages get conflated rather than building up a large queue and hence delays.
> We therefore have the following configuration:
>  - Low prefetch size (100)
>  - Low ConstantPendingMessageLimitStrategy (100)
>  - Conflation when the slow consumer handling kicks in (OldestMessageEvictionStrategy)
> Would like to optimise this further, following Gary's tips in AMQ-3607 I decided to re-enable OptimizeAcknowledge in the hopes that this would further improve latency.
> Following that was seeing that consumers receive a burst of messages, then become a slow consumer (i.e. prefetch is full) and fail to receive any further messages. Had an AbortSlowConsumerStrategy in place which was initially masking the problem as the reconnect caused messages to flow again. However removing this causes the consumers to receive a burst of messages then the flow stops indefinitely (whilst messages get discarded on the server as evidenced by the PendingQueueSize in JMX), so looks like nothing is getting acked. 
> From the discussion on AMQ-3607 I understand that by default once 65% of the prefetch is full (or after a configurable period of time once Gary's patch from AMQ-3607 is available), a batch of acks should be sent.
> Have created a test case to reproduce this (attached):
> {code:title=testWithoutOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> FastConsumer: Receive 50
> Publisher: Send 100
> FastConsumer: Receive 100
> Publisher: Send 150
> FastConsumer: Receive 150
> testWithoutOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithoutOptimiseAcknowledge: Received: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> {code}
> {code:title=testWithOptimiseAcknowledge}
> Publisher: Send 0
> FastConsumer: Receive 0
> Publisher: Send 50
> Publisher: Send 100
> Publisher: Send 150
> testWithOptimiseAcknowledge: Sent: 200 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199]
> testWithOptimiseAcknowledge: Received: 50 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> java.lang.AssertionError: 
> Expected :200
> Actual   :50
> {code}
> Cheers,
> James

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira