You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "michael j. goulish (JIRA)" <qp...@incubator.apache.org> on 2008/11/26 21:22:44 UTC
[jira] Created: (QPID-1488) QueuePolicy serialization fix for
cluster braindump.
QueuePolicy serialization fix for cluster braindump.
----------------------------------------------------
Key: QPID-1488
URL: https://issues.apache.org/jira/browse/QPID-1488
Project: Qpid
Issue Type: Improvement
Components: C++ Broker
Environment: tested on F9
Reporter: michael j. goulish
Priority: Blocker
In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (QPID-1488) QueuePolicy serialization fix for
cluster braindump.
Posted by "michael j. goulish (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
michael j. goulish updated QPID-1488:
-------------------------------------
Attachment: queue_policy_serialization_bug.diff
> QueuePolicy serialization fix for cluster braindump.
> ----------------------------------------------------
>
> Key: QPID-1488
> URL: https://issues.apache.org/jira/browse/QPID-1488
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Environment: tested on F9
> Reporter: michael j. goulish
> Priority: Blocker
> Attachments: queue_policy_serialization_bug.diff
>
>
> In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
> After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
> This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (QPID-1488) QueuePolicy serialization fix for
cluster braindump.
Posted by "Alan Conway (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Conway resolved QPID-1488.
-------------------------------
Resolution: Fixed
Applied in r720979.
> QueuePolicy serialization fix for cluster braindump.
> ----------------------------------------------------
>
> Key: QPID-1488
> URL: https://issues.apache.org/jira/browse/QPID-1488
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Environment: tested on F9
> Reporter: michael j. goulish
> Priority: Blocker
> Attachments: queue_policy_serialization_bug.diff
>
>
> In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
> After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
> The result: what(): framing-error: Unexpected command start frame. (qpid/SessionState.cpp:57)
> This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (QPID-1488) QueuePolicy serialization fix for
cluster braindump.
Posted by "Gordon Sim (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713520#action_12713520 ]
Gordon Sim commented on QPID-1488:
----------------------------------
To see problem (a) described in my comment above:
1. start first cluster node
2. create queue with specific policy (e.g. qpid-config add
queue test-queue --max-queue-count 2 --limit-policy reject)
3. send a message to the queue (e.g. echo one | ./qpid/cpp/src/tests/sender)
4. start second node for cluster
5. send second message (e.g. echo two | ./qpid/cpp/src/tests/sender)
At this point the new member fails to add second message and exits with:
2009-may-27 05:18:50 error Execution exception: resource-limit-exceeded: Policy
exceeded on test-queue by message 2 of size 3 , policy: size: max=104857600,
current=6; count: max=2, current=2; type=reject
(qpid/broker/QueuePolicy.cpp:90)
2009-may-27 05:18:50 critical 100.0.50.15:1404(READY/error) Error 236 outcome
agrees with 100.0.50.15:1404
2009-may-27 05:18:50 critical 100.0.50.15:1404(READY/error) Error 236 did not
occur on 100.0.50.15:1350
2009-may-27 05:18:50 error Error delivering frames: Aborted by local failure
that did not occur on all replicas
> QueuePolicy serialization fix for cluster braindump.
> ----------------------------------------------------
>
> Key: QPID-1488
> URL: https://issues.apache.org/jira/browse/QPID-1488
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Environment: tested on F9
> Reporter: michael j. goulish
> Assignee: Alan Conway
> Priority: Blocker
> Attachments: queue_policy_serialization_bug.diff
>
>
> In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
> After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
> The result: what(): framing-error: Unexpected command start frame. (qpid/SessionState.cpp:57)
> This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Resolved: (QPID-1488) QueuePolicy serialization fix for
cluster braindump.
Posted by "Gordon Sim (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gordon Sim resolved QPID-1488.
------------------------------
Resolution: Fixed
Fix Version/s: 0.6
Fixed by r779183.
> QueuePolicy serialization fix for cluster braindump.
> ----------------------------------------------------
>
> Key: QPID-1488
> URL: https://issues.apache.org/jira/browse/QPID-1488
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Environment: tested on F9
> Reporter: michael j. goulish
> Assignee: Alan Conway
> Priority: Blocker
> Fix For: 0.6
>
> Attachments: queue_policy_serialization_bug.diff
>
>
> In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
> After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
> The result: what(): framing-error: Unexpected command start frame. (qpid/SessionState.cpp:57)
> This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-1488) QueuePolicy serialization fix for
cluster braindump.
Posted by "michael j. goulish (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
michael j. goulish updated QPID-1488:
-------------------------------------
Description:
In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
The result: what(): framing-error: Unexpected command start frame. (qpid/SessionState.cpp:57)
This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
was:
In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
> QueuePolicy serialization fix for cluster braindump.
> ----------------------------------------------------
>
> Key: QPID-1488
> URL: https://issues.apache.org/jira/browse/QPID-1488
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Environment: tested on F9
> Reporter: michael j. goulish
> Priority: Blocker
> Attachments: queue_policy_serialization_bug.diff
>
>
> In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
> After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
> The result: what(): framing-error: Unexpected command start frame. (qpid/SessionState.cpp:57)
> This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (QPID-1488) QueuePolicy serialization fix for
cluster braindump.
Posted by "Gordon Sim (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gordon Sim reopened QPID-1488:
------------------------------
Assignee: Alan Conway
I believe this fix is wrong. The messages on a queue are replicated to the joining member and as part of that the policy will be updated. The only part missing is those messages in the unacked state as these are transferred separately, are not pushed onto the delivery queue and thus not reflected in the computed policy count.
As it stands I believe (a) the policy count may well be incorrect on the new replica as enqueued messages will be counted twice, (b) more seriously, the root issue opens the potential for unacked durable messages to be dequeued without being enqueued first on a new replica.
> QueuePolicy serialization fix for cluster braindump.
> ----------------------------------------------------
>
> Key: QPID-1488
> URL: https://issues.apache.org/jira/browse/QPID-1488
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Environment: tested on F9
> Reporter: michael j. goulish
> Assignee: Alan Conway
> Priority: Blocker
> Attachments: queue_policy_serialization_bug.diff
>
>
> In cluster braindump (when a new member is being added to a cluster) the QueuePolicy is not being serialized out as part of the brain dump. As a result the newbie cluster can get a mistaken idea of the queue size (in bytes).
> After many dequeues, the size can go negative, but since the queue size is an unsigned number it wraps around and look like a large positive. Which sets off the flow-to-disk code, because it thinks that the queue has gotten too large.
> The result: what(): framing-error: Unexpected command start frame. (qpid/SessionState.cpp:57)
> This fix just adds a little serialization of the QueuePolicy on to the end of the serialization of the Queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.