You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Alan Conway (JIRA)" <ji...@apache.org> on 2011/02/21 16:07:38 UTC
[jira] Created: (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Enable producer flow control (QPID-2935) in a cluster
-----------------------------------------------------
Key: QPID-3076
URL: https://issues.apache.org/jira/browse/QPID-3076
Project: Qpid
Issue Type: New Feature
Components: C++ Clustering
Affects Versions: 0.9
Reporter: Alan Conway
Assignee: Alan Conway
Fix For: 0.9
QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Updated] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "Ken Giusti (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ken Giusti updated QPID-3076:
-----------------------------
Attachment: QPID-3076-2.patch
patch against trunk.
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025321#comment-13025321 ]
jiraposter@reviews.apache.org commented on QPID-3076:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/512/
-----------------------------------------------------------
(Updated 2011-04-26 17:04:03.647250)
Review request for Alan Conway and Gordon Sim.
Changes
-------
Folks, this has been languishing on my disk for awhile - I'd like to get it on trunk. This patch is complete and ready to go (passes unit and store tests).
Gordon - note well the changes it makes to the queue observer api: we need some way of replicating queue observer state across the members of a cluster. I thought this approach was generic, though not much of a looker.
Let me know what you think, thanks.
Summary
-------
Hi Alan,
Here's a patch that should give you an idea of the direction I'm going in - could be completely wrong here so I don't want to go too far without checking with you.
bq. From your notes, I've modified a bit those items I *think* we need to replicate based on the changes I've made to increase performance (some items have been removed). Here's what I think we need to cover:
Replicate:
For each Queue:
If Queue has a QueueFlowLimit configured:
QueueFlowLimit::flowStoppedCount
QueueFlowLimit::index set of msg sequence numbers that are pending.
For each msg in the QueueFlowLimit::index set:
Allocate a SessionState::IncompleteIngressMsgXfer context
Attach a SessionState::IncompleteIngressMsgXfer context to messsage's AsyncCompletion callback
Force AsyncCompletion::completion count to 1 (??? or more ???)
For each SessionState:
Replicate the vector of pendingExecutionSyncs SequenceNumbers
Add an AsyncCommandCompleter context
Replicate the vector of completedMsgs
If vector of completedMsgs > 0:
Scheduled requestIOProcessing for callback ????
Let me know what you think, thanks.
This addresses bug QPID-3076.
https://issues.apache.org/jira/browse/QPID-3076
Diffs (updated)
-----
/trunk/qpid/cpp/docs/man/qpidd.1 1096739
/trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1096739
/trunk/qpid/cpp/src/qpid/broker/Queue.h 1096739
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1096739
/trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.h 1096739
/trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp 1096739
/trunk/qpid/cpp/src/qpid/broker/QueueObserver.h 1096739
/trunk/qpid/cpp/src/qpid/cluster/Cluster.cpp 1096739
/trunk/qpid/cpp/src/qpid/cluster/Connection.h 1096739
/trunk/qpid/cpp/src/qpid/cluster/Connection.cpp 1096739
/trunk/qpid/cpp/src/qpid/cluster/UpdateClient.h 1096739
/trunk/qpid/cpp/src/qpid/cluster/UpdateClient.cpp 1096739
/trunk/qpid/cpp/src/tests/QueueFlowLimitTest.cpp 1096739
/trunk/qpid/cpp/src/tests/cluster_tests.py 1096739
/trunk/qpid/cpp/src/tests/queue_flow_limit_tests.py 1096739
/trunk/qpid/cpp/xml/cluster.xml 1096739
Diff: https://reviews.apache.org/r/512/diff
Testing
-------
Barely compiles.
Thanks,
Kenneth
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025814#comment-13025814 ]
jiraposter@reviews.apache.org commented on QPID-3076:
-----------------------------------------------------
bq. On 2011-04-27 09:51:57, Gordon Sim wrote:
bq. > /trunk/qpid/cpp/docs/man/qpidd.1, line 44
bq. > <https://reviews.apache.org/r/512/diff/4/?file=17336#file17336line44>
bq. >
bq. > This is probably unintnetional? I suspect it may just be a result of the build process changing since you created the diff?
bq.
bq. Alan Conway wrote:
bq. You probably need to re-bootstrap and configure to get all the makefiles update.
Totally unintentional - apparently "make" regenerated it. I updated to the latest trunk, re-ran ./bootstrap && ./configure, and I still end up getting a modified qpidd.1.
Definitely not part of the patch - I will remove it.
bq. On 2011-04-27 09:51:57, Gordon Sim wrote:
bq. > /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp, line 165
bq. > <https://reviews.apache.org/r/512/diff/4/?file=17341#file17341line165>
bq. >
bq. > Just curious about removing this check... was it required? Is there ever a case when the payload is not set? If so are we sure it is safe?
I honestly don't know where that check came from. I removed it because all the other implementations derived from QueueObserver do not check payload, neither does the Queue code itself. I'd rather leave it out to be consistent with the other QueueObserver impls, and avoid a needless check in the fastpath.
bq. On 2011-04-27 09:51:57, Gordon Sim wrote:
bq. > /trunk/qpid/cpp/src/qpid/broker/QueueObserver.h, line 59
bq. > <https://reviews.apache.org/r/512/diff/4/?file=17342#file17342line59>
bq. >
bq. > I'm not hugely keen on these changes to QueueObserver. I fully anticipate that the interface will need to be adapted and expanded. However I'd like - if possible - to keep cluster specific concerns out of it.
bq. >
bq. > That is a big 'if' however. One suggestion would be to create a sub-class of QueueObserver - e.g. StatefulQueueObserver - that included these changes. You could then create an instance of that if desired/required. The UpdateClient could also then cast to that interface to determine if there was relevant state.
bq. >
bq. > I'd like at some point to move to a queue factory and have a specific specialisation of that for clustered brokers that could encapsulate any cluster specific shenanigans.
Agreed - those new methods are only for cluster support, which makes them specialized in a manner that won't be applicable to all QueueObservers. I like your suggestion of a sub-class specifically for cluster support. How about I refactor this patch to take that approach?
- Kenneth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/512/#review573
-----------------------------------------------------------
On 2011-04-26 17:04:03, Kenneth Giusti wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/512/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-26 17:04:03)
bq.
bq.
bq. Review request for Alan Conway and Gordon Sim.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Hi Alan,
bq.
bq. Here's a patch that should give you an idea of the direction I'm going in - could be completely wrong here so I don't want to go too far without checking with you.
bq.
bq. From your notes, I've modified a bit those items I *think* we need to replicate based on the changes I've made to increase performance (some items have been removed). Here's what I think we need to cover:
bq.
bq. Replicate:
bq.
bq. For each Queue:
bq. If Queue has a QueueFlowLimit configured:
bq. QueueFlowLimit::flowStoppedCount
bq. QueueFlowLimit::index set of msg sequence numbers that are pending.
bq.
bq. For each msg in the QueueFlowLimit::index set:
bq. Allocate a SessionState::IncompleteIngressMsgXfer context
bq. Attach a SessionState::IncompleteIngressMsgXfer context to messsage's AsyncCompletion callback
bq. Force AsyncCompletion::completion count to 1 (??? or more ???)
bq.
bq. For each SessionState:
bq. Replicate the vector of pendingExecutionSyncs SequenceNumbers
bq. Add an AsyncCommandCompleter context
bq. Replicate the vector of completedMsgs
bq. If vector of completedMsgs > 0:
bq. Scheduled requestIOProcessing for callback ????
bq.
bq. Let me know what you think, thanks.
bq.
bq.
bq. This addresses bug QPID-3076.
bq. https://issues.apache.org/jira/browse/QPID-3076
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /trunk/qpid/cpp/docs/man/qpidd.1 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueObserver.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Cluster.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/QueueFlowLimitTest.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/cluster_tests.py 1096739
bq. /trunk/qpid/cpp/src/tests/queue_flow_limit_tests.py 1096739
bq. /trunk/qpid/cpp/xml/cluster.xml 1096739
bq.
bq. Diff: https://reviews.apache.org/r/512/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Barely compiles.
bq.
bq.
bq. Thanks,
bq.
bq. Kenneth
bq.
bq.
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "Ken Giusti (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010716#comment-13010716 ]
Ken Giusti commented on QPID-3076:
----------------------------------
Proposed patch:
https://reviews.apache.org/r/512/
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Resolved] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "Ken Giusti (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ken Giusti resolved QPID-3076.
------------------------------
Resolution: Fixed
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025970#comment-13025970 ]
jiraposter@reviews.apache.org commented on QPID-3076:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/512/
-----------------------------------------------------------
(Updated 2011-04-27 19:39:49.097658)
Review request for Alan Conway and Gordon Sim.
Changes
-------
This backs out the QueueObserver changes and creates a new Derived object StatefulQueueObserver, which is used by the cluster code for replication. I like this better.
What say ye, Gordon?
Summary
-------
Hi Alan,
Here's a patch that should give you an idea of the direction I'm going in - could be completely wrong here so I don't want to go too far without checking with you.
bq. From your notes, I've modified a bit those items I *think* we need to replicate based on the changes I've made to increase performance (some items have been removed). Here's what I think we need to cover:
Replicate:
For each Queue:
If Queue has a QueueFlowLimit configured:
QueueFlowLimit::flowStoppedCount
QueueFlowLimit::index set of msg sequence numbers that are pending.
For each msg in the QueueFlowLimit::index set:
Allocate a SessionState::IncompleteIngressMsgXfer context
Attach a SessionState::IncompleteIngressMsgXfer context to messsage's AsyncCompletion callback
Force AsyncCompletion::completion count to 1 (??? or more ???)
For each SessionState:
Replicate the vector of pendingExecutionSyncs SequenceNumbers
Add an AsyncCommandCompleter context
Replicate the vector of completedMsgs
If vector of completedMsgs > 0:
Scheduled requestIOProcessing for callback ????
Let me know what you think, thanks.
This addresses bug QPID-3076.
https://issues.apache.org/jira/browse/QPID-3076
Diffs (updated)
-----
/trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1097191
/trunk/qpid/cpp/src/qpid/broker/Queue.h 1097191
/trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.h 1097191
/trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp 1097191
/trunk/qpid/cpp/src/qpid/broker/StatefulQueueObserver.h PRE-CREATION
/trunk/qpid/cpp/src/qpid/cluster/Cluster.cpp 1097191
/trunk/qpid/cpp/src/qpid/cluster/Connection.h 1097191
/trunk/qpid/cpp/src/qpid/cluster/Connection.cpp 1097191
/trunk/qpid/cpp/src/qpid/cluster/UpdateClient.h 1097191
/trunk/qpid/cpp/src/qpid/cluster/UpdateClient.cpp 1097191
/trunk/qpid/cpp/src/tests/QueueFlowLimitTest.cpp 1097191
/trunk/qpid/cpp/src/tests/cluster_tests.py 1097191
/trunk/qpid/cpp/src/tests/queue_flow_limit_tests.py 1097191
/trunk/qpid/cpp/xml/cluster.xml 1097191
Diff: https://reviews.apache.org/r/512/diff
Testing
-------
Barely compiles.
Thanks,
Kenneth
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025803#comment-13025803 ]
jiraposter@reviews.apache.org commented on QPID-3076:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/512/#review575
-----------------------------------------------------------
Ship it!
- Alan
On 2011-04-26 17:04:03, Kenneth Giusti wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/512/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-26 17:04:03)
bq.
bq.
bq. Review request for Alan Conway and Gordon Sim.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Hi Alan,
bq.
bq. Here's a patch that should give you an idea of the direction I'm going in - could be completely wrong here so I don't want to go too far without checking with you.
bq.
bq. From your notes, I've modified a bit those items I *think* we need to replicate based on the changes I've made to increase performance (some items have been removed). Here's what I think we need to cover:
bq.
bq. Replicate:
bq.
bq. For each Queue:
bq. If Queue has a QueueFlowLimit configured:
bq. QueueFlowLimit::flowStoppedCount
bq. QueueFlowLimit::index set of msg sequence numbers that are pending.
bq.
bq. For each msg in the QueueFlowLimit::index set:
bq. Allocate a SessionState::IncompleteIngressMsgXfer context
bq. Attach a SessionState::IncompleteIngressMsgXfer context to messsage's AsyncCompletion callback
bq. Force AsyncCompletion::completion count to 1 (??? or more ???)
bq.
bq. For each SessionState:
bq. Replicate the vector of pendingExecutionSyncs SequenceNumbers
bq. Add an AsyncCommandCompleter context
bq. Replicate the vector of completedMsgs
bq. If vector of completedMsgs > 0:
bq. Scheduled requestIOProcessing for callback ????
bq.
bq. Let me know what you think, thanks.
bq.
bq.
bq. This addresses bug QPID-3076.
bq. https://issues.apache.org/jira/browse/QPID-3076
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /trunk/qpid/cpp/docs/man/qpidd.1 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueObserver.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Cluster.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/QueueFlowLimitTest.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/cluster_tests.py 1096739
bq. /trunk/qpid/cpp/src/tests/queue_flow_limit_tests.py 1096739
bq. /trunk/qpid/cpp/xml/cluster.xml 1096739
bq.
bq. Diff: https://reviews.apache.org/r/512/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Barely compiles.
bq.
bq.
bq. Thanks,
bq.
bq. Kenneth
bq.
bq.
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Assigned: (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "Ken Giusti (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ken Giusti reassigned QPID-3076:
--------------------------------
Assignee: Ken Giusti (was: Alan Conway)
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.9
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026199#comment-13026199 ]
jiraposter@reviews.apache.org commented on QPID-3076:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/512/#review599
-----------------------------------------------------------
Ship it!
- Gordon
On 2011-04-27 19:39:49, Kenneth Giusti wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/512/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-27 19:39:49)
bq.
bq.
bq. Review request for Alan Conway and Gordon Sim.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Hi Alan,
bq.
bq. Here's a patch that should give you an idea of the direction I'm going in - could be completely wrong here so I don't want to go too far without checking with you.
bq.
bq. From your notes, I've modified a bit those items I *think* we need to replicate based on the changes I've made to increase performance (some items have been removed). Here's what I think we need to cover:
bq.
bq. Replicate:
bq.
bq. For each Queue:
bq. If Queue has a QueueFlowLimit configured:
bq. QueueFlowLimit::flowStoppedCount
bq. QueueFlowLimit::index set of msg sequence numbers that are pending.
bq.
bq. For each msg in the QueueFlowLimit::index set:
bq. Allocate a SessionState::IncompleteIngressMsgXfer context
bq. Attach a SessionState::IncompleteIngressMsgXfer context to messsage's AsyncCompletion callback
bq. Force AsyncCompletion::completion count to 1 (??? or more ???)
bq.
bq. For each SessionState:
bq. Replicate the vector of pendingExecutionSyncs SequenceNumbers
bq. Add an AsyncCommandCompleter context
bq. Replicate the vector of completedMsgs
bq. If vector of completedMsgs > 0:
bq. Scheduled requestIOProcessing for callback ????
bq.
bq. Let me know what you think, thanks.
bq.
bq.
bq. This addresses bug QPID-3076.
bq. https://issues.apache.org/jira/browse/QPID-3076
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1097191
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.h 1097191
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.h 1097191
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp 1097191
bq. /trunk/qpid/cpp/src/qpid/broker/StatefulQueueObserver.h PRE-CREATION
bq. /trunk/qpid/cpp/src/qpid/cluster/Cluster.cpp 1097191
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.h 1097191
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.cpp 1097191
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.h 1097191
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.cpp 1097191
bq. /trunk/qpid/cpp/src/tests/QueueFlowLimitTest.cpp 1097191
bq. /trunk/qpid/cpp/src/tests/cluster_tests.py 1097191
bq. /trunk/qpid/cpp/src/tests/queue_flow_limit_tests.py 1097191
bq. /trunk/qpid/cpp/xml/cluster.xml 1097191
bq.
bq. Diff: https://reviews.apache.org/r/512/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Barely compiles.
bq.
bq.
bq. Thanks,
bq.
bq. Kenneth
bq.
bq.
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "Ken Giusti (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ken Giusti updated QPID-3076:
-----------------------------
Affects Version/s: 0.10
Fix Version/s: (was: 0.9)
0.11
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025786#comment-13025786 ]
jiraposter@reviews.apache.org commented on QPID-3076:
-----------------------------------------------------
bq. On 2011-04-27 09:51:57, Gordon Sim wrote:
bq. > /trunk/qpid/cpp/docs/man/qpidd.1, line 44
bq. > <https://reviews.apache.org/r/512/diff/4/?file=17336#file17336line44>
bq. >
bq. > This is probably unintnetional? I suspect it may just be a result of the build process changing since you created the diff?
You probably need to re-bootstrap and configure to get all the makefiles update.
- Alan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/512/#review573
-----------------------------------------------------------
On 2011-04-26 17:04:03, Kenneth Giusti wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/512/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-26 17:04:03)
bq.
bq.
bq. Review request for Alan Conway and Gordon Sim.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Hi Alan,
bq.
bq. Here's a patch that should give you an idea of the direction I'm going in - could be completely wrong here so I don't want to go too far without checking with you.
bq.
bq. From your notes, I've modified a bit those items I *think* we need to replicate based on the changes I've made to increase performance (some items have been removed). Here's what I think we need to cover:
bq.
bq. Replicate:
bq.
bq. For each Queue:
bq. If Queue has a QueueFlowLimit configured:
bq. QueueFlowLimit::flowStoppedCount
bq. QueueFlowLimit::index set of msg sequence numbers that are pending.
bq.
bq. For each msg in the QueueFlowLimit::index set:
bq. Allocate a SessionState::IncompleteIngressMsgXfer context
bq. Attach a SessionState::IncompleteIngressMsgXfer context to messsage's AsyncCompletion callback
bq. Force AsyncCompletion::completion count to 1 (??? or more ???)
bq.
bq. For each SessionState:
bq. Replicate the vector of pendingExecutionSyncs SequenceNumbers
bq. Add an AsyncCommandCompleter context
bq. Replicate the vector of completedMsgs
bq. If vector of completedMsgs > 0:
bq. Scheduled requestIOProcessing for callback ????
bq.
bq. Let me know what you think, thanks.
bq.
bq.
bq. This addresses bug QPID-3076.
bq. https://issues.apache.org/jira/browse/QPID-3076
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /trunk/qpid/cpp/docs/man/qpidd.1 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueObserver.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Cluster.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/QueueFlowLimitTest.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/cluster_tests.py 1096739
bq. /trunk/qpid/cpp/src/tests/queue_flow_limit_tests.py 1096739
bq. /trunk/qpid/cpp/xml/cluster.xml 1096739
bq.
bq. Diff: https://reviews.apache.org/r/512/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Barely compiles.
bq.
bq.
bq. Thanks,
bq.
bq. Kenneth
bq.
bq.
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Commented] (QPID-3076) Enable producer flow control
(QPID-2935) in a cluster
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025701#comment-13025701 ]
jiraposter@reviews.apache.org commented on QPID-3076:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/512/#review573
-----------------------------------------------------------
/trunk/qpid/cpp/docs/man/qpidd.1
<https://reviews.apache.org/r/512/#comment1216>
This is probably unintnetional? I suspect it may just be a result of the build process changing since you created the diff?
/trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp
<https://reviews.apache.org/r/512/#comment1214>
Just curious about removing this check... was it required? Is there ever a case when the payload is not set? If so are we sure it is safe?
/trunk/qpid/cpp/src/qpid/broker/QueueObserver.h
<https://reviews.apache.org/r/512/#comment1215>
I'm not hugely keen on these changes to QueueObserver. I fully anticipate that the interface will need to be adapted and expanded. However I'd like - if possible - to keep cluster specific concerns out of it.
That is a big 'if' however. One suggestion would be to create a sub-class of QueueObserver - e.g. StatefulQueueObserver - that included these changes. You could then create an instance of that if desired/required. The UpdateClient could also then cast to that interface to determine if there was relevant state.
I'd like at some point to move to a queue factory and have a specific specialisation of that for clustered brokers that could encapsulate any cluster specific shenanigans.
- Gordon
On 2011-04-26 17:04:03, Kenneth Giusti wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/512/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-26 17:04:03)
bq.
bq.
bq. Review request for Alan Conway and Gordon Sim.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Hi Alan,
bq.
bq. Here's a patch that should give you an idea of the direction I'm going in - could be completely wrong here so I don't want to go too far without checking with you.
bq.
bq. From your notes, I've modified a bit those items I *think* we need to replicate based on the changes I've made to increase performance (some items have been removed). Here's what I think we need to cover:
bq.
bq. Replicate:
bq.
bq. For each Queue:
bq. If Queue has a QueueFlowLimit configured:
bq. QueueFlowLimit::flowStoppedCount
bq. QueueFlowLimit::index set of msg sequence numbers that are pending.
bq.
bq. For each msg in the QueueFlowLimit::index set:
bq. Allocate a SessionState::IncompleteIngressMsgXfer context
bq. Attach a SessionState::IncompleteIngressMsgXfer context to messsage's AsyncCompletion callback
bq. Force AsyncCompletion::completion count to 1 (??? or more ???)
bq.
bq. For each SessionState:
bq. Replicate the vector of pendingExecutionSyncs SequenceNumbers
bq. Add an AsyncCommandCompleter context
bq. Replicate the vector of completedMsgs
bq. If vector of completedMsgs > 0:
bq. Scheduled requestIOProcessing for callback ????
bq.
bq. Let me know what you think, thanks.
bq.
bq.
bq. This addresses bug QPID-3076.
bq. https://issues.apache.org/jira/browse/QPID-3076
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /trunk/qpid/cpp/docs/man/qpidd.1 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Broker.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.h 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueFlowLimit.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/broker/QueueObserver.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Cluster.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/Connection.cpp 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.h 1096739
bq. /trunk/qpid/cpp/src/qpid/cluster/UpdateClient.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/QueueFlowLimitTest.cpp 1096739
bq. /trunk/qpid/cpp/src/tests/cluster_tests.py 1096739
bq. /trunk/qpid/cpp/src/tests/queue_flow_limit_tests.py 1096739
bq. /trunk/qpid/cpp/xml/cluster.xml 1096739
bq.
bq. Diff: https://reviews.apache.org/r/512/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Barely compiles.
bq.
bq.
bq. Thanks,
bq.
bq. Kenneth
bq.
bq.
> Enable producer flow control (QPID-2935) in a cluster
> -----------------------------------------------------
>
> Key: QPID-3076
> URL: https://issues.apache.org/jira/browse/QPID-3076
> Project: Qpid
> Issue Type: New Feature
> Components: C++ Clustering
> Affects Versions: 0.9, 0.10
> Reporter: Alan Conway
> Assignee: Ken Giusti
> Fix For: 0.11
>
> Attachments: QPID-3076-2.patch
>
>
> QPID-2935 adds producer flow control to the broker, but it is disabled in a cluster. Flow control adds additional broker state that must be replicated in a cluster in order to maintain cluster consistency. Add this replication and enable flow control in a cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org