svn commit: r1186463 - in /qpid/trunk/qpid/cpp/design_docs: new-cluster-design.txt new-cluster-plan.txt

[ac...@apache.org]

Author: aconway
Date: Wed Oct 19 20:34:01 2011
New Revision: 1186463

URL: http://svn.apache.org/viewvc?rev=1186463&view=rev
Log:
QPID-2920: Updates to new-cluster-plan.

Filled out plan to-do list. More implementation detail.

Modified:
    qpid/trunk/qpid/cpp/design_docs/new-cluster-design.txt
    qpid/trunk/qpid/cpp/design_docs/new-cluster-plan.txt

Modified: qpid/trunk/qpid/cpp/design_docs/new-cluster-design.txt
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/design_docs/new-cluster-design.txt?rev=1186463&r1=1186462&r2=1186463&view=diff
==============================================================================
--- qpid/trunk/qpid/cpp/design_docs/new-cluster-design.txt (original)
+++ qpid/trunk/qpid/cpp/design_docs/new-cluster-design.txt Wed Oct 19 20:34:01 2011
@@ -83,20 +83,21 @@ context.
 
 ** A new cluster design.
 
-Clearly defined interface between broker code and cluster plug-in.
+1. Clearly defined interface between broker code and cluster plug-in.
 
-Replicate queue events rather than client data.
-- Broker behavior only needs to match per-queue.
-- Smaller amount of code (queue implementation) that must behave predictably.
-- Events only need be serialized per-queue, allows concurrency between queues
-
-Use a moving queue ownership protocol to agree order of dequeues.
-No longer relies on identical state and lock-step behavior to cause
-identical dequeues on each broker.
-
-Use multiple CPG groups to process different queues in parallel. Use a
-fixed set of groups and hash queue names to choose the group for each
-queue.
+2. Replicate queue events rather than client data.
+ - Only requires consistent enqueue order.
+ - Events only need be serialized per-queue, allows concurrency between queues
+ - Allows for replicated and non-replicated queues.
+
+3. Use a lock protocol to agree order of dequeues: only the broker
+   holding the lock can acqiure & dequeue.  No longer relies on
+   identical state and lock-step behavior to cause identical dequeues
+   on each broker.
+
+4. Use multiple CPG groups to process different queues in
+   parallel. Use a fixed set of groups and hash queue names to choose
+   the group for each queue.
 
 *** Requirements
 

Modified: qpid/trunk/qpid/cpp/design_docs/new-cluster-plan.txt
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/design_docs/new-cluster-plan.txt?rev=1186463&r1=1186462&r2=1186463&view=diff
==============================================================================
--- qpid/trunk/qpid/cpp/design_docs/new-cluster-plan.txt (original)
+++ qpid/trunk/qpid/cpp/design_docs/new-cluster-plan.txt Wed Oct 19 20:34:01 2011
@@ -90,7 +90,9 @@ Independent message IDs that can be gene
 this and potentially allow performance benefits by relaxing total ordering.
 However they imply additional map lookups that might hurt performance.
 
-- [ ] Prototype independent message IDs, check performance.
+- [X] Prototype independent message IDs, check performance.
+Throughput worse by 30% in contented case, 10% in uncontended.
+Sticking with queue sequence numbers.
 
 * Outstanding Tasks
 ** TODO [#A] Defer and async completion of wiring commands.
@@ -152,6 +154,10 @@ Status includes
 - persistent store state (clean, dirty)
 - cluster protocol version.
 
+** TODO [#B] Replace boost::hash with our own hash function.
+The hash function is effectively part of the interface so
+we need to be sure it doesn't change underneath us.
+
 ** TODO [#B] Persistent cluster support.
 Initial status protoocl to support persistent start-up (see existing code)
 
@@ -192,6 +198,16 @@ When this is fixed in the standalone bro
 ** TODO [#B] Network partitions and quorum.
 Re-use existing implementation.
 
+** TODO [#B] Review error handling, put in a consitent model.
+- [ ] Review all asserts, for possible throw.
+- [ ] Decide on fatal vs. non-fatal errors.
+
+** TODO [#B] Implement inconsistent error handling policy.
+What to do if a message is enqueued sucessfully on the local broker,
+but fails on one or more backups - e.g. due to store limits?
+- we have more flexibility, we don't *have* to crash
+- but we've loste some of our redundancy guarantee, how should we inform client? 
+
 ** TODO [#C] Allow non-replicated exchanges, queues.
 
 Set qpid.replicate=false in declare arguments, set flag on Exchange, Queue objects.
@@ -226,23 +242,29 @@ The old cluster has workarounds in the b
 - [ ] drop connections, sessions, management from cluster update.
 - [ ] drop security workarounds: cluster code now operates after message decoding.
 - [ ] drop connection tracking in cluster code.
-- [ ] simper inconsistent-error handling code, no need to stall.
+- [ ] simpler inconsistent-error handling code, no need to stall.
+
 ** TODO [#C] Support for live upgrades.
 
 Allow brokers in a running cluster to be replaced one-by-one with a new version.
+(see new-cluster-design for design notes.)
 
 The old cluster protocol was unstable because any changes in broker
 state caused changes to the cluster protocol.The new design should be
 much more stable.
 
-Points to implement: 
-- Brokers should ignore unknown controls (with a warning) rather than an error.
-- Limit logging frequency for unknown control warnings.
-- Add a version number at front of every CPG message. Determines how the
-  rest of the message is decoded. (allows for entirely new encodings e.g. AMQP 1.0)
+Points to implement in anticipation of live upgrade:
+- Prefix each CPG message with a version number and length.
+  Version number determines how to decode the message.
+- Brokers ignore messages that have a higher version number than they understand.
 - Protocol version XML element in cluster.xml, on each control.
 - Initial status protocol to include protocol version number.
 
+New member udpates: use the store encode/decode for updates, use the
+same backward compatibility strategy as the store. This allows for
+adding new elements to the end of structures but not changing or
+removing new elements.
+
 ** TODO [#C] Support for AMQP 1.0.
 
 * Testing



---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:commits-subscribe@qpid.apache.org