You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@qpid.apache.org by ac...@apache.org on 2010/11/01 15:10:49 UTC
svn commit: r1029671 - in /qpid/trunk/qpid/cpp: design/ src/qpid/cluster/
Author: aconway
Date: Mon Nov 1 14:10:48 2010
New Revision: 1029671
URL: http://svn.apache.org/viewvc?rev=1029671&view=rev
Log:
Moved new cluster design docs into cpp/design.
Added:
qpid/trunk/qpid/cpp/design/
qpid/trunk/qpid/cpp/design/new-cluster-active-passive.txt (contents, props changed)
- copied, changed from r1029670, qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-active-passive.txt
qpid/trunk/qpid/cpp/design/new-cluster-design.txt (contents, props changed)
- copied, changed from r1029670, qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-design.txt
qpid/trunk/qpid/cpp/design/new-cluster-plan.txt (contents, props changed)
- copied, changed from r1029670, qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-plan.txt
Removed:
qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-active-passive.txt
qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-design.txt
qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-plan.txt
Copied: qpid/trunk/qpid/cpp/design/new-cluster-active-passive.txt (from r1029670, qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-active-passive.txt)
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/design/new-cluster-active-passive.txt?p2=qpid/trunk/qpid/cpp/design/new-cluster-active-passive.txt&p1=qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-active-passive.txt&r1=1029670&r2=1029671&rev=1029671&view=diff
==============================================================================
--- qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-active-passive.txt (original)
+++ qpid/trunk/qpid/cpp/design/new-cluster-active-passive.txt Mon Nov 1 14:10:48 2010
@@ -6,9 +6,9 @@
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
-#
+#
# http://www.apache.org/licenses/LICENSE-2.0
-#
+#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -37,7 +37,7 @@ Simpler implementation of broker::Cluste
- can use smaller message IDs: just sequence number. Can be implicit.
Extra requirements:
-- Exactly one broker hast to take over if primary fails.
+- Exactly one broker hast to take over if primary fails.
- Passive members refuse client connections and redirect to active member.
- On failover, clients keep trying till they find the active member.
@@ -49,7 +49,7 @@ Active/active benefits:
- Some load sharing: reading from client + multicast only done on direct node.
- Clients can switch to any broker on failover
- Failure affects a subset of the clients, not all the clients.
-
+
Active/active drawbacks:
- Co-ordinating message allocation impacts performance.
Propchange: qpid/trunk/qpid/cpp/design/new-cluster-active-passive.txt
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: qpid/trunk/qpid/cpp/design/new-cluster-active-passive.txt
------------------------------------------------------------------------------
svn:mime-type = text/plain
Copied: qpid/trunk/qpid/cpp/design/new-cluster-design.txt (from r1029670, qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-design.txt)
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/design/new-cluster-design.txt?p2=qpid/trunk/qpid/cpp/design/new-cluster-design.txt&p1=qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-design.txt&r1=1029670&r2=1029671&rev=1029671&view=diff
==============================================================================
--- qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-design.txt (original)
+++ qpid/trunk/qpid/cpp/design/new-cluster-design.txt Mon Nov 1 14:10:48 2010
@@ -6,9 +6,9 @@
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
-#
+#
# http://www.apache.org/licenses/LICENSE-2.0
-#
+#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -46,7 +46,7 @@ allocated to consumers can cause an inco
- Entire broker state must be faithfully updated to new members.
- Management model also has to be replicated.
- All queues are replicated, can't have unreplicated queues (e.g. for management)
-
+
Events that are not deterministically predictable from the client
input data stream can cause inconsistencies. In particular use of
timers/timestamps require cluster workarounds to synchronize.
@@ -100,7 +100,7 @@ in that queues context, in parallel with
*** Requirements
-The cluster must provide these delivery guarantees:
+The cluster must provide these delivery guarantees:
- client sends transfer: message must be replicated and not lost even if the local broker crashes.
- client acquires a message: message must not be delivered on another broker while acquired.
@@ -113,7 +113,7 @@ Each guarantee takes effect when the cli
for the associated command (transfer, acquire, reject, accept)
*** Broker receiving messages
-
+
On recieving a message transfer, in the connection thread we:
- multicast a message-received event.
- enqueue and complete the transfer when it is self-delivered.
@@ -157,7 +157,7 @@ completed till the message has been forg
On receiving an accept the broker:
- dequeues the message from the local queue
- multicasts an "accept" event
-- completes the accept asynchronously when the dequeue event is self delivered.
+- completes the accept asynchronously when the dequeue event is self delivered.
NOTE: The message store does not currently implement asynchronous
completions of accept, this is a bug.
@@ -208,8 +208,8 @@ Replication events, multicast to cluster
Messages sent on update connection:
- update_front(q,m): during update, receiver pushes m to *front* of q
- update_done(q): during update, update of q is complete.
-
-Updater:
+
+Updater:
- when updatee joins set iterator i = q.end()
- while i != q.begin(): --i; send update_front(q,*i) to updatee
- send update_done(q) to updatee
@@ -232,7 +232,7 @@ Benefits:
# TODO updatee may receive a dequeue for a message it has not yet seen, needs
# to hold on to that so it can drop the message when it is seen.
# Similar problem exists for wiring?
-
+
** Cluster API
The new cluster API is similar to the MessageStore interface.
@@ -263,7 +263,7 @@ protocol-specific state of connections a
protected from future protocol changes (e.g. AMQP 1.0)
A number of specific ways the code will be simplified:
-- drop code to replicate management model.
+- drop code to replicate management model.
- drop timer workarounds for TTL, management, heartbeats.
- drop "cluster-safe assertions" in broker code.
- drop connections, sessions, management from cluster update.
@@ -334,14 +334,14 @@ scale-up.
** Misc outstanding issues & notes
Replicating wiring
-- Need async completion of wiring commands?
+- Need async completion of wiring commands?
- qpid.sequence_counter: need extra work to support in new design, do we care?
Cluster+persistence:
- finish async completion: dequeue completion for store & cluster
- cluster restart from store: clean stores *not* identical, pick 1, all others update.
- need to generate cluster ids for messages recovered from store.
-
+
Live updates: we don't need to stall brokers during an update!
- update on queue-by-queue basis.
- updatee locks queues during update, no dequeue.
@@ -354,7 +354,7 @@ Flow control: need to throttle multicast
2. ensure the entire cluster keeps up, no unbounded "lag"
The existing design uses read-credit to solve 1., and does not solve 2.
New design should stop reading on all connections while flow control
-condition exists?
+condition exists?
Can federation also be unified, at least in configuration?
Propchange: qpid/trunk/qpid/cpp/design/new-cluster-design.txt
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: qpid/trunk/qpid/cpp/design/new-cluster-design.txt
------------------------------------------------------------------------------
svn:mime-type = text/plain
Copied: qpid/trunk/qpid/cpp/design/new-cluster-plan.txt (from r1029670, qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-plan.txt)
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/cpp/design/new-cluster-plan.txt?p2=qpid/trunk/qpid/cpp/design/new-cluster-plan.txt&p1=qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-plan.txt&r1=1029670&r2=1029671&rev=1029671&view=diff
==============================================================================
--- qpid/trunk/qpid/cpp/src/qpid/cluster/new-cluster-plan.txt (original)
+++ qpid/trunk/qpid/cpp/design/new-cluster-plan.txt Mon Nov 1 14:10:48 2010
@@ -7,9 +7,9 @@
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
-#
+#
# http://www.apache.org/licenses/LICENSE-2.0
-#
+#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -21,7 +21,7 @@
Notes on new cluster implementation. See also: new-cluster-design.txt
* Implementation plan.
-
+
Co-existence with old cluster code and tests:
- Separate plugin cluster2, options --cluster2-*. Eventually renamed to replace cluster.
- Double up tests with old version/new version as the new code develops.
@@ -114,7 +114,7 @@ routing(msg)
# Supress everything except enqueues while we are routing.
# We don't want to replicate acquires & dequeues caused by an enqueu,
# e.g. removal of messages from ring/LV queues.
- isRouting = true
+ isRouting = true
enqueue(qmsg):
if noReplicate: return
@@ -132,7 +132,7 @@ acquire(qmsg):
if isRouting: return # Ignore while we are routing a message.
if msg.id: mcast acquire(qmsg)
-release(QueuedMessage)
+release(QueuedMessage)
if noReplicate: return
if isRouting: return # Ignore while we are routing a message.
mcast release(qmsg)
@@ -420,7 +420,7 @@ Cluster needs to complete these asynchro
exist across the cluster when the command completes.
** TODO [#C] Allow non-replicated exchanges, queues.
-
+
Set qpid.replicated=false in declare arguments, set flag on Exchange, Queue objects.
- save replicated status to store.
- support in management tools.
Propchange: qpid/trunk/qpid/cpp/design/new-cluster-plan.txt
------------------------------------------------------------------------------
svn:eol-style = native
Propchange: qpid/trunk/qpid/cpp/design/new-cluster-plan.txt
------------------------------------------------------------------------------
svn:mime-type = text/plain
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:commits-subscribe@qpid.apache.org