[james-project] 01/02: [ADR] Distributed Mailbox Listener Configuration

[bt...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 8c9de9d787f7497e1452c299692fe0bea18b0025
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Apr 23 16:15:46 2020 +0700

    [ADR] Distributed Mailbox Listener Configuration
---
 ...oving-configured-additional-mailboxListeners.md |   2 +
 .../0035-distributed-listeners-configuration.md    | 147 +++++++++++++++++++++
 2 files changed, 149 insertions(+)

diff --git a/src/adr/0026-removing-configured-additional-mailboxListeners.md b/src/adr/0026-removing-configured-additional-mailboxListeners.md
index b9424b0..d1c8ceb 100644
--- a/src/adr/0026-removing-configured-additional-mailboxListeners.md
+++ b/src/adr/0026-removing-configured-additional-mailboxListeners.md
@@ -6,6 +6,8 @@ Date: 2020-04-03
 
 Accepted (lazy consensus)
 
+Superceded by [34. Distributed Mailbox Listener Configuration](0035-distributed-listeners-configuration.md)
+
 ## Context
 
 James enables a user to register additional mailbox listeners.
diff --git a/src/adr/0035-distributed-listeners-configuration.md b/src/adr/0035-distributed-listeners-configuration.md
new file mode 100644
index 0000000..626dfbb
--- /dev/null
+++ b/src/adr/0035-distributed-listeners-configuration.md
@@ -0,0 +1,147 @@
+# 35. Distributed Mailbox Listeners Configuration
+
+Date: 2020-04-23
+
+## Status
+
+Proposed
+
+Supercedes [26. Removing a configured additional MailboxListener](0026-removing-configured-additional-mailboxListeners.md)
+
+## Context
+
+James enables a user to register additional mailbox listeners.
+
+The distributed James server is handling mailbox event processing (mailboxListener execution) using a RabbitMQ work-queue
+per listener.
+
+Mailbox listeners can be registered to be triggered every time an event is generated by user interaction with their 
+mailbox. They are being executed in a distributed fashion following the workqueue messaging pattern. The "group" is an 
+attribute of the mailbox listener identifying to which work queue they belong.
+
+Currently, mailbox listeners are determined by the guice bindings of the server and additional mailbox listeners defined
+via configuration files.
+
+While the configuration might be specific for each James server, what actually is defined in RabbitMQ is common. 
+Heterogeneous configuration might then result in unpredictable RabbitMQ resource status. This was left as a limitation
+of [26. Removing a configured additional MailboxListener](0026-removing-configured-additional-mailboxListeners.md).
+
+## Decision
+
+We need to centralize the definition of mailbox listeners.
+
+An event sourcing system will track the configured mailbox listeners.
+
+It will have the following commands:
+
+ - **AddListener**: Add a given listener. This should be rejected if the group is already used.
+ - **RemoveListener**: Remove a given listener.
+ 
+Configuration changes are not supported. The administrator is expected to remove the given listener, then add it again.
+
+It will have the following events:
+
+ - **ListenerAdded**: A mailbox listener is added
+ - **ListenerRemoved**: A mailbox listener is removed
+
+A subscriber will react to these events to modify the RabbitMQ resource accordingly by adding queues, adding or removing
+bindings.
+
+This event sourcing system differs from the one defined in
+[26. Removing a configured additional MailboxListener](0026-removing-configured-additional-mailboxListeners.md) by the
+fact that we should also keep track of listener configuration.
+
+Upon start, James will ensure the **configured mailbox listener event sourcing system** contains the guice injected 
+listeners, and add them if missing (handling the RabbitMQ bindings by this mean), then starts the eventBus which will
+consume the given queues.
+
+If a listener is configured with a class unknown to James, the start-up fails and James starts in a degraded state 
+allowing to unconfigure the faulty listener. This downgraded state will be described in a separate ADR and the link will
+be updated here.
+
+This differs from [26. Removing a configured additional MailboxListener](0026-removing-configured-additional-mailboxListeners.md)
+by the fact we no longer need to register all listeners at once.
+
+A WebAdmin endpoint will allow:
+
+ - **to add a listener** to the one configured. Such a call:
+    - Will fail if the listener class is not on the local classpath, or if the corresponding group already used within 
+    the **configured mailbox listener aggregate**.
+    - Upon success the listener is added to the **configured mailbox listener aggregate**, and the listener is 
+    registered locally.
+ - **to remove a listener**. Such a call:
+    - Will fail if the listener is required by Guice bindings on the current server or if the listener is not configured.
+    - Upon success, the listener is removed from the **configured mailbox listener aggregate**, and the listener is 
+    unregistered locally.
+    
+A broadcast on the event bus will be attempted to propagate topology changes, by the mean of a common registrationKey 
+to all nodes, a "TopologyChanged" event, and a mailbox listener starting the MailboxListeners on local node upon
+topology changes.
+ 
+If a listener is added but is not in the classpath, an ERROR log is emitted. This can happen during a rolling upgrade,
+which defines a new guice binding for a new mailbox listener. Events will still be emitted (and consumed by other James
+servers) however a local James upgrade will be required to effectively be able to start processing these events. The 
+binding will not need to be redefined.
+
+We will also expose an endpoint listing the groups currently in use, and for each group the associated configuration, if 
+any. This will query the **configured mailbox listener aggregate**.
+
+We will introduce a health check to actually ensure that RabbitMQ resources match the configured listeners, and propose
+a WebAdmin endpoint to add/remove bindings/queue in a similar fashion of what had been proposed in 
+[26. Removing a configured additional MailboxListener](0026-removing-configured-additional-mailboxListeners.md). This 
+can happen if the James server performing the listener registration fails to create the group/queue. This health check 
+will also report if this James server does not succeed to run a given listener, for instance if its class is not on the 
+classpath.
+
+## Consequences
+
+All products other than "Distributed James" are unchanged.
+
+All the currently configured additional listeners will need to be registered.
+
+The definition of mailbox listeners is thus centralized and we are not exposed to an heterogeneous configuration 
+incident.
+
+Mailbox listeners no longer required by guice will still need to be instantiable (even with empty content). They will 
+be considered as additional listener thus requiring explicit admin unconfiguration, which will be mentioned in the 
+related upgrade instructions. Read notes about [rolling upgrade scenarii](#rolling-upgrade-scenari).
+
+[Deploying a new custom listener](#deploying-a-new-custom-listener) also describes how to deploy new custom listeners.
+
+Integration tests relying on additional mailbox listeners of the distributed James product will require to be ported to 
+perform additional mailbox listener registry with this WebAdmin endpoint. JMAP SpamAssassin, quota mailing tests are 
+concerned.
+
+## Notes
+
+## Broadcast of topology changes
+
+### Rolling upgrade scenarii
+
+During a rolling upgrade, the james version is heterogeneous across the cluster, and so might be the mailbox listeners
+required at the Guice level.
+
+**case 1**: James Server version 1 does not require listener A, James server version 2 requires listener A.
+
+Since listener A is registered, James server version 1 cannot be rebooted without being upgraded first. (As listener A 
+cannot be instantiated)
+
+**case 2**: James Server version 1 requires listener A, James server version 2 does not require listener A.
+
+Upgrading to James version 2 means that listener A is still registered as an additional listener, it needs to be 
+manually unconfigured once the rolling upgrade finished. Which is acceptable in upgrade instruction. We need to make 
+sure the listeners could still be instantiated (even with empty code) for a transition period.
+
+## Deploying a new custom listener
+
+Given a new custom listener, not yet deployed in Distributed James cluster,
+
+To deploy it, an admin needs to follow these steps:
+
+ - Add the jar in `extension-jars` folder for each James server
+     - As `extension-jars` is read at instantiation time, no reboot is required to instantiate the new listener.
+ - Call the webadmin endpoint alongside with listener specific configuration to enable the given custom listener. 
+The bindings for the new listener will be created and a listener will be consuming its queue on the James server that 
+had been treating the request.
+ - Broadcast of topology changes will ensure the new custom additional mailbox listener will then be instantiated 
+everywhere without a reboot.


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org