You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by ie...@apache.org on 2020/07/09 22:38:08 UTC
[james-project] 02/02: [JAMES-3302] Migrate ADR docs to asciidoc
using markdown
This is an automated email from the ASF dual-hosted git repository.
ieugen pushed a commit to branch JAMES-3302-migrate-content
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit ac4c21e0e15fcdbe525ceec1ed6b181c422fa306
Author: Eugen Stan <ie...@apache.org>
AuthorDate: Fri Jul 10 01:37:46 2020 +0300
[JAMES-3302] Migrate ADR docs to asciidoc using markdown
---
.../adr/0001-record-architecture-decisions.md.adoc | 39 ++
.../adr/0002-make-taskmanager-distributed.md.adoc | 25 +
.../adr/src/adr/0003-distributed-workqueue.md.adoc | 29 +
.../src/adr/0004-distributed-tasks-listing.md.adoc | 20 +
...ributed-task-termination-ackowledgement.md.adoc | 24 +
.../adr/src/adr/0006-task-serialization.md.adoc | 30 +
.../adr/0007-distributed-task-cancellation.md.adoc | 21 +
.../src/adr/0008-distributed-task-await.md.adoc | 30 +
...9-disable-elasticsearch-dynamic-mapping.md.adoc | 40 ++
.../adr/src/adr/0009-java-11-migration.md.adoc | 23 +
.../adr/0010-enable-elasticsearch-routing.md.adoc | 41 ++
...11-remove-elasticsearch-document-source.md.adoc | 37 ++
.../adr/src/adr/0012-jmap-partial-reads.md.adoc | 50 ++
.../src/adr/0013-precompute-jmap-preview.md.adoc | 56 ++
.../adr/0014-blobstore-storage-policies.md.adoc | 63 ++
.../src/adr/0015-objectstorage-blobid-list.md.adoc | 68 ++
.../adr/src/adr/0016-distributed-workqueue.md.adoc | 29 +
.../adr/0017-file-mail-queue-deprecation.md.adoc | 43 ++
.../adr/src/adr/0018-jmap-new-specs.md.adoc | 65 ++
.../src/adr/0019-reactor-netty-adoption.md.adoc | 40 ++
...20-cassandra-mailbox-object-consistency.md.adoc | 73 +++
.../adr/0021-cassandra-acl-inconsistency.md.adoc | 63 ++
.../0022-cassandra-message-inconsistency.md.adoc | 89 +++
...sandra-mailbox-counters-inconsistencies.md.adoc | 58 ++
.../adr/src/adr/0024-polyglot-strategy.md.adoc | 179 ++++++
.../adr/0025-cassandra-blob-store-cache.md.adoc | 69 +++
...-configured-additional-mailboxListeners.md.adoc | 72 +++
...7-eventBus-error-handling-upon-dispatch.md.adoc | 35 ++
.../src/adr/0028-Recompute-mailbox-quotas.md.adoc | 46 ++
...0029-Cassandra-mailbox-deletion-cleanup.md.adoc | 46 ++
...eparate-attachment-content-and-metadata.md.adoc | 94 +++
.../src/adr/0031-distributed-mail-queue.md.adoc | 122 ++++
.../0032-distributed-mail-queue-cleanup.md.adoc | 50 ++
...033-use-scala-in-event-sourcing-modules.md.adoc | 33 +
.../0034-mailbox-api-visibility-and-usage.md.adoc | 49 ++
...035-distributed-listeners-configuration.md.adoc | 137 ++++
...conditional-statements-in-guice-modules.md.adoc | 112 ++++
.../development/adr/src/adr/0037-eventbus.md.adoc | 59 ++
.../adr/src/adr/0038-distributed-eventbus.md.adoc | 44 ++
...0039-distributed-blob-garbage-collector.md.adoc | 687 +++++++++++++++++++++
migrate-adr.sh | 8 +
.../0009-disable-elasticsearch-dynamic-mapping.md | 16 +-
42 files changed, 2906 insertions(+), 8 deletions(-)
diff --git a/docs/modules/development/adr/src/adr/0001-record-architecture-decisions.md.adoc b/docs/modules/development/adr/src/adr/0001-record-architecture-decisions.md.adoc
new file mode 100644
index 0000000..dac5685
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0001-record-architecture-decisions.md.adoc
@@ -0,0 +1,39 @@
+= 1. [JAMES-2909] Record architecture decisions
+
+Date: 2019-10-02
+
+== Status
+
+Proposed
+
+== Context
+
+In order to be more community-oriented, we should adopt a process to have a structured way to have open architectural decisions.
+
+Using an Architectural Decision Records-based process as a support of discussion on the developers mailing-lists.
+
+== Decision
+
+We will use Architecture Decision Records, as https://web.archive.org/web/20190824074401/http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions[described by Michael Nygard].
+
+Each ADR will be discussed on the Apache James' developers mailing-list before being accepted.
+
+Following https://community.apache.org/committers/decisionMaking.html[Apache Decision Making process], we provide the following possible status, with their associated meaning:
+
+* `Proposed`: The decision is being discussed on the mailing list.
+* `Accepted (lazy consensus)` : the architecture decision was proposed on the mailing list, and a consensus emerged from people involved in the discussion on the mailing list.
+* `Accepted (voted)` : the architecture undergo a voting process.
+* `Rejected` : Consensus built up against that proposal.
+
+== Consequences
+
+See Michael Nygard's article, linked above.
+For a lightweight ADR toolset, see Nat Pryce's https://github.com/npryce/adr-tools[adr-tools].
+
+We should provide in a mutable `References` section links to related JIRA meta-ticket (not necessarily to all related sub-tickets) as well as a link to the mail archive discussion thread.
+
+JIRA tickets implementing that architecture decision should also link the related Architecture Decision Record.
+
+== References
+
+* https://jira.apache.org/jira/browse/JAMES-2909[JAMES-2909]
diff --git a/docs/modules/development/adr/src/adr/0002-make-taskmanager-distributed.md.adoc b/docs/modules/development/adr/src/adr/0002-make-taskmanager-distributed.md.adoc
new file mode 100644
index 0000000..0ce4ce2
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0002-make-taskmanager-distributed.md.adoc
@@ -0,0 +1,25 @@
+= 2. Make TaskManager Distributed
+
+Date: 2019-10-02
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+In order to have a distributed version of James we need to have an homogeneous way to deal with `Task`.
+
+Currently, every James nodes of a cluster have their own instance of `TaskManager` and they have no knowledge of others, making it impossible to orchestrate task execution at the cluster level.
+Tasks are scheduled and ran on the same node they are scheduled.
+
+We are also unable to list or access to the details of all the ``Task``s of a cluster.
+
+== Decision
+
+Create a distribution-aware implementation of `TaskManager`.
+
+== Consequences
+
+* Split the `TaskManager` part dealing with the coordination (`Task` management and view) and the `Task` execution (located in `TaskManagerWorker`)
+* The distributed `TaskManager` will rely on RabbitMQ to coordinate and the event system to synchronize states
diff --git a/docs/modules/development/adr/src/adr/0003-distributed-workqueue.md.adoc b/docs/modules/development/adr/src/adr/0003-distributed-workqueue.md.adoc
new file mode 100644
index 0000000..9e7ffa4
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0003-distributed-workqueue.md.adoc
@@ -0,0 +1,29 @@
+= 3. Distributed WorkQueue
+
+Date: 2019-10-02
+
+== Status
+
+Accepted (lazy consensus)
+
+Superceded by xref:0016-distributed-workqueue.adoc[16.
+Distributed WorkQueue]
+
+== Context
+
+By switching the task manager to a distributed implementation, we need to be able to run a `Task` on any node of the cluster.
+
+== Decision
+
+For the time being we will keep the sequential execution property of the task manager.
+This is an intermediate milestone toward the final implementation which will drop this property.
+
+* Use a RabbitMQ queue as a workqueue where only the `Created` events are pushed into.
+This queue will be exclusive and events will be consumed serially.
+Technically this means the queue will be consumed with a `prefetch = 1`.
+The queue will listen to the worker on the same node and will ack the message only once it is finished (`Completed`, `Failed`, `Cancelled`).
+
+== Consequences
+
+* It's a temporary and not safe to use in production solution: if the node promoted to exclusive listener of the queue dies, no more tasks will be run
+* The serial execution of tasks does not leverage cluster scalability.
diff --git a/docs/modules/development/adr/src/adr/0004-distributed-tasks-listing.md.adoc b/docs/modules/development/adr/src/adr/0004-distributed-tasks-listing.md.adoc
new file mode 100644
index 0000000..917b003
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0004-distributed-tasks-listing.md.adoc
@@ -0,0 +1,20 @@
+= 4. Distributed Tasks listing
+
+Date: 2019-10-02
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+By switching the task manager to a distributed implementation, we need to be able to `list` all ``Task``s running on the cluster.
+
+== Decision
+
+* Read a Cassandra projection to get all ``Task``s and their `Status`
+
+== Consequences
+
+* A Cassandra projection has to be done
+* The `EventSourcingSystem` should have a `Listener` updating the `Projection`
diff --git a/docs/modules/development/adr/src/adr/0005-distributed-task-termination-ackowledgement.md.adoc b/docs/modules/development/adr/src/adr/0005-distributed-task-termination-ackowledgement.md.adoc
new file mode 100644
index 0000000..87ff4f8
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0005-distributed-task-termination-ackowledgement.md.adoc
@@ -0,0 +1,24 @@
+= 5. Distributed Task termination ackowledgement
+
+Date: 2019-10-02
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+By switching the task manager to a distributed implementation, we need to be able to execute a `Task` on any node of the cluster.
+We need a way for nodes to be signaled of any termination event so that we can notify blocking clients.
+
+== Decision
+
+* Creating a `RabbitMQEventHandler` which publish ``Event``s pushed to the task manager's event system to RabbitMQ
+* All the events which end a `Task` (`Completed`, `Failed`, and `Canceled`) have to be transmitted to other nodes
+
+== Consequences
+
+* A new kind of ``Event``s should be created: `TerminationEvent` which includes `Completed`, `Failed`, and `Canceled`
+* ``TerminationEvent``s will be broadcasted on an exchange which will be bound to all interested components later
+* `EventSourcingSystem.dipatch` should use `RabbitMQ` to dispatch ``Event``s instead of triggering local ``Listener``s
+* Any node can be notified when a `Task` emits a termination event
diff --git a/docs/modules/development/adr/src/adr/0006-task-serialization.md.adoc b/docs/modules/development/adr/src/adr/0006-task-serialization.md.adoc
new file mode 100644
index 0000000..bae833f
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0006-task-serialization.md.adoc
@@ -0,0 +1,30 @@
+= 6. Task serialization
+
+Date: 2019-10-02
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+By switching the task manager to a distributed implementation, we need to be able to execute a `Task` on any node of the cluster.
+We need to have a way to describe the `Task` to be executed and serialize it in order to be able to store it in the `Created` event.
+Which will be persisted in the Event Store, and will be send in the event bus.
+
+At this point in time a `Task` can contain any arbitrary code.
+It's not an element of a finite set of actions.
+
+== Decision
+
+* Create a `Factory` for one `Task`
+* Inject a `Factory` `Registry` via a Guice Module
+* The `Task` `Serialization` will be done in JSON, We will get inspired by `EventSerializer`
+* Every ``Task``s should have a specific integration test demonstrating that serialization works
+* Each `Task` is responsible of eventually dealing with the different versions of the serialized information
+
+== Consequences
+
+* Every ``Task``s should be serializable.
+* Every ``Task``s should provide a `Factory` which would be responsible to deserialize the task and instantiate it.
+* Every `Factory` should be registered through a Guice module to be created for each project containing a `Factory`
diff --git a/docs/modules/development/adr/src/adr/0007-distributed-task-cancellation.md.adoc b/docs/modules/development/adr/src/adr/0007-distributed-task-cancellation.md.adoc
new file mode 100644
index 0000000..7e0f6b1
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0007-distributed-task-cancellation.md.adoc
@@ -0,0 +1,21 @@
+= 7. Distributed Task cancellation
+
+Date: 2019-10-02
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+A `Task` could be run on any node of the cluster.
+To interrupt it we need to notify all nodes of the cancel request.
+
+== Decision
+
+* We will add an EventHandler to broadcast the `CancelRequested` event to all the workers listening on a RabbitMQ broadcasting exchange.
+* The `TaskManager` should register to the exchange and will apply `cancel` on the `TaskManagerWorker` if the `Task` is waiting or in progress on it.
+
+== Consequences
+
+* The task manager's event system should be bound to the RabbitMQ exchange which publish the ``TerminationEvent``s
diff --git a/docs/modules/development/adr/src/adr/0008-distributed-task-await.md.adoc b/docs/modules/development/adr/src/adr/0008-distributed-task-await.md.adoc
new file mode 100644
index 0000000..06424d5
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0008-distributed-task-await.md.adoc
@@ -0,0 +1,30 @@
+= 8. Distributed Task await
+
+Date: 2019-10-02
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+By switching the task manager to a distributed implementation, we need to be able to `await` a `Task` running on any node of the cluster.
+
+== Decision
+
+* Broadcast ``Event``s in `RabbitMQ`
+
+== Consequences
+
+* {blank}
++
+[cols=3*]
+|===
+| `RabbitMQTaskManager` should broadcast termination ``Event``s (`Completed`
+| `Failed`
+| `Canceled`)
+|===
+
+* `RabbitMQTaskManager.await` should: first, check the ``Task``'s state;
+and if it's not terminated, listen to RabbitMQ
+* The await should have a timeout limit
diff --git a/docs/modules/development/adr/src/adr/0009-disable-elasticsearch-dynamic-mapping.md.adoc b/docs/modules/development/adr/src/adr/0009-disable-elasticsearch-dynamic-mapping.md.adoc
new file mode 100644
index 0000000..b82e965
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0009-disable-elasticsearch-dynamic-mapping.md.adoc
@@ -0,0 +1,40 @@
+= 9. Disable ElasticSearch dynamic mapping
+
+Date: 2019-10-10
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+We rely on dynamic mappings to expose our mail headers as a JSON map.
+Dynamic mapping is enabled for adding not yet encountered headers in the mapping.
+
+This causes a serie of functional issues:
+
+* Maximum field count can easily be exceeded
+* Field type 'guess' can be wrong, leading to subsequent headers omissions (see JAMES-2078)
+* Document indexation needs to be paused at the index level during mapping changes to avoid concurrent changes, impacting negatively performance.
+
+== Decision
+
+Rely on nested objects to represent mail headers within a mapping
+
+== Consequences
+
+The index needs to be re-created.
+Document reIndexation is needed.
+
+This solves the aforementionned bugs (see JAMES-2078).
+
+Regarding performance:
+
+* Default message list performance is unimpacted
+* We noticed a 4% performance improvment upon indexing throughput
+* We noticed a 7% increase regarding space per message
+
+== References
+
+* https://github.com/linagora/james-project/pull/2726[JAMES-2078] JAMES-2078 Add an integration test to prove that dynamic mapping can lead to ignored header fields
+* https://issues.apache.org/jira/browse/JAMES-2078[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0009-java-11-migration.md.adoc b/docs/modules/development/adr/src/adr/0009-java-11-migration.md.adoc
new file mode 100644
index 0000000..4960085
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0009-java-11-migration.md.adoc
@@ -0,0 +1,23 @@
+= 9. Migration to Java Runtime Environment 11
+
+Date: 2019-10-24
+
+== Status
+
+Proposed
+
+== Context
+
+Java 11 is the only "Long Term Support" java release right now so more and more people will use it exclusively.
+
+James is known to build with Java Compiler 11 for some weeks.
+
+== Decision
+
+We adopt Java Runtime Environment 11 for James as a runtime to benefits from a supported runtime and new features of the languages and the platform.
+
+== Consequences
+
+* It requires the upgrade of Spring to 4.3.x.
+* All docker images should be updated to adoptopenjdk 11.
+* The documentation should be updated accordingly.
diff --git a/docs/modules/development/adr/src/adr/0010-enable-elasticsearch-routing.md.adoc b/docs/modules/development/adr/src/adr/0010-enable-elasticsearch-routing.md.adoc
new file mode 100644
index 0000000..bbac6c2
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0010-enable-elasticsearch-routing.md.adoc
@@ -0,0 +1,41 @@
+= 10 Enable ElasticSearch routing
+
+Date: 2019-10-17
+
+== Status
+
+Accepted (lazy consensus)
+
+Additional performance testing is required for adoption.
+
+== Context
+
+Our queries are mostly bounded to a mailbox or a user.
+We can easily limit the number of ElasticSearch nodes involved in a given query by grouping the underlying documents on the same node using a routing key.
+
+Without a routing key, each shard needs to execute the query.
+The coordinator needs also to be waiting for the slowest shard.
+
+Using the routing key unlocks significant throughput enhancement (proportional to the number of shards) and also a possible high percentile latency enhancement.
+
+As most requests are restricted to a single coordination, most search requests will hit a single shard, as opposed to non routed searches which would have hit each shards (each shard would return the number of searched documents, to be ordered and limited again in the coordination node).
+This allows to be more linearly scalable.
+
+== Decision
+
+Enable ElasticSearch routing.
+
+Messages should be indexed by mailbox.
+
+Quota Ratio should be indexed by user.
+
+== Consequences
+
+A data reindex is needed.
+
+On a single ElasticSearch node with 5 shards, we noticed latency reduction for mailbox search (2x mean time and 3x 99 percentile reduction)
+
+== References
+
+* https://www.elastic.co/guide/en/elasticsearch/reference/6.3/mapping-routing-field.html
+* https://issues.apache.org/jira/browse/JAMES-2917[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0011-remove-elasticsearch-document-source.md.adoc b/docs/modules/development/adr/src/adr/0011-remove-elasticsearch-document-source.md.adoc
new file mode 100644
index 0000000..bf5ff12
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0011-remove-elasticsearch-document-source.md.adoc
@@ -0,0 +1,37 @@
+= 11. Disable ElasticSearch source
+
+Date: 2019-10-17
+
+== Status
+
+Rejected
+
+The benefits do not outweigh the costs.
+
+== Context
+
+Though very handy to have around, the source field does incur storage overhead within the index.
+
+== Decision
+
+Disable `_source` for ElasticSearch indexed documents.
+
+== Consequences
+
+Given a dataset composed of small text/plain messages, we notice a 20% space reduction of data stored on ElasticSearch.
+
+However, patch updates can no longer be performed upon flags updates.
+Upon flag update we need to fully read the mail content, then mime-parse it, potentially html parse it, extract attachment content again and finally index again the full document.
+
+Without `_source` field, flags update is two times slower, 99 percentile 4 times slower, and this impact negatively other requests.
+
+Note please that `_source` allows admin flexibility like performing index level changes without downtime, amongst others:
+
+* Increase shards
+* Modifying replication factor
+* Changing analysers (IE allows an admin to configure FR analyser instead of EN analyser)
+
+== References
+
+* https://www.elastic.co/guide/en/elasticsearch/reference/6.3/mapping-source-field.html
+* https://issues.apache.org/jira/browse/JAMES-2906[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0012-jmap-partial-reads.md.adoc b/docs/modules/development/adr/src/adr/0012-jmap-partial-reads.md.adoc
new file mode 100644
index 0000000..d78610e
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0012-jmap-partial-reads.md.adoc
@@ -0,0 +1,50 @@
+= 12. Projections for JMAP Messages
+
+Date: 2019-10-09
+
+== Status
+
+Accepted
+
+== Context
+
+JMAP core RFC8620 requires that the server responds only properties requested by the client.
+
+James currently computes all of the properties regardless of their cost, and if it had been asked by the client.
+
+Clearly we can save some latencies and resources by avoiding reading/computing expensive properties that had not been explicitly requested by the client.
+
+== Decision
+
+Introduce two new datastructures representing JMAP messages:
+
+* One with only metadata
+* One with metadata + headers
+
+Given the properties requested by the client, the most appropriate message datastructure will be computed, on top of existing message storage APIs that should remain unchanged.
+
+Some performance tests will be run in order to evaluate the improvements.
+
+== Consequences
+
+GetMessages with a limited set of requested properties no longer result necessarily in full database message read.
+We thus have a significant improvement, for instance when only metadata are requested.
+
+Given the following scenario played by 5000 users per hour (constant rate)
+
+* Authenticate
+* List mailboxes
+* List messages in one of their mailboxes
+* Get 10 times the mailboxIds and keywords of the given messages
+
+We went from:
+
+* A 20% failure and timeout rate before this change to no failure
+* Mean time for GetMessages went from 27 159 ms to 27 ms (1000 time improvment), for all operation from 27 591 ms to 60 ms (460 time improvment)
+* P99 is a metric that did not make sense because the initial simulation exceeded Gatling (the performance measuring tool we use) timeout (60s) at the p50 percentile.
+After this proposal p99 for the entire scenario is of 1 383 ms
+
+== References
+
+* /get method: https://tools.ietf.org/html/rfc8620#section-5.1
+* https://issues.apache.org/jira/browse/JAMES-2919[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0013-precompute-jmap-preview.md.adoc b/docs/modules/development/adr/src/adr/0013-precompute-jmap-preview.md.adoc
new file mode 100644
index 0000000..4758f08
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0013-precompute-jmap-preview.md.adoc
@@ -0,0 +1,56 @@
+= 13. Precompute JMAP Email preview
+
+Date: 2019-10-09
+
+== Status
+
+Accepted
+
+== Context
+
+JMAP messages have a handy preview property displaying the firsts 256 characters of meaningful test of a message.
+
+This property is often displayed for message listing in JMAP clients, thus it is queried a lot.
+
+Currently, to get the preview, James retrieves the full message body, parse it using MIME parsers, removes HTML and keep meaningful text.
+
+== Decision
+
+We should pre-compute message preview.
+
+A MailboxListener will compute the preview and store it in a MessagePreviewStore.
+
+We should have a Cassandra and memory implementation.
+
+When the preview is precomputed then for these messages we can consider the "preview" property as a metadata.
+
+When the preview is not precomputed then we should compute the preview for these messages, and save the result for later.
+
+We should provide a webAdmin task allowing to rebuild the projection.
+The computing and storing in MessagePreviewStore is idempotent and the task can be run in live without any concurrency problem.
+
+Some performance tests will be run in order to evaluate the improvements.
+
+== Consequences
+
+Given the following scenario played by 2500 users per hour (constant rate)
+
+* Authenticate
+* List mailboxes
+* List messages in one of their mailboxes
+* Get 8 times the properties expected to be fast to fetch with JMAP
+
+We went from:
+
+* A 7% failure and timeout rate before this change to almost no failure
+* Mean time for GetMessages went from 9 710 ms to 434 ms (22 time improvment), for all operation from 12 802 ms to 407 ms (31 time improvment)
+* P99 is a metric that did not make sense because the initial simulation exceeded Gatling (the performance measuring tool we use) timeout (60s) at the p95 percentile.
+After this proposal p99 for the entire scenario is of 1 747 ms
+
+As such, this changeset significantly increases the JMAP performance.
+
+== References
+
+* https://jmap.io/server.html#1-emails JMAP client guice states that preview needs to be quick to retrieve
+* Similar decision had been taken at FastMail: https://fastmail.blog/2014/12/15/dec-15-putting-the-fast-in-fastmail-loading-your-mailbox-quickly/
+* https://issues.apache.org/jira/browse/JAMES-2919[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0014-blobstore-storage-policies.md.adoc b/docs/modules/development/adr/src/adr/0014-blobstore-storage-policies.md.adoc
new file mode 100644
index 0000000..819109d
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0014-blobstore-storage-policies.md.adoc
@@ -0,0 +1,63 @@
+= 14. Add storage policies for BlobStore
+
+Date: 2019-10-09
+
+== Status
+
+Proposed
+
+Adoption needs to be backed by some performance tests, as well as data repartition between Cassandra and object storage shifts.
+
+== Context
+
+James exposes a simple BlobStore API for storing raw data.
+However such raw data often vary in size and access patterns.
+
+As an example:
+
+* Mailbox message headers are expected to be small and frequently accessed
+* Mailbox message body are expected to have sizes ranging from small to big but are unfrequently accessed
+* DeletedMessageVault message headers are expected to be small and unfrequently accessed
+
+Also, the capabilities of the various implementations of BlobStore have different strengths:
+
+* CassandraBlobStore is efficient for small blobs and offers low latency.
+However it is known to be expensive for big blobs.
+Cassandra storage is expensive.
+* Object Storage blob store is good at storing big blobs, but it induces higher latencies than Cassandra for small blobs for a cost gain that isn't worth it.
+
+Thus, significant performance and cost ratio refinement could be unlocked by using the right blob store for the right blob.
+
+== Decision
+
+Introduce StoragePolicies at the level of the BlobStore API.
+
+The proposed policies include:
+
+* SizeBasedStoragePolicy: The blob underlying storage medium will be chosen depending on its size.
+* LowCostStoragePolicy: The blob is expected to be saved in low cost storage.
+Access is expected to be unfrequent.
+* PerformantStoragePolicy: The blob is expected to be saved in performant storage.
+Access is expected to be frequent.
+
+An HybridBlobStore will replace current UnionBlobStore and will allow to choose between Cassandra and ObjectStorage implementations depending on the policies.
+
+DeletedMessageVault, BlobExport & MailRepository will rely on LowCostStoragePolicy.
+Other BlobStore users will rely on SizeBasedStoragePolicy.
+
+Some performance tests will be run in order to evaluate the improvements.
+
+== Consequences
+
+We expect small frequently accessed blobs to be located in Cassandra, allowing ObjectStorage to be used mainly for large costly blobs.
+
+In case of a less than 5% improvement, the code will not be added to the codebase and the proposal will get the status 'rejected'.
+
+We expect more data to be stored in Cassandra.
+We need to quantify this for adoption.
+
+As reads will be reading the two blobStores, no migration is required to use this composite blobstore on top an existing implementation, however we will benefits of the performance enhancements only for newly stored blobs.
+
+== References
+
+* https://issues.apache.org/jira/browse/JAMES-2921[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0015-objectstorage-blobid-list.md.adoc b/docs/modules/development/adr/src/adr/0015-objectstorage-blobid-list.md.adoc
new file mode 100644
index 0000000..7c94209
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0015-objectstorage-blobid-list.md.adoc
@@ -0,0 +1,68 @@
+= 15. Persist BlobIds for avoiding persisting several time the same blobs within ObjectStorage
+
+Date: 2019-10-09
+
+== Status
+
+Proposed
+
+Adoption needs to be backed by some performance tests.
+
+== Context
+
+A given mail is often written to the blob store by different components.
+And mail traffic is heavily duplicated (several recipients receiving similar email, same attachments).
+This causes a given blob to often be persisted several times.
+
+Cassandra was the first implementation of the blobStore.
+Cassandra is a heavily write optimized NoSQL database.
+One can assume writes to be fast on top of Cassandra.
+Thus we assumed we could always overwrite blobs.
+
+This usage pattern was also adopted for BlobStore on top of ObjectStorage.
+
+However writing in Object storage:
+
+* Takes time
+* Is billed by most cloud providers
+
+Thus choosing a right strategy to avoid writing blob twice is desirable.
+
+However, ObjectStorage (OpenStack Swift) `exist` method was not efficient enough to be a real cost and performance saver.
+
+== Decision
+
+Rely on a StoredBlobIdsList API to know which blob is persisted or not in object storage.
+Provide a Cassandra implementation of it.
+Located in blob-api for convenience, this it not a top level API.
+It is intended to be used by some blobStore implementations (here only ObjectStorage).
+We will provide a CassandraStoredBlobIdsList in blob-cassandra project so that guice products combining object storage and Cassandra can define a binding to it.
+
+* When saving a blob with precomputed blobId, we can check the existence of the blob in storage, avoiding possibly the expensive "save".
+* When saving a blob too big to precompute its blobId, once the blob had been streamed using a temporary random blobId, copy operation can be avoided and the temporary blob could be directly removed.
+
+Cassandra is probably faster doing "write every time" rather than "read before write" so we should not use the stored blob projection for it
+
+Some performance tests will be run in order to evaluate the improvements.
+
+== Consequences
+
+We expect to reduce the amount of writes to the object storage.
+This is expected to improve:
+
+* operational costs on cloud providers
+* performance improvement
+* latency reduction under load
+
+As id persistence in StoredBlobIdsList will be done once the blob successfully saved, inconsistencies in StoredBlobIdsList will lead to duplicated saved blobs, which is the current behaviour.
+
+In case of a less than 5% improvement, the code will not be added to the codebase and the proposal will get the status 'rejected'.
+
+== Reference
+
+Previous optimization proposal using blob existence checks before persist.
+This work was done using ObjectStorage exist method and was prooven not efficient enough.
+
+https://github.com/linagora/james-project/pull/2011 (V2)
+
+* https://issues.apache.org/jira/browse/JAMES-2921[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0016-distributed-workqueue.md.adoc b/docs/modules/development/adr/src/adr/0016-distributed-workqueue.md.adoc
new file mode 100644
index 0000000..7f6133f
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0016-distributed-workqueue.md.adoc
@@ -0,0 +1,29 @@
+= 16. Distributed WorkQueue
+
+Date: 2019-12-03
+
+== Status
+
+Accepted (lazy consensus)
+
+Supercedes xref:0003-distributed-workqueue.adoc[3.
+Distributed WorkQueue]
+
+== Context
+
+By switching the task manager to a distributed implementation, we need to be able to run a `Task` on any node of the cluster.
+
+== Decision
+
+For the time being we will keep the sequential execution property of the task manager.
+This is an intermediate milestone toward the final implementation which will drop this property.
+
+* Use a RabbitMQ queue as a workqueue where only the `Created` events are pushed into.
+Instead of using the brittle exclusive queue mechanism described in xref:0003-distributed-workqueue.adoc[3.
+Distributed WorkQueue], we will now use the natively supported https://www.rabbitmq.com/consumers.html#single-active-consumer[Single Active Consumer] mechanism.
+
+== Consequences
+
+* This solution is safer to use in production: if the active consumer dies, an other one is promoted instead.
+* This change needs RabbitMQ version to be at least 3.8.0.
+* The serial execution of tasks still does not leverage cluster scalability.
diff --git a/docs/modules/development/adr/src/adr/0017-file-mail-queue-deprecation.md.adoc b/docs/modules/development/adr/src/adr/0017-file-mail-queue-deprecation.md.adoc
new file mode 100644
index 0000000..ac38a93
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0017-file-mail-queue-deprecation.md.adoc
@@ -0,0 +1,43 @@
+= 17. FileMailQueue deprecation
+
+Date: 2019-12-04
+
+== Status
+
+Proposed
+
+== Context
+
+James offers several implementation for MailQueue, a component allowing asynchronous mail processing upon smtp mail reception.
+These includes:
+
+* Default embedded ActiveMQ mail queue implementation, leveraging the JMS APIs and using the filesystem.
+* RabbitMQMailQueue allowing several James instances to share their MailQueue content.
+* And FileMailQueue directly leveraging the file system.
+
+We introduced a junit5 test contract regarding management features, concurrency issues, and FileMailQueue do not meet this contract.
+This results in some tests being disabled and in an unstable test suite.
+
+FileMailQueue tries to implement a message queue within James code, which does not really makes sense as some other projects already provides one.
+
+== Decision
+
+Deprecate FileMailQueue components.
+
+Disable FileMailQueue tests.
+
+Target a removal as part of 3.6.0.
+
+== Consequences
+
+FileMailQueue is not exposed to the end user, be it over Spring or Guice, the impact of this deprecation + removal should be limited.
+
+We also expect our test suite to be more stable.
+
+== Reference
+
+Issues listing FileMailQueue defects:
+
+* https://issues.apache.org/jira/browse/JAMES-2298 Unsupported remove management feature
+* https://issues.apache.org/jira/browse/JAMES-2954 Incomplete browse implementation + Mixing concurrent operation might lead to a deadlock and missing fields
+* https://issues.apache.org/jira/browse/JAMES-2979 dequeue is not thread safe
diff --git a/docs/modules/development/adr/src/adr/0018-jmap-new-specs.md.adoc b/docs/modules/development/adr/src/adr/0018-jmap-new-specs.md.adoc
new file mode 100644
index 0000000..0fd12b8
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0018-jmap-new-specs.md.adoc
@@ -0,0 +1,65 @@
+= 18. New JMAP specifications adoption
+
+Date: 2020-02-06
+
+== Status
+
+Proposed
+
+== Context
+
+Historically, James has been an early adopter for the JMAP specification, and a first partial implementation was conducted when JMAP was just a draft.
+But with time, the IETF draft went with radical changes and the community could not keep this implementation up to date with the spec changes.
+
+As of summer 2019, JMAP core (https://tools.ietf.org/html/rfc8620[RFC 8620]) and JMAP mail (https://tools.ietf.org/html/rfc8621[RFC 8621]) have been officially published.
+Thus we should implement these new specifications to claim JMAP support.
+
+We need to keep in mind though that part of the community actively relies on the actual 'draft' implementation of JMAP existing in James.
+
+== Decision
+
+We decided to do as follow:
+
+* Rename packages `server/protocols/jmap*` and guice packages `server/container/guice/protocols/jmap*` to `jmap-draft`.
+`JMAPServer` should also be renamed to `JMAPDraftServer` (this has already been contributed https://github.com/apache/james-project/pull/164[here], thanks to @cketti).
+* Port `jmap-draft` to be served with a reactive technology
+* Implement a JMAP meta project to select the JMAP version specified in the accept header and map it to the correct implementation
+* Create a new `jmap` package
+* Implement the new JMAP request structure with the https://jmap.io/spec-core.html#the-coreecho-method[echo] method
+* Implement authentication and session of the new JMAP protocol
+* Implement protocol-level error handling
+* Duplicate and adapt existing mailbox methods of `jmap-draft` to `jmap`
+* Duplicate and adapt existing email methods of `jmap-draft` to `jmap`
+* Duplicate and adapt existing vacation methods of `jmap-draft` to `jmap`
+* Support uploads/downloads
+
+Then when we finish to port our existing methods to the new JMAP specifications, we can implement these new features:
+
+* Accounts
+* Identities
+* EmailSubmission
+* Push and queryChanges
+* Threads
+
+We decided to support `jmap` on top of memory-guice and distributed-james products for now.
+
+We should ensure no changes is done to `jmap-draft` while implementing the new `jmap` one.
+
+Regarding the versioning in the accept headers:
+
+* `Accept: application/json;jmapVersion=draft` would redirect to `jmap-draft`
+* `Accept: application/json;jmapVersion=rfc-8620` would redirect to `jmap`
+* When the `jmapVersion` is omitted, we will redirect first towards `jmap-draft`, then to `jmap` when `jmap-draft` becomes deprecated
+
+It's worth mentioning as well that we took the decision of writing this new implementation using `Scala`.
+
+== Consequences
+
+* Each feature implemented will respect the final specifications of JMAP
+* Getting missing features that are necessary to deliver a better mailing experience with James, like push, query changes and threads
+* Separating the current implementation from the new one will allow existing `jmap-draft` clients to smoothly transition to `jmap`, then trigger the classic "deprecation-then-removal" process.
+
+== References
+
+* A discussion around this already happened in September 2019 on the server-dev mailinglist: https://www.mail-archive.com/server-dev@james.apache.org/msg62072.html[JMAP protocol: Implementing RFC-8620 & RFC-8621]
+* JIRA: https://issues.apache.org/jira/browse/JAMES-2884[JAMES-2884]
diff --git a/docs/modules/development/adr/src/adr/0019-reactor-netty-adoption.md.adoc b/docs/modules/development/adr/src/adr/0019-reactor-netty-adoption.md.adoc
new file mode 100644
index 0000000..4b48e14
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0019-reactor-netty-adoption.md.adoc
@@ -0,0 +1,40 @@
+= 19. Reactor-netty adoption for JMAP server implementation
+
+Date: 2020-02-28
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+After adopting the last specifications of JMAP (see https://github.com/apache/james-project/blob/master/src/adr/0018-jmap-new-specs.md[new JMAP specifications adoption ADR]), it was agreed that we need to be able to serve both `jmap-draft` and the new `jmap` with a reactive server.
+
+The current outdated implementation of JMAP in James is currently using a non-reactive https://www.eclipse.org/jetty/[Jetty server].
+
+There are many possible candidates as reactive servers.
+Among the most popular ones for Java:
+
+* https://spring.io[Spring]
+* https://github.com/reactor/reactor-netty[Reactor-netty]
+* https://doc.akka.io/docs/akka-http/current/introduction.html[Akka HTTP]
+* ...
+
+== Decision
+
+We decide to use `reactor-netty` for the following reasons:
+
+* It's a reactive server
+* It's using https://projectreactor.io/[Reactor], which is the same technology that we use in the rest of our codebase
+* Implementing JMAP does not require high level HTTP server features
+
+== Consequences
+
+* Porting current `jmap-draft` to use a `reactor-netty` server instead of a Jetty server
+* The `reactor-netty` server should serve as well the new `jmap` implementation
+* We will be able to refactor and get end-to-end reactive operations for JMAP, unlocking performance gains
+
+== References
+
+* JIRA: https://issues.apache.org/jira/browse/JAMES-3078[JAMES-3078]
+* JMAP new specifications adoption ADR: https://github.com/apache/james-project/blob/master/src/adr/0018-jmap-new-specs.md
diff --git a/docs/modules/development/adr/src/adr/0020-cassandra-mailbox-object-consistency.md.adoc b/docs/modules/development/adr/src/adr/0020-cassandra-mailbox-object-consistency.md.adoc
new file mode 100644
index 0000000..e637748
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0020-cassandra-mailbox-object-consistency.md.adoc
@@ -0,0 +1,73 @@
+= 20. Cassandra Mailbox object consistency
+
+Date: 2020-02-27
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+Mailboxes are denormalized in Cassandra in order to access them both by their immutable identifier and their mailbox path (name):
+
+* `mailbox` table stores mailboxes by their immutable identifier
+* `mailboxPathV2` table stores mailboxes by their mailbox path
+
+We furthermore maintain two invariants on top of these tables:
+
+* *mailboxPath* unicity.
+Each mailbox path can be used maximum once.
+This is ensured by writing the mailbox path first using Lightweight Transactions.
+* *mailboxId* unicity.
+Each mailbox identifier is used by only a single path.
+We have no real way to ensure a given mailbox is not referenced by two paths.
+
+Failures during the denormalization process will lead to inconsistencies between the two tables.
+
+This can lead to the following user experience:
+
+----
+BOB creates mailbox A
+Denormalization fails and an error is returned to A
+
+BOB retries mailbox A creation
+BOB is being told mailbox A already exist
+
+BOB tries to access mailbox A
+BOB is being told mailbox A does not exist
+----
+
+== Decision
+
+We should provide an offline (meaning absence of user traffic via for exemple SMTP, IMAP or JMAP) webadmin task to solve mailbox object inconsistencies.
+
+This task will read `mailbox` table and adapt path registrations in `mailboxPathV2`:
+
+* Missing registrations will be added
+* Orphan registrations will be removed
+* Mismatch in content between the two tables will require merging the two mailboxes together.
+
+== Consequences
+
+As an administrator, if some of my users reports the bugs mentioned above, I have a way to sanitize my Cassandra mailbox database.
+
+However, due to the two invariants mentioned above, we can not identify a clear source of trust based on existing tables for the mailbox object.
+The task previously mentioned is subject to concurrency issues that might cancel legitimate concurrent user actions.
+
+Hence this task must be run offline (meaning absence of user traffic via for exemple SMTP, IMAP or JMAP).
+This can be achieved via reconfiguration (disabling the given protocols and restarting James) or via firewall rules.
+
+Due to all of those risks, a Confirmation header `I-KNOW-WHAT-I-M-DOING` should be positioned to `ALL-SERVICES-ARE-OFFLINE` in order to prevent accidental calls.
+
+In the future, we should revisit the mailbox object data-model and restructure it, to identify a source of truth to base the inconsistency fixing task on.
+Event sourcing is a good candidate for this.
+
+== References
+
+* https://issues.apache.org/jira/browse/JAMES-3058[JAMES-3058 Webadmin task to solve Cassandra Mailbox inconsistencies]
+* https://github.com/linagora/james-project/pull/3110[Pull Request: mailbox-cassandra utility to solve Mailbox inconsistency]
+* https://github.com/linagora/james-project/pull/3130[Pull Request: JAMES-3058 Concurrency testing for fixing Cassandra mailbox inconsistencies]
+
+This https://github.com/linagora/james-project/pull/3130#discussion_r383349596[thread] provides significant discussions leading to this Architecture Decision Record
+
+* https://www.mail-archive.com/server-dev@james.apache.org/msg64432.html[Discussion on the mailing list]
diff --git a/docs/modules/development/adr/src/adr/0021-cassandra-acl-inconsistency.md.adoc b/docs/modules/development/adr/src/adr/0021-cassandra-acl-inconsistency.md.adoc
new file mode 100644
index 0000000..3411f73
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0021-cassandra-acl-inconsistency.md.adoc
@@ -0,0 +1,63 @@
+= 21. Cassandra ACL inconsistencies
+
+Date: 2020-02-27
+
+== Status
+
+Proposed
+
+== Context
+
+Mailboxes ACLs are denormalized in Cassandra in order to:
+
+* given a mailbox, list its ACL (enforcing rights for example)
+* discover which mailboxes are delegated to a given user (used to list mailboxes)
+
+Here is the tables organisation:
+
+* `acl` stores the ACLs of a given mailbox
+* `UserMailboxACL` stores which mailboxes had been delegated to which user
+
+Failures during the denormalization process will lead to inconsistencies between the two tables.
+
+This can lead to the following user experience:
+
+----
+ALICE delegates her INBOX mailbox to BOB
+The denormalisation process fails
+ALICE INBOX does not appear in BOB mailbox list
+
+Given a delegated mailbox INBOX.delegated
+ALICE undo the sharing of her INBOX.delegated mailbox
+The denormalisation process fails
+ALICE INBOX.delegated mailbox still appears in BOB mailbox list
+When BOB tries to select it, he is being denied
+----
+
+== Decision
+
+We can adopt a retry policy of the `UserMailboxACL` projection update as a mitigation strategy.
+
+Using `acl` table as a source of truth, we can rebuild the `UserMailboxACL` projection:
+
+* Iterating `acl` entries, we can rewrite entries in `UserMailboxACL`
+* Iterating `UserMailboxACL` we can remove entries not referenced in `acl`
+* Adding a delay and a re-check before the actual fix can decrease the occurrence of concurrency issues
+
+We will expose a webAdmin task for doing this.
+
+== Consequences
+
+User actions concurrent to the inconsistency fixing task could result in concurrency issues.
+New inconsistencies could be created.
+However table of truth would not be impacted hence rerunning the inconsistency fixing task will eventually fix all issues.
+
+This task could be run safely online and can be scheduled on a recurring basis outside of peak traffic by an admin to ensure Cassandra acl consistency.
+
+== References
+
+* https://github.com/linagora/james-project/pull/3125[Plan for fixing Cassandra ACL inconsistencies]
+* https://www.mail-archive.com/server-dev@james.apache.org/msg64432.html[General mailing list discussion about inconsistencies]
+* https://github.com/linagora/james-project/pull/3130[Pull Request: JAMES-3058 Concurrency testing for fixing Cassandra mailbox inconsistencies]
+
+The delay strategy to decrease concurrency issue occurrence is described here.
diff --git a/docs/modules/development/adr/src/adr/0022-cassandra-message-inconsistency.md.adoc b/docs/modules/development/adr/src/adr/0022-cassandra-message-inconsistency.md.adoc
new file mode 100644
index 0000000..f30cff8
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0022-cassandra-message-inconsistency.md.adoc
@@ -0,0 +1,89 @@
+= 22. Cassandra Message inconsistencies
+
+Date: 2020-02-27
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+Messages are denormalized in Cassandra in order to:
+
+* access them by their unique identifier (messageId), for example through the JMAP protocol
+* access them by their mailbox identifier and Unique IDentifier within that mailbox (mailboxId + uid), for example through the IMAP protocol
+
+Here is the table organisation:
+
+* `messageIdTable` Holds mailbox and flags for each message, lookup by mailbox ID + UID
+* `imapUidTable` Holds mailbox and flags for each message, lookup by message ID
+
+Failures during the denormalization process will lead to inconsistencies between the two tables.
+
+This can lead to the following user experience:
+
+----
+BOB receives a message
+The denormalization process fails
+BOB can read the message via JMAP
+BOB cannot read the message via IMAP
+
+BOB marks a message as SEEN
+The denormalization process fails
+The message is SEEN in JMAP
+The message is UNSEEN in IMAP
+----
+
+=== Current operations
+
+* Adding a message:
+ ** (CassandraMessageMapper) First reference the message in `messageIdTable` then in `imapUidTable`.
+ ** (CassandraMessageIdMapper) First reference the message in `imapUidTable` then in `messageIdTable`.
+* Deleting a message:
+ ** (CassandraMessageMapper) First delete the message in `imapUidTable` then in `messageIdTable`.
+ ** (CassandraMessageIdMapper) Read the message metadata using `imapUidTable`, then first delete the message in `imapUidTable` then in `messageIdTable`.
+* Copying a message:
+ ** (CassandraMessageMapper) Read the message first, then first reference the message in `messageIdTable` then in `imapUidTable`.
+* Moving a message:
+ ** (CassandraMessageMapper) Logically copy then delete.
+A failure in the chain migh lead to duplicated message (present in both source and destination mailbox) as well as different view in IMAP/JMAP.
+ ** (CassandraMessageIdMapper) First reference the message in `imapUidTable` then in `messageIdTable`.
+* Updating a message flags:
+ ** (CassandraMessageMapper) First update conditionally the message in `imapUidTable` then in `messageIdTable`.
+ ** (CassandraMessageIdMapper) First update conditionally the message in `imapUidTable` then in `messageIdTable`.
+
+== Decision
+
+Adopt `imapUidTable` as a source of truth.
+Because `messageId` allows tracking changes to messages accross mailboxes upon copy and moves.
+Furthermore, that is the table on which conditional flags updates are performed.
+
+All writes will be performed to `imapUidTable` then performed on `messageIdTable` if successful.
+
+We thus need to modify CassandraMessageMapper 'add' + 'copy' to first write to the source of truth (`imapUidTable`)
+
+We can adopt a retry policy of the `messageIdTable` projection update as a mitigation strategy.
+
+Using `imapUidTable` table as a source of truth, we can rebuild the `messageIdTable` projection:
+
+* Iterating `imapUidTable` entries, we can rewrite entries in `messageIdTable`
+* Iterating `messageIdTable` we can remove entries not referenced in `imapUidTable`
+* Adding a delay and a re-check before the actual fix can decrease the occurrence of concurrency issues
+
+We will expose a webAdmin task for doing this.
+
+== Consequences
+
+User actions concurrent to the inconsistency fixing task could result in concurrency issues.
+New inconsistencies could be created.
+However table of truth would not be impacted hence rerunning the inconsistency fixing task will eventually fix all issues.
+
+This task could be run safely online and can be scheduled on a recurring basis outside of peak traffic by an admin to ensure Cassandra message consistency.
+
+== References
+
+* https://github.com/linagora/james-project/pull/3125[Plan for fixing Cassandra ACL inconsistencies]
+* https://www.mail-archive.com/server-dev@james.apache.org/msg64432.html[General mailing list discussion about inconsistencies]
+* https://github.com/linagora/james-project/pull/3130[Pull Request: JAMES-3058 Concurrency testing for fixing Cassandra mailbox inconsistencies]
+
+The delay strategy to decrease concurrency issue occurrence is described here.
diff --git a/docs/modules/development/adr/src/adr/0023-cassandra-mailbox-counters-inconsistencies.md.adoc b/docs/modules/development/adr/src/adr/0023-cassandra-mailbox-counters-inconsistencies.md.adoc
new file mode 100644
index 0000000..faff600
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0023-cassandra-mailbox-counters-inconsistencies.md.adoc
@@ -0,0 +1,58 @@
+= 23. Cassandra Mailbox Counters inconsistencies
+
+Date: 2020-03-07
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+Cassandra maintains a per mailbox projection for message count and unseen message count.
+
+As with any projection, it can go out of sync, leading to inconsistent results being returned to the client, which is not acceptable.
+
+Here is the table organisation:
+
+* `mailbox` Lists the mailboxes
+* `messageIdTable` Holds mailbox and flags for each message, lookup by mailbox ID + UID
+* `imapUidTable` Holds mailbox and flags for each message, lookup by message ID and serves as a source of truth
+* `mailboxCounters` Holds messages count and unseen message count for each mailbox.
+
+Failures during the denormalization process will lead to inconsistencies between the counts and the content of `imapUidTable`
+
+This can lead to the following user experience:
+
+* Invalid message count can be reported in the Mail User Agent (IMAP & JMAP)
+* Invalid message unseen count can be reported in the Mail User Agent (IMAP & JMAP)
+
+== Decision
+
+Implement a webadmin exposed task to recompute mailbox counters.
+
+This endpoints will:
+
+* List existing mailboxes
+* List their messages using `messageIdTable`
+* Check them against their source of truth `imapUidTable`
+* Compute mailbox counter values
+* And reset the value of the counter if needed in `mailboxCounters`
+
+== Consequences
+
+This endpoint is subject to data races in the face of concurrent operations.
+Concurrent increments & decrements will be ignored during a single mailbox processing.
+However the source of truth is unaffected hence, upon rerunning the task, the result will be eventually correct.
+To be noted that Cassandra counters can't be reset in an atomic manner anyway.
+
+We rely on the "listing messages by mailbox" projection (that we recheck).
+Missing entries in there will be ignored until the given projection is healed (currently unsupported).
+
+We furthermore can piggy back a partial check of the message denormalization described in xref:0021-cassandra-acl-inconsistency.adoc[this ADR] upon counter recomputation (partial because we cannot detect missing entries in the "list messages in mailbox" denormalization table)
+
+== References
+
+* https://github.com/linagora/james-project/pull/3125[Plan for fixing Cassandra ACL inconsistencies]
+* https://www.mail-archive.com/server-dev@james.apache.org/msg64432.html[General mailing list discussion about inconsistencies]
+* https://issues.apache.org/jira/browse/JAMES-3105[JAMES-3105 Related JIRA]
+* https://github.com/linagora/james-project/pull/3185[Pull Request: JAMES-3105 Corrective task for fixing mailbox counters]
diff --git a/docs/modules/development/adr/src/adr/0024-polyglot-strategy.md.adoc b/docs/modules/development/adr/src/adr/0024-polyglot-strategy.md.adoc
new file mode 100644
index 0000000..f00ed6f
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0024-polyglot-strategy.md.adoc
@@ -0,0 +1,179 @@
+= 24. Polyglot codebase
+
+Date: 2020-03-17
+
+== Status
+
+Proposed
+
+== Context & Problem Statement
+
+James is written in Java for a very long time.
+In recent years, Java modernized a lot after a decade of slow progress.
+
+However, in the meantime, most software relying on the JVM started supporting alternative JVM languages to keep being relevant.
+
+It includes Groovy, Clojure, Scala and more recently Kotlin, to name a few.
+
+Not being open to those alternative languages can be a problem for James adoption.
+
+== Decision drivers
+
+Nowadays, libraries and framework targeting the JVM are expected to support usage of one or several of these alternative languages.
+
+James being not only a mail server but also a development framework needs to reach those expectations.
+
+At the same time, more and more developers and languages adopt Function Programming (FP) idioms to solve their problems.
+
+== Considered options
+
+=== Strategies
+
+. Let the users figure out how to make polyglot setups
+. Document the usage of polyglot mailets for some popular languages
+. Document the usage of polyglot components for some popular languages
+. Actually implement some mailets in some popular languages
+. Actually implement some components in some popular languages
+
+=== Languages:
+
+[upperroman] I.
+Clojure II.
+Groovy III.
+Kotlin IV.
+Scala
+
+== Decision
+
+We decide for options 4, 5 and IV.
+
+That means we need to write some mailets in Scala and demonstrate how it's done and then used in a running server.
+
+It also means writing and/or refactoring some server components in Scala, starting where it's the most relevant.
+
+### Positive Consequences
+
+* Modernize parts of James code
+* Leverage Scala richer FP ecosystem and language to overcome Java limitations on that topic
+* Should attract people that would not like Java
+
+### Negative Consequences
+
+* Adds even more knowledge requirements to contribute to James
+* Scala build time is longer than Java build time
+
+== Pros and Cons of the Options
+
+=== Option 1: Let the users figure out how to make polyglot setups
+
+Pros:
+
+* We don't have anything new to do
+
+Cons:
+
+* It's boring, we like new challenges
+* Java is declining despite language modernization, it means in the long term, less and less people will contribute to James
+
+=== Option 2: Document the usage of polyglot mailets for some popular languages
+
+Pros:
+
+* It's not a lot of work and yet it opens James to alternatives and can attract people outside Java developers community
+
+Cons:
+
+* Documentation without implementation often gets outdated when things move forward
+* We don't really gain knowledge on the polyglot matters as a community and won't be able to help users much
+
+=== Option 3: Document the usage of polyglot components for some popular languages
+
+Pros:
+
+* It opens James to alternatives and can attract people outside Java developers community
+
+Cons:
+
+* Documentation without implementation often gets outdated when things move forward
+* For such complex subject it's probably harder to do than actually implementing a component in another language
+* We don't really gain knowledge on the polyglot matters as a community and won't be able to help users much
+
+### Option 4: Actually implement some mailets in some popular languages
+
+Pros:
+
+* It's probably not a lot of work, a mailet is just a simple class, probably easy to do in most JVM language
+* It makes us learn how it works and maybe will help us go further than the basic polyglot experience by doing small enhancements to the codebase
+* We can document the process and illustrate with some actual code
+* It opens James to alternatives and can attract people outside Java developers community
+
+Cons:
+
+* It can have a negative impact on the build time and dependencies download
+
+### Option 5: Actually implement some components in some popular languages
+
+Pros:
+
+* Leverage a modern language for some complex components
+* It makes us learn how it works and maybe will help us go further than the basic polyglot experience by doing small enhancements to the codebase
+* We can document the process and illustrate with some actual code
+* It opens James to alternatives and can attract people outside Java developers community
+
+Cons:
+
+* It makes the codebase more complex, requiring knowledge in another language
+* It can have a negative impact on the build time and dependencies download
+
+=== Option I: Clojure
+
+Pros:
+
+* Functional Language
+
+Cons:
+
+* Weak popularity
+* No prior experience among current active commiters
+* Not statically typed hence less likely to fit the size of the project
+
+=== Option II: Groovy
+
+Pros:
+
+* More advanced than current Java on most topics
+
+Cons:
+
+* No prior experience among current active commiters
+* Not very FP
+* Replaced in JVM community by Kotlin last years
+
+### Option III.
+Kotlin
+
+Pros:
+
+* Great Intellij support
+* Most of the good parts of Scala
+* FP-ish with Arrow
+* Coroutine for handling high-performance IOs
+
+Cons:
+
+* No prior experience among current active commiters
+* Lack of some FP constructs like proper Pattern Matching, persistent collections
+* Despite progress done by Arrow, Kotlin community aims mostly at writing "better Java"
+
+==== Option IV. Scala
+
+Pros:
+
+* Rich FP community and ecosystem
+* Existing knowledge among current active commiters
+
+Cons:
+
+* Needs work to master
+* Can be slow to build
+* 3.0 will probably require code changes
diff --git a/docs/modules/development/adr/src/adr/0025-cassandra-blob-store-cache.md.adoc b/docs/modules/development/adr/src/adr/0025-cassandra-blob-store-cache.md.adoc
new file mode 100644
index 0000000..ef8317c
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0025-cassandra-blob-store-cache.md.adoc
@@ -0,0 +1,69 @@
+= 25. Cassandra Blob Store Cache
+
+Date: 2020-04-03
+
+== Status
+
+Proposed
+
+Supercedes xref:0014-blobstore-storage-policies.adoc[14.
+Add storage policies for BlobStore]
+
+== Context
+
+James exposes a simple BlobStore API for storing raw data.
+However such raw data often vary in size and access patterns.
+
+As an example:
+
+* Mailbox message headers are expected to be small and frequently accessed
+* Mailbox message body are expected to have sizes ranging from small to big but are unfrequently accessed
+* DeletedMessageVault message headers are expected to be small and unfrequently accessed
+
+The access pattern of some of these kind of blobs does not fit Object Storage characteristics: good at storing big blobs, but it induces high latencies for reading small blobs.
+We observe latencies of around 50-100ms while Cassandra latency is of 4ms.
+
+This gets some operations slow (for instance IMAP FETCH headers, or listing JMAP messages).
+
+== Decision
+
+Implement a write through cache to have better read latency for smaller objects.
+
+Such a cache needs to be distributed in order to be more efficient.
+
+Given that we don't want to introduce new technologies, we will implement it using Cassandra.
+
+The cache should be implemented as a key-value table on a dedicated 'cache' keyspace, with a replication factor of 1, and be queried with a consistency level of ONE.
+
+We will leverage a configurable TTL as an eviction policy.
+Cache will be populated upon writes and missed read, if the blob size is below a configurable threashold.
+We will use the TimeWindow compaction strategy.
+
+Failure to read the cache, or cache miss will result in a read in the object storage.
+
+== Consequences
+
+Metadata queries are expected not to query the object storage anymore.
+
+https://github.com/linagora/james-project/pull/3031#issuecomment-572865478[Performance tests] proved such strategies to be highly effective.
+We expect comparable performance improvements compared to an un-cached ObjectStorage blob store.
+
+HybridBlobStore should be removed.
+
+== Alternatives
+
+xref:0014-blobstore-storage-policies.adoc[14.
+Add storage policies for BlobStore] proposes to use the CassandraBlobStore to mimic a cache.
+
+This solution needed further work as we decided to add an option to write all blobs to the object storage in order:
+
+* To get a centralized source of truth
+* Being able to instantly rollback Hybrid blob store adoption
+
+See https://github.com/linagora/james-project/pull/3162[this pull request]
+
+With such a proposal there is no eviction policy.
+Also, the storage is done on the main keyspace with a high replication factor, and QUORUM consistency level (high cost).
+
+To be noted, as cached entries are small, we can assume they are small enough to fit in a single Cassandra row.
+This is more optimized than the large blob handling through blobParts the CassandraBlobStore is doing.
diff --git a/docs/modules/development/adr/src/adr/0026-removing-configured-additional-mailboxListeners.md.adoc b/docs/modules/development/adr/src/adr/0026-removing-configured-additional-mailboxListeners.md.adoc
new file mode 100644
index 0000000..aa48fc0
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0026-removing-configured-additional-mailboxListeners.md.adoc
@@ -0,0 +1,72 @@
+= 26. Removing a configured additional MailboxListener
+
+Date: 2020-04-03
+
+== Status
+
+Accepted (lazy consensus)
+
+Superceded by xref:0035-distributed-listeners-configuration.adoc[34.
+Distributed Mailbox Listener Configuration]
+
+== Context
+
+James enables a user to register additional mailbox listeners.
+
+The distributed James server is handling mailbox event processing (mailboxListener execution) using a RabbitMQ work-queue per listener.
+
+The distributed James server then declares a queue upon start for each one of these user registered listeners, that it binds to the main event exchange.
+
+More information about this component, and its distributed, RabbitMQ based implementation, can be found in xref:0037-eventbus.adoc[ADR 0036].
+
+If the user unconfigures the listener, the queue and the binding are still present but not consumed.
+This results in unbounded queue growth eventually causing RabbitMQ resource exhaustion and failure.
+
+== Vocabulary
+
+A *required group* is a group configured within James additional mailbox listener or statically binded via Guice.
+We should have a queue for that mailbox listener binded to the main exchange.
+
+A *registered group* is a group whose queue exists in RabbitMQ and is bound to the exchange, independently of its James usage.
+If it is required, a consumer will consume the queue.
+Otherwise the queue might grow unbounded.
+
+== Decision
+
+We need a clear consensus and auditability across the James cluster about *required groups* (and their changes).
+Thus Event sourcing will maintain an aggregate tracking *required groups* (and their changes).
+Audit will be enabled by adding host and date information upon changes.
+A subscriber will perform changes (binds and unbinds) in registered groups following the changes of the aggregate.
+
+Event sourcing is desirable as it allows:
+
+* Detecting previously removed MailboxListener upon start
+* Audit of unbind decisions
+* Enables writing more complex business rules in the future
+
+The event sourcing system will have the following command:
+
+* *RequireGroups* the groups that the *EventBus* is starting with.
+
+And the following events:
+
+* *RequiredGroupAdded* a group is added to the required groups.
+* *RequiredGroupRemoved* a group is removed from the required groups.
+
+Upon start the aggregate will be updated if needed and bindings will be adapted accordingly.
+
+Note that upon failure, registered groups will diverge from required groups.
+We will add a health check to diagnose such issues.
+Eventually, we will expose a webadmin task to reset registered groups to required groups.
+
+The queues should not be deleted to prevent message loss.
+
+Given a James topology with a non uniform configuration, the effective RabbitMQ routing will be the one of the latest started James server.
+
+== Alternatives
+
+We could also consider adding a webadmin endpoint to sanitize eventBus bindings, allowing more predictability than the above solution but it would require admin intervention.
+
+== References
+
+* https://github.com/linagora/james-project/pull/3280[Discussion] around the overall design proposed here.
diff --git a/docs/modules/development/adr/src/adr/0027-eventBus-error-handling-upon-dispatch.md.adoc b/docs/modules/development/adr/src/adr/0027-eventBus-error-handling-upon-dispatch.md.adoc
new file mode 100644
index 0000000..64b89f2
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0027-eventBus-error-handling-upon-dispatch.md.adoc
@@ -0,0 +1,35 @@
+= 27. EventBus error handling upon dispatch
+
+Date: 2020-04-03
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+James allows asynchronous processing for mailbox events via MailboxListener.
+This processing is abstracted by the EventBus.
+
+If the processing of an event via a mailbox listener fails, it is retried, until it succeeds.
+If a maxRetries parameter is exceeded, the event is stored in deadLetter and no further processing is attended.
+
+The administrator can then look at the content of deadLetter to diagnose processing issues and schedule a reDelivery in order to retry their processing via webAdmin APIs.
+
+However no such capabilities are supported upon dispatching the event on the eventbus.
+A failed dispatch will result in message loss.
+
+More information about this component can be found in xref:0037-eventbus.adoc[ADR 0036].
+
+== Decision
+
+Upon dispatch failure, the eventBus should save events in dead letter using a dedicated group.
+
+Reprocessing this group an admin can re-trigger these events dispatch.
+
+In order to ensure auto healing, James will periodically check the corresponding group in deadLetter is empty.
+If not a re-dispatching of these events will be attempted.
+
+== Consequence
+
+In distributed James Guice project an administrator have a way to be eventually consistent upon rabbitMQ failure.
diff --git a/docs/modules/development/adr/src/adr/0028-Recompute-mailbox-quotas.md.adoc b/docs/modules/development/adr/src/adr/0028-Recompute-mailbox-quotas.md.adoc
new file mode 100644
index 0000000..75cdd6b
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0028-Recompute-mailbox-quotas.md.adoc
@@ -0,0 +1,46 @@
+= 28. Recompute mailbox quotas
+
+Date: 2020-04-03
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+JMAP custom quota extension, as well as IMAP https://tools.ietf.org/html/rfc2087[RFC-2087] enables a user to monitor the amount of space and message count he is allowed to use, and that he is effectively using.
+
+To track the quota values a user is effectively using, James relies on the link:../site/markdown/server/manage-guice-distributed-james.md#mailbox-event-bus[eventBus] to increment a Cassandra counter corresponding to this user.
+
+However, upon Cassandra failure, this value can be incorrect, hence the need of correcting it.
+
+== Data model details
+
+Table: imapUidTable: Holds mailbox and flags for each message, lookup by message ID
+
+Table: messageV2: Holds message metadata, independently of any mailboxes.
+Content of messages is stored in `blobs` and `blobparts` tables.
+
+Table: currentQuota: Holds per quota-root current values.
+Quota-roots defines groups of mailboxes which share quotas limitations.
+
+Operation:
+
+* Quota updates are done asynchronously (event bus + listener) for successful mailbox operations.
+ ** If the quota update is not applied, then we are inconsistent
+ ** EventBus errors are retried upon errors, counters being non-indempotent, this can result in inconsistent quotas
+
+== Decision
+
+We will implement a generic corrective task exposed via webadmin.
+
+This task can reuse the `CurrentQuotaCalculator` and call it for each and every quotaRoot of each user.
+
+This way, non-Cassandra implementation will also benefit from this task.
+
+== Consequences
+
+This task is not concurrent-safe.
+Concurrent operations will result in an invalid quota to be persisted.
+
+However, as the source of truth is not altered, re-running this task will eventually return the correct result.
diff --git a/docs/modules/development/adr/src/adr/0029-Cassandra-mailbox-deletion-cleanup.md.adoc b/docs/modules/development/adr/src/adr/0029-Cassandra-mailbox-deletion-cleanup.md.adoc
new file mode 100644
index 0000000..e89ce54
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0029-Cassandra-mailbox-deletion-cleanup.md.adoc
@@ -0,0 +1,46 @@
+= 29. Cassandra mailbox deletion cleanup
+
+Date: 2020-04-12
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+Cassandra is used within distributed James product to hold messages and mailboxes metadata.
+
+Cassandra holds the following tables:
+
+* mailboxPathV2 + mailbox allowing to retrieve mailboxes informations
+* acl + UserMailboxACL hold denormalized information
+* messageIdTable & imapUidTable allow to retrieve mailbox context information
+* messageV2 table holds message metadata
+* attachmentV2 holds attachments for messages
+* References to these attachments are contained within the attachmentOwner and attachmentMessageId tables
+
+Currently, the deletion only deletes the first level of metadata.
+Lower level metadata stay unreachable.
+The data looks deleted but references are actually still present.
+
+Concretely:
+
+* Upon mailbox deletion, only mailboxPathV2 & mailbox content is deleted.
+messageIdTable, imapUidTable, messageV2, attachmentV2 & attachmentMessageId metadata are left undeleted.
+* Upon mailbox deletion, acl + UserMailboxACL are not deleted.
+* Upon message deletion, only messageIdTable & imapUidTable content are deleted.
+messageV2, attachmentV2 & attachmentMessageId metadata are left undeleted.
+
+This jeopardize efforts to regain disk space and privacy, for example through blobStore garbage collection.
+
+== Decision
+
+We need to cleanup Cassandra metadata.
+They can be retrieved from dandling metadata after the delete operation had been conducted out.
+We need to delete the lower levels first so that upon failures undeleted metadata can still be reached.
+
+This cleanup is not needed for strict correctness from a MailboxManager point of view thus it could be carried out asynchronously, via mailbox listeners so that it can be retried.
+
+== Consequences
+
+Mailbox listener failures lead to eventBus retrying their execution, we need to ensure the result of the deletion to be idempotent.
diff --git a/docs/modules/development/adr/src/adr/0030-separate-attachment-content-and-metadata.md.adoc b/docs/modules/development/adr/src/adr/0030-separate-attachment-content-and-metadata.md.adoc
new file mode 100644
index 0000000..3bbdb89
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0030-separate-attachment-content-and-metadata.md.adoc
@@ -0,0 +1,94 @@
+= 30. Separate attachment content and metadata
+
+Date: 2020-04-13
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+Some mailbox implementations of James store already parsed attachments for faster retrieval.
+
+This attachment storage capabilities are required for two features:
+
+* JMAP attachment download
+* JMAP message search "attachment content" criteria
+
+Only Memory and Cassandra backends can be relied upon as a JMAP backend.
+
+Other protocols relies on dynamic EML parsing to expose message subparts (IMAP)
+
+Here are the POJOs related to these attachments:
+
+* *Attachment* : holds an attachmentId, the attachment content, as well as the content type
+* *MessageAttachment* : composes an attachment with its disposition within a message (cid, inline and name)
+* *Message* exposes its list of MessageAttachment when it is read with FetchType Full.
+* *Blob* represents some downloadable content, and can be either an attachment or a message.
+Blob has a byte array payload too.
+
+The following classes work with the aforementioned POJOs:
+
+* *AttachmentMapper* and *AttachmentManager* are responsible of storing and retrieving an attachment content.
+* *BlobManager* is used by JMAP to allow blob downloads.
+* Mailbox search exposes attachment content related criteria.
+These criteria are used by the JMAP protocol.
+
+This organisation causes attachment content to be loaded every time a message is fully read (which happens for instance when you open a message using JMAP) despite the fact that it is not needed, as attachments are downloadable through a separate JMAP endpoint, their content is not attached to the JMAP message JSON.
+
+Also, the content being loaded "at once", we allocate memory space to store the whole attachment, which is sub-optimal.
+We want to keep the consumed memory low per-message because a given server should be able to handle a high number of messages at a given time.
+
+To be noted that JPA and maildir mailbox implementations do not support attachment storage.
+To retrieve attachments of a message, these implementations parse the messages to extract their attachments.
+
+Cassandra mailbox prior schema version 4 stored attachment and its metadata in the same table, but from version 5 relies on the blobStore to store the attachment content.
+
+== Decision
+
+Enforce cassandra schema version to be 5 from James release 3.5.0.
+This allows to drop attachment management prior version 5.
+
+We will re-organize the attachment POJOs:
+
+* *Attachment* should hold an attachmentId, a content type, and a size.
+It will no longer hold the content.
+The content can be loaded from its *AttachmentId* via the *AttachmentLoader* API that the *AttachmentManager* implements.
+* *MessageAttachment* : composes an attachment with its disposition within a message (cid, inline and name)
+* *Blob* would no longer hold the content as a byte array but rather a content retriever (`Supplier<InputStream>`)
+* *ParsedAttachment* is the direct result of attachment parsing, and composes a *MessageAttachment* and the corresponding content as byte array.
+This class is only relied upon when saving a message in mailbox.
+This is used as an output of `MessageParser`.
+
+Some adjustments are needed on class working with attachment:
+
+* *AttachmentMapper* and *AttachmentManager* need to allow from an attachmentId to retrieve the attachment content as an `InputStream`.
+This is done through a separate `AttachmentLoader` interface.
+* *AttachmentMapper* and *AttachmentManager* need the Attachment and its content to persist an attachment
+* *MessageManager* then needs to return attachment metadata as a result of Append operation.
+* *InMemoryAttachmentMapper* needs to store attachment content separately.
+* *MessageStorer* will take care of storing a message on the behalf of `MessageManager`.
+This enables to determine if attachment should be parsed or not on an implementation aware fashion, saving attachment parsing upon writes for JPA and Maildir.
+
+Maildir and JPA no longer support attachment content loading.
+Only the JMAP protocol requires attachment content loading, which is not supported on top of these technologies.
+
+Mailbox search attachment content criteria will be supported only on implementation supporting attachment storage.
+
+== Consequences
+
+Users running Cassandra schema version prior version 5 will have to go through James release 3.5.0 to upgrade to a version after version 5 before proceeding with their update.
+
+We noticed performance enhancement when using IMAP FETCH and JMAP GetMessages.
+Running a gatling test suite exercising JMAP getMessages on a dataset containing attachments leads to the following observations:
+
+* Overall better average performance for all JMAP queries (10% global p50 improvement)
+* Sharp decrease in tail latency of getMessages (x40 time faster)
+
+We also expect improvements in James memory allocation.
+
+== References
+
+* https://github.com/linagora/james-project/pull/3061[Contribution on this topic].
+Also contains benchmark for this proposal.
+* https://issues.apache.org/jira/browse/JAMES-2997[JIRA]
diff --git a/docs/modules/development/adr/src/adr/0031-distributed-mail-queue.md.adoc b/docs/modules/development/adr/src/adr/0031-distributed-mail-queue.md.adoc
new file mode 100644
index 0000000..62050e9
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0031-distributed-mail-queue.md.adoc
@@ -0,0 +1,122 @@
+= 31. Distributed Mail Queue
+
+Date: 2020-04-13
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+MailQueue is a central component of SMTP infrastructure allowing asynchronous mail processing.
+This enables a short SMTP reply time despite a potentially longer mail processing time.
+It also works as a buffer during SMTP peak workload to not overload a server.
+
+Furthermore, when used as a Mail Exchange server (MX), the ability to add delays to be observed before dequeing elements allows, among others:
+
+* Delaying retries upon MX delivery failure to a remote site.
+* Throttling, which could be helpful for not being considered a spammer.
+
+A mailqueue also enables advanced administration operations like traffic review, discarding emails, resetting wait delays, purging the queue, etc.
+
+Spring implementation and non distributed implementations rely on an embedded ActiveMQ to implement the MailQueue.
+Emails are being stored in a local file system.
+An administrator wishing to administrate the mailQueue will thus need to interact with all its James servers, which is not friendly in a distributed setup.
+
+Distributed James relies on the following third party softwares (among other):
+
+* *RabbitMQ* for messaging.
+Good at holding a queue, however some advanced administrative operations can't be implemented with this component alone.
+This is the case for `browse`, `getSize` and `arbitrary mail removal`.
+* *Cassandra* is the metadata database.
+Due to *tombstone* being used for delete, queue is a well known anti-pattern.
+* *ObjectStorage* (Swift or S3) holds byte content.
+
+== Decision
+
+Distributed James should ship a distributed MailQueue composing the following softwares with the following responsibilities:
+
+* *RabbitMQ* for messaging.
+A rabbitMQ consumer will trigger dequeue operations.
+* A time series projection of the queue content (order by time list of mail metadata) will be maintained in *Cassandra* (see later).
+Time series avoid the aforementioned tombstone anti-pattern, and no polling is performed on this projection.
+* *ObjectStorage* (Swift or S3) holds large byte content.
+This avoids overwhelming other softwares which do not scale as well in term of Input/Output operation per seconds.
+
+Here are details of the tables composing Cassandra MailQueue View data-model:
+
+* *enqueuedMailsV3* holds the time series.
+The primary key holds the queue name, the (rounded) time of enqueue designed as a slice, and a bucketCount.
+Slicing enables listing a large amount of items from a given point in time, in an fashion that is not achievable with a classic partition approach.
+The bucketCount enables sharding and avoids all writes at a given point in time to go to the same Cassandra partition.
+The clustering key is composed of an enqueueId - a unique identifier.
+The content holds the metadata of the email.
+This table enables, from a starting date, to load all of the emails that have ever been in the mailQueue.
+Its content is never deleted.
+* *deletedMailsV2* tells wether a mail stored in _enqueuedMailsV3_ had been deleted or not.
+The queueName and enqueueId are used as primary key.
+This table is updated upon dequeue and deletes.
+This table is queried upon dequeue to filter out deleted/purged items.
+* *browseStart* store the latest known point in time from which all previous emails had been deleted/dequeued.
+It enables to skip most deleted items upon browsing/deleting queue content.
+Its update is probability based and asynchronously piggy backed on dequeue.
+
+Here are the main mail operation sequences:
+
+* Upon *enqueue* mail content is stored in the _object storage_, an entry is added in _enqueuedMailsV3_ and a message is fired on _rabbitMQ_.
+* *dequeue* is triggered by a rabbitMQ message to be received.
+_deletedMailsV2_ is queried to know if the message had already been deleted.
+If not, the mail content is retrieved from the _object storage_, then an entry is added in _deletedMailsV2_ to notice the email had been dequeued.
+A dequeue has a random probability to trigger a browse start update.
+If so, from current browse start, _enqueuedMailsV3_ content is iterated, and checked against _deletedMailsV2_ until the first non deleted / dequeued email is found.
+This point becomes the new browse start.
+BrowseStart can never point after the start of the current slice.
+A grace period upon browse start update is left to tolerate clock skew.
+Update of the browse start is done randomly as it is a simple way to avoid synchronisation in a distributed system: we ensure liveness while uneeded browseStart updates being triggered would simply waste a few resources.
+* Upon *browse*, _enqueuedMailsV3_ content is iterated, and checked against _deletedMailsV2_, starting from the current browse start.
+* Upon *delete/purge*, _enqueuedMailsV3_ content is iterated, and checked against _deletedMailsV2_.
+Mails matching the condition are marked as deleted in _enqueuedMailsV3_.
+* Upon *getSize*, we perform a browse and count the returned elements.
+
+The distributed mail queue requires a fine tuned configuration, which mostly depends of the count of Cassandra servers, and of the mailQueue throughput:
+
+* *sliceWindow* is the time period of a slice.
+All the elements of *enqueuedMailsV3* sharing the same slice are retrieved at once.
+The bigger, the more elements are going to be read at once, the less frequent browse start update will be.
+Lower values might result in many almost empty slices to be read, generating higher read load.
+We recommend *sliceWindow* to be chosen from users maximum throughput so that approximately 10.000 emails be contained in a slice.
+Only values dividing the current _sliceWindow_ are allowed as new values (otherwize previous slices might not be found).
+* *bucketCount* enables spreading the writes in your Cassandra cluster using a bucketting strategy.
+Low values will lead to workload not to be spread evenly, higher values might result in uneeded reads upon browse.
+The count of Cassandra servers should be a good starting value.
+Only increasing the count of buckets is supported as a configuration update as decreasing the bucket count might result in some buckets to be lost.
+* *updateBrowseStartPace* governs the probability of updating browseStart upon dequeue/deletes.
+We recommend choosing a value guarantying a reasonable probability of updating the browse start every few slices.
+Too big values will lead to uneeded update of not yet finished slices.
+Too low values will end up in a more expensive browseStart update and browse iterating through slices with all their content deleted.
+This value can be changed freely.
+
+We rely on eventSourcing to validate the mailQueue configuration changes upon James start following the aforementioned rules.
+
+== Limitations
+
+Delays are not supported.
+This mail queue implementation is thus not suited for a Mail Exchange (MX) implementation.
+The https://issues.apache.org/jira/browse/JAMES-2896[following proposal] could be a solution to support delays.
+
+*enqueuedMailsV3* and *deletedMailsV2* is never cleaned up and the corresponding blobs are always referenced.
+This is not ideal both from a privacy and space storage costs point of view.
+
+*getSize* operation is sub-optimal and thus not efficient.
+Combined with metric reporting of mail queue size being periodically performed by all James servers this can lead, upon increasing throughput to a Cassandra overload.
+A configuration parameter allows to disable mail queue size reporting as a temporary solution.
+Some alternatives had been presented like https://github.com/linagora/james-project/pull/2565[an eventually consistent per slice counters approach].
+An other proposed solution is https://github.com/linagora/james-project/pull/2325[to rely on RabbitMQ management API to retrieve mail queue size] however by design it cannot take into account purge/delete operations.
+Read https://issues.apache.org/jira/browse/JAMES-2733[the corresponding JIRA].
+
+== Consequences
+
+Distributed mail queue allows a better spreading of Mail processing workload.
+It enables a centralized mailQueue management for all James servers.
+
+Yet some additional work is required to use it as a Mail Exchange scenario.
diff --git a/docs/modules/development/adr/src/adr/0032-distributed-mail-queue-cleanup.md.adoc b/docs/modules/development/adr/src/adr/0032-distributed-mail-queue-cleanup.md.adoc
new file mode 100644
index 0000000..cc8e611
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0032-distributed-mail-queue-cleanup.md.adoc
@@ -0,0 +1,50 @@
+= 32. Distributed Mail Queue Cleanup
+
+Date: 2020-04-13
+
+== Status
+
+Proposed
+
+== Context
+
+Read xref:0031-distributed-mail-queue.adoc[Distributed Mail Queue] for full context.
+
+*enqueuedMailsV3* and *deletedMailsV2* is never cleaned up and the corresponding blobs are always referenced.
+This is not ideal both from a privacy and space storage costs point of view.
+
+Note that *enqueuedMailsV3* and *deletedMailsV2* rely on timeWindowCompactionStrategy.
+
+== Decision
+
+Add a new `contentStart` table referencing the point in time from which a given mailQueue holds data, for each mail queue.
+
+The values contained between `contentStart` and `browseStart` can safely be deleted.
+
+We can perform this cleanup upon `browseStartUpdate`: once finished we can browse then delete content of *enqueuedMailsV3* and *deletedMailsV2* contained between `contentStart` and the new `browseStart` then we can safely set `contentStart` to the new `browseStart`.
+
+Content before `browseStart` can safely be considered deletable, and is applicatively no longer exposed.
+We don't need an additional grace period mechanism for `contentStart`.
+
+Failing cleanup will lead to the content being eventually updated upon next `browseStart` update.
+
+We will furthermore delete blobStore content upon dequeue, also when the mail had been deleted or purged via MailQueue management APIs.
+
+== Consequences
+
+All Cassandra SSTable before `browseStart` can safely be dropped as part of the timeWindowCompactionStrategy.
+
+Updating browse start will then be two times more expensive as we need to unreference passed slices.
+
+Eventually this will allow reclaiming Cassandra disk space and enforce mail privacy by removing dandling metadata.
+
+== Alternative
+
+A https://github.com/linagora/james-project/pull/3291#pullrequestreview-393501339[proposal] was made to piggy back cleanup upon dequeue/delete operations.
+The dequeuer/deleter then directly removes the related metadata from `enqueuedMailsV3` and `deletedMailsV2`.
+This simpler design however have several flaws:
+
+* if the cleanup fails for any reason then it cannot be retried in the future.
+There will be no way of cleaning up the related data.
+* this will end up tumbstoning live slices potentially harming browse/delete/browse start updates performance.
+* this proposition don't leverage as efficiently timeWindowCompactionStrategy.
diff --git a/docs/modules/development/adr/src/adr/0033-use-scala-in-event-sourcing-modules.md.adoc b/docs/modules/development/adr/src/adr/0033-use-scala-in-event-sourcing-modules.md.adoc
new file mode 100644
index 0000000..7b536bb
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0033-use-scala-in-event-sourcing-modules.md.adoc
@@ -0,0 +1,33 @@
+= 33. Use scala in event sourcing modules
+
+Date: 2019-12-13
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+At the time being James use the scala programming language in some parts of its code base, particularily for implementing the Distributed Task Manager, which uses the event sourcing modules.
+
+The module `event-store-memory` already uses Scala.
+
+== Decision
+
+What is proposed here, is to convert in Scala the event sourcing modules.
+The modules concerned by this change are:
+
+* `event-sourcing-core`
+* `event-sourcing-pojo`
+* `event-store-api`
+* `event-store-cassandra`
+
+== Rationales
+
+This will help to standardize the `event-*` modules as `event-store-memory` is already written in Scala.
+This change will avoid interopability concerns with the main consumers of those modules which are already written in Scala: see the distributed task manager.
+In the long run this will allow to have a stronger typing in those parts of the code and to have a much less verbose code.
+
+== Consequences
+
+We will have to mitigate the pervading of the Scale API in the Java code base by implementing Java facade.
diff --git a/docs/modules/development/adr/src/adr/0034-mailbox-api-visibility-and-usage.md.adoc b/docs/modules/development/adr/src/adr/0034-mailbox-api-visibility-and-usage.md.adoc
new file mode 100644
index 0000000..626f3c2
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0034-mailbox-api-visibility-and-usage.md.adoc
@@ -0,0 +1,49 @@
+= 34. Mailbox API visibility and usage
+
+Date: 2020-04-27
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+All mailboxes implementations rely on `mailbox-store` module that defines some common tools to implement the `mailbox-api` (representing the API defining how to use a mailbox).
+
+For example, a `CassandraMailboxmanager` has to extend `StoreMailboxManager` (that implements `Mailboxmanager` from the `mailbox-api`) that requires the implementation of some ``Mapper``s.
+
+``Mapper``s are designed to provide low-level functions and methods on mailboxes.
+It's recurrent that we are tempted in James, outside of the `mailbox` modules, to rely on some of those common tools located in `mailbox-store` to have an easier access on some user's mailboxes or messages.
+
+Like for example, using a `Mapper` outside to be able to retrieve a message with only its `MessageId`, which is not currently possible at the ``Manager``'s level, which tends to violate ``mailbox-api``'s role and primary mission.
+
+As a matter of fact, we have currently such uses of `mailbox-store` in James:
+
+* `mailbox-adapter` because `Authenticator` and `Authorizator` are part of the `mailbox-store`
+
+The manager layer do further validation including right checking, event dispatching (resulting in message search index indexing, current quota calculation mainly), quota validation.
+Not relying on the manager layer is thus error prone and can lead to security vulnerabilities.
+
+== Decision
+
+We should never rely on classes defined in `mailbox-store` outside of the `mailbox` modules (except on some cases limited to the test scope).
+The right way would be to always rely on the ``Manager``s defined in `mailbox-api` module to access mailboxes and messages, as the `mailbox-api` module defines the API on how to use a mailbox.
+
+We should ensure the correctness of ``Manager``s implementations by providing contract tests and not by sharing abstract classes.
+
+Regarding the modules wrongly relying already on `mailbox-store`, we can:
+
+* `mailbox-adapter`: move `Authenticator` and `Authorizator` to `mailbox-api`
+
+== Consequences
+
+We need to introduce some refactorings to be able to rely fully on `mailbox-api` in new emerging cases.
+For example, our `mailbox-api` still lacks APIs to handle messages by their MessageId.
+It creates some issues for rebuilding a single message fast view projection, or the reindexation of a single message.
+
+A refactoring of the session would be thus necessary to bypass such limitation access on a single message without knowing its user from the `mailbox-api` module.
+
+== References
+
+* https://github.com/linagora/james-project/pull/3035#discussion_r363684700[Discussions around rebuild a single message fast view projection]
+* https://www.mail-archive.com/server-dev@james.apache.org/msg64120.html[General mailing list discussion on the session refactoring]
diff --git a/docs/modules/development/adr/src/adr/0035-distributed-listeners-configuration.md.adoc b/docs/modules/development/adr/src/adr/0035-distributed-listeners-configuration.md.adoc
new file mode 100644
index 0000000..025077f
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0035-distributed-listeners-configuration.md.adoc
@@ -0,0 +1,137 @@
+= 35. Distributed Mailbox Listeners Configuration
+
+Date: 2020-04-23
+
+== Status
+
+Proposed
+
+Supercedes xref:0026-removing-configured-additional-mailboxListeners.adoc[26.
+Removing a configured additional MailboxListener]
+
+== Context
+
+James enables a user to register additional mailbox listeners.
+
+The distributed James server is handling mailbox event processing (mailboxListener execution) using a RabbitMQ work-queue per listener.
+
+Mailbox listeners can be registered to be triggered every time an event is generated by user interaction with their mailbox.
+They are being executed in a distributed fashion following the workqueue messaging pattern.
+The "group" is an attribute of the mailbox listener identifying to which work queue they belong.
+
+More information about this component can be found in xref:0037-eventbus.adoc[ADR 0036].
+
+Currently, mailbox listeners are determined by the guice bindings of the server and additional mailbox listeners defined via configuration files.
+
+While the configuration might be specific for each James server, what actually is defined in RabbitMQ is common.
+Heterogeneous configuration might then result in unpredictable RabbitMQ resource status.
+This was left as a limitation of xref:0026-removing-configured-additional-mailboxListeners.adoc[26.
+Removing a configured additional MailboxListener].
+
+== Decision
+
+We need to centralize the definition of mailbox listeners.
+
+An event sourcing system will track the configured mailbox listeners.
+
+It will have the following commands:
+
+* *AddListener*: Add a given listener.
+This should be rejected if the group is already used.
+* *RemoveListener*: Remove a given listener.
+
+Configuration changes are not supported.
+The administrator is expected to remove the given listener, then add it again.
+
+It will have the following events:
+
+* *ListenerAdded*: A mailbox listener is added
+* *ListenerRemoved*: A mailbox listener is removed
+
+A subscriber will react to these events to modify the RabbitMQ resource accordingly by adding queues, adding or removing bindings.
+
+This event sourcing system differs from the one defined in xref:0026-removing-configured-additional-mailboxListeners.adoc[26.
+Removing a configured additional MailboxListener] by the fact that we should also keep track of listener configuration.
+
+Upon start, James will ensure the *configured mailbox listener event sourcing system* contains the guice injected listeners, and add them if missing (handling the RabbitMQ bindings by this mean), then starts the eventBus which will consume the given queues.
+
+If a listener is configured with a class unknown to James, the start-up fails and James starts in a degraded state allowing to unconfigure the faulty listener.
+This downgraded state will be described in a separate ADR and the link will be updated here.
+
+This differs from xref:0026-removing-configured-additional-mailboxListeners.adoc[26.
+Removing a configured additional MailboxListener] by the fact we no longer need to register all listeners at once.
+
+A WebAdmin endpoint will allow:
+
+* *to add a listener* to the one configured.
+Such a call:
+ ** Will fail if the listener class is not on the local classpath, or if the corresponding group already used within the *configured mailbox listener aggregate*.
+ ** Upon success the listener is added to the *configured mailbox listener aggregate*, and the listener is registered locally.
+* *to remove a listener*.
+Such a call:
+ ** Will fail if the listener is required by Guice bindings on the current server or if the listener is not configured.
+ ** Upon success, the listener is removed from the *configured mailbox listener aggregate*, and the listener is unregistered locally.
+
+A broadcast on the event bus will be attempted to propagate topology changes, by the mean of a common registrationKey to all nodes, a "TopologyChanged" event, and a mailbox listener starting the MailboxListeners on local node upon topology changes.
+`registrationKey` concept is explained in xref:0037-eventbus.adoc[ADR 0036].
+
+If a listener is added but is not in the classpath, an ERROR log is emitted.
+This can happen during a rolling upgrade, which defines a new guice binding for a new mailbox listener.
+Events will still be emitted (and consumed by other James servers) however a local James upgrade will be required to effectively be able to start processing these events.
+The binding will not need to be redefined.
+
+We will also expose an endpoint listing the groups currently in use, and for each group the associated configuration, if any.
+This will query the *configured mailbox listener aggregate*.
+
+We will introduce a health check to actually ensure that RabbitMQ resources match the configured listeners, and propose a WebAdmin endpoint to add/remove bindings/queue in a similar fashion of what had been proposed in xref:0026-removing-configured-additional-mailboxListeners.adoc[26.
+Removing a configured additional MailboxListener].
+This can happen if the James server performing the listener registration fails to create the group/queue.
+This health check will also report if this James server does not succeed to run a given listener, for instance if its class is not on the classpath.
+
+== Consequences
+
+All products other than "Distributed James" are unchanged.
+
+All the currently configured additional listeners will need to be registered.
+
+The definition of mailbox listeners is thus centralized and we are not exposed to an heterogeneous configuration incident.
+
+Mailbox listeners no longer required by guice will still need to be instantiable (even with empty content).
+They will be considered as additional listener thus requiring explicit admin unconfiguration, which will be mentioned in the related upgrade instructions.
+Read notes about <<rolling-upgrade-scenari,rolling upgrade scenarii>>.
+
+<<deploying-a-new-custom-listener,Deploying a new custom listener>> also describes how to deploy new custom listeners.
+
+Integration tests relying on additional mailbox listeners of the distributed James product will require to be ported to perform additional mailbox listener registry with this WebAdmin endpoint.
+JMAP SpamAssassin, quota mailing tests are concerned.
+
+== Notes
+
+== Broadcast of topology changes
+
+=== Rolling upgrade scenarii
+
+During a rolling upgrade, the james version is heterogeneous across the cluster, and so might be the mailbox listeners required at the Guice level.
+
+*case 1*: James Server version 1 does not require listener A, James server version 2 requires listener A.
+
+Since listener A is registered, James server version 1 cannot be rebooted without being upgraded first.
+(As listener A cannot be instantiated)
+
+*case 2*: James Server version 1 requires listener A, James server version 2 does not require listener A.
+
+Upgrading to James version 2 means that listener A is still registered as an additional listener, it needs to be manually unconfigured once the rolling upgrade finished.
+Which is acceptable in upgrade instruction.
+We need to make sure the listeners could still be instantiated (even with empty code) for a transition period.
+
+== Deploying a new custom listener
+
+Given a new custom listener, not yet deployed in Distributed James cluster,
+
+To deploy it, an admin needs to follow these steps:
+
+* Add the jar in `extension-jars` folder for each James server
+ ** As `extension-jars` is read at instantiation time, no reboot is required to instantiate the new listener.
+* Call the webadmin endpoint alongside with listener specific configuration to enable the given custom listener.
+The bindings for the new listener will be created and a listener will be consuming its queue on the James server that had been treating the request.
+* Broadcast of topology changes will ensure the new custom additional mailbox listener will then be instantiated everywhere without a reboot.
diff --git a/docs/modules/development/adr/src/adr/0036-against-use-of-conditional-statements-in-guice-modules.md.adoc b/docs/modules/development/adr/src/adr/0036-against-use-of-conditional-statements-in-guice-modules.md.adoc
new file mode 100644
index 0000000..47e9bc0
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0036-against-use-of-conditional-statements-in-guice-modules.md.adoc
@@ -0,0 +1,112 @@
+= 36. Against the use of conditional statements in Guice modules
+
+Date: 2019-12-29
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+James products rely historically on Spring for dependency injection.
+It doesn't use last major Spring version (4.x instead of 5.x).
+James uses Spring in a way that enables overriding any class via a configuration file thus endangering overall correctness by giving too much power to the user.
+
+James proposes several implementations for each of the interfaces it defines.
+The number of possible combinations of implementations is thus really high (like factorial(n) with n > 10).
+It makes it unpractical to run tests for each possible component combination.
+We run integration tests for combinations that we decide brings the more value to the users.
+Spring product defeats this testing logic by allowing the user arbitrary classes combination, which is likely not being tested.
+
+Instead of having a single product allowing all component combination, we rather have several products each one exposing a single component combination.
+Components are defined by code in a static fashion.
+We thus can provide a decent level of QA for these products.
+Overriding components requires explicit code modification and recompilation, warning the user about the impact of the choices he does, and lowering the project's responsibility.
+Guice had been enacted as a way to reach that goal.
+
+With Guice we expose only supported, well tested combinations of components, thus addressing the combination issue.
+
+Spring application often bring dependencies conflicts, for example between Lucene and ElasticSearch components, leading to potential runtime or compile time issues.
+Instead of having a single big application being able to instantiate each and every component application, we have several products defining their dependencies in a minimalistic way, relying only on the components implementation that are needed.
+
+Here is the list of products we provide:
+
+* In-Memory: A memory based James server, mainly for testing purposes
+* Distributed James: A scalable James server, storing data in various data stores.
+Cassandra is used for metadata, ElasticSearch for search, RabbitMQ for messaging, and ObjectStorage for blobs.
+* Cassandra: An implementation step toward Distributed James.
+It does not include messaging and ObjectStorage and should not be run in a cluster way but is still relevant for good performance.
+* JPA: A JPA and Lucene based implementation of James.
+Only Derby driver is currently supported.
+* JPA with SMTP only using Derby: A minimalist SMTP server based on JPA storage technology and Derby driver
+* JPA with SMTP only using MariaDB: A minimalist SMTP server based on JPA storage technology and MariaDB driver
+
+Some components however do have several implementations a user can choose from in a given product.
+This is the case for:
+
+* BlobExport: Exporting a blob from the blobStore to an external user.
+Two implementations are currently supported: localFiles and LinShare.
+* Text extraction: Extracting text from attachment to enable attachment search.
+There is a Tika implementation, but lighter JSOUP based, as well as no text extraction options are also available.
+
+In order to keep the number of products low, we decided to use conditional statements in modules based on the configuration to select which one to enable at runtime.
+Eventually defeating the Guice adoption goals mentioned above.
+
+Finally, Blob Storing technology offers a wide combination of technologies:
+
+* ObjectStorage in itself could implement either Swift APIs or Amazon S3 APIs
+* We decided to keep supporting Cassandra for blob storing as an upgrade solution from Cassandra product to Distributed James for existing users.
+This option also makes sense for small data-sets (typically less than a TB) where storage cost are less of an issue and don't need to be taken into account when reasoning about performance.
+* Proposals such as xref:0014-blobstore-storage-policies.adoc[HybridBlobStore] and then xref:0025-cassandra-blob-store-cache.adoc[Cassandra BlobStore cache] proposed to leverage Cassandra as a performance (latency) enhancer for ObjectStorage technologies.
+
+Yet again it had been decided to use conditional statements in modules in order to lower the number of products.
+
+However, some components requires expensive resource initialization.
+These operations are performed via a separate module that needs to be installed based on the configuration.
+For instance xref:0025-cassandra-blob-store-cache.adoc[Cassandra BlobStore cache] requires usage of an additional cache keyspace that represents a cost and an inconvenience we don't want to pay if we don't rely on that cache.
+Not having the cache module thus enables quickly auditing that the caching cassandra session is not initialized.
+See https://github.com/linagora/james-project/pull/3261#pullrequestreview-389804841[this comment] as well as https://github.com/linagora/james-project/pull/3261#issuecomment-613911695[this comment].
+
+=== Audit
+
+The following modules perform conditional statements upon injection time:
+
+* BlobExportMechanismModule : Choice of the export mechanism
+* ObjectStorageDependenciesModule::selectBlobStoreBuilder: Choice between S3 and Swift ObjectStorage technologies
+* TikaMailboxModule::provideTextExtractor: Choice of text extraction technology
+* BlobStoreChoosingModule::provideBlobStore: Choice of BlobStore technology: Cassandra, ObjectStorage or Hybrid
+* https://github.com/linagora/james-project/pull/3319[Cached blob store] represents a similar problem: should the blobStore be wrapped by a caching layer?
+
+Cassandra and Distributed products are furthermore duplicated to offer a version supporting LDAP authentication.
+JPA product does not offer LDAP support.
+
+== Decision
+
+We should no longer rely on conditional statements in Guice module.
+
+Guice modules combination choice should be decided before starting the dependency injection stage.
+
+Each component choice needs to be abstracted by a related configuration POJO.
+
+Products will, given the set of configuration POJOs, generated the modules it should rely on during the dependency injection stage.
+
+An INFO log with the list of modules used to create its Guice injector.
+This enables easy diagnose of the running components via the selected module list.
+It exposes tested, safe choices to the user while limiting the Guice products count.
+
+== Consequences
+
+Component combination count keeps unchanged for Guice products, but the run combination is explicit.
+QA needs are unchanged.
+
+Integration tests needs to be adapted to accept component choice configuration POJO.
+
+The following conditional statements in guice modules needs to be removed :
+
+* https://github.com/linagora/james-project/pull/3319[Cached blob store pull request] addresses ObjectStorageDependenciesModule::selectBlobStoreBuilder and Cassandra Blob Store Cache conditional statement.
+* https://github.com/linagora/james-project/pull/3099[S3 native blobStore implementation] along side with S3 endpoints support as part of Swift removes the need to select the Object Storage implementation.
+* Follow up work needs to be plan concerning `BlobExportMechanismModule` and `TikaMailboxModule::provideTextExtractor`.
+
+We furthermore need to enable a module choice for LDAP on top of other existing products.
+We should remove LDAP variations for LDAP products.
+Corresponding docker image will be based on their non LDAP version, overriding the `usersrepository.xml` configuration file, be marked as deprecated and eventually removed.
diff --git a/docs/modules/development/adr/src/adr/0037-eventbus.md.adoc b/docs/modules/development/adr/src/adr/0037-eventbus.md.adoc
new file mode 100644
index 0000000..5a8b2e5
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0037-eventbus.md.adoc
@@ -0,0 +1,59 @@
+= 37. Event bus
+
+Date: 2020-05-05
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+Many features rely on behaviors triggered by interactions with the mailbox API main interfaces (`RightManager`, `MailboxManager`, `MessageManager`, `MessageIdManager`).
+We need to provide a convenient extension mechanism for organizing the execution of these behaviours, provide retries and advanced error handling.
+
+Also, protocols enable notifications upon mailbox modifications.
+This is for instance the case for `RFC-2177 IMAP IDLE`, leveraged for `RFC-3501 IMAP unsolicitated notifications` when selecting a Mailbox, as well as maintaining the `+IMAP Message Sequence Number <-> Unique IDentifier+` MSN \<-> UID mapping.
+Changes happening for a specific entity (mailbox) need to be propagated to the relevant listeners.
+
+== Decision
+
+James mailbox component, a core component of James handling the storage of mails and mailboxes, should use an event driven architecture.
+
+It means every meaningful action on mailboxes or messages triggers an event for any component to react to that event.
+
+`MailboxListener` allows executing actions upon mailbox events.
+They could be used for a wide variety of purposes, like enriching mailbox managers features or enabling user notifications upon mailboxes operations performed by other devices via other protocol sessions.
+
+Interactions happen via the managers (`RightManager`, `MailboxManager`, `MessageManager`, `MessageIdManager`) which emit an event on the `EventBus`, which will ensure the relevant ``MailboxListener``s will be executed at least once.
+
+`MailboxListener` can be registered in a work queue fashion on the `EventBus`.
+Each work queue corresponds to a given MailboxListener class with the same configuration, identified by their group.
+Each event is executed at least once within a James cluster, errors are retried with an exponential back-off delay.
+If the execution keeps failing, the event is stored in `DeadLetter` for later reprocessing, triggered via WebAdmin.
+
+Guice products enable the registration of additional mailbox listeners.
+A user can furthermore define its own mailboxListeners via the use of `extension-jars`.
+
+MailboxListener can also be registered to be executed only on events concerning a specific entity (eg.
+a mailbox).
+The `registrationKey` is identifying entities concerned by the event.
+Upon event emission, the manager will indicate the `registrationKey` this event should be sent to.
+A mailboxListener will thus only receive the event for the registration key it is registered to, in an at least once fashion.
+
+== Consequences
+
+We need to provide an `In VM` implementation of the EventBus for single server deployments.
+
+We also need to provide xref:0038-distributed-eventbus.adoc[a distributed event bus implementation].
+
+== Current usages
+
+The following features are implemented as Group mailbox listeners:
+
+* Email indexing in Lucene or ElasticSearch
+* Deletion of mailbox annotations
+* Cassandra Message metadata cleanup upon deletion
+* Quota updates
+* Quota indexing
+* Over Quota mailing
+* SpamAssassin Spam/Ham reporting
diff --git a/docs/modules/development/adr/src/adr/0038-distributed-eventbus.md.adoc b/docs/modules/development/adr/src/adr/0038-distributed-eventbus.md.adoc
new file mode 100644
index 0000000..d96b64e
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0038-distributed-eventbus.md.adoc
@@ -0,0 +1,44 @@
+= 38. Distributed Event bus
+
+Date: 2020-05-25
+
+== Status
+
+Accepted (lazy consensus)
+
+== Context
+
+Read xref:0037-eventbus.adoc[Event Bus ADR] for context.
+
+Given several James servers, we need them to share a common EventBus.
+
+This:
+
+* Ensures a better load balancing for `group mailbox listners`.
+* Is required for correctness of notifications (like IMAP IDLE).
+
+== Decision
+
+Provide a distributed implementation of the EventBus leveraging RabbitMQ.
+
+Events are emitted to a single Exchange.
+
+Each group will have a corresponding queue, bound to the main exchange, with a default routing key.
+Each eventBus will consume this queue and execute the relevant listener, ensuring at least once execution at the cluster level.
+
+Retries are managed via a dedicated exchange for each group: as we need to count retries, the message headers need to be altered and we cannot rely on rabbitMQ build in retries.
+Each time the execution fails locally, a new event is emitted via the dedicated exchange, and the original event is acknowledged.
+
+Each eventBus will have a dedicated exclusive queue, bound to the main exchange with the `registrationKeys` used by local notification mailboxListeners (to only receive the corresponding subset of events).
+Errors are not retried for notifications, failures are not persisted within `DeadLetter`, achieving at most once event delivery.
+
+== Related ADRs
+
+The implementation of the the distributed EventBus suffers from the following flows:
+
+* xref:0026-removing-configured-additional-mailboxListeners.adoc[Removing a configured additional MailboxListener]
+* xref:0035-distributed-listeners-configuration.adoc[Distributed Mailbox Listeners Configuration] also covers more in details topology changes and supersedes ADR 0026.
+
+The following enhancement have furthermore been contributed:
+
+* xref:0027-eventBus-error-handling-upon-dispatch.adoc[EventBus error handling upon dispatch]
diff --git a/docs/modules/development/adr/src/adr/0039-distributed-blob-garbage-collector.md.adoc b/docs/modules/development/adr/src/adr/0039-distributed-blob-garbage-collector.md.adoc
new file mode 100644
index 0000000..e19a8bb
--- /dev/null
+++ b/docs/modules/development/adr/src/adr/0039-distributed-blob-garbage-collector.md.adoc
@@ -0,0 +1,687 @@
+= 39. Distributed blob garbage collector
+
+Date: 2020-02-18
+
+== Status
+
+Proposed
+
+== Context
+
+The body, headers, attachments of the mails are stored as blobs in a blob store.
+In order to save space in those stores, those blobs are de-duplicated using a hash of their content.
+To attain that the current blob store will read the content of the blob before saving it, and generate its id based on a hash of this content.
+This way two blobs with the same content will share the same id and thus be saved only once.
+This makes the safe deletion of one of those blobs a non trivial problem as we can't delete one blob without ensuring that all references to it are themselves deleted.
+For example if two messages share the same blob, when we delete one message there is at the time being no way to tell if the blob is still referenced by another message.
+
+== Decision
+
+To address this issue, we propose to implement a distributed blob garbage collector built upon the previously developed Distributed Task Manager.
+The de-duplicating blob store will keep track of the references pointing toward a blob in a `References` table.
+It will also keep track of the deletion requests for a blob in a `Deletions` table.
+When the garbage collector algorithm runs it will fetch from the `Deletions` table the blobs considered to be effectively deleted, and will check in the `References` table if there are still some references to them.
+If there is no more reference to a blob, it will be effectively deleted from the blob store.
+
+To avoid concurrency issues, where we could garbage collect a blob at the same time a new reference to it appear, a `reference generation` notion will be added.
+The de-duplicating id of the blobs which before where constructed using only the hash of their content, will now include this `reference generation` too.
+At a given interval a new `reference generation` will be emitted, since then all new blobs will point to this new generation.
+
+So a `garbage collection iteration` will run only on the `reference generation` `n-2` to avoid concurrency issues.
+
+The switch of generation will be triggered by a task running on the distributed task manager.
+This task will emit an event into the event sourcing system to increment the `reference generation`.
+
+== Alternatives
+
+Not de-duplicating the blobs' content, this simple approach which involves storing the same blob a lot of times can in some scenario be really slow and costly.
+Albeit it can in some case be preferred for the sake of simplicity, data security...
+
+== Consequences
+
+This change will necessitate to extract the base blob store responsibilities (store a blob, delete a blob, read a blob) from the current blob store implementation which is doing the de-duplication, id generation...
+The garbage collector will use this low level blob store in order to effectively delete the blobs.
+
+One other consequence of this work, is the fact that there will be no de-duplication on different `reference generation`, i.e two blobs with the same content will be stored twice now, if they were created during two different `reference generation`.
+
+When writing a blob into the de-duplicating blob store, we will need to specify the reference to the object (MessageId, AttachmentId...) we store the blob for.
+This can make some components harder to implement as we will have to propagate the references.
+
+Since we will not build a distributed task scheduler.
+To increment the `reference generation` and launch periodically a `garbage collection iteration`, the scheduling will be done by an external scheduler (cron job, kubernetes cronjob ...) which will call a webadmin endpoint to launch this task periodically.
+
+== Algorithm visualisation
+
+=== Generation 1 and Iteration 1
+
+* Events
+ ** `rg1` reference generation is emitted
+ ** `gci1` garbage collection iteration is emitted
+ ** An email is sent to `user1`, a `m1` message, and a blob `b1` are stored with `rg1`
+ ** An email is sent to `user1` and `user2`, `m2` and `m3` messages, and a blob `b2` are stored with `rg1`
+
+==== Tables
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b1
+| rg1
+
+| b2
+| rg1
+|===
+
+===== References
+
+|===
+| message id | blob id | reference generation id
+
+| m1
+| b1
+| rg1
+
+| m2
+| b2
+| rg1
+
+| m3
+| b2
+| rg1
+|===
+
+===== Deletions
+
+Empty
+
+=== Generation 2 / Iteration 2
+
+* Events
+ ** `rg2` reference generation is emitted
+ ** `gci2` garbage collection iteration is emitted
+ ** An email is sent to `user1`, a `m4` message, and a blob `b3` are stored with `rg2`
+ ** An email is sent to `user1` and `user2`, `m5` and `m6` messages, and a blob `b4` are stored with `rg2`
+
+==== Tables
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+| rg2
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+| gci2
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b1
+| rg1
+
+| b2
+| rg1
+
+| b3
+| rg2
+
+| b4
+| rg2
+|===
+
+===== References
+
+|===
+| message id | blob id | reference generation id
+
+| m1
+| b1
+| rg1
+
+| m2
+| b2
+| rg1
+
+| m3
+| b2
+| rg1
+
+| m4
+| b3
+| rg2
+
+| m5
+| b4
+| rg2
+
+| m6
+| b4
+| rg2
+|===
+
+===== Deletions
+
+Empty
+
+=== Generation 3 / Iteration 3
+
+* Events
+ ** `rg3` reference generation is emitted
+ ** `gci3` garbage collection iteration is emitted
+ ** An email is sent to `user1`, a `m7` message, and a blob `b5` are stored with `rg3`
+ ** An email is sent to `user1` and `user2`, `m8` and `m9` messages, and a blob `b6` are stored with `rg3`
+ ** `user1` deletes `m1`, `m2`, `m7`, and `m8` with `gi3`
+ ** `user2` deletes `m3` with `gi3`
+
+==== Tables: before deletions
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+| rg2
+| rg3
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+| gci2
+| gci3
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b1
+| rg1
+
+| b2
+| rg1
+
+| b3
+| rg2
+
+| b4
+| rg2
+
+| b5
+| rg3
+
+| b6
+| rg3
+|===
+
+===== References
+
+|===
+| message id | blob id | reference generation id
+
+| m1
+| b1
+| rg1
+
+| m2
+| b2
+| rg1
+
+| m3
+| b2
+| rg1
+
+| m4
+| b3
+| rg2
+
+| m5
+| b4
+| rg2
+
+| m6
+| b4
+| rg2
+
+| m7
+| b5
+| rg3
+
+| m8
+| b6
+| rg3
+
+| m9
+| b6
+| rg3
+|===
+
+===== Deletions
+
+Empty
+
+==== Tables: after deletions
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+| rg2
+| rg3
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+| gci2
+| gci3
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b1
+| rg1
+
+| b2
+| rg1
+
+| b3
+| rg2
+
+| b4
+| rg2
+
+| b5
+| rg3
+
+| b6
+| rg3
+|===
+
+===== References
+
+|===
+| message id | blob id | reference generation id
+
+| m4
+| b3
+| rg2
+
+| m5
+| b4
+| rg2
+
+| m6
+| b4
+| rg2
+
+| m9
+| b6
+| rg3
+|===
+
+===== Deletions
+
+|===
+| blob id | reference generation id | date | garbage collection iteration id
+
+| b1
+| rg1
+| 10:42
+| gci3
+
+| b2
+| rg1
+| 10:42
+| gci3
+
+| b2
+| rg1
+| 13:37
+| gci3
+
+| b5
+| rg3
+| 10:42
+| gci3
+
+| b6
+| rg3
+| 10:42
+| gci3
+|===
+
+==== Running the algorithm
+
+* fetch `Deletions` for `gci3` in `deletions`
+* find distinct `reference-generation-id` of `deletions` in `generations = {rg1, rg3}`
+* For each generation
+ ** _rg1_
+ *** filter `deletions` to keep only `rg1` entries and extract `blob-ids` in `concernedBlobs = {b1, b2}`
+ *** fetch all references to `concernedBlobs` and build a Bloom-Filter in `foundedReferences = {}`
+ *** filter `concernedBlobs` to keep only those which are not present in `foundedReferences` in `blobsToDelete = {b1, b2}`
+ *** Remove `blobsToDelete` from `Blobs` and `Deletions`
+ ** _rg3_
+ *** filter `deletions` to keep only `rg3` entries and extract `blob-ids` in `concernedBlobs = {b5, b6}`
+ *** fetch all references to `concernedBlobs` and build a Bloom-Filter in `+foundedReferences = {b6}+`
+ *** filter `concernedBlobs` to keep only those which are not present in `foundedReferences` in `+blobsToDelete = {b5}+`
+ *** Remove `blobsToDelete` from `Blobs` and `Deletions`
+
+==== Tables: after garbage collection
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+| rg2
+| rg3
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+| gci2
+| gci3
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b3
+| rg2
+
+| b4
+| rg2
+
+| b6
+| rg3
+|===
+
+===== References
+
+|===
+| message id | blob id | generation id
+
+| m4
+| b3
+| g2
+
+| m5
+| b4
+| g2
+
+| m6
+| b4
+| g2
+
+| m9
+| b6
+| g3
+|===
+
+===== Deletions
+
+|===
+| blob id | reference generation id | date | garbage collection iteration id
+
+| b6
+| rg3
+| 10:42
+| gci3
+|===
+
+=== Generations 4
+
+* Events
+ ** `rg4` reference generation is emitted
+ ** `gci4` garbage collection iteration is emitted
+ ** `user2` deletes `m9` with `gcg4`
+
+==== Tables: before deletions
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+| rg2
+| rg3
+| rg4
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+| gci2
+| gci3
+| gci4
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b3
+| rg2
+
+| b4
+| rg2
+
+| b6
+| rg3
+|===
+
+===== References
+
+|===
+| message id | blob id | reference generation id
+
+| m4
+| b3
+| rg2
+
+| m5
+| b4
+| rg2
+
+| m6
+| b4
+| rg2
+
+| m9
+| b6
+| rg3
+|===
+
+===== Deletions
+
+|===
+| blob id | reference generation id | date | garbage collection iteration id
+
+| b6
+| rg3
+| 10:42
+| gci3
+|===
+
+==== Tables: after deletions
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+| rg2
+| rg3
+| rg4
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+| gci2
+| gci3
+| gci4
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b3
+| rg2
+
+| b4
+| rg2
+
+| b6
+| rg3
+|===
+
+===== References
+
+|===
+| message id | blob id | reference generation id
+
+| m4
+| b3
+| rg2
+
+| m5
+| b4
+| rg2
+
+| m6
+| b4
+| rg2
+|===
+
+===== Deletions
+
+|===
+| blob id | reference generation id | date | garbage collection iteration id |
+
+| b6
+| rg3
+| 10:42
+| gci3
+|
+
+| b6
+| rg3
+| 18:42
+| gci4
+|
+|===
+
+==== Running the algorithm
+
+* fetch `Deletions` for `gci4` in `deletions`
+* find distinct `generation-id` of `deletions` in `+generations = {rg3}+`
+* For each generation
+ ** _rg3_
+ *** filter `deletions` to keep only `rg3` entries and extract `blob-ids` in `+concernedBlobs = {b6}+`
+ *** fetch all references to `concernedBlobs` and build a Bloom-Filter in `foundedReferences = {}`
+ *** filter `concernedBlobs` to keep only those which are not present in `foundedReferences` in `+blobsToDelete = {b6}+`
+ *** Remove `blobsToDelete` from `Blobs` and `Deletions`
+
+==== Tables: after garbage collection
+
+===== Generations
+
+|===
+| reference generation id
+
+| rg1
+| rg2
+| rg3
+| rg4
+|===
+
+|===
+| garbage collection iteration id
+
+| gci1
+| gci2
+| gci3
+| gci4
+|===
+
+===== Blobs
+
+|===
+| blob id | reference generation id
+
+| b3
+| rg2
+
+| b4
+| rg2
+|===
+
+===== References
+
+|===
+| message id | blob id | reference generation id
+
+| m4
+| b3
+| rg2
+
+| m5
+| b4
+| rg2
+
+| m6
+| b4
+| rg2
+|===
+
+===== Deletions
+
+Empty
diff --git a/migrate-adr.sh b/migrate-adr.sh
new file mode 100755
index 0000000..40ab03b
--- /dev/null
+++ b/migrate-adr.sh
@@ -0,0 +1,8 @@
+#!/bin/bash
+#
+# https://matthewsetter.com/technical-documentation/asciidoc/convert-markdown-to-asciidoc-with-kramdoc/
+#
+
+mkdir -p docs/modules/development/adr
+find ./src/adr -name "*.md" -type f -exec sh -c \
+ 'echo "Convert {}" ; kramdoc --format=GFM --wrap=ventilate --output=./docs/modules/development/adr/{}.adoc {}' \;
diff --git a/src/adr/0009-disable-elasticsearch-dynamic-mapping.md b/src/adr/0009-disable-elasticsearch-dynamic-mapping.md
index 5d75a43..a2008f5 100644
--- a/src/adr/0009-disable-elasticsearch-dynamic-mapping.md
+++ b/src/adr/0009-disable-elasticsearch-dynamic-mapping.md
@@ -11,9 +11,9 @@ Accepted (lazy consensus)
We rely on dynamic mappings to expose our mail headers as a JSON map. Dynamic mapping is enabled for adding not yet encountered headers in the mapping.
This causes a serie of functional issues:
- - Maximum field count can easily be exceeded
- - Field type 'guess' can be wrong, leading to subsequent headers omissions [1]
- - Document indexation needs to be paused at the index level during mapping changes to avoid concurrent changes, impacting negatively performance.
+* Maximum field count can easily be exceeded
+* Field type 'guess' can be wrong, leading to subsequent headers omissions (see JAMES-2078)
+* Document indexation needs to be paused at the index level during mapping changes to avoid concurrent changes, impacting negatively performance.
## Decision
@@ -23,14 +23,14 @@ Rely on nested objects to represent mail headers within a mapping
The index needs to be re-created. Document reIndexation is needed.
-This solves the aforementionned bugs [1].
+This solves the aforementionned bugs (see JAMES-2078).
Regarding performance:
- - Default message list performance is unimpacted
- - We noticed a 4% performance improvment upon indexing throughput
- - We noticed a 7% increase regarding space per message
+* Default message list performance is unimpacted
+* We noticed a 4% performance improvment upon indexing throughput
+* We noticed a 7% increase regarding space per message
## References
- - [1]: https://github.com/linagora/james-project/pull/2726 JAMES-2078 Add an integration test to prove that dynamic mapping can lead to ignored header fields
+ - [JAMES-2078](https://github.com/linagora/james-project/pull/2726) JAMES-2078 Add an integration test to prove that dynamic mapping can lead to ignored header fields
- [JIRA](https://issues.apache.org/jira/browse/JAMES-2078)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org