[james-project] 07/27: JAMES-2906 ADR for _source removal rejection

[bt...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 38e564b35a44ab739107794e2de4f8b052fd08df
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Oct 17 09:16:05 2019 +0700

    JAMES-2906 ADR for _source removal rejection
---
 .../0011-remove-elasticsearch-document-source.md   | 38 ++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/src/adr/0011-remove-elasticsearch-document-source.md b/src/adr/0011-remove-elasticsearch-document-source.md
new file mode 100644
index 0000000..30882f3
--- /dev/null
+++ b/src/adr/0011-remove-elasticsearch-document-source.md
@@ -0,0 +1,38 @@
+# 11. Disable ElasticSearch source
+
+Date: 2019-10-17
+
+## Status
+
+Rejected
+
+The benefits do not outweigh the costs.
+
+## Context
+
+Though very handy to have around, the source field does incur storage overhead within the index. 
+
+## Decision
+
+Disable `_source` for ElasticSearch indexed documents.
+
+## Consequences
+
+Given a dataset composed of small text/plain messages, we notice a 20% space reduction of data stored on ElasticSearch.
+
+However, patch updates can no longer be performed upon flags updates. Upon flag update we need to fully read the mail 
+content, then mime-parse it, potentially html parse it, extract attachment content again and finally index again the full 
+document.
+
+Without `_source` field, flags update is two times slower, 99 percentile 4 times slower, and this impact negatively other 
+requests.
+
+Note please that `_source` allows admin flexibility like performing index level changes without downtime, amongst others:
+ - Increase shards
+ - Modifying replication factor
+ - Changing analysers (IE allows an admin to configure FR analyser instead of EN analyser)
+
+## References
+
+ - https://www.elastic.co/guide/en/elasticsearch/reference/6.3/mapping-source-field.html
+ - [JIRA](https://issues.apache.org/jira/browse/JAMES-2906)


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org