[jira] [Comment Edited] (JAMES-2078) disable dynamic parsing in our mapping and changing header structure

["Benoit Tellier (Jira)" <se...@james.apache.org>]

    [ https://issues.apache.org/jira/browse/JAMES-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16944218#comment-16944218 ] 

Benoit Tellier edited comment on JAMES-2078 at 10/4/19 5:10 AM:
----------------------------------------------------------------

https://github.com/linagora/james-project/pull/2726 highlighted functional issues caused by dynamic mappings (and is merged)


was (Author: btellier):
https://github.com/linagora/james-project/pull/2726 highlighted functional issues caused by dynamic mappings

> disable dynamic parsing in our mapping and changing header structure
> --------------------------------------------------------------------
>
>                 Key: JAMES-2078
>                 URL: https://issues.apache.org/jira/browse/JAMES-2078
>             Project: James Server
>          Issue Type: Improvement
>          Components: elasticsearch
>            Reporter: Luc DUZAN
>            Priority: Major
>
> We should turn off dynamic mapping (https://www.elastic.co/guide/en/elasticsearch/reference/2.2/dynamic-field-mapping.html). Dynamic mapping allow ES to automatically guess the type of a field when it indexes a document with a field not described by the mapping. He will also automatically create a index for it. This have two issues:
> * Creating to much index and moreover creating parse index. This will slow down write time
> * Guessing wrong type. Imagine the first time ES receives an unknown field with a number, he will update the mapping and specify this field should be a number. If this is wrong and a new document arrive with this field but with a String, the indexation of this document will fail!
> After changing the mapping to dynamic, we will have an issue when indexing. For the moment the header are stored in the following format inside ES: { headers: { header1: value1, header2: value2}}.
> If we keep this format with dynamic: false, header won't be search-able because they will have no mapping.
> To avoid this issue we will we change the format of the message to: { headers: [{key: "header1", value: "value1"}, {key: "header2", value: "value2"}]}
> This can be express in the mapping with help of "nested": https://www.elastic.co/guide/en/elasticsearch/reference/2.2/nested.html.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org