[jira] [Commented] (JAMES-2078) disable dynamic parsing in our mapping and changing header structure

["Benoit Tellier (Jira)" <se...@james.apache.org>]

    [ https://issues.apache.org/jira/browse/JAMES-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941529#comment-16941529 ] 

Benoit Tellier commented on JAMES-2078:
---------------------------------------

We plan to have a look at this issue.

We will propose an alternative structure (nested objects?), and will evaluate indexation time, dataset size and search time against it.

> disable dynamic parsing in our mapping and changing header structure
> --------------------------------------------------------------------
>
>                 Key: JAMES-2078
>                 URL: https://issues.apache.org/jira/browse/JAMES-2078
>             Project: James Server
>          Issue Type: Improvement
>          Components: elasticsearch
>            Reporter: Luc DUZAN
>            Priority: Major
>
> We should turn off dynamic mapping (https://www.elastic.co/guide/en/elasticsearch/reference/2.2/dynamic-field-mapping.html). Dynamic mapping allow ES to automatically guess the type of a field when it indexes a document with a field not described by the mapping. He will also automatically create a index for it. This have two issues:
> * Creating to much index and moreover creating parse index. This will slow down write time
> * Guessing wrong type. Imagine the first time ES receives an unknown field with a number, he will update the mapping and specify this field should be a number. If this is wrong and a new document arrive with this field but with a String, the indexation of this document will fail!
> After changing the mapping to dynamic, we will have an issue when indexing. For the moment the header are stored in the following format inside ES: { headers: { header1: value1, header2: value2}}.
> If we keep this format with dynamic: false, header won't be search-able because they will have no mapping.
> To avoid this issue we will we change the format of the message to: { headers: [{key: "header1", value: "value1"}, {key: "header2", value: "value2"}]}
> This can be express in the mapping with help of "nested": https://www.elastic.co/guide/en/elasticsearch/reference/2.2/nested.html.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org