You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ponymail.apache.org by hu...@apache.org on 2017/10/30 15:13:45 UTC
[3/5] incubator-ponymail git commit: add fielddata setting,
so word cloud will work
add fielddata setting, so word cloud will work
Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/724837dd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/724837dd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/724837dd
Branch: refs/heads/master
Commit: 724837dda8c757c17c9572ef2c5a34e61a4468b9
Parents: 58d85c4
Author: Daniel Gruno <hu...@apache.org>
Authored: Mon Oct 30 16:11:57 2017 +0100
Committer: Daniel Gruno <hu...@apache.org>
Committed: Mon Oct 30 16:11:57 2017 +0100
----------------------------------------------------------------------
tools/setup.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/724837dd/tools/setup.py
----------------------------------------------------------------------
diff --git a/tools/setup.py b/tools/setup.py
index 57f7c64..38db29a 100755
--- a/tools/setup.py
+++ b/tools/setup.py
@@ -295,7 +295,8 @@ def createIndex():
"type" : "string"
},
"subject" : {
- "type" : "string"
+ "type" : "string",
+ "fielddata": True
},
"to" : {
"type" : "string"
@@ -440,6 +441,7 @@ def createIndex():
},
"subject" : {
"type" : "string",
+ "fielddata": True
# "index" : "not_analyzed"
},
"to" : {
Re: [3/5] incubator-ponymail git commit: add fielddata setting, so
word cloud will work
Posted by sebb <se...@gmail.com>.
Also, note that setting fielddata can consume significant amounts of memory.
Further, the ES docs state that it's rarely appropriate:
https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html#before-enabling-fielddata
Since the word cloud is an optional extra, it seems wrong to
unconditionally enable fielddata.
At the least, it should be optional.
On 30 October 2017 at 16:40, sebb <se...@gmail.com> wrote:
> Does it fix the following bug?
>
> https://github.com/apache/incubator-ponymail/issues/345
>
> On 30 October 2017 at 16:22, sebb <se...@gmail.com> wrote:
>> Is there a bug reference for this?
>>
>> What about the Changelog?
>>
>> On 30 October 2017 at 15:13, <hu...@apache.org> wrote:
>>> add fielddata setting, so word cloud will work
>>>
>>>
>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo
>>> Commit: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/724837dd
>>> Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/724837dd
>>> Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/724837dd
>>>
>>> Branch: refs/heads/master
>>> Commit: 724837dda8c757c17c9572ef2c5a34e61a4468b9
>>> Parents: 58d85c4
>>> Author: Daniel Gruno <hu...@apache.org>
>>> Authored: Mon Oct 30 16:11:57 2017 +0100
>>> Committer: Daniel Gruno <hu...@apache.org>
>>> Committed: Mon Oct 30 16:11:57 2017 +0100
>>>
>>> ----------------------------------------------------------------------
>>> tools/setup.py | 4 +++-
>>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>> ----------------------------------------------------------------------
>>>
>>>
>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/724837dd/tools/setup.py
>>> ----------------------------------------------------------------------
>>> diff --git a/tools/setup.py b/tools/setup.py
>>> index 57f7c64..38db29a 100755
>>> --- a/tools/setup.py
>>> +++ b/tools/setup.py
>>> @@ -295,7 +295,8 @@ def createIndex():
>>> "type" : "string"
>>> },
>>> "subject" : {
>>> - "type" : "string"
>>> + "type" : "string",
>>> + "fielddata": True
>>> },
>>> "to" : {
>>> "type" : "string"
>>> @@ -440,6 +441,7 @@ def createIndex():
>>> },
>>> "subject" : {
>>> "type" : "string",
>>> + "fielddata": True
>>> # "index" : "not_analyzed"
>>> },
>>> "to" : {
>>>
Re: [3/5] incubator-ponymail git commit: add fielddata setting, so
word cloud will work
Posted by sebb <se...@gmail.com>.
On 30 October 2017 at 17:29, Daniel Gruno <hu...@apache.org> wrote:
> On 10/30/2017 05:40 PM, sebb wrote:
>> Does it fix the following bug?
>>
>> https://github.com/apache/incubator-ponymail/issues/345
>
> Yes, we can close that.
> As for memory consumption, that is not really an issue.
> The documentation is there for ES in general, which means it applies to
> clusters with terabytes or petabytes of storage. It WOULD be a problem
> if you had a billion or a trillion emails, but it's not a concern with
> thousands or millions of emails.
Not sure how it's possible to assert that.
Is there any data on the additional memory consumption?
Also, you have not addressed why we should not use the ES suggestion
of adding a keyword field instead.
>>
>> On 30 October 2017 at 16:22, sebb <se...@gmail.com> wrote:
>>> Is there a bug reference for this?
>>>
>>> What about the Changelog?
>>>
>>> On 30 October 2017 at 15:13, <hu...@apache.org> wrote:
>>>> add fielddata setting, so word cloud will work
>>>>
>>>>
>>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo
>>>> Commit: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/724837dd
>>>> Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/724837dd
>>>> Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/724837dd
>>>>
>>>> Branch: refs/heads/master
>>>> Commit: 724837dda8c757c17c9572ef2c5a34e61a4468b9
>>>> Parents: 58d85c4
>>>> Author: Daniel Gruno <hu...@apache.org>
>>>> Authored: Mon Oct 30 16:11:57 2017 +0100
>>>> Committer: Daniel Gruno <hu...@apache.org>
>>>> Committed: Mon Oct 30 16:11:57 2017 +0100
>>>>
>>>> ----------------------------------------------------------------------
>>>> tools/setup.py | 4 +++-
>>>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>>> ----------------------------------------------------------------------
>>>>
>>>>
>>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/724837dd/tools/setup.py
>>>> ----------------------------------------------------------------------
>>>> diff --git a/tools/setup.py b/tools/setup.py
>>>> index 57f7c64..38db29a 100755
>>>> --- a/tools/setup.py
>>>> +++ b/tools/setup.py
>>>> @@ -295,7 +295,8 @@ def createIndex():
>>>> "type" : "string"
>>>> },
>>>> "subject" : {
>>>> - "type" : "string"
>>>> + "type" : "string",
>>>> + "fielddata": True
>>>> },
>>>> "to" : {
>>>> "type" : "string"
>>>> @@ -440,6 +441,7 @@ def createIndex():
>>>> },
>>>> "subject" : {
>>>> "type" : "string",
>>>> + "fielddata": True
>>>> # "index" : "not_analyzed"
>>>> },
>>>> "to" : {
>>>>
>
Re: [3/5] incubator-ponymail git commit: add fielddata setting, so
word cloud will work
Posted by Daniel Gruno <hu...@apache.org>.
On 10/30/2017 05:40 PM, sebb wrote:
> Does it fix the following bug?
>
> https://github.com/apache/incubator-ponymail/issues/345
Yes, we can close that.
As for memory consumption, that is not really an issue.
The documentation is there for ES in general, which means it applies to
clusters with terabytes or petabytes of storage. It WOULD be a problem
if you had a billion or a trillion emails, but it's not a concern with
thousands or millions of emails.
>
> On 30 October 2017 at 16:22, sebb <se...@gmail.com> wrote:
>> Is there a bug reference for this?
>>
>> What about the Changelog?
>>
>> On 30 October 2017 at 15:13, <hu...@apache.org> wrote:
>>> add fielddata setting, so word cloud will work
>>>
>>>
>>> Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo
>>> Commit: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/724837dd
>>> Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/724837dd
>>> Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/724837dd
>>>
>>> Branch: refs/heads/master
>>> Commit: 724837dda8c757c17c9572ef2c5a34e61a4468b9
>>> Parents: 58d85c4
>>> Author: Daniel Gruno <hu...@apache.org>
>>> Authored: Mon Oct 30 16:11:57 2017 +0100
>>> Committer: Daniel Gruno <hu...@apache.org>
>>> Committed: Mon Oct 30 16:11:57 2017 +0100
>>>
>>> ----------------------------------------------------------------------
>>> tools/setup.py | 4 +++-
>>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>> ----------------------------------------------------------------------
>>>
>>>
>>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/724837dd/tools/setup.py
>>> ----------------------------------------------------------------------
>>> diff --git a/tools/setup.py b/tools/setup.py
>>> index 57f7c64..38db29a 100755
>>> --- a/tools/setup.py
>>> +++ b/tools/setup.py
>>> @@ -295,7 +295,8 @@ def createIndex():
>>> "type" : "string"
>>> },
>>> "subject" : {
>>> - "type" : "string"
>>> + "type" : "string",
>>> + "fielddata": True
>>> },
>>> "to" : {
>>> "type" : "string"
>>> @@ -440,6 +441,7 @@ def createIndex():
>>> },
>>> "subject" : {
>>> "type" : "string",
>>> + "fielddata": True
>>> # "index" : "not_analyzed"
>>> },
>>> "to" : {
>>>
Re: [3/5] incubator-ponymail git commit: add fielddata setting, so
word cloud will work
Posted by sebb <se...@gmail.com>.
Does it fix the following bug?
https://github.com/apache/incubator-ponymail/issues/345
On 30 October 2017 at 16:22, sebb <se...@gmail.com> wrote:
> Is there a bug reference for this?
>
> What about the Changelog?
>
> On 30 October 2017 at 15:13, <hu...@apache.org> wrote:
>> add fielddata setting, so word cloud will work
>>
>>
>> Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo
>> Commit: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/724837dd
>> Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/724837dd
>> Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/724837dd
>>
>> Branch: refs/heads/master
>> Commit: 724837dda8c757c17c9572ef2c5a34e61a4468b9
>> Parents: 58d85c4
>> Author: Daniel Gruno <hu...@apache.org>
>> Authored: Mon Oct 30 16:11:57 2017 +0100
>> Committer: Daniel Gruno <hu...@apache.org>
>> Committed: Mon Oct 30 16:11:57 2017 +0100
>>
>> ----------------------------------------------------------------------
>> tools/setup.py | 4 +++-
>> 1 file changed, 3 insertions(+), 1 deletion(-)
>> ----------------------------------------------------------------------
>>
>>
>> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/724837dd/tools/setup.py
>> ----------------------------------------------------------------------
>> diff --git a/tools/setup.py b/tools/setup.py
>> index 57f7c64..38db29a 100755
>> --- a/tools/setup.py
>> +++ b/tools/setup.py
>> @@ -295,7 +295,8 @@ def createIndex():
>> "type" : "string"
>> },
>> "subject" : {
>> - "type" : "string"
>> + "type" : "string",
>> + "fielddata": True
>> },
>> "to" : {
>> "type" : "string"
>> @@ -440,6 +441,7 @@ def createIndex():
>> },
>> "subject" : {
>> "type" : "string",
>> + "fielddata": True
>> # "index" : "not_analyzed"
>> },
>> "to" : {
>>
Re: [3/5] incubator-ponymail git commit: add fielddata setting, so
word cloud will work
Posted by sebb <se...@gmail.com>.
Is there a bug reference for this?
What about the Changelog?
On 30 October 2017 at 15:13, <hu...@apache.org> wrote:
> add fielddata setting, so word cloud will work
>
>
> Project: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/repo
> Commit: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/commit/724837dd
> Tree: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/tree/724837dd
> Diff: http://git-wip-us.apache.org/repos/asf/incubator-ponymail/diff/724837dd
>
> Branch: refs/heads/master
> Commit: 724837dda8c757c17c9572ef2c5a34e61a4468b9
> Parents: 58d85c4
> Author: Daniel Gruno <hu...@apache.org>
> Authored: Mon Oct 30 16:11:57 2017 +0100
> Committer: Daniel Gruno <hu...@apache.org>
> Committed: Mon Oct 30 16:11:57 2017 +0100
>
> ----------------------------------------------------------------------
> tools/setup.py | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
> ----------------------------------------------------------------------
>
>
> http://git-wip-us.apache.org/repos/asf/incubator-ponymail/blob/724837dd/tools/setup.py
> ----------------------------------------------------------------------
> diff --git a/tools/setup.py b/tools/setup.py
> index 57f7c64..38db29a 100755
> --- a/tools/setup.py
> +++ b/tools/setup.py
> @@ -295,7 +295,8 @@ def createIndex():
> "type" : "string"
> },
> "subject" : {
> - "type" : "string"
> + "type" : "string",
> + "fielddata": True
> },
> "to" : {
> "type" : "string"
> @@ -440,6 +441,7 @@ def createIndex():
> },
> "subject" : {
> "type" : "string",
> + "fielddata": True
> # "index" : "not_analyzed"
> },
> "to" : {
>