You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ponymail.apache.org by johnament <gi...@git.apache.org> on 2016/12/30 17:18:32 UTC
[GitHub] incubator-ponymail issue #313: setup.py : prompt for ES index settings
GitHub user johnament opened an issue:
https://github.com/apache/incubator-ponymail/issues/313
setup.py : prompt for ES index settings
When creating the index, there's no options to specify things such as:
1. number of shards
1. number of replicas
We should add those to the setup.
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ponymail issue #313: setup.py : prompt for ES index settings
Posted by sebbASF <gi...@git.apache.org>.
Github user sebbASF commented on the issue:
https://github.com/apache/incubator-ponymail/issues/313
AIUI this is only possible with ES 5.x; previously such settings were done externally.
So in order to implement this, ES 5.x must be made a pre-condition, which is not the case at present.
IMO the defaults should be values that are suitable for test installations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ponymail issue #313: setup.py : prompt for ES index settings
Posted by sebbASF <gi...@git.apache.org>.
Github user sebbASF commented on the issue:
https://github.com/apache/incubator-ponymail/issues/313
Not sure where I got the impression that one could only set the shard count through static config in earlier versions. So ignore the bit about requiring ES 5.0.
Regarding test system: I was thinking of a system mainly used for unit tests, rather than performance or volume testing. I.e. the sort of system one might want to set up to try the code out.
That is likely to have fewer than 1m docs, so a single shard is sufficient. This will make it easier for the first time user. Someone intending to use the system for more exacting purposes would/should review the settings more carefully.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ponymail issue #313: setup.py : prompt for ES index settings
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the issue at:
https://github.com/apache/incubator-ponymail/issues/313
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-ponymail issue #313: setup.py : prompt for ES index settings
Posted by johnament <gi...@git.apache.org>.
Github user johnament commented on the issue:
https://github.com/apache/incubator-ponymail/issues/313
Not sure what you mean. Updating index settings w/ number of replicas has always been configurable on the fly https://www.elastic.co/guide/en/elasticsearch/guide/current/replica-shards.html . Its purposely dynamic since a cluster is meant to grow/shrink. So yes, it probably makes sense that the number of replicas defaults to 0.
Number of shards though is based on your data size, not a "test system" vs "production system." See https://www.elastic.co/guide/en/elasticsearch/guide/current/_index_settings.html for info on setting the value. When you create an index, you can specify the shard count - https://www.elastic.co/guide/en/elasticsearch/reference/2.4/indices-create-index.html - the way it was being done in the module isn't quite right, it's setting a default for the entire node, but each index may need different shard counts. E.g. a single shard may make sense for a single mailing list, or 6 months of history, but when it comes to importing test data sets that span larger amounts multiple shards may make sense (may make sense to use the standard `ceil(doc_count/1000000)` function
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---