You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2020/12/13 07:04:21 UTC
[GitHub] [skywalking] tristan-tsl opened a new issue #6000: restart oap failure in version 8.3-es7
tristan-tsl opened a new issue #6000:
URL: https://github.com/apache/skywalking/issues/6000
Please answer these questions before submitting your issue.
- Why do you submit this issue?
- [ ] Question or discussion
- [x ] Bug
- [ ] Requirement
- [ ] Feature or performance improvement
___
### Question
- What do you want to know?
restart oap failure in version 8.3-es7, should i need to mount data diretory in outer?
___
### Bug
- Which version of SkyWalking, OS and JRE?
8.3
- Which company or project?
oap
- What happened?
If possible, provide a way to reproduce the error. e.g. demo application, component version.
2020-12-13 07:01:45,756 - org.apache.skywalking.oap.server.library.client.elasticsearch.ElasticSearchClient -5138 [main] INFO [] - elasticsearch cluster nodes: 192.168.2.35:9200
2020-12-13 07:01:46,334 - org.apache.skywalking.oap.server.core.storage.model.ModelInstaller -5716 [main] INFO [] - table: endpoint_traffic does not exist
2020-12-13 07:01:46,335 - org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.StorageEsInstaller -5717 [main] INFO [] - index endpoint_traffic's columnTypeEsMapping builder str: {properties={service_id={type=keyword}, name={copy_to=name_match, type=keyword}, time_bucket={type=long}, name_match={analyzer=oap_analyzer, type=text}}}
2020-12-13 07:01:46,559 - org.apache.skywalking.oap.server.starter.OAPServerBootstrap -5941 [main] ERROR [] - Elasticsearch exception [type=validation_exception, reason=Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;]
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=validation_exception, reason=Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;]
at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177) ~[elasticsearch-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1793) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1770) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1527) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1499) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1466) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.IndicesClient.create(IndicesClient.java:131) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch7.client.ElasticSearch7Client.createIndex(ElasticSearch7Client.java:109) ~[storage-elasticsearch7-plugin-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.StorageEsInstaller.createTable(StorageEsInstaller.java:92) ~[storage-elasticsearch-plugin-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.core.storage.model.ModelInstaller.whenCreating(ModelInstaller.java:55) ~[server-core-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.core.storage.model.StorageModels.addModelListener(StorageModels.java:87) ~[server-core-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch7.StorageModuleElasticsearch7Provider.start(StorageModuleElasticsearch7Provider.java:214) ~[storage-elasticsearch7-plugin-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.library.module.BootstrapFlow.start(BootstrapFlow.java:49) ~[library-module-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.library.module.ModuleManager.init(ModuleManager.java:62) ~[library-module-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.starter.OAPServerBootstrap.start(OAPServerBootstrap.java:43) [server-bootstrap-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.starter.OAPServerStartUp.main(OAPServerStartUp.java:27) [server-starter-es7-8.3.0.jar:8.3.0]
Suppressed: org.elasticsearch.client.ResponseException: method [PUT], host [http://192.168.2.35:9200], URI [/endpoint_traffic-20201213?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}],"type":"validation_exception","reason":"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"},"status":400}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:283) ~[elasticsearch-rest-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:261) ~[elasticsearch-rest-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235) ~[elasticsearch-rest-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1514) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1499) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1466) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.elasticsearch.client.IndicesClient.create(IndicesClient.java:131) ~[elasticsearch-rest-high-level-client-7.5.0.jar:7.5.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch7.client.ElasticSearch7Client.createIndex(ElasticSearch7Client.java:109) ~[storage-elasticsearch7-plugin-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch.base.StorageEsInstaller.createTable(StorageEsInstaller.java:92) ~[storage-elasticsearch-plugin-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.core.storage.model.ModelInstaller.whenCreating(ModelInstaller.java:55) ~[server-core-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.core.storage.model.StorageModels.addModelListener(StorageModels.java:87) ~[server-core-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.storage.plugin.elasticsearch7.StorageModuleElasticsearch7Provider.start(StorageModuleElasticsearch7Provider.java:214) ~[storage-elasticsearch7-plugin-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.library.module.BootstrapFlow.start(BootstrapFlow.java:49) ~[library-module-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.library.module.ModuleManager.init(ModuleManager.java:62) ~[library-module-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.starter.OAPServerBootstrap.start(OAPServerBootstrap.java:43) [server-bootstrap-8.3.0.jar:8.3.0]
at org.apache.skywalking.oap.server.starter.OAPServerStartUp.main(OAPServerStartUp.java:27) [server-starter-es7-8.3.0.jar:8.3.0]
___
### Requirement or improvement
- Please describe your requirements or improvement suggestions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] tristan-tsl commented on issue #6000: restart oap failure in version 8.3-es7
Posted by GitBox <gi...@apache.org>.
tristan-tsl commented on issue #6000:
URL: https://github.com/apache/skywalking/issues/6000#issuecomment-743967209
this is problem is by es
visit curl -XGET localhost:9200/_cluster/allocation/explain?pretty to analysis es's status
temp update by request url : curl -X PUT localhost:9200/_cluster/settings -H "Content-Type: application/json" -d '{ "persistent": { "cluster.max_shards_per_node": "3000" } }'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6000: restart oap failure in version 8.3-es7
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6000:
URL: https://github.com/apache/skywalking/issues/6000#issuecomment-743966433
> this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;]
Is this log clear enough?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #6000: restart oap failure in version 8.3-es7
Posted by GitBox <gi...@apache.org>.
wu-sheng closed issue #6000:
URL: https://github.com/apache/skywalking/issues/6000
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org