You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Ulli Franz <ul...@outlook.com> on 2024/05/12 17:41:08 UTC
CREATE via GUI or API fails - BaseHttpSolrClient$RemoteSolrException: Error from server
Hello everyone
I use a 3-node solrCloud with 3 Zookeeper nodes (all on VMWare at one physical machine) and got stuck while creating new collections. From time to time it works via API or GUI, but mostly i receive a BaseHttpSolrClient$RemoteSolrException: Error from server
Here is what I see:
[cid:bc5bb68a-15af-4c1f-aece-4929283ff29f]
These are my already created Collections:
[cid:5dd51a50-a540-4041-919d-a655530e9d2e]
In security.json is set "blockUnknown": False in authentication and at the end of the file, which is already copied to zookeeper.
This is how the zookeeper status looks like:
[cid:d7da20d2-1ba3-4c49-bcc4-9ac3158e55bb]
When trying to create a new collection i use either the GUI or (Python code which works fine e.g. to get something like the Clusterstatus):
createCollection = "http://s03:8983/solr/admin/collections?action=CREATE&name=GPL_COLLECTION_10&replicationFactor=2&numShards=1"
status = requests.post(createCollection, auth=HTTPBasicAuth(user, password))
and receive status 500:
{'responseHeader': {'status': 500, 'QTime': 180017},
'error': {'metadata': ['error-class',
'org.apache.solr.common.SolrException',
'root-error-class',
'org.apache.solr.common.SolrException'],
'msg': 'create the collection time out:180s',
'trace': 'org.apache.solr.common.SolrException: create the collection time out:180s\n\tat org.apache.solr.handler.admin.CollectionsHandler.submitCollectionApiCommand(CollectionsHandler.java:415)\n\tat org.apache.solr.handler.admin.api.CreateCollectionAPI.createCollection(CreateCollectionAPI.java:133)\n\tat org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.lambda$static$0(CollectionsHandler.java:513)\n\tat org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.execute(CollectionsHandler.java:1265)\n\tat org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:315)\n\tat org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:293)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:932)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:880)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:541)\n\tat org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:262)\n\tat org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:219)\n\tat org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:246)\n\tat org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:215)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:213)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)\n\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)\n\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:149)\n\tat org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:228)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:141)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:563)\n\tat org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)\n\tat org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\n',
'code': 500}}
the logfile of the leader says further:
<head>
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://s01:8983/solr/admin/cores: Expected mime type in [application/octet-stream, application/vnd.apache.solr.javabin] but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 403 Forbidden</title>
</head>
<body><h2>HTTP ERROR 403 Forbidden</h2>
<table>
<tr><th>URI:</th><td>/solr/admin/cores</td></tr>
<tr><th>STATUS:</th><td>403</td></tr>
<tr><th>MESSAGE:</th><td>Forbidden</td></tr>
<tr><th>SERVLET:</th><td>-</td></tr>
</table>
</body>
</html>
at org.apache.solr.client.solrj.impl.Http2SolrClient.checkContentType(Http2SolrClient.java:974) ~[?:?]
at org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:863) ~[?:?]
at org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:576) ~[?:?]
at org.apache.solr.client.solrj.impl.Http2SolrClient$1.lambda$onHeaders$0(Http2SolrClient.java:481) ~[?:?]
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:294) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
2024-05-12 17:04:03.874 INFO (zkCallback-11-thread-11) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/GPL_COLLECTION_10/state.json zxid: 438086664484] for collection [GPL_CO>
2024-05-12 17:04:03.875 INFO (zkCallback-11-thread-12) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/GPL_COLLECTION_10/state.json zxid: 438086664484] for collection [GPL_CO>
2024-05-12 17:04:03.904 INFO (OverseerThreadFactory-17-thread-2) [c:GPL_COLLECTION_10] o.a.s.c.a.c.CreateCollectionCmd Cleaned up artifacts for failed create collection for [GPL_COLLECTION_10]
2024-05-12 17:04:03.904 ERROR (OverseerThreadFactory-17-thread-2) [c:GPL_COLLECTION_10] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection GPL_COLLECTION_10}, operation create failed => org.apache.solr.common.SolrException: Underlying core cr>
I am trying to solve this since days. Could you please help me out?
Thanks in andvance, please let me know if I should provide some more information.
Regards, Ulli
Re: CREATE via GUI or API fails - BaseHttpSolrClient$RemoteSolrException: Error from server
Posted by Jan Høydahl <ja...@cominvent.com>.
The images did not make it to the list. Can you share them elsewhere?
Does it work if you disable authentication plugin? I see a HTTP 403 error in the log which suggests there is an auth issue. Are you using forwardCredentials=true?
Which solr version are you on?
Jan Høydahl
> 16. mai 2024 kl. 21:03 skrev Ulli Franz <ul...@outlook.com>:
>
>
> Hello everyone
>
> I use a 3-node solrCloud with 3 Zookeeper nodes (all on VMWare at one physical machine) and got stuck while creating new collections. From time to time it works via API or GUI, but mostly i receive a BaseHttpSolrClient$RemoteSolrException: Error from server
>
> Here is what I see:
>
>
>
> These are my already created Collections:
>
>
>
> In security.json is set "blockUnknown": False in authentication and at the end of the file, which is already copied to zookeeper.
>
> This is how the zookeeper status looks like:
>
>
> When trying to create a new collection i use either the GUI or (Python code which works fine e.g. to get something like the Clusterstatus):
> createCollection = "http://s03:8983/solr/admin/collections?action=CREATE&name=GPL_COLLECTION_10&replicationFactor=2&numShards=1"
> status = requests.post(createCollection, auth=HTTPBasicAuth(user, password))
>
>
> and receive status 500:
>
> {'responseHeader': {'status': 500, 'QTime': 180017},
> 'error': {'metadata': ['error-class',
> 'org.apache.solr.common.SolrException',
> 'root-error-class',
> 'org.apache.solr.common.SolrException'],
> 'msg': 'create the collection time out:180s',
> 'trace': 'org.apache.solr.common.SolrException: create the collection time out:180s\n\tat org.apache.solr.handler.admin.CollectionsHandler.submitCollectionApiCommand(CollectionsHandler.java:415)\n\tat org.apache.solr.handler.admin.api.CreateCollectionAPI.createCollection(CreateCollectionAPI.java:133)\n\tat org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.lambda$static$0(CollectionsHandler.java:513)\n\tat org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.execute(CollectionsHandler.java:1265)\n\tat org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:315)\n\tat org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:293)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:226)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:932)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:880)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:541)\n\tat org.apache.solr.servlet.SolrDispatchFilter.dispatch(SolrDispatchFilter.java:262)\n\tat org.apache.solr.servlet.SolrDispatchFilter.lambda$doFilter$0(SolrDispatchFilter.java:219)\n\tat org.apache.solr.servlet.ServletUtils.traceHttpRequestExecution2(ServletUtils.java:246)\n\tat org.apache.solr.servlet.ServletUtils.rateLimitRequest(ServletUtils.java:215)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:213)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)\n\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:210)\n\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1635)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:527)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:131)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:598)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1384)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1306)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:149)\n\tat org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:228)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:141)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:301)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:822)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:563)\n\tat org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)\n\tat org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277)\n\tat org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\n',
> 'code': 500}}
>
> the logfile of the leader says further:
>
> <head>
> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://s01:8983/solr/admin/cores: Expected mime type in [application/octet-stream, application/vnd.apache.solr.javabin] but got text/html. <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
> <title>Error 403 Forbidden</title>
> </head>
> <body><h2>HTTP ERROR 403 Forbidden</h2>
> <table>
> <tr><th>URI:</th><td>/solr/admin/cores</td></tr>
> <tr><th>STATUS:</th><td>403</td></tr>
> <tr><th>MESSAGE:</th><td>Forbidden</td></tr>
> <tr><th>SERVLET:</th><td>-</td></tr>
> </table>
>
> </body>
> </html>
>
> at org.apache.solr.client.solrj.impl.Http2SolrClient.checkContentType(Http2SolrClient.java:974) ~[?:?]
> at org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:863) ~[?:?]
> at org.apache.solr.client.solrj.impl.Http2SolrClient.processErrorsAndResponse(Http2SolrClient.java:576) ~[?:?]
> at org.apache.solr.client.solrj.impl.Http2SolrClient$1.lambda$onHeaders$0(Http2SolrClient.java:481) ~[?:?]
> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:294) ~[?:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
> at java.lang.Thread.run(Thread.java:829) [?:?]
>
> 2024-05-12 17:04:03.874 INFO (zkCallback-11-thread-11) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/GPL_COLLECTION_10/state.json zxid: 438086664484] for collection [GPL_CO>
> 2024-05-12 17:04:03.875 INFO (zkCallback-11-thread-12) [] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/GPL_COLLECTION_10/state.json zxid: 438086664484] for collection [GPL_CO>
> 2024-05-12 17:04:03.904 INFO (OverseerThreadFactory-17-thread-2) [c:GPL_COLLECTION_10] o.a.s.c.a.c.CreateCollectionCmd Cleaned up artifacts for failed create collection for [GPL_COLLECTION_10]
> 2024-05-12 17:04:03.904 ERROR (OverseerThreadFactory-17-thread-2) [c:GPL_COLLECTION_10] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection GPL_COLLECTION_10}, operation create failed => org.apache.solr.common.SolrException: Underlying core cr>
>
>
> I am trying to solve this since days. Could you please help me out?
> Thanks in andvance, please let me know if I should provide some more information.
>
> Regards, Ulli
>
>
>
>