You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shawn Heisey (JIRA)" <ji...@apache.org> on 2018/04/27 14:45:00 UTC
[jira] [Commented] (SOLR-12283) NULLPointerException Data Import
Handle SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456546#comment-16456546 ]
Shawn Heisey commented on SOLR-12283:
-------------------------------------
Opening a bug is premature. Attaching publicly available jars to the issue is unnecessary.
These exceptions basically shouldn't be happening. The first one is a very fundamental issue with methods that all Java classes should have even if they aren't actually defined. The later ones show that an important data point in the dataimporter class is not defined -- it's null. I do not know how that can happen.
Can you share a solr.log file where the problem has happened? We need to see the errors in context with everything else that's happening.
> NULLPointerException Data Import Handle SolrCloud
> -------------------------------------------------
>
> Key: SOLR-12283
> URL: https://issues.apache.org/jira/browse/SOLR-12283
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - DataImportHandler
> Affects Versions: 6.6.1, 7.3
> Environment: Debian
> SolrCloud
> Reporter: Maxence SAUNIER
> Priority: Blocker
> Attachments: indexation_events.xml, mysql-connector-java-5.1.46-bin.jar, mysql-connector-java-5.1.46.jar, request_handler_config.json, solr-core-7.3.0.jar, solr-dataimporthandler-7.3.0.jar, solr-dataimporthandler-extras-7.3.0.jar, solr-solrj-7.3.0.jar
>
>
> Hello,
> It's been 2 weeks that I try to correct this problem with the community user-solr but no success. I seriously wonder if this is not a problem in the code. I do not have the impression that many people use DIH with Solr's cloud version.
> On Internet, no similar problem.
> For information, the following configuration of DIH comes from DIHs that work in production on a single Solr server. The connections to the databases are therefore correct.
> *Errors messages:*
> {panel:title=DataImporter}
> {code:java}
> Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter
> at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:339)
> at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:420)
> at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
> at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:183)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:530)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
> at org.apache.solr.handler.dataimport.DocBuilder.loadClass(DocBuilder.java:935)
> at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:335)
> ... 42 more
> {code}
> {panel}
> {panel:title=RequestHandlerBase}
> {code:java}
> java.lang.NullPointerException
> at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:429)
> at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
> at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:183)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:530)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> {panel}
> {panel:title=HttpSolrCall}
> {code:java}
> null:java.lang.NullPointerException
> at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:429)
> at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
> at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:183)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:530)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> {panel}
> *Process to reproduce the problem:*
> Create .system collection for the BLOB
> {code}
> curl http://localhost:8983/solr/admin/collections?action=CREATE&name=.system
> {code}
> Add Data Import Handler JAR file.
> {code}
> curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @solr-dataimporthandler-7.3.0.jar http://localhost:8983/solr/.system/blob/data-import-handler
> {code}
> Add the runtime lib on the configoverlay.json with the API
> {code}
> curl http://localhost:8983/solr/unicorn/config -H 'Content-type:application/json' -d '{"add-runtimelib": { "name":"data-import-handler", "version":1 }}'
> {code}
> Send mysql connector on the blob. I have test with the connector-bin.jar but I have the same error.
> {code}
> curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @mysql-connector-java-5.1.46.jar http://localhost:8983/solr/.system/blob/mysql-connector-java
> {code}
> Add runtime
> {code}
> curl http://localhost:8983/solr/unicorn/config -H 'Content-type:application/json' -d '{"add-runtimelib": { "name":"mysql-connector-java", "version":1 }}'
> {code}
> Personally, I create a DIH folder on the collection config path on the cloud. It's not necessary.
> And I put the DIH config file on the cloud with the zkcli (it's a personnal script to overload the zkcli.sh on Solr but it's the same result. My file is on the cloud.)
> {code}
> ./zkcli.py putfile /configs/unicorn/DIH/ DIH/indexation_events.xml
> {code}
> Add request handler config:
> {code}
> curl -X POST http://{solr_server}:8983/solr/{collection_name}/config -H 'Content-type:application/json' -d '
> {
> "add-requesthandler":
> {
> "name": "/test-dih",
> "class": "org.apache.solr.handler.dataimport.DataImportHandler",
> "runtimeLib": true,
> "version": 1,
> "defaults":
> {
> "config": "DIH/indexation_events.xml"
> }
> }
> } '
> {code}
> Reload the config:
> {code}
> curl http://localhost:8983/solr/unicorn/test-dih?command=reload-config&commit=true&debug=true
> {code}
> Import and have error:
> {code}
> curl http://localhost:8983/solr/unicorn/test-dih?command=full-import&commit=true&debug=true
> {code}
> I have test to add on request handler and BLOB this jar files but not change:
> # data-import-handler-extras
> # solr-core
> # solr-solrj
> I have join all files used in this process.
> Thansk you.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org