You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Varun Thacker (JIRA)" <ji...@apache.org> on 2018/08/04 17:38:00 UTC
[jira] [Commented] (SOLR-12615) HashQParserPlugin will throw an NPE
for string hash key and documents have empty value
[ https://issues.apache.org/jira/browse/SOLR-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16569255#comment-16569255 ]
Varun Thacker commented on SOLR-12615:
--------------------------------------
Updated patch. All tests pass. I plan on committing this later today
> HashQParserPlugin will throw an NPE for string hash key and documents have empty value
> --------------------------------------------------------------------------------------
>
> Key: SOLR-12615
> URL: https://issues.apache.org/jira/browse/SOLR-12615
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Varun Thacker
> Priority: Major
> Fix For: master (8.0), 7.5
>
> Attachments: SOLR-12615.patch, SOLR-12615.patch
>
>
> If I index documents where the partitionKeys keys field is missing from a few docs then the stream expression throws an NPE
> docs:
> {code:java}
> [
> {"id" : "1", "search_term_s" : "query1"},
> {"id" : "2", "search_term_s" : "query1"},
> {"id" : "3"},
> {"id" : "4"}
> ]{code}
>
> query
> {code:java}
> search(test_empty, q="*:*", fl="search_term_s,id" , sort="search_term_s desc", qt="/export", partitionKeys="search_term_s"){code}
>
> logs
> {code:java}
> INFO - 2018-08-03 01:44:36.156; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.core.SolrCore.Request; [test_empty_shard1_replica_n1] webapp=/solr path=/stream params={expr=search(test_empty,+q%3D"*:*",+fl%3D"search_term_s,id"+,+sort%3D"search_term_s+desc",+qt%3D"/export",+partitionKeys%3D"search_term_s")&_=1533260573672} status=0 QTime=11
> INFO - 2018-08-03 01:44:36.160; [ ] org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO - 2018-08-03 01:44:36.162; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.common.cloud.ZkStateReader; Updated live nodes from ZooKeeper... (0) -> (1)
> INFO - 2018-08-03 01:44:36.164; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at localhost:9888 ready
> INFO - 2018-08-03 01:44:36.207; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.core.SolrCore.Request; [test_empty_shard1_replica_n1] webapp=/solr path=/export params={q=*:*&distrib=false&indent=off&fl=search_term_s,id&sort=search_term_s+desc&partitionKeys=search_term_s&fq={!hash+workers%3D1+worker%3D0}&wt=json&version=2.2} status=500 QTime=36
> ERROR - 2018-08-03 01:44:36.209; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.servlet.HttpSolrCall; null:java.io.IOException: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
> at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:743)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:463)
> at org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:151)
> at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:140)
> at org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1196)
> at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:836)
> at org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1044)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1563)
> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1439)
> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:586)
> at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1435)
> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:375)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
> at org.apache.solr.handler.ExportHandler.handleRequestBody(ExportHandler.java:37)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:531)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
> at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
> ... 54 more
> Caused by: java.lang.NullPointerException
> at org.apache.lucene.util.CharsRefBuilder.copyUTF8Bytes(CharsRefBuilder.java:122)
> at org.apache.solr.schema.FieldType.indexedToReadable(FieldType.java:378)
> at org.apache.solr.search.HashQParserPlugin$BytesHash.hashCode(HashQParserPlugin.java:300)
> at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
> ... 55 more
> ERROR - 2018-08-03 01:44:36.213; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.client.solrj.io.stream.ExceptionStream; java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:400)
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:275)
> at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54)
> at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:397)
> at org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:83)
> at org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539)
> at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181)
> at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:209)
> at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:325)
> at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:120)
> at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:71)
> at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
> at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:531)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:394)
> ... 49 more
> Caused by: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
> at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:222)
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.next(CloudSolrStream.java:484)
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:507)
> at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:494)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> ... 1 more{code}
> The problem lies with HashQParserPlugin where we try to partition the data and if a document doesn't have a value for the partition field it throws an NPE
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org