You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael Ryan (JIRA)" <ji...@apache.org> on 2014/12/03 21:48:13 UTC
[jira] [Comment Edited] (SOLR-6469) Solr search with multicore +
grouping + highlighting cause NPE
[ https://issues.apache.org/jira/browse/SOLR-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233501#comment-14233501 ]
Michael Ryan edited comment on SOLR-6469 at 12/3/14 8:48 PM:
-------------------------------------------------------------
We ran into this exception today, though for a slightly different reason. Simplified request looks like this:
q=\*:\*&fl=articleId&rows=1&hl=true&shards=foo1,foo2
The workaround in this case is to either request additional fields besides the unique key field, or to not request highlighting.
This hits the same NPE at HighlightComponent.java:189. It throws an NPE because more highlighting data is returned from the shards than are in resultIds. This happens because onePassDistributedQuery is set to true in QueryComponent, and so the highlight data is requested from the shards on the first (single) pass.
The NPE is easy enough to fix for my issue - just add a null check before sdoc is used, and skip it if null. But there is still an underlying inefficiency - many more docs are highlighted on the shards than are actually necessary. If you have a lot of shards, this highlighting becomes non-trivial (e.g., highlighting thousands of docs instead of hundreds). I don't see any way to fix this without doing a second pass. Maybe ignore onePassDistributedQuery if highlights are requested? (That'll make the distributed logic in QueryComponent/HighlightComponent even more fun.)
Should I create another Jira for my issue since it doesn't involve grouping?
was (Author: michaelryan):
We ran into this exception today, though for a slightly different reason. Request looks like this:
q=\*:\*&fl=articleId&rows=1&hl=true&shards=foo1,foo2
The workaround in this case is to either request additional fields besides the unique key field, or to not request highlighting in this case.
This hits the same NPE at HighlightComponent.java:189. It throws an NPE because more highlighting data is returned from the shards than are in resultIds. This happens because onePassDistributedQuery is set to true in QueryComponent, and so the highlight data is requested from the shards on the first (single) pass.
The NPE is easy enough to fix for my issue - just add a null check before sdoc is used, and skip it if null. But there is still an underlying inefficiency - many more docs are highlighted on the shards than are actually necessary. If you have a lot of shards, this highlighting becomes non-trivial (e.g., highlighting thousands of docs instead of hundreds). I don't see any way to fix this without doing a second pass. Maybe ignore onePassDistributedQuery if highlights are requested? (That'll make the distributed logic in QueryComponent/HighlightComponent even more fun.)
Should I create another Jira for my issue since it doesn't involve grouping?
> Solr search with multicore + grouping + highlighting cause NPE
> --------------------------------------------------------------
>
> Key: SOLR-6469
> URL: https://issues.apache.org/jira/browse/SOLR-6469
> Project: Solr
> Issue Type: Bug
> Components: highlighter, multicore, SearchComponents - other
> Affects Versions: 4.8.1
> Environment: Windows 7, Intellij
> Reporter: Shay Sofer
> Assignee: Noble Paul
> Labels: patch
> Attachments: SOLR-6469.patch
>
>
> Integration of Grouping + shards + highlighting cause NullPointerException.
> Query: localhost:8983/solr/Global_A/select?q=%2Btext%3A%28shay*%29+&rows=100&fl=id%2CobjId%2Cnull&shards=http%3A%2F%2F127.0.0.1%3A8983%2Fsolr%2F0_A%2Chttp%3A%2F%2F127.0.0.1%3A8983%2Fsolr%2FGlobal_A&group=true&group.query=name__s%3Ashay&sort=name__s_sort+asc&hl=true
> results:
> java.lang.NullPointerException
> at org.apache.solr.handler.component.HighlightComponent.finishStage(HighlightComponent.java:189)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:330)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)
> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:722)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org