You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "Isabelle Giguere (Jira)" <ji...@apache.org> on 2021/01/29 16:22:00 UTC
[jira] [Commented] (SOLR-8319) NPE when creating pivot
[ https://issues.apache.org/jira/browse/SOLR-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274551#comment-17274551 ]
Isabelle Giguere commented on SOLR-8319:
----------------------------------------
The PR (937) in the linked issue SOLR-13209 suggest throwing a SolrException from SolrIndexSearcher.getDocSet(Query), which is certainly more elegant than an NPE.
I applied PR 937, on Solr 8.7.0, to see if I would get a SolrException for the facet request. I which case, I would have caught the exception, somewhere in the fact pivot processing (we shouldn't send an error message as a response for a valid facet pivot query).
But my facet pivot request still ends with NPE. The request goes through SolrIndexSearcher.numDocs(Query, DocSet) and SolrIndexSearcher.getPositiveDocSet(Query) :
{noformat}
Error from server at null: java.lang.NullPointerException at java.base/java.util.concurrent.ConcurrentHashMap.get(Unknown Source) at org.apache.solr.util.ConcurrentLFUCache.get(ConcurrentLFUCache.java:163) at org.apache.solr.search.LFUCache.get(LFUCache.java:198) at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:846) at org.apache.solr.search.SolrIndexSearcher.numDocs(SolrIndexSearcher.java:2069) at org.apache.solr.handler.component.PivotFacetProcessor.getSubsetSize(PivotFacetProcessor.java:355) at org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:218) at org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:166) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:280) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:360) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:214) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2627) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:415) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
{noformat}
The fact is, in a facet pivot request, the user has no actual control on the Query that ends up in methods of SolrIndexSearcher. The queries are generated from the facet field content. Checking for null, as in the attached patch, may be the only thing we can do.
It is strange, however, that setting a value for facet.limit will return results... or an NPE, depending on the value.
From my experiments, a facet.limit value of -1 returns all results, other values return partial results, but only up to a point, after that, it's NPE.
facet.limit value:
-1 = all results
0 = no results
Other value = partial results OR NPE
1 = 1 result
2 = 2 results
3 = 3 results
4 = NPE
So... Whatever happens when facet.limit=-1 is handled properly, but for values greater than (what ?) are not.
> NPE when creating pivot
> -----------------------
>
> Key: SOLR-8319
> URL: https://issues.apache.org/jira/browse/SOLR-8319
> Project: Solr
> Issue Type: Bug
> Reporter: Neil Ireson
> Priority: Major
> Attachments: SOLR-8319.patch
>
>
> I get a NPE, the trace is shown at the end.
> The problem seems to be this line in the getSubset method:
> Query query = ft.getFieldQuery(null, field, pivotValue);
> Which takes a value from the index and then analyses it to create a query. I believe the problem is that when my analysis process is applied twice it results in a null query. OK this might be seen as my issue because of dodgy analysis, I thought it might be because I have the wrong order with LengthFilterFactory before EnglishPossessiveFilterFactory and KStemFilterFactory, i.e.:
> <filter class="solr.LengthFilterFactory" min="3" max="32"/>
> <filter class="solr.EnglishPossessiveFilterFactory"/>
> <filter class="solr.KStemFilterFactory"/>
> So that "cat's" -> "cat" -> "", however any filter order I tried still resulted in a NPE, and perhaps there is a viable case where parsing a term twice results in a null query.
> The thing is I don't see why when the query term comes from the index it has to undergo any analysis. If the term is from the index can it not simply be created using a TermQuery, which I would imagine would also be faster. I altered the "getFieldQuery" line above to the following and that has fixed my NPE issue.
> Query query = new TermQuery(new Term(field.getName(), pivotValue));
> So far this hasn't caused any other issues but perhaps that is due to my use of Solr, rather than actually fixing an issue.
> o.a.s.c.SolrCore java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:91)
> at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:130)
> at org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.java:1296)
> at org.apache.solr.handler.component.PivotFacetProcessor.getSubset(PivotFacetProcessor.java:375)
> at org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:305)
> at org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:228)
> at org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:170)
> at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:262)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org