[jira] [Commented] (SOLR-13219) LTR FieldLengthFeature causes NullPointerException with documents that do not have the requested field

["Tomasz Elendt (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-13219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611273#comment-17611273 ] 

Tomasz Elendt commented on SOLR-13219:
--------------------------------------

I've been investigating a similar issue, although not in the context of dynamic fields. I noticed that a regular, non-dynamic field could lead to {{NullPointerException}} as well (this could be easily reproduced by setting [{{{}title{}}}/{{{}description{}}} fields from test-files schema|https://github.com/apache/solr/blob/a4905f3/solr/modules/ltr/src/test-files/solr/collection1/conf/schema.xml#L25-L26] as non-stored). Needless to say, {{FieldLengthFeature}} should still work with non-stored fields as long as they have norms (and even if they don't things should not break with NPE).

I debugged the issue and it seems that the culprit is this {{searcher.doc(0).getField(field)}} [line in |https://github.com/apache/solr/blob/1c73e45/solr/modules/ltr/src/java/org/apache/solr/ltr/feature/FieldLengthFeature.java#L137][FieldLengthFeatureScorer constructor|https://github.com/apache/solr/blob/1c73e45/solr/modules/ltr/src/java/org/apache/solr/ltr/feature/FieldLengthFeature.java#L137] returns {{null}} for non-stored fields.

Honestly, I don't understand why {{FieldLengthFeatureScorer}} checks whether the field omits norms or not. I also don't understand why this class is public. It's only used in {{FieldLengthFeature#scorer}} and this already checks whether norms are there or not (for empty norms a constant {{0}} {{ValueFeatureScorer}} is returned).

I can provide a fix for that.

> LTR FieldLengthFeature causes NullPointerException with documents that do not have the requested field
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SOLR-13219
>                 URL: https://issues.apache.org/jira/browse/SOLR-13219
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - LTR
>    Affects Versions: 7.6
>         Environment: OSX - Mojave
>  
>            Reporter: Nick Veenhof
>            Priority: Major
>              Labels: ltr, nullpointerexception
>
> 1) Create an index with a dynamic field, eg tm_my_fulltext_field based on tm_* dynamic field
> 2) Index your documents and make sure not all of the documents have the tm_my_fulltext_field field.
> 3) Set up a feature store as follows
> {code:java}
> {
> "initArgs": {},
> "initializedOn": "2019-01-21T21:09:59.011Z",
> "updatedSinceInit": "2019-01-21T22:01:14.113Z",
> "managedList": [
> {
> "name": "descriptionLength",
> "class": "org.apache.solr.ltr.feature.FieldLengthFeature",
> "params": {
> "field": "tm_my_fulltext_field"
> },
> "store": "_DEFAULT_"
> }
> ]
> }
> {code}
> Try to fetch the feature in a query
>  
> {code:java}
> http://localhost:8983/solr/dropsolid8/select?q=europe&json.nl=flat&omitHeader=true&fl=ss_search_api_id,ss_search_api_language,hash,[features]&start=0&rows=10&wt=json&rq={!ltr efi.query=europe model=originalScoreModel}}&df=tm_X3b_nl_rendered_item
> {code}
> You'll see
>  
> {code:java}
> 2019-02-05 13:45:36.087 ERROR (qtp690521419-21) [ x:dropsolid8] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: java.lang.NullPointerException
> at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:140)
> at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1607)
> at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1421)
> at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:568)
> at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1435)
> at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:375)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:531)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException
> at org.apache.solr.ltr.feature.FieldLengthFeature$FieldLengthFeatureWeight$FieldLengthFeatureScorer.<init>(FieldLengthFeature.java:134)
> at org.apache.solr.ltr.feature.FieldLengthFeature$FieldLengthFeatureWeight.scorer(FieldLengthFeature.java:120)
> at org.apache.solr.ltr.LTRScoringQuery$ModelWeight.scorer(LTRScoringQuery.java:467)
> at org.apache.solr.ltr.LTRRescorer.scoreFeatures(LTRRescorer.java:164)
> at org.apache.solr.ltr.LTRRescorer.rescore(LTRRescorer.java:119)
> at org.apache.solr.search.ReRankCollector.topDocs(ReRankCollector.java:106)
> ... 46 more
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org