You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Alexandros Paramythis <al...@contexity.ch> on 2019/06/26 08:48:13 UTC
Problems using a suggester component in the /select handler in cloud
mode
Hi everyone,
Environment:
Solr 7.5.0, cloud mode (but problem should be identical in multiple
versions, at least in 7.x)
Summary:
We have a Solr configuration that returns suggestions in the course of a
normal search call (i.e., we have a 'suggest' component added to the
'last-components' for '/select' request handler). This does not work in
cloud mode, where we get an NPE in QueryComponent. This problem seems to
have been reported in various forms in the past -- see for example [1]
and [2] (links at the end of this email) -- but we couldn't find any
resolution (or in-depth discussion for that matter).
In more detail:
We have a suggest component configured as follows:
<searchComponent name="suggest" class="solr.SuggestComponent" >
<lst name="suggester">
<str name="name">default</str>
<str
name="classname">org.apache.solr.spelling.suggest.Suggester</str>
<str
name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
<str name="storeDir">dict_default</str>
<str name="suggestAnalyzerFieldType">text_suggest</str>
<str name="field">text_suggest</str>
<str name="buildOnStartup">true</str>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
</lst>
<lst name="suggester">
<str name="name">suggest_phrase</str>
<str
name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
<str name="storeDir">dict_suggest_phrase</str>
<str
name="suggestAnalyzerFieldType">text_suggest_phrase</str>
<str name="field">suggest_phrase</str>
<str name="buildOnStartup">true</str>
<str name="buildOnCommit">true</str>
<str name="buildOnOptimize">true</str>
</lst>
<lst name="suggester">
<str name="name">suggest_infix_shingle</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="indexPath">suggestInfixShingleDir</str>
<str
name="suggestAnalyzerFieldType">text_suggest_phrase</str>
<str name="field">suggest_phrase</str>
<str name="buildOnOptimize">true</str>
<str name="buildOnCommit">true</str>
<str name="buildOnStartup">true</str>
</lst>
<lst name="suggester">
<str name="name">suggest_prefix</str>
<str name="classname">Suggester</str>
<str name="lookupImpl">AnalyzingLookupFactory</str>
<str
name="suggestAnalyzerFieldType">text_suggest_prefix</str>
<str name="field">suggest_prefix</str>
<str name="buildOnOptimize">true</str>
<str name="buildOnCommit">true</str>
<str name="buildOnStartup">true</str>
</lst>
</searchComponent>
This component works without issued both in standalone and cloud mode,
when used as the sole component in a handler, such as in the following
excerpt:
<requestHandler name="/suggest" class="solr.SearchHandler"
startup="lazy">
<lst name="defaults">
<str name="suggest.dictionary">default</str>
<str name="suggest.dictionary">suggest_phrase</str>
<str
name="suggest.dictionary">suggest_infix_shingle</str>
<str name="suggest.dictionary">suggest_prefix</str>
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.highlight">false</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>
It also works when used along with other component in standalone mode,
such as in the following excerpt, where we use the suggest component to
get suggestions during a "normal" search call:
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
<str name="df">text_search</str>
<str name="defType">edismax</str>
<str name="qf">title^5.0 subtitle^3.0
abstract^2.0 text_search</str>
<str name="pf">title^5.0 subtitle^3.0
abstract^2.0 text_search</str>
<str name="ps">4</str>
<str name="spellcheck">on</str>
<str name="spellcheck.dictionary">default</str>
<str name="spellcheck.extendedResults">true</str>
<str name="spellcheck.count">10</str>
<str name="spellcheck.alternativeTermCount">5</str>
<str name="spellcheck.maxResultsForSuggest">5</str>
<str name="spellcheck.collate">true</str>
<str
name="spellcheck.collateExtendedResults">true</str>
<str name="spellcheck.maxCollationTries">10</str>
<str name="spellcheck.maxCollations">5</str>
<str name="suggest.dictionary">default</str>
<str name="suggest.dictionary">suggest_phrase</str>
<str
name="suggest.dictionary">suggest_infix_shingle</str>
<str name="suggest.dictionary">suggest_prefix</str>
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.highlight">false</str>
</lst>
<arr name="last-components">
<str>suggest</str>
<str>spellcheck</str>
</arr>
</requestHandler>
However, the above configuration does not work in cloud mode, where we
get an NPE if a search call is made:
o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
at org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1034)
at org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:885)
at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:531)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
at java.lang.Thread.run(Thread.java:748)
As far as we can tell, this is due to the fact that the failing code in
QueryComponent assumes that the shard response always contains a
'sort_values' section:
NamedList sortFieldValues =
(NamedList)(srsp.getSolrResponse().getResponse().get("sort_values"));
NamedList unmarshalledSortFieldValues = unmarshalSortValues(ss,
sortFieldValues, schema);
(the exception occurs in unmarshalSortValues(), which assumes the
sortFieldValues parameter is not null)
Are we doing something wrong that is preventing 'sort_values' from being
present in the shard response? Or is their absence OK, but the code
fails to account for it? Our guess is the second, because
umarshalSortValues() does check if the values are empty or not, and
handles that.
Can anyone help us with this issue? Thanks in advance for any pointers!
Alex
[1]
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201711.mbox/%3CCACKWnqOv=HFQwTxd7SwEqSTj-im+43FE+c9_zbTCPMWsp8vL_w@mail.gmail.com%3E
[2] https://issues.apache.org/jira/browse/SOLR-12060
Re: Problems using a suggester component in the /select handler in
cloud mode
Posted by Jason Gerlowski <ge...@gmail.com>.
Hi Alexandros,
The first step would be to package up your changes in a patch file, and
upload that to the JIRA you linked to in your initial email. (SOLR-12060).
More detailed instructions can be found here:
https://cwiki.apache.org/confluence/display/solr/HowToContribute#HowToContribute-Generatingapatch.
If you prefer, you can also create a PR on github with your changes, and
put a link to the PR on the JIRA ticket. (There are many guides out there
on creating Github PRs, so I won't get into that.)
Thanks for putting in the effort to share your work. Good luck!
Best,
Jason
On Tue, Jul 16, 2019 at 8:50 AM Alexandros Paramythis <
alexandros.paramythis@contexity.ch> wrote:
> Hi everyone,
>
> We have a fix for the problem described in the message below. Could anyone
> provide pointers to documentation on how we would go about contributing
> this back?
>
> Thanks in advance for your input,
>
> Alex
>
>
> On 26/06/2019 10:48, Alexandros Paramythis wrote:
>
> Hi everyone,
>
> Environment:
>
> Solr 7.5.0, cloud mode (but problem should be identical in multiple
> versions, at least in 7.x)
>
> Summary:
>
> We have a Solr configuration that returns suggestions in the course of a
> normal search call (i.e., we have a 'suggest' component added to the
> 'last-components' for '/select' request handler). This does not work in
> cloud mode, where we get an NPE in QueryComponent. This problem seems to
> have been reported in various forms in the past -- see for example [1] and
> [2] (links at the end of this email) -- but we couldn't find any resolution
> (or in-depth discussion for that matter).
>
> In more detail:
>
> We have a suggest component configured as follows:
>
> <searchComponent name="suggest" class="solr.SuggestComponent" >
>
> <lst name="suggester">
> <str name="name">default</str>
> <str
> name="classname">org.apache.solr.spelling.suggest.Suggester</str>
> <str
> name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
> <str name="storeDir">dict_default</str>
> <str name="suggestAnalyzerFieldType">text_suggest</str>
> <str name="field">text_suggest</str>
> <str name="buildOnStartup">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnOptimize">true</str>
> </lst>
>
> <lst name="suggester">
> <str name="name">suggest_phrase</str>
> <str
> name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
> <str name="storeDir">dict_suggest_phrase</str>
> <str
> name="suggestAnalyzerFieldType">text_suggest_phrase</str>
> <str name="field">suggest_phrase</str>
> <str name="buildOnStartup">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnOptimize">true</str>
> </lst>
>
> <lst name="suggester">
> <str name="name">suggest_infix_shingle</str>
> <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
> <str name="indexPath">suggestInfixShingleDir</str>
> <str
> name="suggestAnalyzerFieldType">text_suggest_phrase</str>
> <str name="field">suggest_phrase</str>
> <str name="buildOnOptimize">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnStartup">true</str>
> </lst>
>
> <lst name="suggester">
> <str name="name">suggest_prefix</str>
> <str name="classname">Suggester</str>
> <str name="lookupImpl">AnalyzingLookupFactory</str>
> <str
> name="suggestAnalyzerFieldType">text_suggest_prefix</str>
> <str name="field">suggest_prefix</str>
> <str name="buildOnOptimize">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnStartup">true</str>
> </lst>
>
> </searchComponent>
>
>
> This component works without issued both in standalone and cloud mode,
> when used as the sole component in a handler, such as in the following
> excerpt:
>
> <requestHandler name="/suggest" class="solr.SearchHandler"
> startup="lazy">
> <lst name="defaults">
> <str name="suggest.dictionary">default</str>
> <str name="suggest.dictionary">suggest_phrase</str>
> <str
> name="suggest.dictionary">suggest_infix_shingle</str>
> <str name="suggest.dictionary">suggest_prefix</str>
> <str name="suggest">true</str>
> <str name="suggest.count">10</str>
> <str name="suggest.highlight">false</str>
> </lst>
> <arr name="components">
> <str>suggest</str>
> </arr>
> </requestHandler>
>
>
> It also works when used along with other component in standalone mode,
> such as in the following excerpt, where we use the suggest component to get
> suggestions during a "normal" search call:
>
> <requestHandler name="/select" class="solr.SearchHandler">
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> <int name="rows">10</int>
> <str name="df">text_search</str>
>
> <str name="defType">edismax</str>
>
> <str name="qf">title^5.0 subtitle^3.0 abstract^2.0
> text_search</str>
> <str name="pf">title^5.0 subtitle^3.0 abstract^2.0
> text_search</str>
> <str name="ps">4</str>
> <str name="spellcheck">on</str>
> <str name="spellcheck.dictionary">default</str>
> <str name="spellcheck.extendedResults">true</str>
> <str name="spellcheck.count">10</str>
> <str
> name="spellcheck.alternativeTermCount">5</str>
> <str
> name="spellcheck.maxResultsForSuggest">5</str>
> <str name="spellcheck.collate">true</str>
> <str
> name="spellcheck.collateExtendedResults">true</str>
> <str name="spellcheck.maxCollationTries">10</str>
> <str name="spellcheck.maxCollations">5</str>
>
> <str name="suggest.dictionary">default</str>
> <str
> name="suggest.dictionary">suggest_phrase</str>
> <str
> name="suggest.dictionary">suggest_infix_shingle</str>
> <str
> name="suggest.dictionary">suggest_prefix</str>
> <str name="suggest">true</str>
> <str name="suggest.count">10</str>
> <str name="suggest.highlight">false</str>
> </lst>
>
> <arr name="last-components">
> <str>suggest</str>
> <str>spellcheck</str>
> </arr>
> </requestHandler>
>
> However, the above configuration does not work in cloud mode, where we get
> an NPE if a search call is made:
>
> o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
> at
> org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1034)
> at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:885)
> at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
> at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:531)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
>
> As far as we can tell, this is due to the fact that the failing code in
> QueryComponent assumes that the shard response always contains a
> 'sort_values' section:
>
> NamedList sortFieldValues =
> (NamedList)(srsp.getSolrResponse().getResponse().get("sort_values"));
> NamedList unmarshalledSortFieldValues = unmarshalSortValues(ss,
> sortFieldValues, schema);
>
> (the exception occurs in unmarshalSortValues(), which assumes the
> sortFieldValues parameter is not null)
>
> Are we doing something wrong that is preventing 'sort_values' from being
> present in the shard response? Or is their absence OK, but the code fails
> to account for it? Our guess is the second, because umarshalSortValues()
> does check if the values are empty or not, and handles that.
>
> Can anyone help us with this issue? Thanks in advance for any pointers!
>
> Alex
>
> [1]
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201711.mbox/%3CCACKWnqOv=HFQwTxd7SwEqSTj-im+43FE+c9_zbTCPMWsp8vL_w@mail.gmail.com%3E
>
> [2] https://issues.apache.org/jira/browse/SOLR-12060
>
>
>
> --
>
> *Dr. Alexandros Paramythis, M.Sc.*
> *CEO*
>
> CONTEXITY AG
> Technoparkstrasse 2
> CH-8406 Winterthur
>
> *T* +41 78 611 6542
> *E* alexandros.paramythis@contexity.ch
> *W* www.contexity.ch
>
>
Re: Problems using a suggester component in the /select handler in
cloud mode
Posted by Alexandros Paramythis <al...@contexity.ch>.
Hi everyone,
We have a fix for the problem described in the message below. Could
anyone provide pointers to documentation on how we would go about
contributing this back?
Thanks in advance for your input,
Alex
On 26/06/2019 10:48, Alexandros Paramythis wrote:
> Hi everyone,
>
> Environment:
>
> Solr 7.5.0, cloud mode (but problem should be identical in multiple
> versions, at least in 7.x)
>
> Summary:
>
> We have a Solr configuration that returns suggestions in the course of
> a normal search call (i.e., we have a 'suggest' component added to the
> 'last-components' for '/select' request handler). This does not work
> in cloud mode, where we get an NPE in QueryComponent. This problem
> seems to have been reported in various forms in the past -- see for
> example [1] and [2] (links at the end of this email) -- but we
> couldn't find any resolution (or in-depth discussion for that matter).
>
> In more detail:
>
> We have a suggest component configured as follows:
>
> <searchComponent name="suggest" class="solr.SuggestComponent" >
>
> <lst name="suggester">
> <str name="name">default</str>
> <str
> name="classname">org.apache.solr.spelling.suggest.Suggester</str>
> <str
> name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
> <str name="storeDir">dict_default</str>
> <str name="suggestAnalyzerFieldType">text_suggest</str>
> <str name="field">text_suggest</str>
> <str name="buildOnStartup">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnOptimize">true</str>
> </lst>
>
> <lst name="suggester">
> <str name="name">suggest_phrase</str>
> <str
> name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
> <str name="storeDir">dict_suggest_phrase</str>
> <str
> name="suggestAnalyzerFieldType">text_suggest_phrase</str>
> <str name="field">suggest_phrase</str>
> <str name="buildOnStartup">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnOptimize">true</str>
> </lst>
>
> <lst name="suggester">
> <str name="name">suggest_infix_shingle</str>
> <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
> <str name="indexPath">suggestInfixShingleDir</str>
> <str
> name="suggestAnalyzerFieldType">text_suggest_phrase</str>
> <str name="field">suggest_phrase</str>
> <str name="buildOnOptimize">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnStartup">true</str>
> </lst>
>
> <lst name="suggester">
> <str name="name">suggest_prefix</str>
> <str name="classname">Suggester</str>
> <str name="lookupImpl">AnalyzingLookupFactory</str>
> <str
> name="suggestAnalyzerFieldType">text_suggest_prefix</str>
> <str name="field">suggest_prefix</str>
> <str name="buildOnOptimize">true</str>
> <str name="buildOnCommit">true</str>
> <str name="buildOnStartup">true</str>
> </lst>
>
> </searchComponent>
>
>
> This component works without issued both in standalone and cloud mode,
> when used as the sole component in a handler, such as in the following
> excerpt:
>
> <requestHandler name="/suggest" class="solr.SearchHandler"
> startup="lazy">
> <lst name="defaults">
> <str name="suggest.dictionary">default</str>
> <str name="suggest.dictionary">suggest_phrase</str>
> <str
> name="suggest.dictionary">suggest_infix_shingle</str>
> <str name="suggest.dictionary">suggest_prefix</str>
> <str name="suggest">true</str>
> <str name="suggest.count">10</str>
> <str name="suggest.highlight">false</str>
> </lst>
> <arr name="components">
> <str>suggest</str>
> </arr>
> </requestHandler>
>
>
> It also works when used along with other component in standalone mode,
> such as in the following excerpt, where we use the suggest component
> to get suggestions during a "normal" search call:
>
> <requestHandler name="/select" class="solr.SearchHandler">
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> <int name="rows">10</int>
> <str name="df">text_search</str>
>
> <str name="defType">edismax</str>
>
> <str name="qf">title^5.0 subtitle^3.0
> abstract^2.0 text_search</str>
> <str name="pf">title^5.0 subtitle^3.0
> abstract^2.0 text_search</str>
> <str name="ps">4</str>
> <str name="spellcheck">on</str>
> <str name="spellcheck.dictionary">default</str>
> <str name="spellcheck.extendedResults">true</str>
> <str name="spellcheck.count">10</str>
> <str
> name="spellcheck.alternativeTermCount">5</str>
> <str
> name="spellcheck.maxResultsForSuggest">5</str>
> <str name="spellcheck.collate">true</str>
> <str
> name="spellcheck.collateExtendedResults">true</str>
> <str name="spellcheck.maxCollationTries">10</str>
> <str name="spellcheck.maxCollations">5</str>
>
> <str name="suggest.dictionary">default</str>
> <str
> name="suggest.dictionary">suggest_phrase</str>
> <str
> name="suggest.dictionary">suggest_infix_shingle</str>
> <str
> name="suggest.dictionary">suggest_prefix</str>
> <str name="suggest">true</str>
> <str name="suggest.count">10</str>
> <str name="suggest.highlight">false</str>
> </lst>
>
> <arr name="last-components">
> <str>suggest</str>
> <str>spellcheck</str>
> </arr>
> </requestHandler>
>
> However, the above configuration does not work in cloud mode, where we
> get an NPE if a search call is made:
>
> o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
> at
> org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1034)
> at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:885)
> at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
> at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
> at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:531)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
> at
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
> at java.lang.Thread.run(Thread.java:748)
>
>
> As far as we can tell, this is due to the fact that the failing code
> in QueryComponent assumes that the shard response always contains a
> 'sort_values' section:
>
> NamedList sortFieldValues =
> (NamedList)(srsp.getSolrResponse().getResponse().get("sort_values"));
> NamedList unmarshalledSortFieldValues =
> unmarshalSortValues(ss, sortFieldValues, schema);
>
> (the exception occurs in unmarshalSortValues(), which assumes the
> sortFieldValues parameter is not null)
>
> Are we doing something wrong that is preventing 'sort_values' from
> being present in the shard response? Or is their absence OK, but the
> code fails to account for it? Our guess is the second, because
> umarshalSortValues() does check if the values are empty or not, and
> handles that.
>
> Can anyone help us with this issue? Thanks in advance for any pointers!
>
> Alex
>
> [1]
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201711.mbox/%3CCACKWnqOv=HFQwTxd7SwEqSTj-im+43FE+c9_zbTCPMWsp8vL_w@mail.gmail.com%3E
>
> [2] https://issues.apache.org/jira/browse/SOLR-12060
>
>
--
*Dr. Alexandros Paramythis, M.Sc.*
/CEO/
CONTEXITY AG
Technoparkstrasse 2
CH-8406 Winterthur
*T* +41 78 611 6542
*E* alexandros.paramythis@contexity.ch
<ma...@contexity.ch>
*W* www.contexity.ch <http://www.contexity.ch>