You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Charles Sanders <cs...@redhat.com> on 2014/12/30 23:20:55 UTC

SpellCheck (AutoComplete) Not Working In Distributed Environment

I'm running Solr 4.8 in a distributed environment (2 shards). I have added the spellcheck component to my request handler. In my test system, which is not distributed, it works. But when I move it to the Dev box, which is distributed, 2 shards, it is not working. Is there something additional I must do to get this to work in a distributed environment? 

<requestHandler default="true" name="standard" class="solr.SearchHandler"> 
<!-- default values for query parameters can be specified, these 
will be overridden by parameters in the request 
--> 
<lst name="defaults"> 
<str name="echoParams">explicit</str> 
<int name="rows">10</int> 
<str name="df">allText</str> 
<!-- default autocomplete settings for this search request handler --> 
<str name="spellcheck">true</str> 
<str name="spellcheck.dictionary">andreasAutoComplete</str> 
<str name="spellcheck.onlyMorePopular">true</str> 
<str name="spellcheck.count">5</str> 
<str name="spellcheck.collate">true</str> 
<str name="spellcheck.maxCollations">5</str> 
</lst> 
<arr name="last-components"> 
<str>autoComplete</str> 
</arr> 
</requestHandler> 

<searchComponent name="autoComplete" class="solr.SpellCheckComponent"> 
<lst name="spellchecker"> 
<str name="name">andreasAutoComplete</str> 
<str name="classname">org.apache.solr.spelling.suggest.Suggester</str> 
<str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookupFactory</str> 
<str name="field">sugg_allText</str> 
<str name="buildOnCommit">true</str> 
<float name="threshold">.005</float> 
<str name="queryAnalyzerFieldType">text_suggest</str> 
</lst> 
</searchComponent> 


Any help greatly appreciated! Thanks, 
-Charles

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Posted by Charles Sanders <cs...@redhat.com>.
Still not able to get my autoComplete component to work in a distributed environment. Works fine on a non-distributed system. Also, on the distributed system, if I include distrib=false, it works. 

I have tried shards.qt and shards parameters, but they make no difference. I should add, I am running SolrCloud and ZooKeeper, if that makes any difference. I have played around with this quite a bit, but nothing seems to work. 

When I add shards.qt=/ac {the name of the request handler}, I get an error in the solr logs. It simply states: java.lang.NullPointerException. That's it nothing more. This is listed as "logger" SolrCore and SolrDispatchFilter. 

Any ideas, suggestions on how I can troubleshoot and find the problem? Is there something specific I should look for? 

Please find attached text file with relevant information from schema.xml and sorlconfig.xml. 

Any help greatly appreciated! Thanks, 
-Charles 



----- Original Message -----

From: "Erick Erickson" <er...@gmail.com> 
To: solr-user@lucene.apache.org 
Sent: Tuesday, December 30, 2014 6:07:13 PM 
Subject: Re: SpellCheck (AutoComplete) Not Working In Distributed Environment 

Did you try the shards parameter? See: 
https://cwiki.apache.org/confluence/display/solr/Spell+Checking#SpellChecking-DistributedSpellCheck 

On Tue, Dec 30, 2014 at 2:20 PM, Charles Sanders <cs...@redhat.com> wrote: 
> I'm running Solr 4.8 in a distributed environment (2 shards). I have added the spellcheck component to my request handler. In my test system, which is not distributed, it works. But when I move it to the Dev box, which is distributed, 2 shards, it is not working. Is there something additional I must do to get this to work in a distributed environment? 
> 
> <requestHandler default="true" name="standard" class="solr.SearchHandler"> 
> <!-- default values for query parameters can be specified, these 
> will be overridden by parameters in the request 
> --> 
> <lst name="defaults"> 
> <str name="echoParams">explicit</str> 
> <int name="rows">10</int> 
> <str name="df">allText</str> 
> <!-- default autocomplete settings for this search request handler --> 
> <str name="spellcheck">true</str> 
> <str name="spellcheck.dictionary">andreasAutoComplete</str> 
> <str name="spellcheck.onlyMorePopular">true</str> 
> <str name="spellcheck.count">5</str> 
> <str name="spellcheck.collate">true</str> 
> <str name="spellcheck.maxCollations">5</str> 
> </lst> 
> <arr name="last-components"> 
> <str>autoComplete</str> 
> </arr> 
> </requestHandler> 
> 
> <searchComponent name="autoComplete" class="solr.SpellCheckComponent"> 
> <lst name="spellchecker"> 
> <str name="name">andreasAutoComplete</str> 
> <str name="classname">org.apache.solr.spelling.suggest.Suggester</str> 
> <str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookupFactory</str> 
> <str name="field">sugg_allText</str> 
> <str name="buildOnCommit">true</str> 
> <float name="threshold">.005</float> 
> <str name="queryAnalyzerFieldType">text_suggest</str> 
> </lst> 
> </searchComponent> 
> 
> 
> Any help greatly appreciated! Thanks, 
> -Charles 
> 
> 
>

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Posted by Shawn Heisey <ap...@elyograg.org>.
On 1/1/2015 1:09 PM, Meraj A. Khan wrote:
> When running SolrCloud do you even have to include the shards parameter
> ,shouldnt only shards.qt parameter suffice?

If you are using SolrCloud, no shards parameter is required ... all
queries sent to either the collection or any shard replica will
automatically use all shards.  You *can* use the shards parameter if you
only want to query a subset of your shards, or distrib=false if you want
to only query the specific shard replica where you sent the request.

If it's not SolrCloud, there is no distributed search without the shards
parameter.

If you're using the /select handler, then shards.qt is not necessary at
all.  I do not know what SolrCloud does with handlers on distributed
requests, whether it uses the same handler name for those requests, or
uses /select by default like non-cloud does.

Thanks,
Shawn

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Posted by "Meraj A. Khan" <me...@gmail.com>.
Shawn,

When running SolrCloud do you even have to include the shards parameter
,shouldnt only shards.qt parameter suffice?
On Dec 30, 2014 7:17 PM, "Shawn Heisey" <ap...@elyograg.org> wrote:

> On 12/30/2014 5:03 PM, Charles Sanders wrote:
> > Thanks for the suggestion.
> >
> > I did not do that originally because the documentation states:
> > This parameter is not required for the /select request handler.
> >
> > Which is what I am using. But I gave it a go, even though I'm not
> certain of the shard names. Now I have a NPE.
> >
> >
> solr/collection1/select?q=kernel+p&rows=1&wt=json&indent=true&shards.qt=/ac&shards=shard1,shard2
>
> If this is not SolrCloud, then the shards parameter must include most of
> the full base URL for each shard that you will be querying.  You can
> only use a bare shard name if you're running SolrCloud.
>
> The shards.qt parameter that you have used means that when the shards
> are consulted, the /ac handler will be used rather than /select.
>
> Here's an example of a shards parameter that will combine results from
> three cores on two machines.  When not running SolrCloud, this is how
> you do distributed searching:
>
> shards=
> idxa2.example.com:8981/solr/ai-inclive,idxa1.example.com:8981/solr/ai-0live,idxa2.example.com:8981/solr/ai-1live
>
> SolrCloud hides almost all of this complexity.
>
> Thanks,
> Shawn
>
>

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Posted by Charles Sanders <cs...@redhat.com>.
Got it. Thanks for your help everyone. 


----- Original Message -----

From: "Shawn Heisey" <ap...@elyograg.org> 
To: solr-user@lucene.apache.org 
Sent: Tuesday, December 30, 2014 7:16:59 PM 
Subject: Re: SpellCheck (AutoComplete) Not Working In Distributed Environment 

On 12/30/2014 5:03 PM, Charles Sanders wrote: 
> Thanks for the suggestion. 
> 
> I did not do that originally because the documentation states: 
> This parameter is not required for the /select request handler. 
> 
> Which is what I am using. But I gave it a go, even though I'm not certain of the shard names. Now I have a NPE. 
> 
> solr/collection1/select?q=kernel+p&rows=1&wt=json&indent=true&shards.qt=/ac&shards=shard1,shard2 

If this is not SolrCloud, then the shards parameter must include most of 
the full base URL for each shard that you will be querying. You can 
only use a bare shard name if you're running SolrCloud. 

The shards.qt parameter that you have used means that when the shards 
are consulted, the /ac handler will be used rather than /select. 

Here's an example of a shards parameter that will combine results from 
three cores on two machines. When not running SolrCloud, this is how 
you do distributed searching: 

shards=idxa2.example.com:8981/solr/ai-inclive,idxa1.example.com:8981/solr/ai-0live,idxa2.example.com:8981/solr/ai-1live 

SolrCloud hides almost all of this complexity. 

Thanks, 
Shawn

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Posted by Shawn Heisey <ap...@elyograg.org>.
On 12/30/2014 5:03 PM, Charles Sanders wrote:
> Thanks for the suggestion. 
> 
> I did not do that originally because the documentation states: 
> This parameter is not required for the /select request handler. 
> 
> Which is what I am using. But I gave it a go, even though I'm not certain of the shard names. Now I have a NPE. 
> 
> solr/collection1/select?q=kernel+p&rows=1&wt=json&indent=true&shards.qt=/ac&shards=shard1,shard2 

If this is not SolrCloud, then the shards parameter must include most of
the full base URL for each shard that you will be querying.  You can
only use a bare shard name if you're running SolrCloud.

The shards.qt parameter that you have used means that when the shards
are consulted, the /ac handler will be used rather than /select.

Here's an example of a shards parameter that will combine results from
three cores on two machines.  When not running SolrCloud, this is how
you do distributed searching:

shards=idxa2.example.com:8981/solr/ai-inclive,idxa1.example.com:8981/solr/ai-0live,idxa2.example.com:8981/solr/ai-1live

SolrCloud hides almost all of this complexity.

Thanks,
Shawn

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Posted by Charles Sanders <cs...@redhat.com>.
Thanks for the suggestion. 

I did not do that originally because the documentation states: 
This parameter is not required for the /select request handler. 

Which is what I am using. But I gave it a go, even though I'm not certain of the shard names. Now I have a NPE. 

solr/collection1/select?q=kernel+p&rows=1&wt=json&indent=true&shards.qt=/ac&shards=shard1,shard2 

{
  "responseHeader":{
    "status":500,
    "QTime":12,
    "params":{
      "shards":"shard1,shard2",
      "indent":"true",
      "shards.qt":"/ac",
      "q":"kernel p",
      "wt":"json",
      "rows":"1"}},
  "error":{
    "trace":"java.lang.NullPointerException\n\tat org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:901)\n\tat org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:686)\n\tat org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:665)\n\tat org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:325)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)\n\tat org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:787)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:431)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\n\tat com.lucid.servlet.LweSolrDispatchFilter.doFilter(LweSolrDispatchFilter.java:202)\n\tat com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)\n\tat com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)\n\tat com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)\n\tat com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)\n\tat com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)\n\tat com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)\n\tat org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:212)\n\tat org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:179)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:351)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)\n\tat org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)\n\tat org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)\n\tat org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)\n\tat java.lang.Thread.run(Thread.java:744)\n",
    "code":500}} 



----- Original Message -----

From: "Erick Erickson" <er...@gmail.com> 
To: solr-user@lucene.apache.org 
Sent: Tuesday, December 30, 2014 6:07:13 PM 
Subject: Re: SpellCheck (AutoComplete) Not Working In Distributed Environment 

Did you try the shards parameter? See: 
https://cwiki.apache.org/confluence/display/solr/Spell+Checking#SpellChecking-DistributedSpellCheck 

On Tue, Dec 30, 2014 at 2:20 PM, Charles Sanders <cs...@redhat.com> wrote: 
> I'm running Solr 4.8 in a distributed environment (2 shards). I have added the spellcheck component to my request handler. In my test system, which is not distributed, it works. But when I move it to the Dev box, which is distributed, 2 shards, it is not working. Is there something additional I must do to get this to work in a distributed environment? 
> 
> <requestHandler default="true" name="standard" class="solr.SearchHandler"> 
> <!-- default values for query parameters can be specified, these 
> will be overridden by parameters in the request 
> --> 
> <lst name="defaults"> 
> <str name="echoParams">explicit</str> 
> <int name="rows">10</int> 
> <str name="df">allText</str> 
> <!-- default autocomplete settings for this search request handler --> 
> <str name="spellcheck">true</str> 
> <str name="spellcheck.dictionary">andreasAutoComplete</str> 
> <str name="spellcheck.onlyMorePopular">true</str> 
> <str name="spellcheck.count">5</str> 
> <str name="spellcheck.collate">true</str> 
> <str name="spellcheck.maxCollations">5</str> 
> </lst> 
> <arr name="last-components"> 
> <str>autoComplete</str> 
> </arr> 
> </requestHandler> 
> 
> <searchComponent name="autoComplete" class="solr.SpellCheckComponent"> 
> <lst name="spellchecker"> 
> <str name="name">andreasAutoComplete</str> 
> <str name="classname">org.apache.solr.spelling.suggest.Suggester</str> 
> <str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookupFactory</str> 
> <str name="field">sugg_allText</str> 
> <str name="buildOnCommit">true</str> 
> <float name="threshold">.005</float> 
> <str name="queryAnalyzerFieldType">text_suggest</str> 
> </lst> 
> </searchComponent> 
> 
> 
> Any help greatly appreciated! Thanks, 
> -Charles 
> 
> 
>

Re: SpellCheck (AutoComplete) Not Working In Distributed Environment

Posted by Erick Erickson <er...@gmail.com>.
Did you try the shards parameter? See:
https://cwiki.apache.org/confluence/display/solr/Spell+Checking#SpellChecking-DistributedSpellCheck

On Tue, Dec 30, 2014 at 2:20 PM, Charles Sanders <cs...@redhat.com> wrote:
> I'm running Solr 4.8 in a distributed environment (2 shards). I have added the spellcheck component to my request handler. In my test system, which is not distributed, it works. But when I move it to the Dev box, which is distributed, 2 shards, it is not working. Is there something additional I must do to get this to work in a distributed environment?
>
> <requestHandler default="true" name="standard" class="solr.SearchHandler">
> <!-- default values for query parameters can be specified, these
> will be overridden by parameters in the request
> -->
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> <int name="rows">10</int>
> <str name="df">allText</str>
> <!-- default autocomplete settings for this search request handler -->
> <str name="spellcheck">true</str>
> <str name="spellcheck.dictionary">andreasAutoComplete</str>
> <str name="spellcheck.onlyMorePopular">true</str>
> <str name="spellcheck.count">5</str>
> <str name="spellcheck.collate">true</str>
> <str name="spellcheck.maxCollations">5</str>
> </lst>
> <arr name="last-components">
> <str>autoComplete</str>
> </arr>
> </requestHandler>
>
> <searchComponent name="autoComplete" class="solr.SpellCheckComponent">
> <lst name="spellchecker">
> <str name="name">andreasAutoComplete</str>
> <str name="classname">org.apache.solr.spelling.suggest.Suggester</str>
> <str name="lookupImpl">org.apache.solr.spelling.suggest.tst.TSTLookupFactory</str>
> <str name="field">sugg_allText</str>
> <str name="buildOnCommit">true</str>
> <float name="threshold">.005</float>
> <str name="queryAnalyzerFieldType">text_suggest</str>
> </lst>
> </searchComponent>
>
>
> Any help greatly appreciated! Thanks,
> -Charles
>
>
>