You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Eric Wilson <wi...@gmail.com> on 2012/06/19 15:54:23 UTC
Solr spellchecking fails on sharded query
I have a Solr application that is distributed into 11 shards, using Solr
version 4.0.0.2011.07.26.16.34.16
In the solrconfig.xml for each shard, I have configured a spellcheck
component:
<searchComponent name="spellcheck" class="solr.SpellCheckComponent">
<str name="queryAnalyzerFieldType">textSpell</str>
<lst name="spellchecker">
<str name="name">cn_spell</str>
<str name="field">company_name_spell</str>
<float name="threshold">0.0001</float>
<str name="buildOnOptimize">true</str>
<str name="spellcheckIndexDir">./spellchecker_cn_spell</str>
</lst>
</searchComponent>
I have built the dictionary for each shard, and verified that each shard
will return suggestions for misspellings. Moreover, it is evident that a
different dictionary is being used for the various shards.
The problem comes when I submit a sharded query. In that case the result
comes back with the following:
<lst name="spellcheck">
<lst name="suggestions"/>
</lst>
In other words, the list of words for which there are suggestions is empty.
Is there a trick to sharded spellchecking? I appreciate any suggestions.
Eric
Re: Solr spellchecking fails on sharded query
Posted by fabio curti <fa...@gmail.com>.
Hi,
i found this article about your issue.
http://wiki.apache.org/solr/SpellCheckComponent#Distributed_Search_Support
Fabio
2012/6/19 Eric Wilson <wi...@gmail.com>
> I have a Solr application that is distributed into 11 shards, using Solr
> version 4.0.0.2011.07.26.16.34.16
>
> In the solrconfig.xml for each shard, I have configured a spellcheck
> component:
>
> <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
>
> <str name="queryAnalyzerFieldType">textSpell</str>
>
> <lst name="spellchecker">
>
> <str name="name">cn_spell</str>
>
> <str name="field">company_name_spell</str>
>
> <float name="threshold">0.0001</float>
>
> <str name="buildOnOptimize">true</str>
>
> <str name="spellcheckIndexDir">./spellchecker_cn_spell</str>
>
> </lst>
>
> </searchComponent>
>
> I have built the dictionary for each shard, and verified that each shard
> will return suggestions for misspellings. Moreover, it is evident that a
> different dictionary is being used for the various shards.
>
> The problem comes when I submit a sharded query. In that case the result
> comes back with the following:
>
> <lst name="spellcheck">
> <lst name="suggestions"/>
> </lst>
>
> In other words, the list of words for which there are suggestions is empty.
>
> Is there a trick to sharded spellchecking? I appreciate any suggestions.
>
> Eric
>
Re: Solr spellchecking fails on sharded query
Posted by fabio curti <fa...@gmail.com>.
Hi,
it seems the shards suggestion working fine if i set the select RH as
follow ( <arr name="last-components"> instead of <arr name="components"> )
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
</lst>
<arr name="last-components">
<str>spellcheck</str>
</arr>
</requestHandler>
Now suggestion is populated!
Fabio
2012/6/22 fabio curti <fa...@gmail.com>
> I did as you suggest enabling "spellcheck" component in select RH.
>
> <requestHandler name="/select" class="solr.SearchHandler">
> <lst name="defaults">
> <str name="echoParams">explicit</str>
> <int name="rows">10</int>
> </lst>
> <arr name="components">
> <str>spellcheck</str>
> </arr>
> </requestHandler>
>
> Response contains error 500
> <response>
> <lst name="responseHeader">
> <int name="status">500</int>
> <int name="QTime">29</int>
> <lst name="params">
> <str name="spellcheck.dictionary">file</str>
> <str name="spellcheck">true</str>
> <str name="shards">
>
> fc:8900/solr/commenti,fc:7500/solr/commenti,fc:8584/solr/commenti,fc:7574/solr/commenti
> </str>
> <str name="q">piza</str>
> <str name="spellcheck.q">piza</str>
> </lst>
> </lst>
> <lst name="error">
> <str name="trace">
> java.lang.NullPointerException at
> org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:819)
> at
> org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626)
> at
> org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:307)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1566) at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> at org.eclipse.jetty.server.Server.handle(Server.java:351) at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
> at java.lang.Thread.run(Thread.java:679)
> </str>
> <int name="code">500</int>
> </lst>
> </response>
>
> Fabio
>
> 2012/6/22 Markus Jelsma <ma...@openindex.io>
>
>> Hi,
>>
>> The spellcheck component must be enabled in your default request handler
>> otherwise your suggestions list is empty.
>>
>> Cheers,
>>
>>
>>
>> -----Original message-----
>> > From:fabio curti <fa...@gmail.com>
>> > Sent: Fri 22-Jun-2012 09:34
>> > To: solr-user@lucene.apache.org
>> > Subject: Re: Solr spellchecking fails on sharded query
>> >
>> > Hi,
>> > i try solr shards configuration ( SolrCloud ) and request settings as
>> > suggested in
>> >
>> http://wiki.apache.org/solr/SpellCheckComponent#Distributed_Search_Supportfor
>> > shards spelling.
>> > Suggestion is empty as Eric said.
>> >
>> > Any idea?
>> >
>> > Fabio
>> >
>> > 2012/6/19 Eric Wilson <wi...@gmail.com>
>> >
>> > > I have a Solr application that is distributed into 11 shards, using
>> Solr
>> > > version 4.0.0.2011.07.26.16.34.16
>> > >
>> > > In the solrconfig.xml for each shard, I have configured a spellcheck
>> > > component:
>> > >
>> > > <searchComponent name="spellcheck"
>> class="solr.SpellCheckComponent">
>> > >
>> > > <str name="queryAnalyzerFieldType">textSpell</str>
>> > >
>> > > <lst name="spellchecker">
>> > >
>> > > <str name="name">cn_spell</str>
>> > >
>> > > <str name="field">company_name_spell</str>
>> > >
>> > > <float name="threshold">0.0001</float>
>> > >
>> > > <str name="buildOnOptimize">true</str>
>> > >
>> > > <str name="spellcheckIndexDir">./spellchecker_cn_spell</str>
>> > >
>> > > </lst>
>> > >
>> > > </searchComponent>
>> > >
>> > > I have built the dictionary for each shard, and verified that each
>> shard
>> > > will return suggestions for misspellings. Moreover, it is evident
>> that a
>> > > different dictionary is being used for the various shards.
>> > >
>> > > The problem comes when I submit a sharded query. In that case the
>> result
>> > > comes back with the following:
>> > >
>> > > <lst name="spellcheck">
>> > > <lst name="suggestions"/>
>> > > </lst>
>> > >
>> > > In other words, the list of words for which there are suggestions is
>> empty.
>> > >
>> > > Is there a trick to sharded spellchecking? I appreciate any
>> suggestions.
>> > >
>> > > Eric
>> > >
>> >
>>
>
>
Re: Solr spellchecking fails on sharded query
Posted by fabio curti <fa...@gmail.com>.
I did as you suggest enabling "spellcheck" component in select RH.
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
</lst>
<arr name="components">
<str>spellcheck</str>
</arr>
</requestHandler>
Response contains error 500
<response>
<lst name="responseHeader">
<int name="status">500</int>
<int name="QTime">29</int>
<lst name="params">
<str name="spellcheck.dictionary">file</str>
<str name="spellcheck">true</str>
<str name="shards">
fc:8900/solr/commenti,fc:7500/solr/commenti,fc:8584/solr/commenti,fc:7574/solr/commenti
</str>
<str name="q">piza</str>
<str name="spellcheck.q">piza</str>
</lst>
</lst>
<lst name="error">
<str name="trace">
java.lang.NullPointerException at
org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:819)
at
org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:626)
at
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:605)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:307)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1566) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:351) at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634) at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230) at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
at java.lang.Thread.run(Thread.java:679)
</str>
<int name="code">500</int>
</lst>
</response>
Fabio
2012/6/22 Markus Jelsma <ma...@openindex.io>
> Hi,
>
> The spellcheck component must be enabled in your default request handler
> otherwise your suggestions list is empty.
>
> Cheers,
>
>
>
> -----Original message-----
> > From:fabio curti <fa...@gmail.com>
> > Sent: Fri 22-Jun-2012 09:34
> > To: solr-user@lucene.apache.org
> > Subject: Re: Solr spellchecking fails on sharded query
> >
> > Hi,
> > i try solr shards configuration ( SolrCloud ) and request settings as
> > suggested in
> >
> http://wiki.apache.org/solr/SpellCheckComponent#Distributed_Search_Supportfor
> > shards spelling.
> > Suggestion is empty as Eric said.
> >
> > Any idea?
> >
> > Fabio
> >
> > 2012/6/19 Eric Wilson <wi...@gmail.com>
> >
> > > I have a Solr application that is distributed into 11 shards, using
> Solr
> > > version 4.0.0.2011.07.26.16.34.16
> > >
> > > In the solrconfig.xml for each shard, I have configured a spellcheck
> > > component:
> > >
> > > <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
> > >
> > > <str name="queryAnalyzerFieldType">textSpell</str>
> > >
> > > <lst name="spellchecker">
> > >
> > > <str name="name">cn_spell</str>
> > >
> > > <str name="field">company_name_spell</str>
> > >
> > > <float name="threshold">0.0001</float>
> > >
> > > <str name="buildOnOptimize">true</str>
> > >
> > > <str name="spellcheckIndexDir">./spellchecker_cn_spell</str>
> > >
> > > </lst>
> > >
> > > </searchComponent>
> > >
> > > I have built the dictionary for each shard, and verified that each
> shard
> > > will return suggestions for misspellings. Moreover, it is evident that
> a
> > > different dictionary is being used for the various shards.
> > >
> > > The problem comes when I submit a sharded query. In that case the
> result
> > > comes back with the following:
> > >
> > > <lst name="spellcheck">
> > > <lst name="suggestions"/>
> > > </lst>
> > >
> > > In other words, the list of words for which there are suggestions is
> empty.
> > >
> > > Is there a trick to sharded spellchecking? I appreciate any
> suggestions.
> > >
> > > Eric
> > >
> >
>
RE: Solr spellchecking fails on sharded query
Posted by Markus Jelsma <ma...@openindex.io>.
Hi,
The spellcheck component must be enabled in your default request handler otherwise your suggestions list is empty.
Cheers,
-----Original message-----
> From:fabio curti <fa...@gmail.com>
> Sent: Fri 22-Jun-2012 09:34
> To: solr-user@lucene.apache.org
> Subject: Re: Solr spellchecking fails on sharded query
>
> Hi,
> i try solr shards configuration ( SolrCloud ) and request settings as
> suggested in
> http://wiki.apache.org/solr/SpellCheckComponent#Distributed_Search_Support for
> shards spelling.
> Suggestion is empty as Eric said.
>
> Any idea?
>
> Fabio
>
> 2012/6/19 Eric Wilson <wi...@gmail.com>
>
> > I have a Solr application that is distributed into 11 shards, using Solr
> > version 4.0.0.2011.07.26.16.34.16
> >
> > In the solrconfig.xml for each shard, I have configured a spellcheck
> > component:
> >
> > <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
> >
> > <str name="queryAnalyzerFieldType">textSpell</str>
> >
> > <lst name="spellchecker">
> >
> > <str name="name">cn_spell</str>
> >
> > <str name="field">company_name_spell</str>
> >
> > <float name="threshold">0.0001</float>
> >
> > <str name="buildOnOptimize">true</str>
> >
> > <str name="spellcheckIndexDir">./spellchecker_cn_spell</str>
> >
> > </lst>
> >
> > </searchComponent>
> >
> > I have built the dictionary for each shard, and verified that each shard
> > will return suggestions for misspellings. Moreover, it is evident that a
> > different dictionary is being used for the various shards.
> >
> > The problem comes when I submit a sharded query. In that case the result
> > comes back with the following:
> >
> > <lst name="spellcheck">
> > <lst name="suggestions"/>
> > </lst>
> >
> > In other words, the list of words for which there are suggestions is empty.
> >
> > Is there a trick to sharded spellchecking? I appreciate any suggestions.
> >
> > Eric
> >
>
Re: Solr spellchecking fails on sharded query
Posted by fabio curti <fa...@gmail.com>.
Hi,
i try solr shards configuration ( SolrCloud ) and request settings as
suggested in
http://wiki.apache.org/solr/SpellCheckComponent#Distributed_Search_Support for
shards spelling.
Suggestion is empty as Eric said.
Any idea?
Fabio
2012/6/19 Eric Wilson <wi...@gmail.com>
> I have a Solr application that is distributed into 11 shards, using Solr
> version 4.0.0.2011.07.26.16.34.16
>
> In the solrconfig.xml for each shard, I have configured a spellcheck
> component:
>
> <searchComponent name="spellcheck" class="solr.SpellCheckComponent">
>
> <str name="queryAnalyzerFieldType">textSpell</str>
>
> <lst name="spellchecker">
>
> <str name="name">cn_spell</str>
>
> <str name="field">company_name_spell</str>
>
> <float name="threshold">0.0001</float>
>
> <str name="buildOnOptimize">true</str>
>
> <str name="spellcheckIndexDir">./spellchecker_cn_spell</str>
>
> </lst>
>
> </searchComponent>
>
> I have built the dictionary for each shard, and verified that each shard
> will return suggestions for misspellings. Moreover, it is evident that a
> different dictionary is being used for the various shards.
>
> The problem comes when I submit a sharded query. In that case the result
> comes back with the following:
>
> <lst name="spellcheck">
> <lst name="suggestions"/>
> </lst>
>
> In other words, the list of words for which there are suggestions is empty.
>
> Is there a trick to sharded spellchecking? I appreciate any suggestions.
>
> Eric
>