You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Dmitry Kan <so...@gmail.com> on 2015/12/15 16:02:24 UTC
similarity as a parameter
Hi guys,
Is there a way to alter the similarity class at runtime, with a parameter?
--
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info
Re: similarity as a parameter
Posted by Dmitry Kan <so...@gmail.com>.
Hi Hoss,
Thanks for sharing the knowledge on dangerous zones, will try to avoid
them. #2 is quite probable way of implementing this in my case, as many
Query objects are custom (although not all). But #1 is compelling too and
sounds like a bit less trouble.
On Tue, Dec 15, 2015 at 8:13 PM, Chris Hostetter <ho...@fucit.org>
wrote:
>
> : I think this is a legitimate request. Majority of the similarities are
> : compatible index wise. I think the only exception is sweet spot
> : similarity.
>
> I think you are grossly underestimating the risk of arbitrarily using diff
> Similarities between index time and query time -- particulaly in how norms
> are computed. But for the sake of argument let's assuming you know what
> you are doing, you are careful, and you know that the index time
> similarity you used is compatibly with N diff query time similarities you
> want to choose between...
>
> : I wonder what solr-plugin would be best for this functionality.
> : How about a custom search component, in its prepare method?
> :
> : I think we can access (Solr)IndexSearcher inside a SearchComponent.
> : setSimilarity in the process method should work.
>
> this owuld be *very* dangerous to do, because the SolrIndexSearcher is
> shared across all requests -- so you'd get race conditions and
> non-deterministic behavior from diff queries not getting the similarity
> they expected.
>
> The only sane way to do this on a per-request basis would either be...
>
> 1) wrap the IndexSearcher in a new IndexSearcherWrapper that returned a
> per-request Similarity.
>
> 2) modify the Query objects themselves so that createWeight use the
> similarity you want instead of delegating to INdexSearcher.getSimilarity
> (see for example how BooleanQuery/BooleanWeight use the "disableCoord"
> property of the query to decide wether or not to care about
> Similarity.coord
>
>
> Depending on your real world usecase / goal, i would suspect that either
> way a QParser that wraps the constructed query is going to be the
> simplest/cleanest solution regardless of wether #1 or #2 makes the most
> sense -- perhaps even achieving #2 by using #1 so that createWeight in
> your new QueryWrapper class does the IndexSearcher wrapping before
> delegating.
>
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
--
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info
Re: similarity as a parameter
Posted by Chris Hostetter <ho...@fucit.org>.
: I think this is a legitimate request. Majority of the similarities are
: compatible index wise. I think the only exception is sweet spot
: similarity.
I think you are grossly underestimating the risk of arbitrarily using diff
Similarities between index time and query time -- particulaly in how norms
are computed. But for the sake of argument let's assuming you know what
you are doing, you are careful, and you know that the index time
similarity you used is compatibly with N diff query time similarities you
want to choose between...
: I wonder what solr-plugin would be best for this functionality.
: How about a custom search component, in its prepare method?
:
: I think we can access (Solr)IndexSearcher inside a SearchComponent.
: setSimilarity in the process method should work.
this owuld be *very* dangerous to do, because the SolrIndexSearcher is
shared across all requests -- so you'd get race conditions and
non-deterministic behavior from diff queries not getting the similarity
they expected.
The only sane way to do this on a per-request basis would either be...
1) wrap the IndexSearcher in a new IndexSearcherWrapper that returned a
per-request Similarity.
2) modify the Query objects themselves so that createWeight use the
similarity you want instead of delegating to INdexSearcher.getSimilarity
(see for example how BooleanQuery/BooleanWeight use the "disableCoord"
property of the query to decide wether or not to care about Similarity.coord
Depending on your real world usecase / goal, i would suspect that either
way a QParser that wraps the constructed query is going to be the
simplest/cleanest solution regardless of wether #1 or #2 makes the most
sense -- perhaps even achieving #2 by using #1 so that createWeight in
your new QueryWrapper class does the IndexSearcher wrapping before
delegating.
-Hoss
http://www.lucidworks.com/
Re: similarity as a parameter
Posted by Dmitry Kan <so...@gmail.com>.
Markus, Jack,
I think Ahmet nails it pretty nicely: the similarity functions in question
are compatible on the index level. So it is not necessary to create a
separate search field.
Ahmet, I like your idea. Will take a look, thanks.
Rgds,
Dmitry
On Tue, Dec 15, 2015 at 7:58 PM, Ahmet Arslan <io...@yahoo.com.invalid>
wrote:
>
>
> I wonder what solr-plugin would be best for this functionality.
> How about a custom search component, in its prepare method?
>
> I think we can access (Solr)IndexSearcher inside a SearchComponent.
> setSimilarity in the process method should work.
>
> Ahmet
>
>
> On Tuesday, December 15, 2015 7:43 PM, Ahmet Arslan
> <io...@yahoo.com.INVALID> wrote:
> Hi Dmitry,
>
> I think this is a legitimate request. Majority of the similarities are
> compatible index wise. I think the only exception is sweet spot similarity.
>
> In Lucene, it can be changed on the fly with a new Searcher. It should be
> possible to do so in solr.
>
> Thanks,
> Ahmet
>
>
>
>
> On Tuesday, December 15, 2015 6:08 PM, Jack Krupansky <
> jack.krupansky@gmail.com> wrote:
> You would need to define an alternate field which copied a base field but
> then had the desired alternate similarity, using SchemaSimilarityFactory.
>
> See:
> https://cwiki.apache.org/confluence/display/solr/Other+Schema+Elements
>
>
> -- Jack Krupansky
>
>
> On Tue, Dec 15, 2015 at 10:02 AM, Dmitry Kan <so...@gmail.com> wrote:
>
> > Hi guys,
> >
> > Is there a way to alter the similarity class at runtime, with a
> parameter?
> >
> > --
> > Dmitry Kan
> > Luke Toolbox: http://github.com/DmitryKey/luke
> > Blog: http://dmitrykan.blogspot.com
> > Twitter: http://twitter.com/dmitrykan
> > SemanticAnalyzer: www.semanticanalyzer.info
> >
>
--
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info
Re: similarity as a parameter
Posted by Ahmet Arslan <io...@yahoo.com.INVALID>.
I wonder what solr-plugin would be best for this functionality.
How about a custom search component, in its prepare method?
I think we can access (Solr)IndexSearcher inside a SearchComponent.
setSimilarity in the process method should work.
Ahmet
On Tuesday, December 15, 2015 7:43 PM, Ahmet Arslan <io...@yahoo.com.INVALID> wrote:
Hi Dmitry,
I think this is a legitimate request. Majority of the similarities are compatible index wise. I think the only exception is sweet spot similarity.
In Lucene, it can be changed on the fly with a new Searcher. It should be possible to do so in solr.
Thanks,
Ahmet
On Tuesday, December 15, 2015 6:08 PM, Jack Krupansky <ja...@gmail.com> wrote:
You would need to define an alternate field which copied a base field but
then had the desired alternate similarity, using SchemaSimilarityFactory.
See:
https://cwiki.apache.org/confluence/display/solr/Other+Schema+Elements
-- Jack Krupansky
On Tue, Dec 15, 2015 at 10:02 AM, Dmitry Kan <so...@gmail.com> wrote:
> Hi guys,
>
> Is there a way to alter the similarity class at runtime, with a parameter?
>
> --
> Dmitry Kan
> Luke Toolbox: http://github.com/DmitryKey/luke
> Blog: http://dmitrykan.blogspot.com
> Twitter: http://twitter.com/dmitrykan
> SemanticAnalyzer: www.semanticanalyzer.info
>
Re: similarity as a parameter
Posted by Ahmet Arslan <io...@yahoo.com.INVALID>.
Hi Dmitry,
I think this is a legitimate request. Majority of the similarities are compatible index wise. I think the only exception is sweet spot similarity.
In Lucene, it can be changed on the fly with a new Searcher. It should be possible to do so in solr.
Thanks,
Ahmet
On Tuesday, December 15, 2015 6:08 PM, Jack Krupansky <ja...@gmail.com> wrote:
You would need to define an alternate field which copied a base field but
then had the desired alternate similarity, using SchemaSimilarityFactory.
See:
https://cwiki.apache.org/confluence/display/solr/Other+Schema+Elements
-- Jack Krupansky
On Tue, Dec 15, 2015 at 10:02 AM, Dmitry Kan <so...@gmail.com> wrote:
> Hi guys,
>
> Is there a way to alter the similarity class at runtime, with a parameter?
>
> --
> Dmitry Kan
> Luke Toolbox: http://github.com/DmitryKey/luke
> Blog: http://dmitrykan.blogspot.com
> Twitter: http://twitter.com/dmitrykan
> SemanticAnalyzer: www.semanticanalyzer.info
>
Re: similarity as a parameter
Posted by Jack Krupansky <ja...@gmail.com>.
You would need to define an alternate field which copied a base field but
then had the desired alternate similarity, using SchemaSimilarityFactory.
See:
https://cwiki.apache.org/confluence/display/solr/Other+Schema+Elements
-- Jack Krupansky
On Tue, Dec 15, 2015 at 10:02 AM, Dmitry Kan <so...@gmail.com> wrote:
> Hi guys,
>
> Is there a way to alter the similarity class at runtime, with a parameter?
>
> --
> Dmitry Kan
> Luke Toolbox: http://github.com/DmitryKey/luke
> Blog: http://dmitrykan.blogspot.com
> Twitter: http://twitter.com/dmitrykan
> SemanticAnalyzer: www.semanticanalyzer.info
>