You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Markus, Sascha" <sa...@uberresearch.com> on 2016/02/15 10:28:07 UTC
Why QueryWeight with Custom Similarity
Hi,
I created a custom similarity and factory which extends
DefaultSimilarity/-Factory to have
to achive this I my similarity overwrites idfExplain like this and also the
method for an array of terms.
public Explanation idfExplain(CollectionStatistics collectionStats,
TermStatistics termStats) {
final long df = lookUpDocumentFrequency(termStats);
final long max = getDocumentCountForIDF();
final float idf = idf(df, max);
log.debug("term:" + termStats.term() + " idf(docFreq=" + df + ",
maxDocs=" + max + ") -> " + idf);
return Explanation.match(idf, "idf(docFreq=" + df + ", maxDocs=" +
max + ")");
}
I configured my similarity for one field in the schema.
Without my plugin the score just uses the fieldWeight.
But when my similarity is enabled scores are calculated with the
fieldWeight multiplied by a queryWeight.
And this is done for ALL FIELDS queried, not only the one with my
similarity.
Why does this happen and is there a possibility to get around this.
>From a solr point of view this is probably ok because the score is not
meant to be absolute.
But in our application a user may set a threshold which is used as a filter
query like {!frange l=0.31}query($q).
Any hints?
Cheers,
Sascha
Re: Why QueryWeight with Custom Similarity
Posted by "Markus, Sascha" <sa...@uberresearch.com>.
Sorry clicked send to early :-)
with the above additional code the calculation is done by the default
similarity and the behaviour is as expected.
I think this is an issue of the implementation but I didn't find one in
jira. Should I create one?
Cheers
Sascha
On Thu, Feb 25, 2016 at 11:12 PM, Markus, Sascha <sa...@uberresearch.com>
wrote:
> Hi,
> I finally found the source of the problem I'm having with the custom
> similarity.
>
> The setting:
> - Solr 5.4.1
> - the SpecialSimilarity extends ClassicSimilarity
> - for one field this similarity is configured. Everything else uses
> ClassicSimilarity because of <similarity
> class="solr.SchemaSimilarityFactory"/>
>
> Result:
> - most calculation is done by the correct similarity for each field
> - but the method public float queryNorm(float valueForNormalization) is
> always called on the base class Similarity which always returns 1f
>
> My workaround:
> I changed PerFieldSimilarityWrapper and added
> @Override
> public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization);
> }
>
>
>
>
> On Mon, Feb 15, 2016 at 10:28 AM, Markus, Sascha <sa...@uberresearch.com>
> wrote:
>
>> Hi,
>> I created a custom similarity and factory which extends
>> DefaultSimilarity/-Factory to have
>>
>> to achive this I my similarity overwrites idfExplain like this and also
>> the method for an array of terms.
>> public Explanation idfExplain(CollectionStatistics collectionStats,
>> TermStatistics termStats) {
>> final long df = lookUpDocumentFrequency(termStats);
>> final long max = getDocumentCountForIDF();
>> final float idf = idf(df, max);
>> log.debug("term:" + termStats.term() + " idf(docFreq=" + df + ",
>> maxDocs=" + max + ") -> " + idf);
>> return Explanation.match(idf, "idf(docFreq=" + df + ", maxDocs="
>> + max + ")");
>> }
>>
>> I configured my similarity for one field in the schema.
>>
>> Without my plugin the score just uses the fieldWeight.
>> But when my similarity is enabled scores are calculated with the
>> fieldWeight multiplied by a queryWeight.
>> And this is done for ALL FIELDS queried, not only the one with my
>> similarity.
>>
>> Why does this happen and is there a possibility to get around this.
>> From a solr point of view this is probably ok because the score is not
>> meant to be absolute.
>> But in our application a user may set a threshold which is used as a
>> filter query like {!frange l=0.31}query($q).
>>
>> Any hints?
>>
>> Cheers,
>> Sascha
>>
>>
>>
>>
>
Re: Why QueryWeight with Custom Similarity
Posted by "Markus, Sascha" <sa...@uberresearch.com>.
Hi,
I finally found the source of the problem I'm having with the custom
similarity.
The setting:
- Solr 5.4.1
- the SpecialSimilarity extends ClassicSimilarity
- for one field this similarity is configured. Everything else uses
ClassicSimilarity because of <similarity
class="solr.SchemaSimilarityFactory"/>
Result:
- most calculation is done by the correct similarity for each field
- but the method public float queryNorm(float valueForNormalization) is
always called on the base class Similarity which always returns 1f
My workaround:
I changed PerFieldSimilarityWrapper and added
@Override
public float queryNorm(float valueForNormalization) {
return get("").queryNorm(valueForNormalization);
}
On Mon, Feb 15, 2016 at 10:28 AM, Markus, Sascha <sa...@uberresearch.com>
wrote:
> Hi,
> I created a custom similarity and factory which extends
> DefaultSimilarity/-Factory to have
>
> to achive this I my similarity overwrites idfExplain like this and also
> the method for an array of terms.
> public Explanation idfExplain(CollectionStatistics collectionStats,
> TermStatistics termStats) {
> final long df = lookUpDocumentFrequency(termStats);
> final long max = getDocumentCountForIDF();
> final float idf = idf(df, max);
> log.debug("term:" + termStats.term() + " idf(docFreq=" + df + ",
> maxDocs=" + max + ") -> " + idf);
> return Explanation.match(idf, "idf(docFreq=" + df + ", maxDocs=" +
> max + ")");
> }
>
> I configured my similarity for one field in the schema.
>
> Without my plugin the score just uses the fieldWeight.
> But when my similarity is enabled scores are calculated with the
> fieldWeight multiplied by a queryWeight.
> And this is done for ALL FIELDS queried, not only the one with my
> similarity.
>
> Why does this happen and is there a possibility to get around this.
> From a solr point of view this is probably ok because the score is not
> meant to be absolute.
> But in our application a user may set a threshold which is used as a
> filter query like {!frange l=0.31}query($q).
>
> Any hints?
>
> Cheers,
> Sascha
>
>
>
>