You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by elisabeth benoit <el...@gmail.com> on 2015/12/21 10:07:06 UTC
solr 4.10 I change slop in pf2 pf3 and query norm changes
Hello all,
I am using solr 4.10.1 and I have configured my pf2 pf3 like this
<str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
<str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
my search field (qf) is my catchall field
I'v been trying to change slop in pf2, pf3 for catchall and synonyms (going
from 0, or default value for synonyms, to 1)
pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
but some results are not ordered the same way anymore even if I get the
same MATCH values in debugQuery output
For instance, for a doc matching bastill in catchall field (and nothing to
do with pf2, pf3!)
with first pf2, pf3
0.5163083 = (MATCH) weight(catchall:bastill in 105256) [NoTFIDFSimilarity],
result of:
* 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
), product of:
* 0.5163083 = queryWeight,* product of:
1.0 = idf(docFreq=134, maxDocs=12258543)
0.5163083 = queryNorm
1.0 = fieldWeight in 105256, product of:
1.0 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
1.0 = idf(docFreq=134, maxDocs=12258543)
1.0 = fieldNorm(doc=105256)
0.5163083 = (MATCH) weight(catchall:paris in 105256)
[NoTFIDFSimilarity], result of:
0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
and when I change pf2 pf3 (the only change, same query, same docs)
0.47504464 = (MATCH) weight(catchall:paris in 105256) [NoTFIDFSimilarity],
result of:
* 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
), product of:
* 0.47504464 = queryWeight*, product of:
1.0 = idf(docFreq=10958, maxDocs=12258543)
0.47504464 = queryNorm
1.0 = fieldWeight in 105256, product of:
1.0 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
1.0 = idf(docFreq=10958, maxDocs=12258543)
1.0 = fieldNorm(doc=105256)
so in the end, with same MATCH results, in first case I get two documents
with same score, and in second case, one document has a higher score.
This seem very very strange. Does anyone have a clue what's going on?
Thanks
Elisabeth
Re: solr 4.10 I change slop in pf2 pf3 and query norm changes
Posted by Binoy Dalal <bi...@gmail.com>.
Oh OK. I see. So you want both the docs to have the same score?
In that case, I don't think there's any other way of going about it other
than writing your own custom similarity class.
Maybe someone else can suggest something better.
On Tue, 22 Dec 2015, 13:08 elisabeth benoit <el...@gmail.com>
wrote:
> hello,
>
> That's what I did, like I wrote in my mail yesterday. In first case, solr
> computes max. In second case, he sums both results.
>
> That's why I dont get the same relative scoring between docs with the same
> query.
>
> 2015-12-22 8:30 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
>
> > Unless the content for both the docs is exactly the same it is highly
> > unlikely that you will get the same score for the docs under different
> > querying conditions. What you saw in the first case may have been a happy
> > coincidence.
> > Other than that it is very difficult to say why the scoring is different
> > without getting a look at the actual query and the doc content.
> >
> > If you still wish to dig deeper, try to understand how solr actually
> scores
> > documents that match your query. It takes into account a variety of
> factors
> > to compute the cosine similarity to find the best match.
> > You can find this formula and a decent explanation for it in the book
> solr
> > in action or online in the lucene docs:
> >
> >
> https://lucene.apache.org/core/3_5_0/api/core/org/apache/lucene/search/Similarity.html
> >
> > On Tue, 22 Dec 2015, 11:10 elisabeth benoit <el...@gmail.com>
> > wrote:
> >
> > > hello,
> > >
> > > yes in the second case I get one document with a higher score. the
> > relative
> > > scoring between documents is not the same anymore.
> > >
> > > best regards,
> > > elisabeth
> > >
> > > 2015-12-22 4:39 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> > >
> > > > I have one query.
> > > > In the second case do you get two records with the same lower scores
> or
> > > > just one record with a lower score and the other with a higher one?
> > > >
> > > > On Mon, 21 Dec 2015, 18:45 elisabeth benoit <
> elisaelisaelisa@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I don't think the query is important in this case.
> > > > >
> > > > > After checking out solr's debug output, I dont think the query norm
> > is
> > > > > relevant either.
> > > > >
> > > > > I think the scoring changes because
> > > > >
> > > > > 1) in first case, I have same slop for catchall and name fields.
> Bot
> > > > match
> > > > > pf2 pf3. In this case, solr uses max of both for scoring pf2 pf3
> > > results.
> > > > >
> > > > > 2) In second case, I have different slopes, then solr uses sum of
> > > values
> > > > > instead of max.
> > > > >
> > > > >
> > > > >
> > > > > If anyone knows how to work around this, please let me know.
> > > > >
> > > > > Elisabeth
> > > > >
> > > > > 2015-12-21 11:22 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> > > > >
> > > > > > What is your query?
> > > > > >
> > > > > > On Mon, 21 Dec 2015, 14:37 elisabeth benoit <
> > > elisaelisaelisa@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello all,
> > > > > > >
> > > > > > > I am using solr 4.10.1 and I have configured my pf2 pf3 like
> this
> > > > > > >
> > > > > > > <str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > > > > > <str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > > > > >
> > > > > > > my search field (qf) is my catchall field
> > > > > > >
> > > > > > > I'v been trying to change slop in pf2, pf3 for catchall and
> > > synonyms
> > > > > > (going
> > > > > > > from 0, or default value for synonyms, to 1)
> > > > > > >
> > > > > > > pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > > > > > pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > > > > >
> > > > > > > but some results are not ordered the same way anymore even if I
> > get
> > > > the
> > > > > > > same MATCH values in debugQuery output
> > > > > > >
> > > > > > > For instance, for a doc matching bastill in catchall field (and
> > > > nothing
> > > > > > to
> > > > > > > do with pf2, pf3!)
> > > > > > >
> > > > > > > with first pf2, pf3
> > > > > > >
> > > > > > > 0.5163083 = (MATCH) weight(catchall:bastill in 105256)
> > > > > > [NoTFIDFSimilarity],
> > > > > > > result of:
> > > > > > > * 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
> > > > > > > ), product of:
> > > > > > > * 0.5163083 = queryWeight,* product of:
> > > > > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > > > > 0.5163083 = queryNorm
> > > > > > > 1.0 = fieldWeight in 105256, product of:
> > > > > > > 1.0 = tf(freq=2.0), with freq of:
> > > > > > > 2.0 = termFreq=2.0
> > > > > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > > > > 1.0 = fieldNorm(doc=105256)
> > > > > > > 0.5163083 = (MATCH) weight(catchall:paris in 105256)
> > > > > > > [NoTFIDFSimilarity], result of:
> > > > > > > 0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
> > > > > > >
> > > > > > > and when I change pf2 pf3 (the only change, same query, same
> > docs)
> > > > > > >
> > > > > > > 0.47504464 = (MATCH) weight(catchall:paris in 105256)
> > > > > > [NoTFIDFSimilarity],
> > > > > > > result of:
> > > > > > > * 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
> > > > > > > ), product of:
> > > > > > > * 0.47504464 = queryWeight*, product of:
> > > > > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > > > > 0.47504464 = queryNorm
> > > > > > > 1.0 = fieldWeight in 105256, product of:
> > > > > > > 1.0 = tf(freq=6.0), with freq of:
> > > > > > > 6.0 = termFreq=6.0
> > > > > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > > > > 1.0 = fieldNorm(doc=105256)
> > > > > > >
> > > > > > > so in the end, with same MATCH results, in first case I get two
> > > > > documents
> > > > > > > with same score, and in second case, one document has a higher
> > > score.
> > > > > > >
> > > > > > > This seem very very strange. Does anyone have a clue what's
> going
> > > on?
> > > > > > >
> > > > > > > Thanks
> > > > > > > Elisabeth
> > > > > > >
> > > > > > --
> > > > > > Regards,
> > > > > > Binoy Dalal
> > > > > >
> > > > >
> > > > --
> > > > Regards,
> > > > Binoy Dalal
> > > >
> > >
> > --
> > Regards,
> > Binoy Dalal
> >
>
--
Regards,
Binoy Dalal
Re: solr 4.10 I change slop in pf2 pf3 and query norm changes
Posted by elisabeth benoit <el...@gmail.com>.
hello,
That's what I did, like I wrote in my mail yesterday. In first case, solr
computes max. In second case, he sums both results.
That's why I dont get the same relative scoring between docs with the same
query.
2015-12-22 8:30 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> Unless the content for both the docs is exactly the same it is highly
> unlikely that you will get the same score for the docs under different
> querying conditions. What you saw in the first case may have been a happy
> coincidence.
> Other than that it is very difficult to say why the scoring is different
> without getting a look at the actual query and the doc content.
>
> If you still wish to dig deeper, try to understand how solr actually scores
> documents that match your query. It takes into account a variety of factors
> to compute the cosine similarity to find the best match.
> You can find this formula and a decent explanation for it in the book solr
> in action or online in the lucene docs:
>
> https://lucene.apache.org/core/3_5_0/api/core/org/apache/lucene/search/Similarity.html
>
> On Tue, 22 Dec 2015, 11:10 elisabeth benoit <el...@gmail.com>
> wrote:
>
> > hello,
> >
> > yes in the second case I get one document with a higher score. the
> relative
> > scoring between documents is not the same anymore.
> >
> > best regards,
> > elisabeth
> >
> > 2015-12-22 4:39 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> >
> > > I have one query.
> > > In the second case do you get two records with the same lower scores or
> > > just one record with a lower score and the other with a higher one?
> > >
> > > On Mon, 21 Dec 2015, 18:45 elisabeth benoit <elisaelisaelisa@gmail.com
> >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I don't think the query is important in this case.
> > > >
> > > > After checking out solr's debug output, I dont think the query norm
> is
> > > > relevant either.
> > > >
> > > > I think the scoring changes because
> > > >
> > > > 1) in first case, I have same slop for catchall and name fields. Bot
> > > match
> > > > pf2 pf3. In this case, solr uses max of both for scoring pf2 pf3
> > results.
> > > >
> > > > 2) In second case, I have different slopes, then solr uses sum of
> > values
> > > > instead of max.
> > > >
> > > >
> > > >
> > > > If anyone knows how to work around this, please let me know.
> > > >
> > > > Elisabeth
> > > >
> > > > 2015-12-21 11:22 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> > > >
> > > > > What is your query?
> > > > >
> > > > > On Mon, 21 Dec 2015, 14:37 elisabeth benoit <
> > elisaelisaelisa@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hello all,
> > > > > >
> > > > > > I am using solr 4.10.1 and I have configured my pf2 pf3 like this
> > > > > >
> > > > > > <str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > > > > <str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > > > >
> > > > > > my search field (qf) is my catchall field
> > > > > >
> > > > > > I'v been trying to change slop in pf2, pf3 for catchall and
> > synonyms
> > > > > (going
> > > > > > from 0, or default value for synonyms, to 1)
> > > > > >
> > > > > > pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > > > > pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > > > >
> > > > > > but some results are not ordered the same way anymore even if I
> get
> > > the
> > > > > > same MATCH values in debugQuery output
> > > > > >
> > > > > > For instance, for a doc matching bastill in catchall field (and
> > > nothing
> > > > > to
> > > > > > do with pf2, pf3!)
> > > > > >
> > > > > > with first pf2, pf3
> > > > > >
> > > > > > 0.5163083 = (MATCH) weight(catchall:bastill in 105256)
> > > > > [NoTFIDFSimilarity],
> > > > > > result of:
> > > > > > * 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
> > > > > > ), product of:
> > > > > > * 0.5163083 = queryWeight,* product of:
> > > > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > > > 0.5163083 = queryNorm
> > > > > > 1.0 = fieldWeight in 105256, product of:
> > > > > > 1.0 = tf(freq=2.0), with freq of:
> > > > > > 2.0 = termFreq=2.0
> > > > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > > > 1.0 = fieldNorm(doc=105256)
> > > > > > 0.5163083 = (MATCH) weight(catchall:paris in 105256)
> > > > > > [NoTFIDFSimilarity], result of:
> > > > > > 0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
> > > > > >
> > > > > > and when I change pf2 pf3 (the only change, same query, same
> docs)
> > > > > >
> > > > > > 0.47504464 = (MATCH) weight(catchall:paris in 105256)
> > > > > [NoTFIDFSimilarity],
> > > > > > result of:
> > > > > > * 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
> > > > > > ), product of:
> > > > > > * 0.47504464 = queryWeight*, product of:
> > > > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > > > 0.47504464 = queryNorm
> > > > > > 1.0 = fieldWeight in 105256, product of:
> > > > > > 1.0 = tf(freq=6.0), with freq of:
> > > > > > 6.0 = termFreq=6.0
> > > > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > > > 1.0 = fieldNorm(doc=105256)
> > > > > >
> > > > > > so in the end, with same MATCH results, in first case I get two
> > > > documents
> > > > > > with same score, and in second case, one document has a higher
> > score.
> > > > > >
> > > > > > This seem very very strange. Does anyone have a clue what's going
> > on?
> > > > > >
> > > > > > Thanks
> > > > > > Elisabeth
> > > > > >
> > > > > --
> > > > > Regards,
> > > > > Binoy Dalal
> > > > >
> > > >
> > > --
> > > Regards,
> > > Binoy Dalal
> > >
> >
> --
> Regards,
> Binoy Dalal
>
Re: solr 4.10 I change slop in pf2 pf3 and query norm changes
Posted by Binoy Dalal <bi...@gmail.com>.
Unless the content for both the docs is exactly the same it is highly
unlikely that you will get the same score for the docs under different
querying conditions. What you saw in the first case may have been a happy
coincidence.
Other than that it is very difficult to say why the scoring is different
without getting a look at the actual query and the doc content.
If you still wish to dig deeper, try to understand how solr actually scores
documents that match your query. It takes into account a variety of factors
to compute the cosine similarity to find the best match.
You can find this formula and a decent explanation for it in the book solr
in action or online in the lucene docs:
https://lucene.apache.org/core/3_5_0/api/core/org/apache/lucene/search/Similarity.html
On Tue, 22 Dec 2015, 11:10 elisabeth benoit <el...@gmail.com>
wrote:
> hello,
>
> yes in the second case I get one document with a higher score. the relative
> scoring between documents is not the same anymore.
>
> best regards,
> elisabeth
>
> 2015-12-22 4:39 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
>
> > I have one query.
> > In the second case do you get two records with the same lower scores or
> > just one record with a lower score and the other with a higher one?
> >
> > On Mon, 21 Dec 2015, 18:45 elisabeth benoit <el...@gmail.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I don't think the query is important in this case.
> > >
> > > After checking out solr's debug output, I dont think the query norm is
> > > relevant either.
> > >
> > > I think the scoring changes because
> > >
> > > 1) in first case, I have same slop for catchall and name fields. Bot
> > match
> > > pf2 pf3. In this case, solr uses max of both for scoring pf2 pf3
> results.
> > >
> > > 2) In second case, I have different slopes, then solr uses sum of
> values
> > > instead of max.
> > >
> > >
> > >
> > > If anyone knows how to work around this, please let me know.
> > >
> > > Elisabeth
> > >
> > > 2015-12-21 11:22 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> > >
> > > > What is your query?
> > > >
> > > > On Mon, 21 Dec 2015, 14:37 elisabeth benoit <
> elisaelisaelisa@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello all,
> > > > >
> > > > > I am using solr 4.10.1 and I have configured my pf2 pf3 like this
> > > > >
> > > > > <str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > > > <str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > > >
> > > > > my search field (qf) is my catchall field
> > > > >
> > > > > I'v been trying to change slop in pf2, pf3 for catchall and
> synonyms
> > > > (going
> > > > > from 0, or default value for synonyms, to 1)
> > > > >
> > > > > pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > > > pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > > >
> > > > > but some results are not ordered the same way anymore even if I get
> > the
> > > > > same MATCH values in debugQuery output
> > > > >
> > > > > For instance, for a doc matching bastill in catchall field (and
> > nothing
> > > > to
> > > > > do with pf2, pf3!)
> > > > >
> > > > > with first pf2, pf3
> > > > >
> > > > > 0.5163083 = (MATCH) weight(catchall:bastill in 105256)
> > > > [NoTFIDFSimilarity],
> > > > > result of:
> > > > > * 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
> > > > > ), product of:
> > > > > * 0.5163083 = queryWeight,* product of:
> > > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > > 0.5163083 = queryNorm
> > > > > 1.0 = fieldWeight in 105256, product of:
> > > > > 1.0 = tf(freq=2.0), with freq of:
> > > > > 2.0 = termFreq=2.0
> > > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > > 1.0 = fieldNorm(doc=105256)
> > > > > 0.5163083 = (MATCH) weight(catchall:paris in 105256)
> > > > > [NoTFIDFSimilarity], result of:
> > > > > 0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
> > > > >
> > > > > and when I change pf2 pf3 (the only change, same query, same docs)
> > > > >
> > > > > 0.47504464 = (MATCH) weight(catchall:paris in 105256)
> > > > [NoTFIDFSimilarity],
> > > > > result of:
> > > > > * 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
> > > > > ), product of:
> > > > > * 0.47504464 = queryWeight*, product of:
> > > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > > 0.47504464 = queryNorm
> > > > > 1.0 = fieldWeight in 105256, product of:
> > > > > 1.0 = tf(freq=6.0), with freq of:
> > > > > 6.0 = termFreq=6.0
> > > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > > 1.0 = fieldNorm(doc=105256)
> > > > >
> > > > > so in the end, with same MATCH results, in first case I get two
> > > documents
> > > > > with same score, and in second case, one document has a higher
> score.
> > > > >
> > > > > This seem very very strange. Does anyone have a clue what's going
> on?
> > > > >
> > > > > Thanks
> > > > > Elisabeth
> > > > >
> > > > --
> > > > Regards,
> > > > Binoy Dalal
> > > >
> > >
> > --
> > Regards,
> > Binoy Dalal
> >
>
--
Regards,
Binoy Dalal
Re: solr 4.10 I change slop in pf2 pf3 and query norm changes
Posted by elisabeth benoit <el...@gmail.com>.
hello,
yes in the second case I get one document with a higher score. the relative
scoring between documents is not the same anymore.
best regards,
elisabeth
2015-12-22 4:39 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> I have one query.
> In the second case do you get two records with the same lower scores or
> just one record with a lower score and the other with a higher one?
>
> On Mon, 21 Dec 2015, 18:45 elisabeth benoit <el...@gmail.com>
> wrote:
>
> > Hello,
> >
> > I don't think the query is important in this case.
> >
> > After checking out solr's debug output, I dont think the query norm is
> > relevant either.
> >
> > I think the scoring changes because
> >
> > 1) in first case, I have same slop for catchall and name fields. Bot
> match
> > pf2 pf3. In this case, solr uses max of both for scoring pf2 pf3 results.
> >
> > 2) In second case, I have different slopes, then solr uses sum of values
> > instead of max.
> >
> >
> >
> > If anyone knows how to work around this, please let me know.
> >
> > Elisabeth
> >
> > 2015-12-21 11:22 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> >
> > > What is your query?
> > >
> > > On Mon, 21 Dec 2015, 14:37 elisabeth benoit <elisaelisaelisa@gmail.com
> >
> > > wrote:
> > >
> > > > Hello all,
> > > >
> > > > I am using solr 4.10.1 and I have configured my pf2 pf3 like this
> > > >
> > > > <str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > > <str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > >
> > > > my search field (qf) is my catchall field
> > > >
> > > > I'v been trying to change slop in pf2, pf3 for catchall and synonyms
> > > (going
> > > > from 0, or default value for synonyms, to 1)
> > > >
> > > > pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > > pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > >
> > > > but some results are not ordered the same way anymore even if I get
> the
> > > > same MATCH values in debugQuery output
> > > >
> > > > For instance, for a doc matching bastill in catchall field (and
> nothing
> > > to
> > > > do with pf2, pf3!)
> > > >
> > > > with first pf2, pf3
> > > >
> > > > 0.5163083 = (MATCH) weight(catchall:bastill in 105256)
> > > [NoTFIDFSimilarity],
> > > > result of:
> > > > * 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
> > > > ), product of:
> > > > * 0.5163083 = queryWeight,* product of:
> > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > 0.5163083 = queryNorm
> > > > 1.0 = fieldWeight in 105256, product of:
> > > > 1.0 = tf(freq=2.0), with freq of:
> > > > 2.0 = termFreq=2.0
> > > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > > 1.0 = fieldNorm(doc=105256)
> > > > 0.5163083 = (MATCH) weight(catchall:paris in 105256)
> > > > [NoTFIDFSimilarity], result of:
> > > > 0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
> > > >
> > > > and when I change pf2 pf3 (the only change, same query, same docs)
> > > >
> > > > 0.47504464 = (MATCH) weight(catchall:paris in 105256)
> > > [NoTFIDFSimilarity],
> > > > result of:
> > > > * 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
> > > > ), product of:
> > > > * 0.47504464 = queryWeight*, product of:
> > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > 0.47504464 = queryNorm
> > > > 1.0 = fieldWeight in 105256, product of:
> > > > 1.0 = tf(freq=6.0), with freq of:
> > > > 6.0 = termFreq=6.0
> > > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > > 1.0 = fieldNorm(doc=105256)
> > > >
> > > > so in the end, with same MATCH results, in first case I get two
> > documents
> > > > with same score, and in second case, one document has a higher score.
> > > >
> > > > This seem very very strange. Does anyone have a clue what's going on?
> > > >
> > > > Thanks
> > > > Elisabeth
> > > >
> > > --
> > > Regards,
> > > Binoy Dalal
> > >
> >
> --
> Regards,
> Binoy Dalal
>
Re: solr 4.10 I change slop in pf2 pf3 and query norm changes
Posted by Binoy Dalal <bi...@gmail.com>.
I have one query.
In the second case do you get two records with the same lower scores or
just one record with a lower score and the other with a higher one?
On Mon, 21 Dec 2015, 18:45 elisabeth benoit <el...@gmail.com>
wrote:
> Hello,
>
> I don't think the query is important in this case.
>
> After checking out solr's debug output, I dont think the query norm is
> relevant either.
>
> I think the scoring changes because
>
> 1) in first case, I have same slop for catchall and name fields. Bot match
> pf2 pf3. In this case, solr uses max of both for scoring pf2 pf3 results.
>
> 2) In second case, I have different slopes, then solr uses sum of values
> instead of max.
>
>
>
> If anyone knows how to work around this, please let me know.
>
> Elisabeth
>
> 2015-12-21 11:22 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
>
> > What is your query?
> >
> > On Mon, 21 Dec 2015, 14:37 elisabeth benoit <el...@gmail.com>
> > wrote:
> >
> > > Hello all,
> > >
> > > I am using solr 4.10.1 and I have configured my pf2 pf3 like this
> > >
> > > <str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > > <str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > >
> > > my search field (qf) is my catchall field
> > >
> > > I'v been trying to change slop in pf2, pf3 for catchall and synonyms
> > (going
> > > from 0, or default value for synonyms, to 1)
> > >
> > > pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > > pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > >
> > > but some results are not ordered the same way anymore even if I get the
> > > same MATCH values in debugQuery output
> > >
> > > For instance, for a doc matching bastill in catchall field (and nothing
> > to
> > > do with pf2, pf3!)
> > >
> > > with first pf2, pf3
> > >
> > > 0.5163083 = (MATCH) weight(catchall:bastill in 105256)
> > [NoTFIDFSimilarity],
> > > result of:
> > > * 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
> > > ), product of:
> > > * 0.5163083 = queryWeight,* product of:
> > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > 0.5163083 = queryNorm
> > > 1.0 = fieldWeight in 105256, product of:
> > > 1.0 = tf(freq=2.0), with freq of:
> > > 2.0 = termFreq=2.0
> > > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > > 1.0 = fieldNorm(doc=105256)
> > > 0.5163083 = (MATCH) weight(catchall:paris in 105256)
> > > [NoTFIDFSimilarity], result of:
> > > 0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
> > >
> > > and when I change pf2 pf3 (the only change, same query, same docs)
> > >
> > > 0.47504464 = (MATCH) weight(catchall:paris in 105256)
> > [NoTFIDFSimilarity],
> > > result of:
> > > * 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
> > > ), product of:
> > > * 0.47504464 = queryWeight*, product of:
> > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > 0.47504464 = queryNorm
> > > 1.0 = fieldWeight in 105256, product of:
> > > 1.0 = tf(freq=6.0), with freq of:
> > > 6.0 = termFreq=6.0
> > > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > > 1.0 = fieldNorm(doc=105256)
> > >
> > > so in the end, with same MATCH results, in first case I get two
> documents
> > > with same score, and in second case, one document has a higher score.
> > >
> > > This seem very very strange. Does anyone have a clue what's going on?
> > >
> > > Thanks
> > > Elisabeth
> > >
> > --
> > Regards,
> > Binoy Dalal
> >
>
--
Regards,
Binoy Dalal
Re: solr 4.10 I change slop in pf2 pf3 and query norm changes
Posted by elisabeth benoit <el...@gmail.com>.
Hello,
I don't think the query is important in this case.
After checking out solr's debug output, I dont think the query norm is
relevant either.
I think the scoring changes because
1) in first case, I have same slop for catchall and name fields. Bot match
pf2 pf3. In this case, solr uses max of both for scoring pf2 pf3 results.
2) In second case, I have different slopes, then solr uses sum of values
instead of max.
If anyone knows how to work around this, please let me know.
Elisabeth
2015-12-21 11:22 GMT+01:00 Binoy Dalal <bi...@gmail.com>:
> What is your query?
>
> On Mon, 21 Dec 2015, 14:37 elisabeth benoit <el...@gmail.com>
> wrote:
>
> > Hello all,
> >
> > I am using solr 4.10.1 and I have configured my pf2 pf3 like this
> >
> > <str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> > <str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> >
> > my search field (qf) is my catchall field
> >
> > I'v been trying to change slop in pf2, pf3 for catchall and synonyms
> (going
> > from 0, or default value for synonyms, to 1)
> >
> > pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> > pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> >
> > but some results are not ordered the same way anymore even if I get the
> > same MATCH values in debugQuery output
> >
> > For instance, for a doc matching bastill in catchall field (and nothing
> to
> > do with pf2, pf3!)
> >
> > with first pf2, pf3
> >
> > 0.5163083 = (MATCH) weight(catchall:bastill in 105256)
> [NoTFIDFSimilarity],
> > result of:
> > * 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
> > ), product of:
> > * 0.5163083 = queryWeight,* product of:
> > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > 0.5163083 = queryNorm
> > 1.0 = fieldWeight in 105256, product of:
> > 1.0 = tf(freq=2.0), with freq of:
> > 2.0 = termFreq=2.0
> > 1.0 = idf(docFreq=134, maxDocs=12258543)
> > 1.0 = fieldNorm(doc=105256)
> > 0.5163083 = (MATCH) weight(catchall:paris in 105256)
> > [NoTFIDFSimilarity], result of:
> > 0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
> >
> > and when I change pf2 pf3 (the only change, same query, same docs)
> >
> > 0.47504464 = (MATCH) weight(catchall:paris in 105256)
> [NoTFIDFSimilarity],
> > result of:
> > * 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
> > ), product of:
> > * 0.47504464 = queryWeight*, product of:
> > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > 0.47504464 = queryNorm
> > 1.0 = fieldWeight in 105256, product of:
> > 1.0 = tf(freq=6.0), with freq of:
> > 6.0 = termFreq=6.0
> > 1.0 = idf(docFreq=10958, maxDocs=12258543)
> > 1.0 = fieldNorm(doc=105256)
> >
> > so in the end, with same MATCH results, in first case I get two documents
> > with same score, and in second case, one document has a higher score.
> >
> > This seem very very strange. Does anyone have a clue what's going on?
> >
> > Thanks
> > Elisabeth
> >
> --
> Regards,
> Binoy Dalal
>
Re: solr 4.10 I change slop in pf2 pf3 and query norm changes
Posted by Binoy Dalal <bi...@gmail.com>.
What is your query?
On Mon, 21 Dec 2015, 14:37 elisabeth benoit <el...@gmail.com>
wrote:
> Hello all,
>
> I am using solr 4.10.1 and I have configured my pf2 pf3 like this
>
> <str name="pf2">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
> <str name="pf3">catchall~0^0.2 name~0^0.21 synonyms^0.2</str>
>
> my search field (qf) is my catchall field
>
> I'v been trying to change slop in pf2, pf3 for catchall and synonyms (going
> from 0, or default value for synonyms, to 1)
>
> pf2=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
> pf3=catchall~1^0.2 name~0^0.21 synonyms~1^0.2
>
> but some results are not ordered the same way anymore even if I get the
> same MATCH values in debugQuery output
>
> For instance, for a doc matching bastill in catchall field (and nothing to
> do with pf2, pf3!)
>
> with first pf2, pf3
>
> 0.5163083 = (MATCH) weight(catchall:bastill in 105256) [NoTFIDFSimilarity],
> result of:
> * 0.5163083 = score(doc=105256,freq=2.0 = termFreq=2.0*
> ), product of:
> * 0.5163083 = queryWeight,* product of:
> 1.0 = idf(docFreq=134, maxDocs=12258543)
> 0.5163083 = queryNorm
> 1.0 = fieldWeight in 105256, product of:
> 1.0 = tf(freq=2.0), with freq of:
> 2.0 = termFreq=2.0
> 1.0 = idf(docFreq=134, maxDocs=12258543)
> 1.0 = fieldNorm(doc=105256)
> 0.5163083 = (MATCH) weight(catchall:paris in 105256)
> [NoTFIDFSimilarity], result of:
> 0.5163083 = score(doc=105256,freq=6.0 = termFreq=6.0
>
> and when I change pf2 pf3 (the only change, same query, same docs)
>
> 0.47504464 = (MATCH) weight(catchall:paris in 105256) [NoTFIDFSimilarity],
> result of:
> * 0.47504464 = score(doc=105256,freq=6.0 = termFreq=6.0*
> ), product of:
> * 0.47504464 = queryWeight*, product of:
> 1.0 = idf(docFreq=10958, maxDocs=12258543)
> 0.47504464 = queryNorm
> 1.0 = fieldWeight in 105256, product of:
> 1.0 = tf(freq=6.0), with freq of:
> 6.0 = termFreq=6.0
> 1.0 = idf(docFreq=10958, maxDocs=12258543)
> 1.0 = fieldNorm(doc=105256)
>
> so in the end, with same MATCH results, in first case I get two documents
> with same score, and in second case, one document has a higher score.
>
> This seem very very strange. Does anyone have a clue what's going on?
>
> Thanks
> Elisabeth
>
--
Regards,
Binoy Dalal