Solr 7.1 and 5.4 differences in bf

[Homero Gonzalez <hg...@costar.com>]

Hi,
I am working on the migration of SOLR 5.4 to 7.1 and I have not been able to get the same order in the results. Looks like the problem is with the bf parameter.

We use edismax with both boost and bf functions. It is important to have some functions as bf so they add to the score and the impact is less than the ones in the boost parameter.

I have tried overriding the similarity class by specifying the following in schema.xml but I am still not able to get the same score and order of results with equal queries in 5.4 and 7.1
<similarity class="org.apache.lucene.search.similarities.ClassicSimilarity" />

We are also using qf to boost some query fields.

Enabling debug=true I noticed that 5.4 uses the queryNorm as the boost value of the bf formulas. Version 7.1 is using a boost value of 1 in all bf formulas.

Is there a way to configure 7.1 so it uses queryNorm in bf and produces the same results as 5.4?

Thanks,
Homero.

RE: Solr 7.1 and 5.4 differences in bf

[Homero Gonzalez <hg...@costar.com>]

Hi Erick,
I am ok with getting differences because lucene uses different similarity algorithm in qf that in the new version may be better.

The problem I am reporting is with the bf behavior. Since queryNorm varies from query to query I have not found a way to have consistent boost results between 5.4 and 7.1.

Is there a way to get the same boost results with the same formulas in bf between 5.4 and 7.1?

Thanks,
Homero.

-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: Friday, March 23, 2018 9:33 AM
To: solr-user
Subject: Re: Solr 7.1 and 5.4 differences in bf

I would not focus at all on getting the same ordering. There are ongoing improvements and changes, for instance:
LUCENE-7368: Removed query normalization

Instead, I'd focus on the question "is the ranking I'm seeing in 7.x better or worse than 5.4 and tune until you could say "yes"...

Best,
Erick

On Fri, Mar 23, 2018 at 8:15 AM, Homero Gonzalez <hg...@costar.com> wrote:
> Hi,
> I am working on the migration of SOLR 5.4 to 7.1 and I have not been able to get the same order in the results. Looks like the problem is with the bf parameter.
>
> We use edismax with both boost and bf functions. It is important to have some functions as bf so they add to the score and the impact is less than the ones in the boost parameter.
>
> I have tried overriding the similarity class by specifying the 
> following in schema.xml but I am still not able to get the same score 
> and order of results with equal queries in 5.4 and 7.1 <similarity 
> class="org.apache.lucene.search.similarities.ClassicSimilarity" />
>
> We are also using qf to boost some query fields.
>
> Enabling debug=true I noticed that 5.4 uses the queryNorm as the boost value of the bf formulas. Version 7.1 is using a boost value of 1 in all bf formulas.
>
> Is there a way to configure 7.1 so it uses queryNorm in bf and produces the same results as 5.4?
>
> Thanks,
> Homero.

Re: Solr 7.1 and 5.4 differences in bf

[Erick Erickson <er...@gmail.com>]

I would not focus at all on getting the same ordering. There are
ongoing improvements and changes, for instance:
LUCENE-7368: Removed query normalization

Instead, I'd focus on the question "is the ranking I'm seeing in 7.x
better or worse than 5.4 and tune until you could
say "yes"...

Best,
Erick

On Fri, Mar 23, 2018 at 8:15 AM, Homero Gonzalez <hg...@costar.com> wrote:
> Hi,
> I am working on the migration of SOLR 5.4 to 7.1 and I have not been able to get the same order in the results. Looks like the problem is with the bf parameter.
>
> We use edismax with both boost and bf functions. It is important to have some functions as bf so they add to the score and the impact is less than the ones in the boost parameter.
>
> I have tried overriding the similarity class by specifying the following in schema.xml but I am still not able to get the same score and order of results with equal queries in 5.4 and 7.1
> <similarity class="org.apache.lucene.search.similarities.ClassicSimilarity" />
>
> We are also using qf to boost some query fields.
>
> Enabling debug=true I noticed that 5.4 uses the queryNorm as the boost value of the bf formulas. Version 7.1 is using a boost value of 1 in all bf formulas.
>
> Is there a way to configure 7.1 so it uses queryNorm in bf and produces the same results as 5.4?
>
> Thanks,
> Homero.