You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Christian Kohlschütter (JIRA)" <ji...@apache.org> on 2007/07/09 18:53:04 UTC
[jira] Created: (LUCENE-954) Toggle score normalization in Hits
Toggle score normalization in Hits
----------------------------------
Key: LUCENE-954
URL: https://issues.apache.org/jira/browse/LUCENE-954
Project: Lucene - Java
Issue Type: Improvement
Components: Search
Affects Versions: 2.2
Environment: any
Reporter: Christian Kohlschütter
Fix For: 2.2
The current implementation of the "Hits" class sometimes performs score normalization.
In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
In this case, Hits may return different score results than TopDocs-based methods.
In my scenario (a federated search system), Hits delievered just plain wrong results.
I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
I suggest to add a switch to enable/disable this score-normalization at runtime.
My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-954) Toggle score normalization in Hits
Posted by "Christian Kohlschütter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kohlschütter updated LUCENE-954:
------------------------------------------
Fix Version/s: 2.4
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
Affects Version/s: 2.3
2.3.1
changed affected versions
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571363#action_12571363 ]
Grant Ingersoll commented on LUCENE-954:
----------------------------------------
{quote}
This is why the scores from Hits are currently not comparable to each other
{quote}
Do you mean across queries or within queries? Even if you have raw scores, they still won't be comparable across queries, or at least that is my understanding of the literature. In your original case of federated search across several sources, each with their own stats it is not well understood what scores mean. Not saying it can't be done, it really is the only way to do federated search, just not sure one can try to read too much into the scores. Of course, this is more of a discussion for the user list than a JIRA issue, so I'd be happy to discuss more there and hear other thoughts. It has been a while since I have read anything on it.
That also isn't to say that your patch isn't worthwhile, just wondering whether the change is actually meaningful for your use case.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Christian Kohlschütter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571339#action_12571339 ]
Christian Kohlschütter commented on LUCENE-954:
-----------------------------------------------
You are right, Yonik.
Hits currently tries to "hide" this by normalizing the scores to a maximum of 1, simply by dividing the "raw" scores by the maximum score returned.
This is why the scores from Hits are currently not comparable to each other. The suggested patch resolves this problem.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Yonik Seeley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12569698#action_12569698 ]
Yonik Seeley commented on LUCENE-954:
-------------------------------------
Normalization is only applied to the queryWeight part of the score (the part the same for all documents), but not to the fieldWeight. idf and norms can both be > 1.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-954) Toggle score normalization in Hits
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Otis Gospodnetic updated LUCENE-954:
------------------------------------
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
Assignee: Otis Gospodnetic
Unit test passes.
Will commit next week.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Assignee: Otis Gospodnetic
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571553#action_12571553 ]
Doug Cutting commented on LUCENE-954:
-------------------------------------
Disabling score normalization in Hits seems like a reasonable feature to me. +1
The indentation in this patch uses tabs. -1
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600366#action_12600366 ]
Otis Gospodnetic commented on LUCENE-954:
-----------------------------------------
I suppose there is now suddenly no need to work on Hits. I'll resolve this as Won't Fix in a few days, unless somebody has some more thoughts on this.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Assignee: Otis Gospodnetic
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Closed: (LUCENE-954) Toggle score normalization in Hits
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Busch closed LUCENE-954.
--------------------------------
Resolution: Won't Fix
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
Hits is deprecated.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Assignee: Otis Gospodnetic
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Hoss Man (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579277#action_12579277 ]
Hoss Man commented on LUCENE-954:
---------------------------------
bq. I hate to rain on the parade, but maybe instead of making small modifications to the way Hits works, it's time to deprecate it?
I agree with your sentiment, but it's somewhat orthogonal to this issue.
If someone opens a Jira issue to deprecate the Hits class, and attaches a patch that does so and replaces examples of it's usage in the demo and tutorial, i'll certainly vote for it -- but until then, if people want to try and improve Hits, there's little reason not to do so.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-954) Toggle score normalization in Hits
Posted by "Christian Kohlschütter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kohlschütter updated LUCENE-954:
------------------------------------------
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
Affects Version/s: 2.4
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by Grant Ingersoll <gs...@apache.org>.
On Feb 22, 2008, at 7:10 AM, Christian Kohlschütter (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571367
> #action_12571367 ]
>
> Christian Kohlschütter commented on LUCENE-954:
> -----------------------------------------------
>
> Grant,
>
> sorry I was perhaps not too clear about it.
>
> The distribution of scores of one Hits instance is currently not
> comparable to another distribution of scores of another Hits object,
> even if the underlying statistics are comparable/compatible/
> identical. This is due to the case that the values are always
> normalized to a maximum of 1.0.
>
> As I said, my Federated Search system provides homogeneous
> statistics (but not via MultiSearcher). In fact, it does not use
> MultiSearcher for this, but a variant of the SRU/SRW/XCQL protocols
> ("SRX/FS"), where all communication is done via HTTP and XML. This
> includes the exchange of Term/DF statistics. At the end, the system
> makes several distributed Indexes appear as a single (read:
> federated) index. In order to merge the results from each index,
> Hits is used.
Gotcha. It wasn't clear that you were exchanging the searchers were
aware of global statistics. I think the use of "federated" is what
through me off. I have understood your description to be better
called "distributed search", whereas federated search is more along
the lines of disparate indexes that don't share statistics. I know,
"you say tomato,..."
In the distributed case, yes, it does make sense to be able to compare
the scores.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Christian Kohlschütter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571367#action_12571367 ]
Christian Kohlschütter commented on LUCENE-954:
-----------------------------------------------
Grant,
sorry I was perhaps not too clear about it.
The distribution of scores of one Hits instance is currently not comparable to another distribution of scores of another Hits object, even if the underlying statistics are comparable/compatible/identical. This is due to the case that the values are always normalized to a maximum of 1.0.
As I said, my Federated Search system provides homogeneous statistics (but not via MultiSearcher). In fact, it does not use MultiSearcher for this, but a variant of the SRU/SRW/XCQL protocols ("SRX/FS"), where all communication is done via HTTP and XML. This includes the exchange of Term/DF statistics. At the end, the system makes several distributed Indexes appear as a single (read: federated) index. In order to merge the results from each index, Hits is used.
In the simplest case, the results from every Hits object (one per source) are simply merged by score in descending order. With the current implementation of Lucene Hits, these scores are not comparable across instances. With the patch, they are (at least when score normalization is turned off).
If you need more information about the Federated Search system, we can indeed move the discussion to the mailing list. However, I think the problem is not really specific to my needs. Even if you have two Hits instances locally, you might want to be able to compare the scores (or merge the results) from Hits instance A to those from Hits instance B (in particular, when they are from the same index). This is also not possible right now.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620057#action_12620057 ]
Mark Miller commented on LUCENE-954:
------------------------------------
Looks like this guy is ready to be resolved.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Assignee: Otis Gospodnetic
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Christian Kohlschütter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579367#action_12579367 ]
Christian Kohlschütter commented on LUCENE-954:
-----------------------------------------------
I agree with Hoss. Please file a new issue if you want to see Hits (and consequently also Hit/HitIterator) being deprecated. I do not see any reason for this, though.
This patch is meant for helping Lucene users who currently use the Hits class and particularly have problems with the built-in score normalization, and not with its performance.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-954) Toggle score normalization in Hits
Posted by "Christian Kohlschütter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kohlschütter updated LUCENE-954:
------------------------------------------
Attachment: LUCENE-954.patch
Reworked patch, now against SVN Trunk (also works with 2.3), and without tabs.
The patch file also includes a new testcase which demonstrates the new feature.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598092#action_12598092 ]
Michael Busch commented on LUCENE-954:
--------------------------------------
{quote}
If someone opens a Jira issue to deprecate the Hits class, and attaches a patch that does so and replaces examples of it's usage in the demo and tutorial, i'll certainly vote for it.
{quote}
Done with LUCENE-1290.
I'm waiting for your vote, Hoss ;-)
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Assignee: Otis Gospodnetic
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Nadav Har'El (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579258#action_12579258 ]
Nadav Har'El commented on LUCENE-954:
-------------------------------------
I hate to rain on the parade, but maybe instead of making small modifications to the way Hits works, it's time to deprecate it?
Hits has numerous flaws compared to the alternative interface (Searcher.search(Query, HitCollector), with TopDocCollector). It tries to "guess" in advance the number of results it should calculate (usually calculating too many, or too few and having to run the search again). It does bizarre normalization of the score (as this patch points out). It is harder to extend the way the HitCollector interface can be (for an example, see the recently checked-in timed hit collector, replacing yet another suggest improvements to the Hits interface).
So I say - it's time to deprecate the Hits search(Query) method, to change the tutorials to recommend TopDocCollector instead, and to stop trying to improve Hits.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-954) Toggle score normalization in Hits
Posted by "Christian Kohlschütter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian Kohlschütter updated LUCENE-954:
------------------------------------------
Attachment: hits-scoreNorm.patch
Adds a switch to enable/disable Hits-based score normalization.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Fix For: 2.2
>
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-954) Toggle score normalization in Hits
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated LUCENE-954:
-----------------------------------
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
Fix Version/s: (was: 2.2)
change the version
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Michael Busch (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12579264#action_12579264 ]
Michael Busch commented on LUCENE-954:
--------------------------------------
{quote}
So I say - it's time to deprecate the Hits search(Query) method, to change the tutorials to recommend TopDocCollector instead, and to stop trying to improve Hits.
{quote}
+1
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2, 2.3, 2.3.1, 2.4
> Environment: any
> Reporter: Christian Kohlschütter
> Fix For: 2.4
>
> Attachments: hits-scoreNorm.patch, LUCENE-954.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-954) Toggle score normalization in Hits
Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12569656#action_12569656 ]
Otis Gospodnetic commented on LUCENE-954:
-----------------------------------------
I don't see any harm in adding this, though I don't understand why the top-score was greater than 1 (original description above mentions this) in the first place, since scores in Hits are normalized and thus should always be less than 1.
> Toggle score normalization in Hits
> ----------------------------------
>
> Key: LUCENE-954
> URL: https://issues.apache.org/jira/browse/LUCENE-954
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Search
> Affects Versions: 2.2
> Environment: any
> Reporter: Christian Kohlschütter
> Attachments: hits-scoreNorm.patch
>
>
> The current implementation of the "Hits" class sometimes performs score normalization.
> In particular, whenever the top-ranked score is bigger than 1.0, it is normalized to a maximum of 1.0.
> In this case, Hits may return different score results than TopDocs-based methods.
> In my scenario (a federated search system), Hits delievered just plain wrong results.
> I was merging results from several sources, all having homogeneous statistics (similar to MultiSearcher, but over the Internet using HTTP/XML-based protocols).
> Sometimes, some of the sources had a top-score greater than 1, so I ended up with garbled results.
> I suggest to add a switch to enable/disable this score-normalization at runtime.
> My patch (attached) has an additional peformance benefit, since score normalization now occurs only when Hits#score() is called, not when creating the Hits result list. Whenever scores are not required, you save one multiplication per retrieved hit (i.e., at least 100 multiplications with the current implementation of Hits).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org