[jira] Updated: (LUCENE-2091) Add BM25 Scoring to Lucene

["Robert Muir (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robert Muir updated LUCENE-2091:
--------------------------------

    Attachment: BM25SimilarityProvider.java

Hello, I think there might be a bug in the formula here.

While working on LUCENE-2392, I have implemented several formula, including BM25 and I(Ne)C2/I(Ne)B2 from the divergence-from-randomness models.

I first implemented DFR and after verifying the relevance was correct against published results, I noticed BM25 (with this patch) had much lower relevance scores. This didn't make a lot of sense to me.

So I implemented BM25 from scratch, in terms of the stats framework of LUCENE-2392. 

Here are some MAP scores on an english text collection (I have tested several and this is the consistent behavior):
|DefaultSimilarity|0.4333|
|BM25 (LUCENE-2091)|0.4131|
|BM25 (LUCENE-2392)|0.4395|
|BM25* (LUCENE-2392)|0.4401|
|DFR I(Ne)C2|0.4453|
|DFR I(Ne)B2|0.4475|

Notes: 
* BM25* is what is actually attached, as it uses the flooring for IDF, but i also tested without this (BM25)
* I used the same defaults as this patch for k1 and b parameters.

I've attached my implementation for the sake of it, although the APIs are pretty meaningless right now, but just so you can see how I implemented the equation.

> Add BM25 Scoring to Lucene
> --------------------------
>
>                 Key: LUCENE-2091
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2091
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: contrib/*
>            Reporter: Yuval Feinstein
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: BM25SimilarityProvider.java, LUCENE-2091.patch, persianlucene.jpg
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> http://nlp.uned.es/~jperezi/Lucene-BM25/ describes an implementation of Okapi-BM25 scoring in the Lucene framework,
> as an alternative to the standard Lucene scoring (which is a version of mixed boolean/TFIDF).
> I have refactored this a bit, added unit tests and improved the runtime somewhat.
> I would like to contribute the code to Lucene under contrib. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org