You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by David Ryan <he...@gmail.com> on 2011/10/05 20:23:36 UTC
New scoring models in LUCENE/SOLR (LUCENE-2959)
Hi,
According to the IRA issue 2959,
https://issues.apache.org/jira/browse/LUCENE-2959
BM25 will be included in the next release of LUCENE.
1). Will BM25F be included in the next release as well as part
of LUCENE-2959?
2). What's the timeline of the next release that new scoring modules will
be available?
Re: New scoring models in LUCENE/SOLR (LUCENE-2959)
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Oct 5, 2011 at 3:03 PM, David Ryan <he...@gmail.com> wrote:
> Do you mean both BM25 and BM25F?
>
>
No, BM25F and other "fielded" or structured models are somewhat different.
In these model, if you have two fields (body/title) you are saying
that "dogs" in body is actually the same term as "dogs" in title. This
is only appropriate in these cases, but not for all fields in the
document (e.g. many solr users use copyField and analyzer content in
different ways, so the terms are different, or put different languages
in different fields).
In my opinion, to support models like this we should add a "structured
query" (and ideally queryparser hooks) representing this intent, that
works for a term across multiple fields where you declare this is the
case. This would be a future improvement, and i'm not sure BM25F is
ever a good fit because it wants a document-level idf (not really a
practical thing for lucene, unless we come up with some cool
approximation), but newer models like pl2f/mdl2 use corpus total term
frequency instead which we can compute from the components
efficiently.
--
lucidimagination.com
Re: New scoring models in LUCENE/SOLR (LUCENE-2959)
Posted by David Ryan <he...@gmail.com>.
Do you mean both BM25 and BM25F?
On Wed, Oct 5, 2011 at 11:44 AM, Robert Muir <rc...@gmail.com> wrote:
> On Wed, Oct 5, 2011 at 2:23 PM, David Ryan <he...@gmail.com> wrote:
> > Hi,
> >
> > According to the IRA issue 2959,
> > https://issues.apache.org/jira/browse/LUCENE-2959
> >
> > BM25 will be included in the next release of LUCENE.
> >
> > 1). Will BM25F be included in the next release as well as part
> > of LUCENE-2959?
>
> should be in the 4.0 release, as the solr integration has already been
> added (https://issues.apache.org/jira/browse/SOLR-2754)
> So if you checkout trunk from svn, you can specify these factories in
> your schema.xml to use these new models.
>
> > 2). What's the timeline of the next release that new scoring modules
> will
> > be available?
> >
>
> unfortunately, nobody can answer this...
>
> --
> lucidimagination.com
>
Re: New scoring models in LUCENE/SOLR (LUCENE-2959)
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Oct 5, 2011 at 2:23 PM, David Ryan <he...@gmail.com> wrote:
> Hi,
>
> According to the IRA issue 2959,
> https://issues.apache.org/jira/browse/LUCENE-2959
>
> BM25 will be included in the next release of LUCENE.
>
> 1). Will BM25F be included in the next release as well as part
> of LUCENE-2959?
should be in the 4.0 release, as the solr integration has already been
added (https://issues.apache.org/jira/browse/SOLR-2754)
So if you checkout trunk from svn, you can specify these factories in
your schema.xml to use these new models.
> 2). What's the timeline of the next release that new scoring modules will
> be available?
>
unfortunately, nobody can answer this...
--
lucidimagination.com