TREC-style IR experiments

[Ismo Raitanen <is...@gmail.com>]

Hi,

I'm planning to do some information retrieval experiments with Solr.
I'd like to compare different IR methods. I have a test collection
with topics and judgements available. I'm considering using Solr (and
not Lemur/Indri etc.) for the tests, because Solr supports several
nice methods out-of-the-box, e.g. n-grams.

Finally, I plan to evaluate the different methods and their results
with trec_eval or similar program. What I need is a program, which
puts Solr results in a suitable format for trec_eval. I think I can
get the Solr search results in that format quite easily by using the
solr-php-client library.

Have any of you run "TREC-style" IR experiments with Solr and what are
your experiences with that? Do you have any suggestion for that kind
of tests with Solr?

Kind regards,
Ismo

Re: TREC-style IR experiments

[Ismo Raitanen <is...@gmail.com>]

>> I'm planning to do some information retrieval experiments with Solr.

> There some existing implementations in Lucene
> http://lucene.apache.org/java/3_0_2/api/contrib-benchmark/org/apache/lucene/benchmark/quality/trec/package-summary.html

Have you used that with Solr? How?

//Ismo

Re: TREC-style IR experiments

[Ahmet Arslan <io...@yahoo.com>]

> I'm planning to do some information retrieval experiments
> with Solr.
> I'd like to compare different IR methods. I have a test
> collection
> with topics and judgements available. I'm considering using
> Solr (and
> not Lemur/Indri etc.) for the tests, because Solr supports
> several
> nice methods out-of-the-box, e.g. n-grams.
> 
> Finally, I plan to evaluate the different methods and their
> results
> with trec_eval or similar program. What I need is a
> program, which
> puts Solr results in a suitable format for trec_eval. I
> think I can
> get the Solr search results in that format quite easily by
> using the
> solr-php-client library.
> 
> Have any of you run "TREC-style" IR experiments with Solr
> and what are
> your experiences with that? Do you have any suggestion for
> that kind
> of tests with Solr?

There some existing implementations in Lucene

http://lucene.apache.org/java/3_0_2/api/contrib-benchmark/org/apache/lucene/benchmark/quality/trec/package-summary.html