You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Michael Preminger <Mi...@hioa.no> on 2013/10/30 15:52:20 UTC
Evaluating a SOLR index with trec_eval
Hello!
Is there a simple way to evaluate a SOLR index with TREC_EVAL?
I mean:
* preparing a query file in some format Solr will understand, but where each query has an ID
* getting results out in trec format, with these query IDs attached
Thanks
Michael
SV: Evaluating a SOLR index with trec_eval
Posted by Michael Preminger <Mi...@hioa.no>.
... AND appologies to everyone for erroneously posting irrelevant stuff on the list.
Michael
________________________________________
Fra: Michael Preminger [Michael.Preminger@hioa.no]
Sendt: 30. oktober 2013 20:34
To: solr-user@lucene.apache.org
Emne: SV: Evaluating a SOLR index with trec_eval
Hi, Tom!
Thanks alot. Ill check Ian's stuff and anticpate yours ...
As you know, the ProveIt is now terminated as an INEX track, but we still hope to write a paper to a journal, summarizing what was done, and it would be nice to have you on.
AND, youll be happy (or shocked) to know that this week I used your INEX paper from 2011 as an example
of "practice-near" research in a seminar I was running for them, and they had an assignment to right a "reflection note" in advance where they associate your stuff with their own assignment.
Michael
________________________________________
Fra: Tom Burton-West [tburtonw@umich.edu]
Sendt: 30. oktober 2013 20:26
To: solr-user@lucene.apache.org
Emne: Re: Evaluating a SOLR index with trec_eval
Hi Michael,
I know you are asking about Solr, but in case you haven't seen it, Ian
Soboroff has a nice little demo for Lucene:
https://github.com/isoboroff/trec-demo.
There is also the lucene benchmark code:
http://lucene.apache.org/core/4_5_1/benchmark/org/apache/lucene/benchmark/quality/package-summary.html
Otherwise, all I can think of is writing an app layer that keeps track of
the id, sends the query to Solr, parses the search results and spits out
results in the trec format. I'd love to find some open-source code that
does what you ask.
I did a quick and dirty version of something like that for the INEX book
track. I'll see if I can find the code and if it is in any shape to share.
Tom
Tom Burton-West
Information Retrieval Programmer
Digital Library Production Sevice
University of Michigan Library
tburtonw@umich.edu
http://www.hathitrust.org/blogs/large-scale-search
On Wed, Oct 30, 2013 at 10:52 AM, Michael Preminger <
Michael.Preminger@hioa.no> wrote:
> Hello!
>
> Is there a simple way to evaluate a SOLR index with TREC_EVAL?
> I mean:
> * preparing a query file in some format Solr will understand, but where
> each query has an ID
> * getting results out in trec format, with these query IDs attached
>
> Thanks
>
> Michael
>
SV: Evaluating a SOLR index with trec_eval
Posted by Michael Preminger <Mi...@hioa.no>.
Hi, Tom!
Thanks alot. Ill check Ian's stuff and anticpate yours ...
As you know, the ProveIt is now terminated as an INEX track, but we still hope to write a paper to a journal, summarizing what was done, and it would be nice to have you on.
AND, youll be happy (or shocked) to know that this week I used your INEX paper from 2011 as an example
of "practice-near" research in a seminar I was running for them, and they had an assignment to right a "reflection note" in advance where they associate your stuff with their own assignment.
Michael
________________________________________
Fra: Tom Burton-West [tburtonw@umich.edu]
Sendt: 30. oktober 2013 20:26
To: solr-user@lucene.apache.org
Emne: Re: Evaluating a SOLR index with trec_eval
Hi Michael,
I know you are asking about Solr, but in case you haven't seen it, Ian
Soboroff has a nice little demo for Lucene:
https://github.com/isoboroff/trec-demo.
There is also the lucene benchmark code:
http://lucene.apache.org/core/4_5_1/benchmark/org/apache/lucene/benchmark/quality/package-summary.html
Otherwise, all I can think of is writing an app layer that keeps track of
the id, sends the query to Solr, parses the search results and spits out
results in the trec format. I'd love to find some open-source code that
does what you ask.
I did a quick and dirty version of something like that for the INEX book
track. I'll see if I can find the code and if it is in any shape to share.
Tom
Tom Burton-West
Information Retrieval Programmer
Digital Library Production Sevice
University of Michigan Library
tburtonw@umich.edu
http://www.hathitrust.org/blogs/large-scale-search
On Wed, Oct 30, 2013 at 10:52 AM, Michael Preminger <
Michael.Preminger@hioa.no> wrote:
> Hello!
>
> Is there a simple way to evaluate a SOLR index with TREC_EVAL?
> I mean:
> * preparing a query file in some format Solr will understand, but where
> each query has an ID
> * getting results out in trec format, with these query IDs attached
>
> Thanks
>
> Michael
>
Re: Evaluating a SOLR index with trec_eval
Posted by Tom Burton-West <tb...@umich.edu>.
Hi Michael,
I know you are asking about Solr, but in case you haven't seen it, Ian
Soboroff has a nice little demo for Lucene:
https://github.com/isoboroff/trec-demo.
There is also the lucene benchmark code:
http://lucene.apache.org/core/4_5_1/benchmark/org/apache/lucene/benchmark/quality/package-summary.html
Otherwise, all I can think of is writing an app layer that keeps track of
the id, sends the query to Solr, parses the search results and spits out
results in the trec format. I'd love to find some open-source code that
does what you ask.
I did a quick and dirty version of something like that for the INEX book
track. I'll see if I can find the code and if it is in any shape to share.
Tom
Tom Burton-West
Information Retrieval Programmer
Digital Library Production Sevice
University of Michigan Library
tburtonw@umich.edu
http://www.hathitrust.org/blogs/large-scale-search
On Wed, Oct 30, 2013 at 10:52 AM, Michael Preminger <
Michael.Preminger@hioa.no> wrote:
> Hello!
>
> Is there a simple way to evaluate a SOLR index with TREC_EVAL?
> I mean:
> * preparing a query file in some format Solr will understand, but where
> each query has an ID
> * getting results out in trec format, with these query IDs attached
>
> Thanks
>
> Michael
>