You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Yakob <ja...@opensuse-id.org> on 2010/12/01 11:53:02 UTC
Re: precision and recall in lucene
On 11/30/10, Robert Muir <rc...@gmail.com> wrote:
> On Tue, Nov 30, 2010 at 10:46 AM, Yakob <ja...@opensuse-id.org> wrote:
>
>> can you tell me what went wrong? what is the difference between
>> topicsFile and qrelsFile anyway?
>>
>
> well its hard to tell what you are supplying as topics and qrels.
> have a look at /src/lia/benchmark in the LIA2 sample code: it has an
> example topic and 3 rows in qrels for it.
>
> basically, the topicsFile contains the "queries", and the qrelsFile
> contains judgements as to which documents are relevant.
>
> P.S.: once you have an index, a topics, and a qrels file, you can just
> use org.apache.lucene.benchmark.quality.trec.QueryDriver (it has a
> main method)
>
well yes your information is really helpful.I did find a topics and
qrels file that come in /src/lia/benchmark in the LIA2 sample code.
and the result did change slightly.but the precision and recall value
is still zero. I did also happen to use QueryDriver as you're
suggesting and the result is also the same as precision and recall is
still zero. I really need the value of precision and recall to be
anything but zero
I also put this question in stackoverflow so you can check this link
if you wanted to know more of my problem.
http://t.co/t0hat0T
so any further advice would be helpful though :-)
--
http://jacobian.web.id
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: precision and recall in lucene
Posted by Yakob <ja...@opensuse-id.org>.
On 12/1/10, Robert Muir <rc...@gmail.com> wrote:
>
> you fill the topics files with list of queries, like the lia2 example
> that has a single query for "apache source":
>
> <top>
> <num> Number: 0
> <title> apache source
> <desc> Description:
> <narr> Narrative:
> </top>
>
> then you populate the qrels file with the "answers" for your document
> collection:
>
> # qnum 0 doc-name is-relevant
>
> 0 0 apache1.0.txt 1
> 0 0 apache1.1.txt 1
> 0 0 apache2.0.txt 1
>
> this says that these 3 documents are relevant results for the query
> "apache source"
>
OMG, you are really helpful.I just did it.I really think we should be
friend on facebook though.hehe...
thank you. :-)
--
http://jacobian.web.id
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: precision and recall in lucene
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Dec 1, 2010 at 7:25 AM, Yakob <ja...@opensuse-id.org> wrote:
> can you give me an example of how to populate the topics file and
> qrels file other than those on the LIA2 sample code? I still don't
> understand of how these 2 files text work anyway. :-)
>
> let me get this straight. I need to fill topics file with any query
> that I want and qrels file with judgement. but what is the meaning of
> judgement in this case?
>
you fill the topics files with list of queries, like the lia2 example
that has a single query for "apache source":
<top>
<num> Number: 0
<title> apache source
<desc> Description:
<narr> Narrative:
</top>
then you populate the qrels file with the "answers" for your document
collection:
# qnum 0 doc-name is-relevant
0 0 apache1.0.txt 1
0 0 apache1.1.txt 1
0 0 apache2.0.txt 1
this says that these 3 documents are relevant results for the query
"apache source"
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: precision and recall in lucene
Posted by Yakob <ja...@opensuse-id.org>.
On 12/1/10, Robert Muir <rc...@gmail.com> wrote:
>
> well you can't use those files with your own document collection.
> you need to populate the topics file with queries that you care about
> measuring.
> then you need to populate the qrels file with judgements for each
> query, *for your collection*. you are saying this set of documents is
> relevant as a search result to this query.
>
> ---------------------------------------------------------------------
can you give me an example of how to populate the topics file and
qrels file other than those on the LIA2 sample code? I still don't
understand of how these 2 files text work anyway. :-)
let me get this straight. I need to fill topics file with any query
that I want and qrels file with judgement. but what is the meaning of
judgement in this case?
--
http://jacobian.web.id
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: precision and recall in lucene
Posted by Robert Muir <rc...@gmail.com>.
On Wed, Dec 1, 2010 at 5:53 AM, Yakob <ja...@opensuse-id.org> wrote:
>
> well yes your information is really helpful.I did find a topics and
> qrels file that come in /src/lia/benchmark in the LIA2 sample code.
> and the result did change slightly.but the precision and recall value
> is still zero. I did also happen to use QueryDriver as you're
> suggesting and the result is also the same as precision and recall is
> still zero. I really need the value of precision and recall to be
> anything but zero
>
well you can't use those files with your own document collection.
you need to populate the topics file with queries that you care about measuring.
then you need to populate the qrels file with judgements for each
query, *for your collection*. you are saying this set of documents is
relevant as a search result to this query.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org