duplicate results MultiFieldQueryParser

[varun sood <vs...@gmail.com>]

Hi,
  I am searching two fields at a time and using MultiFieldQueryParser to
parse my query (method described in In Action book). I noticed that I am
getting duplicate results. The document exists only once in the index
structure but in results its showing the same doucment multiple times with
same same score and also different score.

Basic way I am doing is

String fields[] = null;

fields[0] = new String("field1");
fields[1] = new String("field2");

Query query = MultiFieldQueryParser.parse(Query, fields , new
StandardAnalyzer());

Hits hits = is.search(query);

using the above code.. I m getting few documents multiple times. Is there
any obvious reason for this?

Thanks for ur replies

Re: duplicate results MultiFieldQueryParser

[varun sood <vs...@gmail.com>]

yeah you are right. I was talking about going through the large index and
discovering the problem as to where else it have occured and how?
But thanks for your tips. I use Luke before and certianly it helped me this
time aswell.  I found the problem. Its not great Lucene. Its me.. error in
understaing nature of data.

Thanks
Varun

On 6/5/06, Erick Erickson <er...@gmail.com> wrote:
>
> Even if the index is huge, you should be able to open it in Luke and
> submit
> a search just like the one that's giving you trouble, and then examine the
> documents. Yes, it'll take a while to open the index, but not an
> outrageous
> amount of time... unless you run out of memory or something.
>
> Best
> Erick
>
>

Re: duplicate results MultiFieldQueryParser

[Erick Erickson <er...@gmail.com>]

Even if the index is huge, you should be able to open it in Luke and submit
a search just like the one that's giving you trouble, and then examine the
documents. Yes, it'll take a while to open the index, but not an outrageous
amount of time... unless you run out of memory or something.

Best
Erick

Re: duplicate results MultiFieldQueryParser

[varun sood <vs...@gmail.com>]

On 6/5/06, Erick Erickson <er...@gmail.com> wrote:
>
> A few thoughts...
>
> 1> are you sure you only indexed the document once? If you indexed the
> same
> data multiple times, you'll have duplicate documents, each of which will
> have a different Lucene ID (i.e. doc()).


Yes.. but I will make sure again.


2> have you examined your index with, say, Luke? I've found that a wonderful
> tool for seeing if the data I *thought* was in my index was actually
> there.



Database is too huge.. I will need some time to go through it to look if I
did any mistake while creating indexes..

3> when you say "the same document", how do you know that? The internal
> Lucene ID or some field you've put in the index? This really as another
> form
> of "are you sure you indexed the data once?" because the internal Lucene
> id
> is what you get back from hits.doc(). If you're getting multiple entries
> like that, then I'm lost.



Same document means.. same path of the document say .. same URL miltiple
times.. well its a good point.. I will check if they all have same docIDs..

thanks for your suggestion.

Re: duplicate results MultiFieldQueryParser

[Erick Erickson <er...@gmail.com>]

A few thoughts...

1> are you sure you only indexed the document once? If you indexed the same
data multiple times, you'll have duplicate documents, each of which will
have a different Lucene ID (i.e. doc()).

2> have you examined your index with, say, Luke? I've found that a wonderful
tool for seeing if the data I *thought* was in my index was actually there.

3> when you say "the same document", how do you know that? The internal
Lucene ID or some field you've put in the index? This really as another form
of "are you sure you indexed the data once?" because the internal Lucene id
is what you get back from hits.doc(). If you're getting multiple entries
like that, then I'm lost.

Best
Erick