You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by Trond Aksel Myklebust <Tr...@idi.ntnu.no> on 2006/04/20 19:00:09 UTC
Missing records from searchresult
Hello,
Been using PyLucene, but for a small project I want to use dotLucene, though
having some difficulties.
I have indexed 732 files and want to do a "text:a*" search against these
indexes files, but dotLucene only returns 530 of them. If I do a "text:s*"
search 732 files are returned as expected. Running the "text:a*" query using
Luke returns 731 matches, running the query using PyLucene returns the same.
Why am I missing 201 matches when using dotLucene, any issues I am not aware
of? If I do a search "text:ar*" files that was missing from the first search
shows up.
dotLucene code that doesnt work:
IndexSearcher searcher = new IndexSearcher(@"C:\Lucene");
Query query = QueryParser.Parse("a*", "text", new WhitespaceAnalyzer());
Hits hits = searcher.Search(query);
searcher.Close()
PyLucene code that works:
import sys
from PyLucene import *
searcher = IndexSearcher("C:\\Lucene")
query = QueryParser.parse("a*", "text", WhitespaceAnalyzer())
hits = searcher.search(query)
searcher.close()
Hope someone got an answer/fix.
Regards
Trond A Myklebust
SV: Missing records from searchresult
Posted by Trond Aksel Myklebust <tr...@idi.ntnu.no>.
Oh, interesting, because I am getting 518 with the searcher.exe app on the
two PC's I've tried it on (both got .NET 2.0), but 715 with Luke. Guess I
should test on a few more systems.
Thanks for testing.
Trond
-----Opprinnelig melding-----
Fra: Eyal Post [mailto:eyalpost@epocalipse.com]
Sendt: 20. april 2006 22:30
Til: lucene-net-user@incubator.apache.org
Emne: RE: Missing records from searchresult
I ran your searcher and got 715 results, I then ran the same query with Luke
and got 715 results too. Seems fine to me..
Regards,
Eyal Post
> -----Original Message-----
> From: Trond Aksel Myklebust
> [mailto:Trond.Aksel.Myklebust@idi.ntnu.no]
> Sent: Thursday, April 20, 2006 21:38 PM
> To: lucene-net-user@incubator.apache.org
> Subject: SV: Missing records from searchresult
>
> I've tried:
> DotLucene 1.4.3 build 004
> DotLucene 1.9 RC1 build 002 alpha
>
> Since Luke and PyLucene returns the correct answers using the index
> dotLucene creates I assume that the problem lies in searching using
> dotLucene.
> The best I can do atm regarding a testcase is a copy of my test Lucene
> index together with a small app that returns the wrong number of
> results compared to Luke and PyLucene.
> http://folk.ntnu.no/trondaks/uploads/LuceneTest.zip
> If I reduce the number of indexed documents the difference between the
> number of hits from dotLucene and Luke gets smaller until they are the
> same, almost like the index for "a" is overpopulated or there is a
> score threshold somewhere.
> According to Luke "a" is the among the most used terms, unsure if
> there is a connection between that and my "text:a*" search.
>
> Not exactly what you ask for but hope it can help someway.
> The index data is extracted from a database:
> IndexWriter writer = new IndexWriter(@"LuceneIndex", new
> WhitespaceAnalyzer(), true); //Get data from database Document doc =
> new Document(); doc.Add(Field.Text("text", document.toLower()));
> writer.AddDocument(doc);
> writer.Close()
>
>
> Trond
>
>
> -----Opprinnelig melding-----
> Fra: Jeff Rodenburg [mailto:jeff.rodenburg@gmail.com]
> Sendt: 20. april 2006 19:25
> Til: lucene-net-user@incubator.apache.org
> Emne: Re: Missing records from searchresult
>
> Trond -
>
> We have several assumptions that have to be made about how your index
> was created, i.e. analyzer in use, contents of fields, etc. Normally,
> if you provide a unit test that mimics the behavior you're seeing,
> it's easier for all of us to provide feedback.
>
> Can you provide some information about the version of dotlucene you're
> using?
>
> -- j
>
> On 4/20/06, Trond Aksel Myklebust
> <Tr...@idi.ntnu.no> wrote:
> >
> > Hello,
> > Been using PyLucene, but for a small project I want to use
> dotLucene,
> > though having some difficulties.
> >
> > I have indexed 732 files and want to do a "text:a*" search against
> > these indexes files, but dotLucene only returns 530 of
> them. If I do a
> "text:s*"
> > search 732 files are returned as expected. Running the
> "text:a*" query
> > using Luke returns 731 matches, running the query using PyLucene
> > returns the same.
> > Why am I missing 201 matches when using dotLucene, any
> issues I am not
> > aware of? If I do a search "text:ar*" files that was
> missing from the
> > first search shows up.
> >
> > dotLucene code that doesnt work:
> > IndexSearcher searcher = new IndexSearcher(@"C:\Lucene");
> Query query
> > = QueryParser.Parse("a*", "text", new
> WhitespaceAnalyzer()); Hits hits
> > = searcher.Search(query);
> > searcher.Close()
> >
> >
> > PyLucene code that works:
> > import sys
> > from PyLucene import *
> > searcher = IndexSearcher("C:\\Lucene") query =
> QueryParser.parse("a*",
> > "text", WhitespaceAnalyzer()) hits = searcher.search(query)
> > searcher.close()
> >
> > Hope someone got an answer/fix.
> >
> > Regards
> > Trond A Myklebust
> >
> >
> >
>
>
>
>
RE: Missing records from searchresult
Posted by Eyal Post <ey...@epocalipse.com>.
I ran your searcher and got 715 results, I then ran the same query with Luke
and got 715 results too. Seems fine to me..
Regards,
Eyal Post
> -----Original Message-----
> From: Trond Aksel Myklebust
> [mailto:Trond.Aksel.Myklebust@idi.ntnu.no]
> Sent: Thursday, April 20, 2006 21:38 PM
> To: lucene-net-user@incubator.apache.org
> Subject: SV: Missing records from searchresult
>
> I've tried:
> DotLucene 1.4.3 build 004
> DotLucene 1.9 RC1 build 002 alpha
>
> Since Luke and PyLucene returns the correct answers using the
> index dotLucene creates I assume that the problem lies in
> searching using dotLucene.
> The best I can do atm regarding a testcase is a copy of my
> test Lucene index together with a small app that returns the
> wrong number of results compared to Luke and PyLucene.
> http://folk.ntnu.no/trondaks/uploads/LuceneTest.zip
> If I reduce the number of indexed documents the difference
> between the number of hits from dotLucene and Luke gets
> smaller until they are the same, almost like the index for
> "a" is overpopulated or there is a score threshold somewhere.
> According to Luke "a" is the among the most used terms,
> unsure if there is a connection between that and my "text:a*" search.
>
> Not exactly what you ask for but hope it can help someway.
> The index data is extracted from a database:
> IndexWriter writer = new IndexWriter(@"LuceneIndex", new
> WhitespaceAnalyzer(), true); //Get data from database
> Document doc = new Document(); doc.Add(Field.Text("text",
> document.toLower())); writer.AddDocument(doc);
> writer.Close()
>
>
> Trond
>
>
> -----Opprinnelig melding-----
> Fra: Jeff Rodenburg [mailto:jeff.rodenburg@gmail.com]
> Sendt: 20. april 2006 19:25
> Til: lucene-net-user@incubator.apache.org
> Emne: Re: Missing records from searchresult
>
> Trond -
>
> We have several assumptions that have to be made about how
> your index was created, i.e. analyzer in use, contents of
> fields, etc. Normally, if you provide a unit test that
> mimics the behavior you're seeing, it's easier for all of us
> to provide feedback.
>
> Can you provide some information about the version of
> dotlucene you're using?
>
> -- j
>
> On 4/20/06, Trond Aksel Myklebust
> <Tr...@idi.ntnu.no> wrote:
> >
> > Hello,
> > Been using PyLucene, but for a small project I want to use
> dotLucene,
> > though having some difficulties.
> >
> > I have indexed 732 files and want to do a "text:a*" search against
> > these indexes files, but dotLucene only returns 530 of
> them. If I do a
> "text:s*"
> > search 732 files are returned as expected. Running the
> "text:a*" query
> > using Luke returns 731 matches, running the query using PyLucene
> > returns the same.
> > Why am I missing 201 matches when using dotLucene, any
> issues I am not
> > aware of? If I do a search "text:ar*" files that was
> missing from the
> > first search shows up.
> >
> > dotLucene code that doesnt work:
> > IndexSearcher searcher = new IndexSearcher(@"C:\Lucene");
> Query query
> > = QueryParser.Parse("a*", "text", new
> WhitespaceAnalyzer()); Hits hits
> > = searcher.Search(query);
> > searcher.Close()
> >
> >
> > PyLucene code that works:
> > import sys
> > from PyLucene import *
> > searcher = IndexSearcher("C:\\Lucene") query =
> QueryParser.parse("a*",
> > "text", WhitespaceAnalyzer()) hits = searcher.search(query)
> > searcher.close()
> >
> > Hope someone got an answer/fix.
> >
> > Regards
> > Trond A Myklebust
> >
> >
> >
>
>
>
>
SV: Missing records from searchresult
Posted by Trond Aksel Myklebust <Tr...@idi.ntnu.no>.
I've tried:
DotLucene 1.4.3 build 004
DotLucene 1.9 RC1 build 002 alpha
Since Luke and PyLucene returns the correct answers using the index
dotLucene creates I assume that the problem lies in searching using
dotLucene.
The best I can do atm regarding a testcase is a copy of my test Lucene index
together with a small app that returns the wrong number of results compared
to Luke and PyLucene.
http://folk.ntnu.no/trondaks/uploads/LuceneTest.zip
If I reduce the number of indexed documents the difference between the
number of hits from dotLucene and Luke gets smaller until they are the same,
almost like the index for "a" is overpopulated or there is a score threshold
somewhere. According to Luke "a" is the among the most used terms, unsure if
there is a connection between that and my "text:a*" search.
Not exactly what you ask for but hope it can help someway. The index data is
extracted from a database:
IndexWriter writer = new IndexWriter(@"LuceneIndex", new
WhitespaceAnalyzer(), true);
//Get data from database
Document doc = new Document();
doc.Add(Field.Text("text", document.toLower()));
writer.AddDocument(doc);
writer.Close()
Trond
-----Opprinnelig melding-----
Fra: Jeff Rodenburg [mailto:jeff.rodenburg@gmail.com]
Sendt: 20. april 2006 19:25
Til: lucene-net-user@incubator.apache.org
Emne: Re: Missing records from searchresult
Trond -
We have several assumptions that have to be made about how your index was
created, i.e. analyzer in use, contents of fields, etc. Normally, if you
provide a unit test that mimics the behavior you're seeing, it's easier for
all of us to provide feedback.
Can you provide some information about the version of dotlucene you're
using?
-- j
On 4/20/06, Trond Aksel Myklebust <Tr...@idi.ntnu.no> wrote:
>
> Hello,
> Been using PyLucene, but for a small project I want to use dotLucene,
> though having some difficulties.
>
> I have indexed 732 files and want to do a "text:a*" search against
> these indexes files, but dotLucene only returns 530 of them. If I do a
"text:s*"
> search 732 files are returned as expected. Running the "text:a*" query
> using Luke returns 731 matches, running the query using PyLucene
> returns the same.
> Why am I missing 201 matches when using dotLucene, any issues I am not
> aware of? If I do a search "text:ar*" files that was missing from the
> first search shows up.
>
> dotLucene code that doesnt work:
> IndexSearcher searcher = new IndexSearcher(@"C:\Lucene"); Query query
> = QueryParser.Parse("a*", "text", new WhitespaceAnalyzer()); Hits hits
> = searcher.Search(query);
> searcher.Close()
>
>
> PyLucene code that works:
> import sys
> from PyLucene import *
> searcher = IndexSearcher("C:\\Lucene") query = QueryParser.parse("a*",
> "text", WhitespaceAnalyzer()) hits = searcher.search(query)
> searcher.close()
>
> Hope someone got an answer/fix.
>
> Regards
> Trond A Myklebust
>
>
>
Re: Missing records from searchresult
Posted by Jeff Rodenburg <je...@gmail.com>.
Trond -
We have several assumptions that have to be made about how your index was
created, i.e. analyzer in use, contents of fields, etc. Normally, if you
provide a unit test that mimics the behavior you're seeing, it's easier for
all of us to provide feedback.
Can you provide some information about the version of dotlucene you're
using?
-- j
On 4/20/06, Trond Aksel Myklebust <Tr...@idi.ntnu.no> wrote:
>
> Hello,
> Been using PyLucene, but for a small project I want to use dotLucene,
> though
> having some difficulties.
>
> I have indexed 732 files and want to do a "text:a*" search against these
> indexes files, but dotLucene only returns 530 of them. If I do a "text:s*"
> search 732 files are returned as expected. Running the "text:a*" query
> using
> Luke returns 731 matches, running the query using PyLucene returns the
> same.
> Why am I missing 201 matches when using dotLucene, any issues I am not
> aware
> of? If I do a search "text:ar*" files that was missing from the first
> search
> shows up.
>
> dotLucene code that doesnt work:
> IndexSearcher searcher = new IndexSearcher(@"C:\Lucene");
> Query query = QueryParser.Parse("a*", "text", new WhitespaceAnalyzer());
> Hits hits = searcher.Search(query);
> searcher.Close()
>
>
> PyLucene code that works:
> import sys
> from PyLucene import *
> searcher = IndexSearcher("C:\\Lucene")
> query = QueryParser.parse("a*", "text", WhitespaceAnalyzer())
> hits = searcher.search(query)
> searcher.close()
>
> Hope someone got an answer/fix.
>
> Regards
> Trond A Myklebust
>
>
>