You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Parminder Singh <ps...@mahindrabt.com> on 2004/02/26 05:55:56 UTC
Database
I've a CMS application that deploys metadata to a database. Is it possible to use lucene to search this database instead of it's (lucene's) index. If you could tell me the steps that would be involved in doing this, it'd be great help. I'm new to Lucene.
Thank You.
Parminder Singh
--------------------------------------------------------------------------------
In war: resolution. In defeat: defiance. In victory: magnanimity. In peace: goodwill. - Sir Winston Leonard Spencer Churchill
*********************************************************
Disclaimer
This message (including any attachments) contains
confidential information intended for a specific
individual and purpose, and is protected by law.
If you are not the intended recipient, you should
delete this message and are hereby notified that
any disclosure, copying, or distribution of this
message, or the taking of any action based on it,
is strictly prohibited.
*********************************************************
Visit us at http://www.mahindrabt.com
Re: Database
Posted by David Spencer <da...@tropo.com>.
Parminder Singh wrote:
>I've a CMS application that deploys metadata to a database. Is it possible to use lucene to search this database instead of it's (lucene's) index. If you could tell me the steps that would be involved in doing this, it'd be great help. I'm new to Lucene.
>
>
I've done this extensively. Basically you create documents out of the
database and in my case I generated a URL for each doc which would be
fed to users after a query. This is one of those things where Lucene
stands apart from what seemed to be the alternatives a few years ago
(htdig was one thing I used) -- it doesn't have to spider a web site,
and if you have a dynamic web site (pages generated from db queries)
then the indexing is in a way more efficient as you don't have to parse
html to extract what may or may not be the actual text - the db will
have your exact content so you index off the db, not from web pages..
>Thank You.
>
>Parminder Singh
>
>--------------------------------------------------------------------------------
>
>In war: resolution. In defeat: defiance. In victory: magnanimity. In peace: goodwill. - Sir Winston Leonard Spencer Churchill
>
>*********************************************************
>Disclaimer
>
>This message (including any attachments) contains
>confidential information intended for a specific
>individual and purpose, and is protected by law.
>If you are not the intended recipient, you should
>delete this message and are hereby notified that
>any disclosure, copying, or distribution of this
>message, or the taking of any action based on it,
>is strictly prohibited.
>
>*********************************************************
>Visit us at http://www.mahindrabt.com
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Iterating TermEnum backwards
Posted by David Spencer <da...@tropo.com>.
Matt Quail wrote:
> Hi all,
>
> Is there any way to iterate through a TermEnum backwards? Okay, I know
> that there isn't a way to do this via the TermEnum class, but is it
> "implementable" on top of the underlying Lucene datastore?
>
> My particular problem is this:
>
> I have an index of documents, each document has a "date" field (I'm
> using DateField). Most documents have a different date, so the number of
> unique dates is close to the number of documents.
>
> I want to find the top N most recent dates, but I don't want to have to
> iterate through ALL of them first. NB: With DateField, the earlier dates
> are lexocographically smaller. (I also want to find the most recent N
> less than some date D).
>
> I know I could "invert" my dates (something like MAX_LONG - date) to get
> the REVERSE order, but I want to be able to do "least recent" and "most
> recent".
Maybe add another field, a "backwards" date, such that existing api will
return more recent ones first?!
>
> Is iterating over a TermEnum "impossible", or should I dig further, and
> possibly submit a patch?
>
>
> =Matt
>
> PS: my current solution is to do a binary search between MIN and MAX,
> halving my search space until I find close to N matching documents.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Iterating TermEnum backwards
Posted by Matt Quail <ma...@ctx.com.au>.
>> I know I could "invert" my dates (something like MAX_LONG - date) to get
>> the REVERSE order, but I want to be able to do "least recent" and "most
>> recent".
>
>
> Why not have two date fields, one inverted and one not?
>
>> PS: my current solution is to do a binary search between MIN and MAX,
>> halving my search space until I find close to N matching documents.
>
>
> That doesn't sound like a bad solution.
Cool, thanks for all your suggestions. I'm getting adequate performance
from my binary search now, and if it really becomes a performance
problem, I'll just index an inverted version of the date (we all have
diskspace to spare!).
=Matt
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Iterating TermEnum backwards
Posted by Doug Cutting <cu...@apache.org>.
Matt Quail wrote:
> Is there any way to iterate through a TermEnum backwards? Okay, I know
> that there isn't a way to do this via the TermEnum class, but is it
> "implementable" on top of the underlying Lucene datastore?
Not really. The best you can do is skip back to the previous "indexed"
term in TermInfosReader.indexTerms, and scan forward from there. You
could try adding a method to that class like:
final synchronized void seekBefore(Term term) throws IOException {
int offset = getIndexOffset(term);
seekEnum(offset > 0 ? offset - 1 : offset);
}
Then you'd need to add stuff to MultiReader, SegmentReader and
IndexReader, to take advantage of this. It could get a little tricky,
but it is possible. I'm not convinced this is your best route.
> My particular problem is this:
>
> I have an index of documents, each document has a "date" field (I'm
> using DateField). Most documents have a different date, so the number of
> unique dates is close to the number of documents.
Are you adding documents in date order? If so, then you could look at
the date of the document numbered maxDoc() - N and scan forward from
there. To be safe, you could start at maxDoc() - N*2 or something.
> I want to find the top N most recent dates, but I don't want to have to
> iterate through ALL of them first. NB: With DateField, the earlier dates
> are lexocographically smaller. (I also want to find the most recent N
> less than some date D).
>
> I know I could "invert" my dates (something like MAX_LONG - date) to get
> the REVERSE order, but I want to be able to do "least recent" and "most
> recent".
Why not have two date fields, one inverted and one not?
> PS: my current solution is to do a binary search between MIN and MAX,
> halving my search space until I find close to N matching documents.
That doesn't sound like a bad solution.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Iterating TermEnum backwards
Posted by Matt Quail <ma...@ctx.com.au>.
Hi all,
Is there any way to iterate through a TermEnum backwards? Okay, I know
that there isn't a way to do this via the TermEnum class, but is it
"implementable" on top of the underlying Lucene datastore?
My particular problem is this:
I have an index of documents, each document has a "date" field (I'm
using DateField). Most documents have a different date, so the number of
unique dates is close to the number of documents.
I want to find the top N most recent dates, but I don't want to have to
iterate through ALL of them first. NB: With DateField, the earlier dates
are lexocographically smaller. (I also want to find the most recent N
less than some date D).
I know I could "invert" my dates (something like MAX_LONG - date) to get
the REVERSE order, but I want to be able to do "least recent" and "most
recent".
Is iterating over a TermEnum "impossible", or should I dig further, and
possibly submit a patch?
=Matt
PS: my current solution is to do a binary search between MIN and MAX,
halving my search space until I find close to N matching documents.
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org