You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Dragon Fly <dr...@hotmail.com> on 2008/10/30 20:24:50 UTC
Read all the data from an index
Hi,
I have an old index that was built a few months ago. The data that I used to build the index has been deleted from the database. I'd like to read all the data from the old index to build a new index. Which Lucene API calls should I use to read all the data from the old index? Thank you in advance for your help.
_________________________________________________________________
When your life is on the go—take your life with you.
http://clk.atdmt.com/MRT/go/115298558/direct/01/
RE: Read all the data from an index
Posted by Dragon Fly <dr...@hotmail.com>.
Thank you both for your help.
> Date: Fri, 31 Oct 2008 09:06:50 +0100
> From: ab@getopt.org
> To: java-user@lucene.apache.org
> Subject: Re: Read all the data from an index
>
> Erick Erickson wrote:
> > I'm not sure what *could* be easier than looping with IndexSearcher.doc(),
> > looping from 1 to maxDoc. Of course you'll have to pay some attention to
> > whether you get a document back or not, and I'm not quite sure whether you'd
> > have to worry about getting deleted documents. But I don't think either of
> > these
> > really count if the index was optimized
>
> Document numbers start at 0. You will never get a document marked
> "deleted" from either IndexReader or IndexSearcher.
>
> Why use IndexSearcher and not IndexReader?
>
> IndexReader reader = IndexReader.open(....);
>
> for (int i = 0; i < reader.maxDoc(); i++) {
> if (reader.isDeleted(i)) {
> continue;
> }
> Document doc = reader.document(i);
> ...
> }
>
> Hint: if you have an unoptimized index with deleted documents, and you
> want to retrieve also the content of these deleted documents, call first
> IndexReader.undeleteAll().
>
> --
> Best regards,
> Andrzej Bialecki <><
> ___. ___ ___ ___ _ _ __________________________________
> [__ || __|__/|__||\/| Information Retrieval, Semantic Web
> ___|||__|| \| || | Embedded Unix, System Integration
> http://www.sigram.com Contact: info at sigram dot com
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
_________________________________________________________________
Want to read Hotmail messages in Outlook? The Wordsmiths show you how.
http://windowslive.com/connect/post/wedowindowslive.spaces.live.com-Blog-cns!20EE04FBC541789!167.entry?ocid=TXT_TAGLM_WL_hotmail_092008
Re: Read all the data from an index
Posted by Andrzej Bialecki <ab...@getopt.org>.
Erick Erickson wrote:
> I'm not sure what *could* be easier than looping with IndexSearcher.doc(),
> looping from 1 to maxDoc. Of course you'll have to pay some attention to
> whether you get a document back or not, and I'm not quite sure whether you'd
> have to worry about getting deleted documents. But I don't think either of
> these
> really count if the index was optimized
Document numbers start at 0. You will never get a document marked
"deleted" from either IndexReader or IndexSearcher.
Why use IndexSearcher and not IndexReader?
IndexReader reader = IndexReader.open(....);
for (int i = 0; i < reader.maxDoc(); i++) {
if (reader.isDeleted(i)) {
continue;
}
Document doc = reader.document(i);
...
}
Hint: if you have an unoptimized index with deleted documents, and you
want to retrieve also the content of these deleted documents, call first
IndexReader.undeleteAll().
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Read all the data from an index
Posted by Erick Erickson <er...@gmail.com>.
I'm not sure what *could* be easier than looping with IndexSearcher.doc(),
looping from 1 to maxDoc. Of course you'll have to pay some attention to
whether you get a document back or not, and I'm not quite sure whether you'd
have to worry about getting deleted documents. But I don't think either of
these
really count if the index was optimized
Best
Erick
On Thu, Oct 30, 2008 at 4:25 PM, Dragon Fly <dr...@hotmail.com>wrote:
> I'll double check but I believe all the fields in my index are stored.
> Should I just loop using indexSearcher.doc() or is there a faster way?
> Thanks.
>
> > Date: Thu, 30 Oct 2008 16:09:47 -0400
> > From: erickerickson@gmail.com
> > To: java-user@lucene.apache.org
> > Subject: Re: Read all the data from an index
> >
> > Well, that's trickier than you might think. You can easily get
> > all the STORED data just by getting doc IDs 1-MaxDoc(). But
> > reconstructing the data from data that is NOT stored is more
> > difficult. Luke tries, but it may be a lossy process.
> >
> > Best
> > Erick
> >
> > On Thu, Oct 30, 2008 at 3:24 PM, Dragon Fly <dragon-fly999@hotmail.com
> >wrote:
> >
> > > Hi,
> > >
> > > I have an old index that was built a few months ago. The data that I
> used
> > > to build the index has been deleted from the database. I'd like to
> read all
> > > the data from the old index to build a new index. Which Lucene API
> calls
> > > should I use to read all the data from the old index? Thank you in
> advance
> > > for your help.
> > >
> > > _________________________________________________________________
> > > When your life is on the go—take your life with you.
> > > http://clk.atdmt.com/MRT/go/115298558/direct/01/
>
> _________________________________________________________________
> Stay organized with simple drag and drop from Windows Live Hotmail.
> http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_102008
>
RE: Read all the data from an index
Posted by Dragon Fly <dr...@hotmail.com>.
I'll double check but I believe all the fields in my index are stored. Should I just loop using indexSearcher.doc() or is there a faster way? Thanks.
> Date: Thu, 30 Oct 2008 16:09:47 -0400
> From: erickerickson@gmail.com
> To: java-user@lucene.apache.org
> Subject: Re: Read all the data from an index
>
> Well, that's trickier than you might think. You can easily get
> all the STORED data just by getting doc IDs 1-MaxDoc(). But
> reconstructing the data from data that is NOT stored is more
> difficult. Luke tries, but it may be a lossy process.
>
> Best
> Erick
>
> On Thu, Oct 30, 2008 at 3:24 PM, Dragon Fly <dr...@hotmail.com>wrote:
>
> > Hi,
> >
> > I have an old index that was built a few months ago. The data that I used
> > to build the index has been deleted from the database. I'd like to read all
> > the data from the old index to build a new index. Which Lucene API calls
> > should I use to read all the data from the old index? Thank you in advance
> > for your help.
> >
> > _________________________________________________________________
> > When your life is on the go—take your life with you.
> > http://clk.atdmt.com/MRT/go/115298558/direct/01/
_________________________________________________________________
Stay organized with simple drag and drop from Windows Live Hotmail.
http://windowslive.com/Explore/hotmail?ocid=TXT_TAGLM_WL_hotmail_102008
Re: Read all the data from an index
Posted by Erick Erickson <er...@gmail.com>.
Well, that's trickier than you might think. You can easily get
all the STORED data just by getting doc IDs 1-MaxDoc(). But
reconstructing the data from data that is NOT stored is more
difficult. Luke tries, but it may be a lossy process.
Best
Erick
On Thu, Oct 30, 2008 at 3:24 PM, Dragon Fly <dr...@hotmail.com>wrote:
> Hi,
>
> I have an old index that was built a few months ago. The data that I used
> to build the index has been deleted from the database. I'd like to read all
> the data from the old index to build a new index. Which Lucene API calls
> should I use to read all the data from the old index? Thank you in advance
> for your help.
>
> _________________________________________________________________
> When your life is on the go—take your life with you.
> http://clk.atdmt.com/MRT/go/115298558/direct/01/