IndexWriter stopped before commit

[liat oren <or...@gmail.com>]

Hi all,

I ran a code that joins a list of indexes to one index.
The commit and close of the writer is done when we finish looping on the
list of the original paths.
An error occured before it arrived to the commit part.
The index size is big, so it looks like the data is there, but when I check
the number of documents, it shows 0.
I unlocked the index, and I also tried to commit it, but it still shows 0.
Is there a way to "save" the data instaed of re-running the join code?

Thanks,
Liat

Re: IndexWriter stopped before commit

[liat oren <or...@gmail.com>]

Thanks Mike, I will do that

2009/5/13 Michael McCandless <lu...@mikemccandless.com>

> Unfortunately, no.
>
> If the JRE crashes/exits without IndexWriter.commit (or close) being
> called, then the index will reflect none of the changes during that
> session.
>
> There will be partial files in there (that's why you see so much disk
> space being used) but there's no segments file describing what these
> files are; when you next open a writer on that indexed, these
> unreferenced files will be cleaned up.
>
> Maybe you could add the indexes one by one, calling commit() after
> each addition?  This way if a crash happens, the index will just
> fallback to the last commit.
>
> Mike
>
> On Wed, May 13, 2009 at 3:08 AM, liat oren <or...@gmail.com> wrote:
> > Hi all,
> >
> > I ran a code that joins a list of indexes to one index.
> > The commit and close of the writer is done when we finish looping on the
> > list of the original paths.
> > An error occured before it arrived to the commit part.
> > The index size is big, so it looks like the data is there, but when I
> check
> > the number of documents, it shows 0.
> > I unlocked the index, and I also tried to commit it, but it still shows
> 0.
> > Is there a way to "save" the data instaed of re-running the join code?
> >
> > Thanks,
> > Liat
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Re: IndexWriter stopped before commit

[Michael McCandless <lu...@mikemccandless.com>]

Unfortunately, no.

If the JRE crashes/exits without IndexWriter.commit (or close) being
called, then the index will reflect none of the changes during that
session.

There will be partial files in there (that's why you see so much disk
space being used) but there's no segments file describing what these
files are; when you next open a writer on that indexed, these
unreferenced files will be cleaned up.

Maybe you could add the indexes one by one, calling commit() after
each addition?  This way if a crash happens, the index will just
fallback to the last commit.

Mike

On Wed, May 13, 2009 at 3:08 AM, liat oren <or...@gmail.com> wrote:
> Hi all,
>
> I ran a code that joins a list of indexes to one index.
> The commit and close of the writer is done when we finish looping on the
> list of the original paths.
> An error occured before it arrived to the commit part.
> The index size is big, so it looks like the data is there, but when I check
> the number of documents, it shows 0.
> I unlocked the index, and I also tried to commit it, but it still shows 0.
> Is there a way to "save" the data instaed of re-running the join code?
>
> Thanks,
> Liat
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org