You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Michael Dodson <mg...@mac.com> on 2006/03/16 21:40:36 UTC
re: restart interrupted index
I'm relatively new to Lucene and I've been trying to index a large
number of html files. If my operation is interrupted the index
appears to be corrupted. I can no longer open it for searching with
IndexSearcher (and no amount of toying with Luke's options seems to
help if I try to browse the index by that route). I would like to be
able to restart an interrupted process. Should I be able to do this?
Cheers,
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: restart interrupted index
Posted by Rob Young <bu...@gmail.com>.
Paulo Silveira wrote:
>Chris,
>
>I really would like only this extra files, but I have the same problem here.
>
>If I interrupt my IndexWriter with a kill signal, must of the time I
>will be left with a lock file AND corrupted index files (the searcher
>will throw some IllegalStateExceptions after the lock file is
>deleted).
>
>
I have found that if I send a HUP it shuts down cleanly and leaves no
lock file (assuming you're on a *nix machine)
kill -HUP [pid]
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: restart interrupted index
Posted by Chris Hostetter <ho...@fucit.org>.
: If I interrupt my IndexWriter with a kill signal, must of the time I
: will be left with a lock file AND corrupted index files (the searcher
: will throw some IllegalStateExceptions after the lock file is
: deleted).
if you are trying to deal with teh possibility that your indexing process
might crash hard, then there's not much you can do about the possibiity of
your index getting corrupted ... that's where backups are useful.
if you are trying to have a way to "cancel" your indexing process and then
start again later that's not really an issue with lucene -- you just need
a cleaner way to notify your application that you want it to stop, close
whatever IndexWriter/IndexReader you may be useing and shutdown cleanly.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: restart interrupted index
Posted by Paulo Silveira <pa...@caelum.com.br>.
Chris,
I really would like only this extra files, but I have the same problem here.
If I interrupt my IndexWriter with a kill signal, must of the time I
will be left with a lock file AND corrupted index files (the searcher
will throw some IllegalStateExceptions after the lock file is
deleted).
Paulo
On 3/16/06, Chris Hostetter <ho...@fucit.org> wrote:
>
> : I'm relatively new to Lucene and I've been trying to index a large
> : number of html files. If my operation is interrupted the index
> : appears to be corrupted. I can no longer open it for searching with
> : IndexSearcher (and no amount of toying with Luke's options seems to
> : help if I try to browse the index by that route). I would like to be
> : able to restart an interrupted process. Should I be able to do this?
> :
>
> what exactly isthe exception you are getting?
>
> more then likely, there is a lock file left over from your last process,
> if you manually remove it, new process can open your index, but there is
> no way to know exactly what docs might not have been successfully added
> ... there may also be unused files cluttering up the directory, but they
> won't hurt anything (they just take up space)
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Paulo E. A. Silveira
Caelum Ensino e Soluções em Java
http://www.caelum.com.br/
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: serializable RAMDirectory
Posted by Aditya Liviandi <ad...@i2r.a-star.edu.sg>.
Because I'm embedding the index inside another file...
So that file is self-contained, containing both the payload (which might
not be text) and the index...
But I figured out how to do it already... I just made RAMDirectory and
RAMFile Serializable and create my own build of lucene...
-----Original Message-----
From: Erik Hatcher [mailto:erik@ehatchersolutions.com]
Sent: Monday, March 20, 2006 7:21 PM
To: java-user@lucene.apache.org
Subject: Re: serializable RAMDirectory
On Mar 20, 2006, at 1:05 AM, Aditya Liviandi wrote:
> Is there any implementation of lucene that allows the index to be
> portable? It seems pointless that I have to do the indexing
> operation to
> a directory with FSDirectory, and then copy the directory over to the
> portable file, and unpack the file whenever I want to search the
> directory at another place...
Could you be more specific about what you want that Lucene does not
already provide?
FSDirectory is essentially a serialized RAMDirectory. What do you
mean by "unpack the file"? There is nothing special needed to move
an index from one machine to another, simply copy the entire
directory and use your searching code to refer to its location.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
--------------------------------------------------- I²R Disclaimer ------------------------------
This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately. Please do not copy or use it for any purpose, or disclose its contents to any other person. Thank you.
-------------------------------------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: serializable RAMDirectory
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On Mar 20, 2006, at 1:05 AM, Aditya Liviandi wrote:
> Is there any implementation of lucene that allows the index to be
> portable? It seems pointless that I have to do the indexing
> operation to
> a directory with FSDirectory, and then copy the directory over to the
> portable file, and unpack the file whenever I want to search the
> directory at another place...
Could you be more specific about what you want that Lucene does not
already provide?
FSDirectory is essentially a serialized RAMDirectory. What do you
mean by "unpack the file"? There is nothing special needed to move
an index from one machine to another, simply copy the entire
directory and use your searching code to refer to its location.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
serializable RAMDirectory
Posted by Aditya Liviandi <ad...@i2r.a-star.edu.sg>.
Is there any implementation of lucene that allows the index to be
portable? It seems pointless that I have to do the indexing operation to
a directory with FSDirectory, and then copy the directory over to the
portable file, and unpack the file whenever I want to search the
directory at another place...
Can anyone help me?
--------------------------------------------------- I²R Disclaimer ------------------------------
This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately. Please do not copy or use it for any purpose, or disclose its contents to any other person. Thank you.
-------------------------------------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
re: restart interrupted index
Posted by Chris Hostetter <ho...@fucit.org>.
: I'm relatively new to Lucene and I've been trying to index a large
: number of html files. If my operation is interrupted the index
: appears to be corrupted. I can no longer open it for searching with
: IndexSearcher (and no amount of toying with Luke's options seems to
: help if I try to browse the index by that route). I would like to be
: able to restart an interrupted process. Should I be able to do this?
:
what exactly isthe exception you are getting?
more then likely, there is a lock file left over from your last process,
if you manually remove it, new process can open your index, but there is
no way to know exactly what docs might not have been successfully added
... there may also be unused files cluttering up the directory, but they
won't hurt anything (they just take up space)
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org