You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Paul Friedman <pa...@yahoo.com> on 2002/02/28 21:23:58 UTC
optimizing index - too many open files
Hello all,
I am running into an error:
java.io.FileNotFoundException: /lucene/index/_2vx.tii ( too many open
files )
after my class calls IndexWriter.optimize().
Does anybody know what causes this error?
Any help is appreciated.
( By the way, the site that I am indexing is huge.
I have a crawler run through the site calling many .jsps, .pdfs, and .html
docs.
It ran fine two days ago after indexing 3700+ pages. )
Could the index be too large for Lucene to handle?
The error:
java.io.FileNotFoundException: /lucene/index/-2vx.tii ( too many open
files )
at java.io.RandomAccessFile.open( Native Method )
at java.io.RandomAccessFile.<init>
at java.io.RandomAccessFile.<init>
at org.apache.lucene.store.FSInputStream$Descriptor.<init>
at org.apache.lucene.store.FSInputStream.<init>
at org.apache.lucene.store.FSDirectory.openFile
at org.apache.lucene.index.TermInfosReader.readIndex
at org.apache.lucene.index.TermInfosReader.<init>
at org.apache.lucene.index.SegmentReader.<init>
at org.apache.lucene.index.IndexWriter.mergeSegments
at org.apache.lucene.index.IndexWriter.optimize
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: optimizing index - too many open files
Posted by Karl Øie <ka...@gan.no>.
I have to index 1650mb of documents, and eventually i will get "out of
memory" with a RAMDir and get "too many open files" with a FSdir, so to get
around this i am indexing 100 documents at a time in a RAMDir, then merge
this RAMDir into a FSDir before i index the next set of 100 files. This made
me work around both "out of memory" and "too many files" exceptions...
mvh karl øie
-----Original Message-----
From: Paul Friedman [mailto:paulsanfordfriedman@yahoo.com]
Sent: 28. februar 2002 21:38
To: Lucene Users List
Subject: Re: optimizing index - too many open files
Sorry to bother y'all again.
Found an answer in the archives under the Thread "Indexing problem".
About to try using RAMDirectory first.
pax et bonum. p.
----- Original Message -----
From: "Paul Friedman" <pa...@yahoo.com>
To: "Lucene Users List" <lu...@jakarta.apache.org>
Sent: Thursday, February 28, 2002 1:23 PM
Subject: optimizing index - too many open files
> Hello all,
>
> I am running into an error:
> java.io.FileNotFoundException: /lucene/index/_2vx.tii ( too many open
> files )
> after my class calls IndexWriter.optimize().
>
> Does anybody know what causes this error?
> Any help is appreciated.
>
> ( By the way, the site that I am indexing is huge.
> I have a crawler run through the site calling many .jsps, .pdfs, and
.html
> docs.
> It ran fine two days ago after indexing 3700+ pages. )
>
> Could the index be too large for Lucene to handle?
>
> The error:
> java.io.FileNotFoundException: /lucene/index/-2vx.tii ( too many open
> files )
> at java.io.RandomAccessFile.open( Native Method )
> at java.io.RandomAccessFile.<init>
> at java.io.RandomAccessFile.<init>
> at org.apache.lucene.store.FSInputStream$Descriptor.<init>
> at org.apache.lucene.store.FSInputStream.<init>
> at org.apache.lucene.store.FSDirectory.openFile
> at org.apache.lucene.index.TermInfosReader.readIndex
> at org.apache.lucene.index.TermInfosReader.<init>
> at org.apache.lucene.index.SegmentReader.<init>
> at org.apache.lucene.index.IndexWriter.mergeSegments
> at org.apache.lucene.index.IndexWriter.optimize
>
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
--
To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail:
<ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: optimizing index - too many open files
Posted by Paul Friedman <pa...@yahoo.com>.
Sorry to bother y'all again.
Found an answer in the archives under the Thread "Indexing problem".
About to try using RAMDirectory first.
pax et bonum. p.
----- Original Message -----
From: "Paul Friedman" <pa...@yahoo.com>
To: "Lucene Users List" <lu...@jakarta.apache.org>
Sent: Thursday, February 28, 2002 1:23 PM
Subject: optimizing index - too many open files
> Hello all,
>
> I am running into an error:
> java.io.FileNotFoundException: /lucene/index/_2vx.tii ( too many open
> files )
> after my class calls IndexWriter.optimize().
>
> Does anybody know what causes this error?
> Any help is appreciated.
>
> ( By the way, the site that I am indexing is huge.
> I have a crawler run through the site calling many .jsps, .pdfs, and
.html
> docs.
> It ran fine two days ago after indexing 3700+ pages. )
>
> Could the index be too large for Lucene to handle?
>
> The error:
> java.io.FileNotFoundException: /lucene/index/-2vx.tii ( too many open
> files )
> at java.io.RandomAccessFile.open( Native Method )
> at java.io.RandomAccessFile.<init>
> at java.io.RandomAccessFile.<init>
> at org.apache.lucene.store.FSInputStream$Descriptor.<init>
> at org.apache.lucene.store.FSInputStream.<init>
> at org.apache.lucene.store.FSDirectory.openFile
> at org.apache.lucene.index.TermInfosReader.readIndex
> at org.apache.lucene.index.TermInfosReader.<init>
> at org.apache.lucene.index.SegmentReader.<init>
> at org.apache.lucene.index.IndexWriter.mergeSegments
> at org.apache.lucene.index.IndexWriter.optimize
>
>
> _________________________________________________________
> Do You Yahoo!?
> Get your free @yahoo.com address at http://mail.yahoo.com
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>