You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Cheng <zh...@gmail.com> on 2013/01/22 14:20:25 UTC
IndexReader.open and CorruptIndexException
Hi,
I run a Lucene application on Tomcat. The app will try to open a Linux
directory, and sometime returns CorruptIndexException error.
Shortly after I restart Tomcat (nothing else changes), the app can be run
on the fly. I am using the following statements to open a directory:
try {
searcher = new IndexSearcher(IndexReader.open(new NIOFSDirectory(new
File("/home/user/"+ type))));
} catch (IOException e) {
throw new Exception("[" + type + "] Cannot open index folder...");
}
I would like to know how to tackle this problem.
Many thanks!
Re: IndexReader.open and CorruptIndexException
Posted by Ian Lea <ia...@gmail.com>.
Well, raising the limits is one option but there may be better ones.
There's an FAQ entry on this:
http://wiki.apache.org/lucene-java/LuceneFAQ#Why_am_I_getting_an_IOException_that_says_.22Too_many_open_files.22.3F
Take a look at org.apache.lucene.search.SearcherManager "Utility class
to safely share IndexSearcher instances across multiple threads".
--
Ian.
On Thu, Jan 24, 2013 at 9:18 AM, Rafał Kuć <r....@solr.pl> wrote:
> Hello!
>
> You need to allow the user that is running Lucene to open more files.
> There are plenty of tutorials available on the web. Modify your
> /etc/security/limits.conf and if for example your user is lucene add
> the following (or modify if those already exists):
>
> lucene soft nofile 64000
> lucene hard nofile 64000
>
> Relog and run:
>
> sudo -u lucene -s "ulimit -Sn"
>
> To see if the limits are the ones you set. If they are not, check if
> you don't have pam_limits.so commented in the /etc/pam.d/
>
> --
> Regards,
> Rafał Kuć
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
>
>> Here is the log:
>
>> Jan 24, 2013 4:10:33 AM
>> org.apache.tomcat.util.net.AprEndpoint$Acceptor run
>> SEVERE: Socket accept failed
>> org.apache.tomcat.jni.Error: 24: Too many open files
>> at org.apache.tomcat.jni.Socket.accept(Native Method)
>> at
>> org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:990)
>> at java.lang.Thread.run(Thread.java:722)
>
>
>
>> Too many open files... How to solve it?
>
>
>> On Tue, Jan 22, 2013 at 10:52 PM, Michael McCandless <
>> lucene@mikemccandless.com> wrote:
>
>>> Can you post the full stack trace of the CorruptIndexException?
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>> On Tue, Jan 22, 2013 at 8:20 AM, Cheng <zh...@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I run a Lucene application on Tomcat. The app will try to open a Linux
>>> > directory, and sometime returns CorruptIndexException error.
>>> >
>>> > Shortly after I restart Tomcat (nothing else changes), the app can be run
>>> > on the fly. I am using the following statements to open a directory:
>>> >
>>> >
>>> > try {
>>> > searcher = new IndexSearcher(IndexReader.open(new
>>> NIOFSDirectory(new
>>> > File("/home/user/"+ type))));
>>> > } catch (IOException e) {
>>> > throw new Exception("[" + type + "] Cannot open index folder...");
>>> > }
>>> >
>>> > I would like to know how to tackle this problem.
>>> >
>>> > Many thanks!
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IndexReader.open and CorruptIndexException
Posted by Rafał Kuć <r....@solr.pl>.
Hello!
You need to allow the user that is running Lucene to open more files.
There are plenty of tutorials available on the web. Modify your
/etc/security/limits.conf and if for example your user is lucene add
the following (or modify if those already exists):
lucene soft nofile 64000
lucene hard nofile 64000
Relog and run:
sudo -u lucene -s "ulimit -Sn"
To see if the limits are the ones you set. If they are not, check if
you don't have pam_limits.so commented in the /etc/pam.d/
--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Here is the log:
> Jan 24, 2013 4:10:33 AM
> org.apache.tomcat.util.net.AprEndpoint$Acceptor run
> SEVERE: Socket accept failed
> org.apache.tomcat.jni.Error: 24: Too many open files
> at org.apache.tomcat.jni.Socket.accept(Native Method)
> at
> org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:990)
> at java.lang.Thread.run(Thread.java:722)
> Too many open files... How to solve it?
> On Tue, Jan 22, 2013 at 10:52 PM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>> Can you post the full stack trace of the CorruptIndexException?
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>> On Tue, Jan 22, 2013 at 8:20 AM, Cheng <zh...@gmail.com> wrote:
>> > Hi,
>> >
>> > I run a Lucene application on Tomcat. The app will try to open a Linux
>> > directory, and sometime returns CorruptIndexException error.
>> >
>> > Shortly after I restart Tomcat (nothing else changes), the app can be run
>> > on the fly. I am using the following statements to open a directory:
>> >
>> >
>> > try {
>> > searcher = new IndexSearcher(IndexReader.open(new
>> NIOFSDirectory(new
>> > File("/home/user/"+ type))));
>> > } catch (IOException e) {
>> > throw new Exception("[" + type + "] Cannot open index folder...");
>> > }
>> >
>> > I would like to know how to tackle this problem.
>> >
>> > Many thanks!
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: IndexReader.open and CorruptIndexException
Posted by Cheng <zh...@gmail.com>.
Here is the log:
Jan 24, 2013 4:10:33 AM org.apache.tomcat.util.net.AprEndpoint$Acceptor run
SEVERE: Socket accept failed
org.apache.tomcat.jni.Error: 24: Too many open files
at org.apache.tomcat.jni.Socket.accept(Native Method)
at org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:990)
at java.lang.Thread.run(Thread.java:722)
Too many open files... How to solve it?
On Tue, Jan 22, 2013 at 10:52 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:
> Can you post the full stack trace of the CorruptIndexException?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Tue, Jan 22, 2013 at 8:20 AM, Cheng <zh...@gmail.com> wrote:
> > Hi,
> >
> > I run a Lucene application on Tomcat. The app will try to open a Linux
> > directory, and sometime returns CorruptIndexException error.
> >
> > Shortly after I restart Tomcat (nothing else changes), the app can be run
> > on the fly. I am using the following statements to open a directory:
> >
> >
> > try {
> > searcher = new IndexSearcher(IndexReader.open(new
> NIOFSDirectory(new
> > File("/home/user/"+ type))));
> > } catch (IOException e) {
> > throw new Exception("[" + type + "] Cannot open index folder...");
> > }
> >
> > I would like to know how to tackle this problem.
> >
> > Many thanks!
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: IndexReader.open and CorruptIndexException
Posted by Michael McCandless <lu...@mikemccandless.com>.
Can you post the full stack trace of the CorruptIndexException?
Mike McCandless
http://blog.mikemccandless.com
On Tue, Jan 22, 2013 at 8:20 AM, Cheng <zh...@gmail.com> wrote:
> Hi,
>
> I run a Lucene application on Tomcat. The app will try to open a Linux
> directory, and sometime returns CorruptIndexException error.
>
> Shortly after I restart Tomcat (nothing else changes), the app can be run
> on the fly. I am using the following statements to open a directory:
>
>
> try {
> searcher = new IndexSearcher(IndexReader.open(new NIOFSDirectory(new
> File("/home/user/"+ type))));
> } catch (IOException e) {
> throw new Exception("[" + type + "] Cannot open index folder...");
> }
>
> I would like to know how to tackle this problem.
>
> Many thanks!
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org