You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Leonardo Barbosa <ls...@gmail.com> on 2005/04/18 22:47:30 UTC
when searching -> java.io.IOException: File does not exist
Hi,
I'm having this problem when I try to search something after my tomcat
is runing for 2 or 3 days.
Just after build the index and start tomcat, everything is fine, but
after sometime ( I couldn't find what is the trigger), I start to
have this exception.
here is the stack:
java.io.IOException: File does not exist
at net.nutch.fs.LocalFileSystem.open(LocalFileSystem.java:77)
at net.nutch.io.SequenceFile$Reader.<init>(SequenceFile.java:143)
at net.nutch.io.SequenceFile$Reader.<init>(SequenceFile.java:136)
at net.nutch.io.MapFile$Reader.<init>(MapFile.java:171)
at net.nutch.io.MapFile$Reader.<init>(MapFile.java:160)
at net.nutch.io.ArrayFile$Reader.<init>(ArrayFile.java:37)
at net.nutch.searcher.FetchedSegments$Segment.getParseText(FetchedSegments.java:74)
at net.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:131)
at net.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:140)
at net.nutch.searcher.NutchBean.getSummary(NutchBean.java:240)
....
It tries to open my segment dir to get the summary. I openned the
index with Luke, and the segment name in the index is right! It really
exists in my disk. Its readble, etc. So, why can't nutch read it?
I've found a bug at sun reporting that after sometime, the "user.dir"
(default in the nutch-default.xml) is lost by the JVM:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4117557
it seams that getCanonicalPath() prints the write directory, but the
other functions in File.class wont access the write one.
Did anyone have this problem?
[]s
Leonardo Barbosa
Re: when searching -> java.io.IOException: File does not exist
Posted by Leonardo Barbosa <ls...@gmail.com>.
Hi,
Sorry, just forget my last email.
I just read the lucene's source code and understood that it always
closes the directory (except when "open(Directory)" is called).
So, I just created a new method IndexSearch.closeReaders() the calls
IndexReader's close method.
And now, when I'm creating a new NutchBean (refresh parameter), if
"searcher" is instanceof IndexSearcher, I call this new closeReaders
method.
Lets see if I stop to get TooManyOpenFiles after a while "realoading"
my NutchBean in the servletContext.
Best Regards,
Leonardo Barbosa
On 4/21/05, Leonardo Barbosa <ls...@gmail.com> wrote:
> Hello,
>
> It's me again.. sorry if I'm bothering you with this question, but like
> Richard I don't want to reload tomcat every time I change the index,
> so I created a simple request parameter "refresh", and changed
> NutchBean.java to create a new NuchBean in the servletContext if it
> doesn't exists OR if the parameter refresh was true.
> I was working fine, but after some time I got this error above. Now
> I've found that it's because "Too many files open".
> Looking to the IndexSearcher code I've found that it doesn't tell
> lucene IndexReader to close the directories (second optional argument
> in the constructor).
> Is ther any special reason for this? Is it ok to create a close method
> in the IndexSearcher to close all IndexReader's?
>
> Thanks,
> Leonardo Barbosa
>
>
> On 4/18/05, Leonardo Barbosa <ls...@gmail.com> wrote:
> > Hi,
> >
> > I'm having this problem when I try to search something after my tomcat
> > is runing for 2 or 3 days.
> > Just after build the index and start tomcat, everything is fine, but
> > after sometime ( I couldn't find what is the trigger), I start to
> > have this exception.
> >
> > here is the stack:
> >
> > java.io.IOException: File does not exist
> > at net.nutch.fs.LocalFileSystem.open(LocalFileSystem.java:77)
> > at net.nutch.io.SequenceFile$Reader.<init>(SequenceFile.java:143)
> > at net.nutch.io.SequenceFile$Reader.<init>(SequenceFile.java:136)
> > at net.nutch.io.MapFile$Reader.<init>(MapFile.java:171)
> > at net.nutch.io.MapFile$Reader.<init>(MapFile.java:160)
> > at net.nutch.io.ArrayFile$Reader.<init>(ArrayFile.java:37)
> > at net.nutch.searcher.FetchedSegments$Segment.getParseText(FetchedSegments.java:74)
> > at net.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:131)
> > at net.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:140)
> > at net.nutch.searcher.NutchBean.getSummary(NutchBean.java:240)
> > ....
> >
> > It tries to open my segment dir to get the summary. I openned the
> > index with Luke, and the segment name in the index is right! It really
> > exists in my disk. Its readble, etc. So, why can't nutch read it?
> >
> > I've found a bug at sun reporting that after sometime, the "user.dir"
> > (default in the nutch-default.xml) is lost by the JVM:
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4117557
> >
> > it seams that getCanonicalPath() prints the write directory, but the
> > other functions in File.class wont access the write one.
> >
> > Did anyone have this problem?
> >
> > []s
> > Leonardo Barbosa
> >
>
> --
> ------------------------------------------------------------------------------------------
> Encumbered forever by desire and ambition
> There's a hunger still unsatisfied
> Our weary eyes still stray to the horizon
> Though down this road we've been so many times
>
> Pink Floyd (David Gilmour/Polly Samson) - High Hopes
> ------------------------------------------------------------------------------------------
>
--
------------------------------------------------------------------------------------------
Encumbered forever by desire and ambition
There's a hunger still unsatisfied
Our weary eyes still stray to the horizon
Though down this road we've been so many times
Pink Floyd (David Gilmour/Polly Samson) - High Hopes
------------------------------------------------------------------------------------------
Re: when searching -> java.io.IOException: File does not exist
Posted by Leonardo Barbosa <ls...@gmail.com>.
Hello,
It's me again.. sorry if I'm bothering you with this question, but like
Richard I don't want to reload tomcat every time I change the index,
so I created a simple request parameter "refresh", and changed
NutchBean.java to create a new NuchBean in the servletContext if it
doesn't exists OR if the parameter refresh was true.
I was working fine, but after some time I got this error above. Now
I've found that it's because "Too many files open".
Looking to the IndexSearcher code I've found that it doesn't tell
lucene IndexReader to close the directories (second optional argument
in the constructor).
Is ther any special reason for this? Is it ok to create a close method
in the IndexSearcher to close all IndexReader's?
Thanks,
Leonardo Barbosa
On 4/18/05, Leonardo Barbosa <ls...@gmail.com> wrote:
> Hi,
>
> I'm having this problem when I try to search something after my tomcat
> is runing for 2 or 3 days.
> Just after build the index and start tomcat, everything is fine, but
> after sometime ( I couldn't find what is the trigger), I start to
> have this exception.
>
> here is the stack:
>
> java.io.IOException: File does not exist
> at net.nutch.fs.LocalFileSystem.open(LocalFileSystem.java:77)
> at net.nutch.io.SequenceFile$Reader.<init>(SequenceFile.java:143)
> at net.nutch.io.SequenceFile$Reader.<init>(SequenceFile.java:136)
> at net.nutch.io.MapFile$Reader.<init>(MapFile.java:171)
> at net.nutch.io.MapFile$Reader.<init>(MapFile.java:160)
> at net.nutch.io.ArrayFile$Reader.<init>(ArrayFile.java:37)
> at net.nutch.searcher.FetchedSegments$Segment.getParseText(FetchedSegments.java:74)
> at net.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:131)
> at net.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:140)
> at net.nutch.searcher.NutchBean.getSummary(NutchBean.java:240)
> ....
>
> It tries to open my segment dir to get the summary. I openned the
> index with Luke, and the segment name in the index is right! It really
> exists in my disk. Its readble, etc. So, why can't nutch read it?
>
> I've found a bug at sun reporting that after sometime, the "user.dir"
> (default in the nutch-default.xml) is lost by the JVM:
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4117557
>
> it seams that getCanonicalPath() prints the write directory, but the
> other functions in File.class wont access the write one.
>
> Did anyone have this problem?
>
> []s
> Leonardo Barbosa
>
--
------------------------------------------------------------------------------------------
Encumbered forever by desire and ambition
There's a hunger still unsatisfied
Our weary eyes still stray to the horizon
Though down this road we've been so many times
Pink Floyd (David Gilmour/Polly Samson) - High Hopes
------------------------------------------------------------------------------------------