still getting intermittent index error on linux

[Michael Neale <mi...@gmail.com>]

ERROR 20-04 17:25:35,776 (LazyQueryResultImpl.java:getResults:266)
-Exception while executing query:
java.io.IOException: No such file or directory
    at java.io.UnixFileSystem.createFileExclusively(Native Method)
    at java.io.File.createNewFile(File.java:850)
    at org.apache.jackrabbit.core.query.lucene.FSDirectory$1.obtain(
FSDirectory.java:119)
    at org.apache.lucene.store.Lock.obtain(Lock.java:51)
    at org.apache.lucene.store.Lock$With.run(Lock.java:98)

I now know I can just blow away the lucene index directory, and then it will
start up fine (and recreate).
Am I somehow not "shutting down" things correctly?

Michael.

Re: still getting intermittent index error on linux

[Michael Neale <mi...@gmail.com>]

Marcel, I have attached an example to reproduce this. Whilst I believe the
code for locking is slightly flawed (but fixable) it is not the cause of
this problem - the problem is looking for content/directory that is not
there (and only doing that rarely).

See the attachment in: https://issues.apache.org/jira/browse/JCR-778

I look forward to hearing some ideas, this is a little bit troubling to me,
as it seems to get the database into quite a state, and I am worried this is
some kind of corruption which could cause problems for heavy use, so I need
to get to the bottom of it.

Michael.

On 4/22/07, Michael Neale <mi...@gmail.com> wrote:
>
> Hi Marcel - ok I can reproduce it on MacOSX, a few flavours of linux and
> Windows XP - I am now working on a fix.
>
> I have noted the main issues in the JIRA, but basically, its incorrect use
> of createNewFile in File - according to the java specification, so I have
> re-worked locks to use NIO (FileChannel and FileLock).
>
> Now, what is confusing me is that locks are being requested for
> *directories* that do not exist - not sure what to do in that case, and that
> is causing failures (as it won't created the directory automatically).
>
> Ideas?
>
> A few pointers, and I should have a patch for this (testing is a bitch,
> but its reproducible at least, just not in a junit test - you have to start
> up and stop a lot of times to see this).
>
> Michael.
>
>
> On 4/20/07, Marcel Reutegger <ma...@gmx.net> wrote:
> >
> > Hi Michael,
> >
> > do you have a test case that you can attach to issue JCR-778?
> >
> > Michael Neale wrote:
> > > ERROR 20-04 17:25:35,776 (LazyQueryResultImpl.java:getResults:266)
> > > -Exception while executing query:
> > > java.io.IOException: No such file or directory
> > >    at java.io.UnixFileSystem.createFileExclusively(Native Method)
> > >    at java.io.File.createNewFile(File.java:850)
> > >    at org.apache.jackrabbit.core.query.lucene.FSDirectory$1.obtain (
> > > FSDirectory.java:119)
> > >    at org.apache.lucene.store.Lock.obtain(Lock.java:51)
> > >    at org.apache.lucene.store.Lock$With.run(Lock.java:98)
> > >
> > > I now know I can just blow away the lucene index directory, and then
> > it
> > > will
> > > start up fine (and recreate).
> > > Am I somehow not "shutting down" things correctly?
> >
> > that's not very likely. unless you have some sort of cleanup script that
> > removes
> > files while the test is still running.
> >
> > regards
> >   marcel
> >
>
>

Re: still getting intermittent index error on linux

[Michael Neale <mi...@gmail.com>]

Hi Marcel - ok I can reproduce it on MacOSX, a few flavours of linux and
Windows XP - I am now working on a fix.

I have noted the main issues in the JIRA, but basically, its incorrect use
of createNewFile in File - according to the java specification, so I have
re-worked locks to use NIO (FileChannel and FileLock).

Now, what is confusing me is that locks are being requested for
*directories* that do not exist - not sure what to do in that case, and that
is causing failures (as it won't created the directory automatically).

Ideas?

A few pointers, and I should have a patch for this (testing is a bitch, but
its reproducible at least, just not in a junit test - you have to start up
and stop a lot of times to see this).

Michael.


On 4/20/07, Marcel Reutegger <ma...@gmx.net> wrote:
>
> Hi Michael,
>
> do you have a test case that you can attach to issue JCR-778?
>
> Michael Neale wrote:
> > ERROR 20-04 17:25:35,776 (LazyQueryResultImpl.java:getResults:266)
> > -Exception while executing query:
> > java.io.IOException: No such file or directory
> >    at java.io.UnixFileSystem.createFileExclusively(Native Method)
> >    at java.io.File.createNewFile(File.java:850)
> >    at org.apache.jackrabbit.core.query.lucene.FSDirectory$1.obtain (
> > FSDirectory.java:119)
> >    at org.apache.lucene.store.Lock.obtain(Lock.java:51)
> >    at org.apache.lucene.store.Lock$With.run(Lock.java:98)
> >
> > I now know I can just blow away the lucene index directory, and then it
> > will
> > start up fine (and recreate).
> > Am I somehow not "shutting down" things correctly?
>
> that's not very likely. unless you have some sort of cleanup script that
> removes
> files while the test is still running.
>
> regards
>   marcel
>

Re: still getting intermittent index error on linux

[Marcel Reutegger <ma...@gmx.net>]

Hi Michael,

do you have a test case that you can attach to issue JCR-778?

Michael Neale wrote:
> ERROR 20-04 17:25:35,776 (LazyQueryResultImpl.java:getResults:266)
> -Exception while executing query:
> java.io.IOException: No such file or directory
>    at java.io.UnixFileSystem.createFileExclusively(Native Method)
>    at java.io.File.createNewFile(File.java:850)
>    at org.apache.jackrabbit.core.query.lucene.FSDirectory$1.obtain(
> FSDirectory.java:119)
>    at org.apache.lucene.store.Lock.obtain(Lock.java:51)
>    at org.apache.lucene.store.Lock$With.run(Lock.java:98)
> 
> I now know I can just blow away the lucene index directory, and then it 
> will
> start up fine (and recreate).
> Am I somehow not "shutting down" things correctly?

that's not very likely. unless you have some sort of cleanup script that removes 
files while the test is still running.

regards
  marcel