Index corrupted / no file removal FIX

[Krzysztof Kowalczykiewicz <kr...@cs.put.poznan.pl>]

Hi!

After few ours of debugging, logging and VM hacking we figured out what
causes situation described in my previous message (index corruption because
of not removed files):
http://marc.theaimsgroup.com/?l=xindice-users&m=101342784112788&w=2

Apparently it seems that there are 2 file descriptors opened
(RandomAccessFile objects) for newly created collection files (*.tbl).

We tracked that it is done within Collection's setConfig method line
188-192, where the filer object is first create()d if not exists, and then
open()ed. It looks good, but within Paged class implementation both create()
and open() functions create new RandomAccessFile instances. Moreover
create() method despite opening file, doesn't set opened flag what prevents
from using close() method.

I think it is now for you to decide which solution to use. We don't know the
sources very well, but we propose:
1. Fix Collection.java(188-) setConfig to
            if ( !filer.exists() )
               filer.create();
+          else                  // else added
               filer.open();

but it requires to set opened flag within Paged's create() method

or

2. Close file in the end of create() (caused some exception...).

or

3. close file after create() within setConfig()

After those modifications collections were correctly removed from disk, no
files were left locked.

I'm really amazed it worked sometimes (maybe GC swept both file objects).

We're waiting for official CVS update.

regards

Krzysztof Kowalczykiewicz, POLAND
Dawid Weiss, POLAND

Re: Index corrupted / no file removal FIX

[Krzysztof Kowalczykiewicz <kr...@cs.put.poznan.pl>]

> On UNIX if you unlink a file, it will continue to be available until all 
> descriptors are closed...  On Windows, I think deletion is forbidden if 
> any process has a descriptor open, which is probably why it isn't 
> happening on UNIX from what I can tell.  I'll make the modification 
> today.
> 
> I've already got a fix for it, but won't be able to do a checkin until 
> thursday.  The 'actual' fix is to add raf.close to the create method in 
> Paged.java, because the Filer interface doesn't require that the create 
> method leave the file open after creation.

Thank you very much. Just the way we've solved it...

best regards

Krzysztof Kowalczykiewicz

Re: Index corrupted / no file removal FIX

[Tom Bradford <to...@mac.com>]

On Tuesday, February 12, 2002, at 01:59 AM, Kimbro Staken wrote:
> Thanks for looking into this, this seems to be a pretty common problem 
> now.
>
> Tom, are you going to have time to tackle this? If not a pointer in the 
> preferred direction would be good and I can work on a fix.

I sent this email yesterday:

On UNIX if you unlink a file, it will continue to be available until all 
descriptors are closed...  On Windows, I think deletion is forbidden if 
any process has a descriptor open, which is probably why it isn't 
happening on UNIX from what I can tell.  I'll make the modification 
today.

I've already got a fix for it, but won't be able to do a checkin until 
thursday.  The 'actual' fix is to add raf.close to the create method in 
Paged.java, because the Filer interface doesn't require that the create 
method leave the file open after creation.

--
Tom Bradford - http://www.tbradford.org
Apache Xindice (Native XML Database) - http://xml.apache.org
Project Labrador (Web Services Framework) - http://notdotnet.org

Re: Index corrupted / no file removal FIX

[Kimbro Staken <ks...@dbxmlgroup.com>]

Thanks for looking into this, this seems to be a pretty common problem now.

Tom, are you going to have time to tackle this? If not a pointer in the 
preferred direction would be good and I can work on a fix.


On Monday, February 11, 2002, at 06:58 AM, Krzysztof Kowalczykiewicz wrote:

> Hi!
>
> After few ours of debugging, logging and VM hacking we figured out what
> causes situation described in my previous message (index corruption 
> because
> of not removed files):
> http://marc.theaimsgroup.com/?l=xindice-users&m=101342784112788&w=2
>
> Apparently it seems that there are 2 file descriptors opened
> (RandomAccessFile objects) for newly created collection files (*.tbl).
>
> We tracked that it is done within Collection's setConfig method line
> 188-192, where the filer object is first create()d if not exists, and then
> open()ed. It looks good, but within Paged class implementation both 
> create()
> and open() functions create new RandomAccessFile instances. Moreover
> create() method despite opening file, doesn't set opened flag what 
> prevents
> from using close() method.
>
> I think it is now for you to decide which solution to use. We don't know 
> the
> sources very well, but we propose:
> 1. Fix Collection.java(188-) setConfig to
>             if ( !filer.exists() )
>                filer.create();
> +          else                  // else added
>                filer.open();
>
> but it requires to set opened flag within Paged's create() method
>
> or
>
> 2. Close file in the end of create() (caused some exception...).
>
> or
>
> 3. close file after create() within setConfig()
>
> After those modifications collections were correctly removed from disk, no
> files were left locked.
>
> I'm really amazed it worked sometimes (maybe GC swept both file objects).
>
> We're waiting for official CVS update.
>
> regards
>
> Krzysztof Kowalczykiewicz, POLAND
> Dawid Weiss, POLAND
>
>
>
>
Kimbro Staken
XML Database Software, Consulting and Writing
http://www.xmldatabases.org/