Multiple Creation of Writers

[David Townsend <da...@magus.co.uk>]

In my system indices are created and updated by multiple threads.  I need to check if an index exists to decide whether to pass true or false to the IndexWriter constructor.

new IndexWriter(FSDirectory, Analyzer, boolean);

The problem arises when two threads attempt to create the same index after simultaneously finding that the index does not exist.  This problem can be reproduced in a single thread by

writerA = new IndexWriter(new File("c:/import/test"), new StandardAnalyzer(), true);
writerB = new IndexWriter(new File("c:/import/test"), new StandardAnalyzer(), true);
add1000Docs(writerA);
add1000Docs(writerB);

this will throw an IOException

C:\import\test\_a.fnm (The system cannot find the file specified)

The only solution I can think of is to create a database/file lock to get around this, or change the Lucene code to obtain a lock before creating an index.  Any ideas?

David








---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Re: Multiple Creation of Writers

[Michael Giles <mg...@visionstudio.com>]

Couldn't you solve this by creating your own synchronized getWriter 
method?  I'm thinking something like (pseudo code):

protected void myProgram() {
     ...
     File dir = new File("c:/import/test");
     IndexWriter wrt = getWriter(dir, new StandardAnalyzer(), create(dir));
     ...
}

protected boolean create(String dir) {
     return true if index doesn't exist;
}

protected synchronized IndexWriter getWriter(dir, anlyz, crt) {
     if (crt && !create(dir)) {
         crt = false;
     }

     return new IndexWriter(dir, anylz, crt);
}

You would actually want to get fancier on the synchronization and grab a 
mutex that is specific to the directory name, but you get the idea.

-Mike

At 12:48 PM 1/14/2004, David Townsend wrote:
>In my system indices are created and updated by multiple threads.  I need 
>to check if an index exists to decide whether to pass true or false to the 
>IndexWriter constructor.
>
>new IndexWriter(FSDirectory, Analyzer, boolean);
>
>The problem arises when two threads attempt to create the same index after 
>simultaneously finding that the index does not exist.  This problem can be 
>reproduced in a single thread by
>
>writerA = new IndexWriter(new File("c:/import/test"), new 
>StandardAnalyzer(), true);
>writerB = new IndexWriter(new File("c:/import/test"), new 
>StandardAnalyzer(), true);
>add1000Docs(writerA);
>add1000Docs(writerB);
>
>this will throw an IOException
>
>C:\import\test\_a.fnm (The system cannot find the file specified)
>
>The only solution I can think of is to create a database/file lock to get 
>around this, or change the Lucene code to obtain a lock before creating an 
>index.  Any ideas?
>
>David
>
>
>
>
>
>
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: lucene-user-help@jakarta.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org