Problem with indexing/merging indices - documents not indexed.

["amigo@max3d.com" <am...@max3d.com>]

Hello all

After reading the list for more than a year, I've finally decided (got 
courage) to post my first question.
I'm not an expert in Lucene or Java, but I can find my way around it and 
right now I'm having a problem
that I hope this list could help me out with.

I'm using MySQL to store document records (9 document types, about 30-40 
fields per document type, various lengths and purpose)
which are also indexed with Lucene upon creation and after updates. This 
is done by calling a JAVA class I wrote, from PHP and passing
on necessary parameters for indexing. But for whatever reason documents 
are not being indexed or indices being merged for that matter.

Attached is a test.java source that I wrote based on my existing code 
for indexing, as a command line test class
that indexes single documents based on two arguments: doctype and 
fileno. I have tried indexing directly into the existing
index or indexing into RAMDirectory and then merging that index with the 
existing index and in both cases the document
does not get merged into the existing FS index (count of documents does 
not increase in the existing index, and in case of a new index
it's 0 and the only file in the index directory is segments)

I have read Otis's article "Advanced Text Indexing with Lucene" and 
that's where I got the idea to try to first index into RAM and then merge
indices instead of directly indexing, just for a sanity test, and while 
Otis's code works fine, my alteration does not.

I would appreciate any feedback on my code and whether I'm doing 
something in a wrong way, because I'm at a total loss right now
as to why documents are not being indexed at all.


thanks in advance,

-pedja

Re: Problem with indexing/merging indices - documents not indexed.

["amigo@max3d.com" <am...@max3d.com>]

Hi Chris

actually for merging indices that's how Otis did it in the article I quoted:

            // if -r argument was specified, use RAMDirectory
            RAMDirectory ramDir    = new RAMDirectory();
            IndexWriter  ramWriter = new IndexWriter(ramDir, analyzer, 
true);
            addDocs(ramWriter, docsInIndex);
            IndexWriter fsWriter   = new IndexWriter(indexDir, analyzer, 
true);
            fsWriter.addIndexes(new Directory[] { ramDir });
            ramWriter.close();
            fsWriter.close();

..which works great, and all I've done is replace the addDocs with my 
MySQL version of the function.

I do know about having to close and re-open to find the document count, 
which I've also tried but it didnt yield any difference,
a simple look in the index directory shows no files there except 
segements, even though it should've merged the RAMDir index into the fsDir.


thanks

-pedja



Chris Hostetter said the following on 12/6/2004 6:09 PM:

>: I would appreciate any feedback on my code and whether I'm doing
>: something in a wrong way, because I'm at a total loss right now
>: as to why documents are not being indexed at all.
>
>I didn't try running your code (because i don't have a DB to test it with)
>but a quick read gives me a good guess as to your problem:
>
>I believe you to call...
>	ramWriter.close();
>...before you call...
>	fsWriter.addIndexes(new Directory[] { ramDir });
>
>(I've never played with merging indexes, so i could be completley wrong)
>
>Everything I've ever read/seen/tried has indicated that untill you "close"
>your IndexWritter, nothing you do will be visible to anybody else who
>opens that "Directory"
>
>I'm also guessing that when you were trying to add the docs to fsWriter
>directly, you were using an IndexReader you had opened prior to calling
>fsWriter.close() to check the number of docs ... that won't work for hte
>same reason.
>
>
>
>
>-Hoss
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
>
>  
>

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Re: Problem with indexing/merging indices - documents not indexed.

[Chris Hostetter <ho...@fucit.org>]

: I would appreciate any feedback on my code and whether I'm doing
: something in a wrong way, because I'm at a total loss right now
: as to why documents are not being indexed at all.

I didn't try running your code (because i don't have a DB to test it with)
but a quick read gives me a good guess as to your problem:

I believe you to call...
	ramWriter.close();
...before you call...
	fsWriter.addIndexes(new Directory[] { ramDir });

(I've never played with merging indexes, so i could be completley wrong)

Everything I've ever read/seen/tried has indicated that untill you "close"
your IndexWritter, nothing you do will be visible to anybody else who
opens that "Directory"

I'm also guessing that when you were trying to add the docs to fsWriter
directly, you were using an IndexReader you had opened prior to calling
fsWriter.close() to check the number of docs ... that won't work for hte
same reason.




-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org