You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by miztaken <ma...@ccnep.com.np> on 2007/07/26 08:27:59 UTC
Strange Error while deleting Documents from index while indexing.
Hi,
I am dumping the database tables into lucene documents.
I am doing like this:
1. Get the rowset from database to be stored as Lucene Document.
2. Open IndexReader and check if they are already indexed.
If Indexed, delete them and add the new rowset.
Continue this till the end
3. Close IndexReader
4. Open IndexWriter
5. Write the same rowset in the index.
6. delete the rowset from database..
7. Repeat the same process[step 1 - step 7 ] till there are records in
database.
Like this i am doing Indexing and deletion.
Some key points:
1. New indexWriter is opened when there is not instance of indexwriter
available,but if available it makes use of the same IndexWriter. i.e. My
index Writer opens once in Step 4 and after that the whole process makes use
of it.
2. But i open indexReader for each deletion and close.
3. I optimize IndexWriter after certain threshold is crossed.
Now my problem is:
In the first deletion of document (if present) in step 2 and closing of
indexreader in step 3. I get no error.
But in the second loop, i get the error while trying to close the
IndexReader.
The error is :
Unable to cast object of type 'System.Collections.DictionaryEntry' to type
'System.String'.
Stack Trace:
at Lucene.Net.Index.IndexFileDeleter.DeleteFiles(ArrayList files)
at Lucene.Net.Index.IndexFileDeleter.DeleteFiles()
at Lucene.Net.Index.IndexFileDeleter.CommitPendingFiles()
at Lucene.Net.Index.IndexReader.Commit()
at Lucene.Net.Index.IndexReader.Close()
at QueryDatabaseForIndexing.Program.Main(String[] args) in E:\Test
Applications\ORS Lucene Developments\July 25\
TotalIndexingAndSearching_25_july\T otalIndexingAndSearching\
QueryDatabaseForIndexing \Program2.cs:line 159
I dont know whats the cause of this error.
I am in real need of help.
Please help me find error.
--
View this message in context: http://www.nabble.com/Strange-Error-while-deleting-Documents-from-index-while-indexing.-tf4149570.html#a11804824
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange Error while deleting Documents from index while indexing.
Posted by Chris Hostetter <ho...@fucit.org>.
: Where shall i post this issue.
you are currently posting to a list named "java-user" this is for "user"
related questions about the "java" lucene project.
if you have questions about "Lucene.Net" you should be asking them on the
"Lucene.Net" user list...
http://incubator.apache.org/lucene.net/
http://mail-archives.apache.org/mod_mbox/incubator-lucene-net-user/
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange Error while deleting Documents from index while indexing.
Posted by karl wettin <ka...@gmail.com>.
27 jul 2007 kl. 13.43 skrev miztaken:
>
>> Can you use IndexWriter#deleteDocument instead?
>
> No i cant use this method.
> I dont know docid and i dont wanna search for it. It will only add
> extra
> time.
> I am deleting the document on the basis of unique key field.
You can do that with IndexWriter#deleteDocuments(Term)
And if Lucene.Net does not support that, can you try doing it by
searching and deleting just to see how it changes the outcome?
>> Can you please supply an isolated and working test case that
>> demonstrate your problem?
>
> Find the attachment here: http://www.nabble.com/file/p11827583/
> Program2.cs
That is a fairly large amount of code. And as it is C# I have nothing
to test it on. Too much for me to compile and run in the head. I was
hoping for a few line of code that demonstrated the problem and
nothing else.
--
karl
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange Error while deleting Documents from index while
indexing.
Posted by miztaken <ma...@ccnep.com.np>.
>Can you use IndexWriter#deleteDocument instead?
No i cant use this method.
I dont know docid and i dont wanna search for it. It will only add extra
time.
I am deleting the document on the basis of unique key field.
>Can you please supply an isolated and working test case that
>demonstrate your problem?
Well my application is used to index the database rowsets so i wont be able
to give you running code but i have tried my best to keep it elaborative.
Please check the attachment.
I have attached a C# file.
Please open with any text editor of your choice.
Find the attachment here: http://www.nabble.com/file/p11827583/Program2.cs
Program2.cs
Please help me.
--
View this message in context: http://www.nabble.com/Strange-Error-while-deleting-Documents-from-index-while-indexing.-tf4149570.html#a11827583
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange Error while deleting Documents from index while indexing.
Posted by karl wettin <ka...@gmail.com>.
27 jul 2007 kl. 10.50 skrev miztaken:
>
> My application simply shut downs.
> After that when i try to open the same index using IndexReader and
> fetch the
> document then it says "trying to access deleted document". After
> getting
> such error, i opened the indexWriter, optimized and then closed it.
> Then again i tried to get the documents using indexreader and its
> working
> all fine.
>
> What can be the problem.
>
> Well the pseudo code will be like:
Can you please supply an isolated and working test case that
demonstrate your problem?
Can you use IndexWriter#deleteDocument instead?
--
karl
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange Error while deleting Documents from index while
indexing.
Posted by miztaken <ma...@ccnep.com.np>.
Where shall i post this issue.
I am naive to Lucene.
And about IndexWriter Closing.
Now i am trying like this:
1. Open New IndexReader.
2. Delete Documents.
3. Close IndexReader.
4. Open New IndexWriter.
5. Write Documents.
6. Close IndexWriter.
7. Repeat the process for n times the in nth time optimize the index before
closing indexwriter.
Is it acceptable.
But according to the http://wiki.apache.org/lucene-java/ImproveIndexingSpeed
It says to use only once instance of IndexWriter.
But in my case, after each iteration i have to close my IndexWriter as you
have suggested.
Is this the only way to do this..
Or is it that, using one instance of IndexWriter is only applicable while
doing indexing without deleting documents..?
Please make me clear.
And another question..?
What can be the possible scenario for threaded indexWriting and deleting
context..?
--
View this message in context: http://www.nabble.com/Strange-Error-while-deleting-Documents-from-index-while-indexing.-tf4149570.html#a11823559
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange Error while deleting Documents from index while
indexing.
Posted by miztaken <ma...@ccnep.com.np>.
Where shall i post this issue.
I am naive to Lucene.
Now i have switched to lucene 1.9.1.4
And about IndexWriter Closing.
Now i am trying like this:
1. Open New IndexReader.
2. Delete Documents.
3. Close IndexReader.
4. Open New IndexWriter.
5. Write Documents.
6. Close IndexWriter.
7. Repeat the process for n times the in nth time optimize the index before
closing indexwriter.
By doing this i am not getting any of my previous problems but i got the new
problem.
When i close my application then i call indexwriter.optimize() and then
indexwriter.close() eventually.
For small amount of indexing, things are working fine but for larger
indexing indexwriter.optimze() terminated unexpectedly and it also didnt
threw any exception.
My application simply shut downs.
After that when i try to open the same index using IndexReader and fetch the
document then it says "trying to access deleted document". After getting
such error, i opened the indexWriter, optimized and then closed it.
Then again i tried to get the documents using indexreader and its working
all fine.
What can be the problem.
Well the pseudo code will be like:
Not Working pseudo-code
1. Open Application (declare int countWriting=0;)
2. Open IndexReader
3. Delete Document
4. Close IndexReader.
5. Open IndexWriter
6. Add Documents
7. countWriting++;
8. Close IndexWriter.
9. If countWriting==10 then optimize();
10. if user donot cacel, repeat the whole process.
11. else try optimize
The program terminates abnormally in step 11.
But afterwards in the same index, when i try to open indexReader to fetch
documents then it says "trying to access deleted documents" but when i open
indexwriter optimize the index and then close it and open indexreader then
everything works fine.
--
View this message in context: http://www.nabble.com/Strange-Error-while-deleting-Documents-from-index-while-indexing.-tf4149570.html#a11823559
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Strange Error while deleting Documents from index while indexing.
Posted by Doron Cohen <DO...@il.ibm.com>.
Seems like a Lucene.Net issue, better post there to
solve this. One comment - must close the writer
between iterations otherwise next attempt to delete
a document with a reader will fail to obtain
a write lock.
miztaken <ma...@ccnep.com.np> wrote on 25/07/2007 23:27:59:
>
> Hi,
> I am dumping the database tables into lucene documents.
> I am doing like this:
>
> 1. Get the rowset from database to be stored as Lucene Document.
> 2. Open IndexReader and check if they are already indexed.
> If Indexed, delete them and add the new rowset.
> Continue this till the end
> 3. Close IndexReader
> 4. Open IndexWriter
> 5. Write the same rowset in the index.
> 6. delete the rowset from database..
> 7. Repeat the same process[step 1 - step 7 ] till there are records in
> database.
>
>
> Like this i am doing Indexing and deletion.
> Some key points:
> 1. New indexWriter is opened when there is not instance of indexwriter
> available,but if available it makes use of the same IndexWriter. i.e. My
> index Writer opens once in Step 4 and after that the whole
> process makes use
> of it.
> 2. But i open indexReader for each deletion and close.
> 3. I optimize IndexWriter after certain threshold is crossed.
>
> Now my problem is:
> In the first deletion of document (if present) in step 2 and closing of
> indexreader in step 3. I get no error.
> But in the second loop, i get the error while trying to close the
> IndexReader.
>
> The error is :
> Unable to cast object of type 'System.Collections.
> DictionaryEntry' to type
> 'System.String'.
>
> Stack Trace:
> at Lucene.Net.Index.IndexFileDeleter.DeleteFiles(ArrayList files)
> at Lucene.Net.Index.IndexFileDeleter.DeleteFiles()
> at Lucene.Net.Index.IndexFileDeleter.CommitPendingFiles()
> at Lucene.Net.Index.IndexReader.Commit()
> at Lucene.Net.Index.IndexReader.Close()
> at QueryDatabaseForIndexing.Program.Main(String[] args) in E:\Test
> Applications\ORS Lucene Developments\July 25\
> TotalIndexingAndSearching_25_july\T otalIndexingAndSearching\
> QueryDatabaseForIndexing \Program2.cs:line 159
>
> I dont know whats the cause of this error.
>
> I am in real need of help.
> Please help me find error.
>
>
>
> --
> View this message in context: http://www.nabble.com/Strange-
> Error-while-deleting-Documents-from-index-while-indexing.-
> tf4149570.html#a11804824
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org