You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Oscar Picasso <os...@yahoo.com> on 2005/01/21 02:59:35 UTC
Closed IndexWriter reuse
Hi,
Is it safe to add documents to an IndexWriter that has been closed?
>From what I have seen, the close method flush the changes, closes the files but
it creates new files allowing to add new documents.
Am I right?
Thanks.
__________________________________
Do you Yahoo!?
Yahoo! Mail - Easier than ever with enhanced search. Learn more.
http://info.mail.yahoo.com/mail_250
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Closed IndexWriter reuse
Posted by Oscar Picasso <os...@yahoo.com>.
> --- Otis Gospodnetic <ot...@yahoo.com> wrote:
>
> > No, you can't add documents to an index once you close the IndexWriter.
> > You can re-open the IndexWriter and add more documents, of course.
> >
> > Otis
After my previous post I have made some further tests with multithreading and
effectively it randomly throw NullPointerExceptions and Lock exceptions when
reusing a closed IndexWriter.
My example was bad because based on a very simple single thread.
But wouldn't it be safer if IndexWriter rose immediatly an Exception when
trying to use its modifying methods after is has been closed?
__________________________________
Do you Yahoo!?
Yahoo! Mail - 250MB free storage. Do more. Manage less.
http://info.mail.yahoo.com/mail_250
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Closed IndexWriter reuse
Posted by Oscar Picasso <os...@yahoo.com>.
--- Otis Gospodnetic <ot...@yahoo.com> wrote:
> No, you can't add documents to an index once you close the IndexWriter.
> You can re-open the IndexWriter and add more documents, of course.
>
> Otis
That's what I expected at first, but:
1- It's a disappointment, because such a 'feature' would have made IndexeWriter
management much easier. You would open an IndexWriter at startup and reuse it
during all the life of the application, just flushing on a regular base using
the close() method and without worrying if other objects are currently using
the writer.
2- When you say you can't add, do you mean it's impossible or that you
shouldn't because for example it could corrupt the index?
Maybe I'm wrong, but I think it's possible. Let's look at the follwoing code:
"
public static void main(String[] args) throws IOException
{
final IndexWriter writer1 = new IndexWriter("/tmp/test-reuse", new
StandardAnalyzer(), true);
// First write with the writer
Document doc = new Document();
doc.add(new Field("name", "John", Field.Store.YES, Field.Index.UN_TOKENIZED));
writer1.addDocument(doc);
System.out.println("1 ---- After first write, before closing the writer ---");
Searcher searcher = new IndexSearcher("/tmp/test-reuse");
Query query = new TermQuery(new Term("name", "John"));
Hits hits = searcher.search(query);
System.out.println("===> hits: " + hits.length());
System.out.println();
// CLOSING THE WRITER ONCE
writer1.close();
System.out.println("2 ---- After first write, after closing the writer ---");
searcher = new IndexSearcher("/tmp/test-reuse");
hits = searcher.search(query);
System.out.println("===> hits: " + hits.length());
System.out.println();
// Second write, THE WRITER HAS ALREADY BEEN CLOSED ONCE
writer1.addDocument(doc);
System.out.println("3 ---- After second write, the writer has been closed once
---");
hits = searcher.search(query);
System.out.println("===> hits: " + hits.length());
System.out.println();
// Closing the writer again
writer1.close();
System.out.println("4 ---- After second write, the writer has been closed
twice ---");
searcher = new IndexSearcher("/tmp/test-reuse");
hits = searcher.search(query);
System.out.println("===> hits: " + hits.length());
}
== Results ==
1 ---- After first write, before closing the writer ---
===> hits: 0
2 ---- After first write, after closing the writer ---
===> hits: 1
3 ---- After second write, the writer has been closed once ---
===> hits: 1
4 ---- After second write, the writer has been closed twice ---
===> hits: 2
As your can see, not only does the code above execute without complain but it
also gives the right results.
Thanks for your comments.
__________________________________
Do you Yahoo!?
Yahoo! Mail - Easier than ever with enhanced search. Learn more.
http://info.mail.yahoo.com/mail_250
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Suggestion needed for extranet search
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Free as in orange juice.
Otis
--- "Ranjan K. Baisak" <ra...@yahoo.com> wrote:
> Otis,
> Thanks for your help. Is nutch a freeware tool?
>
> regards,
> Ranjan
> --- Otis Gospodnetic <ot...@yahoo.com>
> wrote:
>
> > Hi Ranjan,
> >
> > It sounds like you are should look at and use Nutch:
> > http://www.nutch.org
> >
> > Otis
> >
> > --- "Ranjan K. Baisak" <ra...@yahoo.com>
> > wrote:
> >
> > > I am planning to move to Lucene but not have much
> > > knowledge on the same. The search engine which I
> > had
> > > developed is searching some extranet URLs e.g.
> > > codeguru.com/index.html. Is is possible to get the
> > > same functionality using Lucene. So basically can
> > I
> > > make Lucene as a search engine to search
> > extranets.
> > >
> > > regards,
> > > Ranjan
> > >
> > > __________________________________________________
> > > Do You Yahoo!?
> > > Tired of spam? Yahoo! Mail has the best spam
> > protection around
> > > http://mail.yahoo.com
> > >
> > >
> >
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail:
> > lucene-user-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail:
> > lucene-user-help@jakarta.apache.org
> > >
> > >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> > lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> > lucene-user-help@jakarta.apache.org
> >
> >
>
>
>
>
> __________________________________
> Do you Yahoo!?
> The all-new My Yahoo! - What will yours do?
> http://my.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Suggestion needed for extranet search
Posted by "Ranjan K. Baisak" <ra...@yahoo.com>.
Otis,
Thanks for your help. Is nutch a freeware tool?
regards,
Ranjan
--- Otis Gospodnetic <ot...@yahoo.com>
wrote:
> Hi Ranjan,
>
> It sounds like you are should look at and use Nutch:
> http://www.nutch.org
>
> Otis
>
> --- "Ranjan K. Baisak" <ra...@yahoo.com>
> wrote:
>
> > I am planning to move to Lucene but not have much
> > knowledge on the same. The search engine which I
> had
> > developed is searching some extranet URLs e.g.
> > codeguru.com/index.html. Is is possible to get the
> > same functionality using Lucene. So basically can
> I
> > make Lucene as a search engine to search
> extranets.
> >
> > regards,
> > Ranjan
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam? Yahoo! Mail has the best spam
> protection around
> > http://mail.yahoo.com
> >
> >
>
---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
> >
> >
>
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
> lucene-user-help@jakarta.apache.org
>
>
__________________________________
Do you Yahoo!?
The all-new My Yahoo! - What will yours do?
http://my.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Suggestion needed for extranet search
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Hi Ranjan,
It sounds like you are should look at and use Nutch:
http://www.nutch.org
Otis
--- "Ranjan K. Baisak" <ra...@yahoo.com> wrote:
> I am planning to move to Lucene but not have much
> knowledge on the same. The search engine which I had
> developed is searching some extranet URLs e.g.
> codeguru.com/index.html. Is is possible to get the
> same functionality using Lucene. So basically can I
> make Lucene as a search engine to search extranets.
>
> regards,
> Ranjan
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Suggestion needed for extranet search
Posted by "Ranjan K. Baisak" <ra...@yahoo.com>.
I am planning to move to Lucene but not have much
knowledge on the same. The search engine which I had
developed is searching some extranet URLs e.g.
codeguru.com/index.html. Is is possible to get the
same functionality using Lucene. So basically can I
make Lucene as a search engine to search extranets.
regards,
Ranjan
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Closed IndexWriter reuse
Posted by Otis Gospodnetic <ot...@yahoo.com>.
No, you can't add documents to an index once you close the IndexWriter.
You can re-open the IndexWriter and add more documents, of course.
Otis
--- Oscar Picasso <os...@yahoo.com> wrote:
> Hi,
>
> Is it safe to add documents to an IndexWriter that has been closed?
>
> From what I have seen, the close method flush the changes, closes the
> files but
> it creates new files allowing to add new documents.
>
> Am I right?
>
> Thanks.
>
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - Easier than ever with enhanced search. Learn more.
> http://info.mail.yahoo.com/mail_250
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org