You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Pranav goyal <pr...@gmail.com> on 2011/06/06 08:29:05 UTC
Lucene Indexing
Hi all,
I am a newbie to lucene.
I have successfully created my lucene index. But I am not getting how to
invalidate previous indexes whenever I add/delete/update any field in my
lucene index.
Please help me out.
for better understanding I have wrote my indexing function :
StandardAnalyzer analyzer = new StandardAnalyzer(Version.LUCENE_31);
IndexWriter writer = null;
File indexDir = new File("./indexDir");
try {
writer = new IndexWriter(FSDirectory.open(indexDir),
analyzer, IndexWriter.MaxFieldLength.UNLIMITED);
}
catch (IOException e1) {
e1.printStackTrace();
}
//Adding New fields to the document
Document d = new Document();
d.add(new Field("key", key , Field.Store.YES ,
Field.Index.ANALYZED));
// d.add(Field.Text("obj", attribute.toString()));
// index in Lucene here
if (attribute instanceof String) {
System.out.println("IN CMnString ");
d.add(new Field("attribute",attribute.toString(),
Field.Store.YES, Field.Index.NO));
System.out.println("key is ->"+key+"...Attribute is
"+attribute);
}
else if (attribute instanceof CMnMoney) {
System.out.println("IN CMnMoney ");
d.add(new Field("attribute",attribute.toString(),
Field.Store.YES, Field.Index.NO));
System.out.println("key is ->"+key+"...Value is
"+attribute);
}
else if (attribute instanceof CMnQuantity) {
System.out.println("IN CMnQuantity ");
d.add(new Field("attribute",attribute.toString(),
Field.Store.YES, Field.Index.NO));
System.out.println("key is ->"+key+"...Attribute is
"+attribute);
}
else if (attribute instanceof CMnEnum) {
System.out.println("IN CMnEnum ");
d.add(new Field("attribute",attribute.toString(),
Field.Store.YES, Field.Index.NO));
System.out.println("key is ->"+key+"...Attribute is
"+attribute);
}
else {
// see what else is there....
}
try {
writer.addDocument(d);
writer.optimize() ;
writer.close();
}
catch (IOException e1) {
e1.printStackTrace();
}
}
Please guide me further as I got stuck at this point.
I have searched a lot about this and all I got is I have to first delete
that document and then add that document again to my index.
Re: Lucene Indexing
Posted by Anshum <an...@gmail.com>.
Yes,
You'd need to delete the document and then re-add a newly created document
object. You may use the key and delete the doc using the Term(key, value).
--
Anshum Gupta
http://ai-cafe.blogspot.com
On Mon, Jun 6, 2011 at 4:45 PM, Pranav goyal <pr...@gmail.com>wrote:
> Hi Anshum,
>
> Thanks for answering my question. By this I got to know that I cannot
> update
> without deleting my document.
>
> So whenever I am indexing the documents first I need to check whether the
> particular key exists in the document or not and if it exists I need to
> delete it and add the updated one?
>
> Sounds Cool?
>
> On Mon, Jun 6, 2011 at 4:10 PM, Anshum <an...@gmail.com> wrote:
>
> > Hii Pranav,
> > By what you've mentioned, it looks like you want to modify a particular
> > document (or all docs) by adding a particular field in the document(s).
> As
> > of right now, its not possible to modify a document inside a lucene
> index.
> > That is due to the way the index is structured. The only way as of now to
> > modify a record is to delete it, create a new doc and then add the new
> doc.
> > Also, the lucene's delete document function does not physically free up
> the
> > space for the document unless you optimize the index so you may have to
> do
> > that periodically, depending upon your update frequency.
> > Even the updateDocument function as of now would internally delete the
> > document and add the new supplied document.
> > Hope this answer helps.
> >
> > --
> > Anshum Gupta
> > http://ai-cafe.blogspot.com
> >
> >
> > On Mon, Jun 6, 2011 at 11:59 AM, Pranav goyal <
> pranavgoyal40341@gmail.com
> > >wrote:
> >
> > > Hi all,
> > >
> > > I am a newbie to lucene.
> > > I have successfully created my lucene index. But I am not getting how
> to
> > > invalidate previous indexes whenever I add/delete/update any field in
> my
> > > lucene index.
> > > Please help me out.
> > >
> > > for better understanding I have wrote my indexing function :
> > >
> > > StandardAnalyzer analyzer = new StandardAnalyzer(Version.LUCENE_31);
> > > IndexWriter writer = null;
> > > File indexDir = new File("./indexDir");
> > > try {
> > > writer = new
> > IndexWriter(FSDirectory.open(indexDir),
> > > analyzer, IndexWriter.MaxFieldLength.UNLIMITED);
> > > }
> > > catch (IOException e1) {
> > > e1.printStackTrace();
> > > }
> > >
> > >
> > > //Adding New fields to the document
> > > Document d = new Document();
> > > d.add(new Field("key", key , Field.Store.YES ,
> > > Field.Index.ANALYZED));
> > >
> > >
> > > // d.add(Field.Text("obj", attribute.toString()));
> > > // index in Lucene here
> > > if (attribute instanceof String) {
> > > System.out.println("IN CMnString ");
> > > d.add(new
> Field("attribute",attribute.toString(),
> > > Field.Store.YES, Field.Index.NO));
> > > System.out.println("key is ->"+key+"...Attribute
> > is
> > > "+attribute);
> > > }
> > > else if (attribute instanceof CMnMoney) {
> > > System.out.println("IN CMnMoney ");
> > > d.add(new
> Field("attribute",attribute.toString(),
> > > Field.Store.YES, Field.Index.NO));
> > > System.out.println("key is ->"+key+"...Value is
> > > "+attribute);
> > > }
> > > else if (attribute instanceof CMnQuantity) {
> > > System.out.println("IN CMnQuantity ");
> > > d.add(new
> Field("attribute",attribute.toString(),
> > > Field.Store.YES, Field.Index.NO));
> > > System.out.println("key is ->"+key+"...Attribute
> > is
> > > "+attribute);
> > > }
> > > else if (attribute instanceof CMnEnum) {
> > > System.out.println("IN CMnEnum ");
> > > d.add(new
> Field("attribute",attribute.toString(),
> > > Field.Store.YES, Field.Index.NO));
> > > System.out.println("key is ->"+key+"...Attribute
> > is
> > > "+attribute);
> > > }
> > > else {
> > > // see what else is there....
> > > }
> > > try {
> > > writer.addDocument(d);
> > > writer.optimize() ;
> > > writer.close();
> > > }
> > > catch (IOException e1) {
> > > e1.printStackTrace();
> > > }
> > > }
> > >
> > >
> > >
> > > Please guide me further as I got stuck at this point.
> > >
> > > I have searched a lot about this and all I got is I have to first
> delete
> > > that document and then add that document again to my index.
> > >
> >
>
>
>
> --
> I'm very responsible, when ever something goes wrong they always say I'm
> responsible --
>
Re: Lucene Indexing
Posted by Pranav goyal <pr...@gmail.com>.
Hi Anshum,
Thanks for answering my question. By this I got to know that I cannot update
without deleting my document.
So whenever I am indexing the documents first I need to check whether the
particular key exists in the document or not and if it exists I need to
delete it and add the updated one?
Sounds Cool?
On Mon, Jun 6, 2011 at 4:10 PM, Anshum <an...@gmail.com> wrote:
> Hii Pranav,
> By what you've mentioned, it looks like you want to modify a particular
> document (or all docs) by adding a particular field in the document(s). As
> of right now, its not possible to modify a document inside a lucene index.
> That is due to the way the index is structured. The only way as of now to
> modify a record is to delete it, create a new doc and then add the new doc.
> Also, the lucene's delete document function does not physically free up the
> space for the document unless you optimize the index so you may have to do
> that periodically, depending upon your update frequency.
> Even the updateDocument function as of now would internally delete the
> document and add the new supplied document.
> Hope this answer helps.
>
> --
> Anshum Gupta
> http://ai-cafe.blogspot.com
>
>
> On Mon, Jun 6, 2011 at 11:59 AM, Pranav goyal <pranavgoyal40341@gmail.com
> >wrote:
>
> > Hi all,
> >
> > I am a newbie to lucene.
> > I have successfully created my lucene index. But I am not getting how to
> > invalidate previous indexes whenever I add/delete/update any field in my
> > lucene index.
> > Please help me out.
> >
> > for better understanding I have wrote my indexing function :
> >
> > StandardAnalyzer analyzer = new StandardAnalyzer(Version.LUCENE_31);
> > IndexWriter writer = null;
> > File indexDir = new File("./indexDir");
> > try {
> > writer = new
> IndexWriter(FSDirectory.open(indexDir),
> > analyzer, IndexWriter.MaxFieldLength.UNLIMITED);
> > }
> > catch (IOException e1) {
> > e1.printStackTrace();
> > }
> >
> >
> > //Adding New fields to the document
> > Document d = new Document();
> > d.add(new Field("key", key , Field.Store.YES ,
> > Field.Index.ANALYZED));
> >
> >
> > // d.add(Field.Text("obj", attribute.toString()));
> > // index in Lucene here
> > if (attribute instanceof String) {
> > System.out.println("IN CMnString ");
> > d.add(new Field("attribute",attribute.toString(),
> > Field.Store.YES, Field.Index.NO));
> > System.out.println("key is ->"+key+"...Attribute
> is
> > "+attribute);
> > }
> > else if (attribute instanceof CMnMoney) {
> > System.out.println("IN CMnMoney ");
> > d.add(new Field("attribute",attribute.toString(),
> > Field.Store.YES, Field.Index.NO));
> > System.out.println("key is ->"+key+"...Value is
> > "+attribute);
> > }
> > else if (attribute instanceof CMnQuantity) {
> > System.out.println("IN CMnQuantity ");
> > d.add(new Field("attribute",attribute.toString(),
> > Field.Store.YES, Field.Index.NO));
> > System.out.println("key is ->"+key+"...Attribute
> is
> > "+attribute);
> > }
> > else if (attribute instanceof CMnEnum) {
> > System.out.println("IN CMnEnum ");
> > d.add(new Field("attribute",attribute.toString(),
> > Field.Store.YES, Field.Index.NO));
> > System.out.println("key is ->"+key+"...Attribute
> is
> > "+attribute);
> > }
> > else {
> > // see what else is there....
> > }
> > try {
> > writer.addDocument(d);
> > writer.optimize() ;
> > writer.close();
> > }
> > catch (IOException e1) {
> > e1.printStackTrace();
> > }
> > }
> >
> >
> >
> > Please guide me further as I got stuck at this point.
> >
> > I have searched a lot about this and all I got is I have to first delete
> > that document and then add that document again to my index.
> >
>
--
I'm very responsible, when ever something goes wrong they always say I'm
responsible --
Re: Lucene Indexing
Posted by Anshum <an...@gmail.com>.
Hii Pranav,
By what you've mentioned, it looks like you want to modify a particular
document (or all docs) by adding a particular field in the document(s). As
of right now, its not possible to modify a document inside a lucene index.
That is due to the way the index is structured. The only way as of now to
modify a record is to delete it, create a new doc and then add the new doc.
Also, the lucene's delete document function does not physically free up the
space for the document unless you optimize the index so you may have to do
that periodically, depending upon your update frequency.
Even the updateDocument function as of now would internally delete the
document and add the new supplied document.
Hope this answer helps.
--
Anshum Gupta
http://ai-cafe.blogspot.com
On Mon, Jun 6, 2011 at 11:59 AM, Pranav goyal <pr...@gmail.com>wrote:
> Hi all,
>
> I am a newbie to lucene.
> I have successfully created my lucene index. But I am not getting how to
> invalidate previous indexes whenever I add/delete/update any field in my
> lucene index.
> Please help me out.
>
> for better understanding I have wrote my indexing function :
>
> StandardAnalyzer analyzer = new StandardAnalyzer(Version.LUCENE_31);
> IndexWriter writer = null;
> File indexDir = new File("./indexDir");
> try {
> writer = new IndexWriter(FSDirectory.open(indexDir),
> analyzer, IndexWriter.MaxFieldLength.UNLIMITED);
> }
> catch (IOException e1) {
> e1.printStackTrace();
> }
>
>
> //Adding New fields to the document
> Document d = new Document();
> d.add(new Field("key", key , Field.Store.YES ,
> Field.Index.ANALYZED));
>
>
> // d.add(Field.Text("obj", attribute.toString()));
> // index in Lucene here
> if (attribute instanceof String) {
> System.out.println("IN CMnString ");
> d.add(new Field("attribute",attribute.toString(),
> Field.Store.YES, Field.Index.NO));
> System.out.println("key is ->"+key+"...Attribute is
> "+attribute);
> }
> else if (attribute instanceof CMnMoney) {
> System.out.println("IN CMnMoney ");
> d.add(new Field("attribute",attribute.toString(),
> Field.Store.YES, Field.Index.NO));
> System.out.println("key is ->"+key+"...Value is
> "+attribute);
> }
> else if (attribute instanceof CMnQuantity) {
> System.out.println("IN CMnQuantity ");
> d.add(new Field("attribute",attribute.toString(),
> Field.Store.YES, Field.Index.NO));
> System.out.println("key is ->"+key+"...Attribute is
> "+attribute);
> }
> else if (attribute instanceof CMnEnum) {
> System.out.println("IN CMnEnum ");
> d.add(new Field("attribute",attribute.toString(),
> Field.Store.YES, Field.Index.NO));
> System.out.println("key is ->"+key+"...Attribute is
> "+attribute);
> }
> else {
> // see what else is there....
> }
> try {
> writer.addDocument(d);
> writer.optimize() ;
> writer.close();
> }
> catch (IOException e1) {
> e1.printStackTrace();
> }
> }
>
>
>
> Please guide me further as I got stuck at this point.
>
> I have searched a lot about this and all I got is I have to first delete
> that document and then add that document again to my index.
>