You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by 黄靖宇 <qi...@gmail.com> on 2011/01/21 10:04:37 UTC
How to rebuild index
Hi,
I am new to lucene. Recently I was assigned for some lucene related
workitems.
Now there is one problem. Before, we use StandardAnalyzer in our
application, and our application has been online for about two years.
Now, we must to write a Custom Analyzer to replace the StandarAnalyzer for
enhancement requirement.
But there are already much exists production data and the lucene
index(generated with StandardAnalyzer).
What I need to do if I change to use the new Custom Analyzer?
>From the book "Lucene In Action". I see follow sentence:
"If you’re changing analyzers, you should rebuild your index using the new
analyzer so that all documents are analyzed in the same manner."
But I don't know how to rebuild the exists index with a new Analyzer. I
can't find a API related to this.
Any help will be appreciated.
Thanks In Advance.
Re: How to rebuild index
Posted by Erdinc Akkaya <hz...@gmail.com>.
First of all try it on different folder than your current index folder. new
analyzer will make different index but same data. First you should create
index on different folder than just replace your new index with current
index files. If it fits, then replace the code and it will work.
2011/1/21 黄靖宇 <qi...@gmail.com>
> Hmm, I see. Thanks very much.
>
> 2011/1/21 Uwe Schindler <uw...@thetaphi.de>
>
> > Hi,
> >
> > > "If you’re changing analyzers, you should rebuild your index using the
> > new analyzer so that all documents are analyzed in the same manner."
> >
> > It says everything: Take your original data and re-create the index.
> > Indexing is a lossy operation, so you must recreate the index using *all*
> > the original data like it was done the last 2 years. Just use another
> > analyzer.
> >
> > Uwe
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> > > -----Original Message-----
> > > From: 黄靖宇 [mailto:qingyujingyu@gmail.com]
> > > Sent: Friday, January 21, 2011 10:05 AM
> > > To: java-user@lucene.apache.org
> > > Subject: How to rebuild index
> > >
> > > Hi,
> > > I am new to lucene. Recently I was assigned for some lucene related
> > > workitems.
> > > Now there is one problem. Before, we use StandardAnalyzer in our
> > > application, and our application has been online for about two years.
> > > Now, we must to write a Custom Analyzer to replace the StandarAnalyzer
> > for
> > > enhancement requirement.
> > > But there are already much exists production data and the lucene
> > > index(generated with StandardAnalyzer).
> > > What I need to do if I change to use the new Custom Analyzer?
> > > From the book "Lucene In Action". I see follow sentence:
> > > "If you’re changing analyzers, you should rebuild your index using the
> > new
> > > analyzer so that all documents are analyzed in the same manner."
> > >
> > > But I don't know how to rebuild the exists index with a new Analyzer. I
> > can't
> > > find a API related to this.
> > > Any help will be appreciated.
> > > Thanks In Advance.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>
Re: How to rebuild index
Posted by 黄靖宇 <qi...@gmail.com>.
Hmm, I see. Thanks very much.
2011/1/21 Uwe Schindler <uw...@thetaphi.de>
> Hi,
>
> > "If you’re changing analyzers, you should rebuild your index using the
> new analyzer so that all documents are analyzed in the same manner."
>
> It says everything: Take your original data and re-create the index.
> Indexing is a lossy operation, so you must recreate the index using *all*
> the original data like it was done the last 2 years. Just use another
> analyzer.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
> > -----Original Message-----
> > From: 黄靖宇 [mailto:qingyujingyu@gmail.com]
> > Sent: Friday, January 21, 2011 10:05 AM
> > To: java-user@lucene.apache.org
> > Subject: How to rebuild index
> >
> > Hi,
> > I am new to lucene. Recently I was assigned for some lucene related
> > workitems.
> > Now there is one problem. Before, we use StandardAnalyzer in our
> > application, and our application has been online for about two years.
> > Now, we must to write a Custom Analyzer to replace the StandarAnalyzer
> for
> > enhancement requirement.
> > But there are already much exists production data and the lucene
> > index(generated with StandardAnalyzer).
> > What I need to do if I change to use the new Custom Analyzer?
> > From the book "Lucene In Action". I see follow sentence:
> > "If you’re changing analyzers, you should rebuild your index using the
> new
> > analyzer so that all documents are analyzed in the same manner."
> >
> > But I don't know how to rebuild the exists index with a new Analyzer. I
> can't
> > find a API related to this.
> > Any help will be appreciated.
> > Thanks In Advance.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
RE: How to rebuild index
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
> "If you’re changing analyzers, you should rebuild your index using the
new analyzer so that all documents are analyzed in the same manner."
It says everything: Take your original data and re-create the index.
Indexing is a lossy operation, so you must recreate the index using *all*
the original data like it was done the last 2 years. Just use another
analyzer.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: 黄靖宇 [mailto:qingyujingyu@gmail.com]
> Sent: Friday, January 21, 2011 10:05 AM
> To: java-user@lucene.apache.org
> Subject: How to rebuild index
>
> Hi,
> I am new to lucene. Recently I was assigned for some lucene related
> workitems.
> Now there is one problem. Before, we use StandardAnalyzer in our
> application, and our application has been online for about two years.
> Now, we must to write a Custom Analyzer to replace the StandarAnalyzer for
> enhancement requirement.
> But there are already much exists production data and the lucene
> index(generated with StandardAnalyzer).
> What I need to do if I change to use the new Custom Analyzer?
> From the book "Lucene In Action". I see follow sentence:
> "If you’re changing analyzers, you should rebuild your index using the
new
> analyzer so that all documents are analyzed in the same manner."
>
> But I don't know how to rebuild the exists index with a new Analyzer. I
can't
> find a API related to this.
> Any help will be appreciated.
> Thanks In Advance.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: How to rebuild index
Posted by Jack Krupansky <ja...@lucidimagination.com>.
The best thing is to re-index from your original source data, but if that is
not available, you can also re-index stored fields, assuming that you
created the index using stored fields for text fields. You would have to
write custom code to retrieve the stored values (not the actual terms since
those were analyzed and won't be guaranteed to be re-analyzed compatibly
with a different analyzer.) For non-text fields such as numbers and raw
strings you can of course just retrieve the raw term values and use them for
re-indexing. None of this is easy or automatic, but at least you can
retrieve the text to be re-indexed - if and only if it was indexed in stored
fields.
-- Jack Krupansky
-----Original Message-----
From: 黄靖宇
Sent: Friday, January 21, 2011 4:04 AM
To: java-user@lucene.apache.org
Subject: How to rebuild index
Hi,
I am new to lucene. Recently I was assigned for some lucene related
workitems.
Now there is one problem. Before, we use StandardAnalyzer in our
application, and our application has been online for about two years.
Now, we must to write a Custom Analyzer to replace the StandarAnalyzer for
enhancement requirement.
But there are already much exists production data and the lucene
index(generated with StandardAnalyzer).
What I need to do if I change to use the new Custom Analyzer?
>From the book "Lucene In Action". I see follow sentence:
"If you’re changing analyzers, you should rebuild your index using the new
analyzer so that all documents are analyzed in the same manner."
But I don't know how to rebuild the exists index with a new Analyzer. I
can't find a API related to this.
Any help will be appreciated.
Thanks In Advance.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: How to rebuild index
Posted by Lahiru Samarakoon <la...@gmail.com>.
HI
You were using a system for two years and it used an index created using
lucene with the StandardAnalyzer. So, There must be an index creation code
with your system.
Anyway,Since you have the book “*Lucene in action*” you can find how to
create an index by reading chapter 2 (Indexing). Please refer that section
and you will have a better understanding. It also contains the API details.
Thank you,
Best Regards,
Lahiru
On Fri, Jan 21, 2011 at 2:34 PM, 黄靖宇 <qi...@gmail.com> wrote:
> Hi,
> I am new to lucene. Recently I was assigned for some lucene related
> workitems.
> Now there is one problem. Before, we use StandardAnalyzer in our
> application, and our application has been online for about two years.
> Now, we must to write a Custom Analyzer to replace the StandarAnalyzer for
> enhancement requirement.
> But there are already much exists production data and the lucene
> index(generated with StandardAnalyzer).
> What I need to do if I change to use the new Custom Analyzer?
> From the book "Lucene In Action". I see follow sentence:
> "If you’re changing analyzers, you should rebuild your index using the new
> analyzer so that all documents are analyzed in the same manner."
>
> But I don't know how to rebuild the exists index with a new Analyzer. I
> can't find a API related to this.
> Any help will be appreciated.
> Thanks In Advance.
>