You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Ravi Solr <ra...@gmail.com> on 2011/05/09 19:24:02 UTC
Solr 3.1 Upgrade - Reindex necessary ?
Hello All,
I am planning to upgrade from Solr 1.4.1 to Solr 3.1. I
saw some deprecation warnings in the log as shown below
[#|2011-05-09T12:37:18.762-0400|WARNING|sun-appserver9.1|org.apache.solr.analysis.BaseTokenStreamFactory|_ThreadID=53;_ThreadName=httpSSLWorkerThread-9001-13
;_RequestID=de32fd3f-e968-4228-a071-9bb175bfb549;|StopFilterFactory is
using deprecated LUCENE_24 emulation. You should at some point declare
and reindex to
at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0|#]
[#|2011-05-09T12:37:18.765-0400|WARNING|sun-appserver9.1|org.apache.solr.analysis.BaseTokenStreamFactory|_ThreadID=53;_ThreadName=httpSSLWorkerThread-9001-13
;_RequestID=de32fd3f-e968-4228-a071-9bb175bfb549;|WordDelimiterFilterFactory
is using deprecated LUCENE_24 emulation. You should at some point
declare and re
index to at least 3.0, because 2.x emulation is deprecated and will be
removed in 4.0|#]
[#|2011-05-09T12:37:18.767-0400|WARNING|sun-appserver9.1|org.apache.solr.analysis.BaseTokenStreamFactory|_ThreadID=53;_ThreadName=httpSSLWorkerThread-9001-13
;_RequestID=de32fd3f-e968-4228-a071-9bb175bfb549;|EnglishPorterFilterFactory
is using deprecated LUCENE_24 emulation. You should at some point
declare and re
index to at least 3.0, because 2.x emulation is deprecated and will be
removed in 4.0|#]
so I would love the experts advise on the following questions
1. Do we have to reindex all content again to use Solr 3.1 ?
2. If we don't reindex all content are there any potential issues ? (I
read somewhere that first commit would change the 1.4.1 format to 3.1.
have the analyzer's behavior changed which warrants reindexing ?)
3. Apart from deploying the new solr 3.1 war; Is it just enough to set
"<luceneMatchVersion>LUCENE_31</luceneMatchVersion>" to get all the
goodies and bug fixes of the LUCENE/SOLR 3.1 ?
Thank You,
Ravi Kiran Bhaskar
Re: Solr 3.1 Upgrade - Reindex necessary ?
Posted by Ravi Solr <ra...@gmail.com>.
Hoss,
Thank you very much for clearly delineating the difference.
Just to be clear - My intent to move to 3.1 was driven by my desire to
improve my replication performance - Deducing from your explanation, I
believe the replication/indexing related changes/bug fixes like the
following will be available to me even without specifying
"<luceneMatchVersion>LUCENE_31</luceneMatchVersion>" am I right ??
faster exact PhraseQuery; merging favors segments with deletions;
primary key lookup is faster; IndexWriter.addIndexes(Directory[]) uses
file copy instead of merging; various Directory performance
improvements; compound file is dynamically turned off for large
segments; fully deleted segments are dropped on commit; faster
snowball analyzers (in contrib); ConcurrentMergeScheduler is more
careful about setting priority of merge threads.
Ravi Kiran Bhaskar
On Tue, May 10, 2011 at 2:49 PM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : Thanks Grijesh for responding. I meant that I will use the Lucene 3.1
> : jars for indexing also from now on. My current index already has a
> : million docs indexed with solr 1.4.1 version, I read somewhere that
> : once server is upgraded to 3.1, it is said that the first commit will
> : change the indexes to 3.1 format automatically. Is this true or do I
> : have to literally reindex the million docs again ?
>
> index versioning happens on a segment basis, so once you start using Solr
> 3.1, as new docs are added and segments are merged those segments will be
> updated to the new file format -- the way to ensure that "all" segments
> are updated is to optimize your index.
>
> : >>1. Do we have to reindex all content again to use Solr 3.1 ?
>
> you should not need to, know.
>
> : >>3. Apart from deploying the new solr 3.1 war; Is it just enough to set
> : >>"<luceneMatchVersion>LUCENE_31</luceneMatchVersion>" to get all the
> : >>goodies and bug fixes of the LUCENE/SOLR 3.1 ?
>
> It's not mandatory to change the <luceneMatchVersion/> to upgrade -- if
> you do want to change the <luceneMatchVersion/> then you should reindex,
> as that change causes analyzers/query parsers to behave differently (in
> ways thta might be incompatible with how they behave previously.
>
> this change is unrelated to the index fileformat -- optimizing your index
> to force the 3.1 fileformat has no impact on how what esoteric/broken
> behavior a tokenizer might have had in the past that changed once the
> <luceneMatchVersion/> setting is updated.
>
> The purpose of <luceneMatchVersion/> is to say "i want the behavior of
> X.Y, even when it's been decided that that behavior was bad, because it's
> what matches the terms i've already indexed"
>
>
> -Hoss
Re: Solr 3.1 Upgrade - Reindex necessary ?
Posted by Chris Hostetter <ho...@fucit.org>.
: Thanks Grijesh for responding. I meant that I will use the Lucene 3.1
: jars for indexing also from now on. My current index already has a
: million docs indexed with solr 1.4.1 version, I read somewhere that
: once server is upgraded to 3.1, it is said that the first commit will
: change the indexes to 3.1 format automatically. Is this true or do I
: have to literally reindex the million docs again ?
index versioning happens on a segment basis, so once you start using Solr
3.1, as new docs are added and segments are merged those segments will be
updated to the new file format -- the way to ensure that "all" segments
are updated is to optimize your index.
: >>1. Do we have to reindex all content again to use Solr 3.1 ?
you should not need to, know.
: >>3. Apart from deploying the new solr 3.1 war; Is it just enough to set
: >>"<luceneMatchVersion>LUCENE_31</luceneMatchVersion>" to get all the
: >>goodies and bug fixes of the LUCENE/SOLR 3.1 ?
It's not mandatory to change the <luceneMatchVersion/> to upgrade -- if
you do want to change the <luceneMatchVersion/> then you should reindex,
as that change causes analyzers/query parsers to behave differently (in
ways thta might be incompatible with how they behave previously.
this change is unrelated to the index fileformat -- optimizing your index
to force the 3.1 fileformat has no impact on how what esoteric/broken
behavior a tokenizer might have had in the past that changed once the
<luceneMatchVersion/> setting is updated.
The purpose of <luceneMatchVersion/> is to say "i want the behavior of
X.Y, even when it's been decided that that behavior was bad, because it's
what matches the terms i've already indexed"
-Hoss
Re: Solr 3.1 Upgrade - Reindex necessary ?
Posted by Ravi Solr <ra...@gmail.com>.
Thanks Grijesh for responding. I meant that I will use the Lucene 3.1
jars for indexing also from now on. My current index already has a
million docs indexed with solr 1.4.1 version, I read somewhere that
once server is upgraded to 3.1, it is said that the first commit will
change the indexes to 3.1 format automatically. Is this true or do I
have to literally reindex the million docs again ?
Thanks,
Ravi Kiran Bhaskar
On Tuesday, May 10, 2011, Grijesh <pi...@gmail.com> wrote:
>>1. Do we have to reindex all content again to use Solr 3.1 ?
>
>>2. If we don't reindex all content are there any potential issues ? (I
>>read somewhere that first commit would change the 1.4.1 format to 3.1.
>>have the analyzer's behavior changed which warrants reindexing ?)
>>3. Apart from deploying the new solr 3.1 war; Is it just enough to set
>>"<luceneMatchVersion>LUCENE_31</luceneMatchVersion>" to get all the
>>goodies and bug fixes of the LUCENE/SOLR 3.1 ?
>
> HI Solr-3.1 version usage the the latest version of Lucene jars so if you
> are planning to Upgrade then it is necessary to Re index all the content
> with Solr3.1 version.
>
> Not re-indexing will possibly cause of index corruption because newer
> version of lucene will create indexes in Newer version which is backward
> compatible for read only.
>
> setting <luceneMatchVersion>LUCENE_31</luceneMatchVersion> is not enough
> because it will not get the lucene 3.1 jar automatically.
>
> -----
> Thanx:
> Grijesh
> www.gettinhahead.co.in
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Solr-3-1-Upgrade-Reindex-necessary-tp2919679p2922645.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Solr 3.1 Upgrade - Reindex necessary ?
Posted by Grijesh <pi...@gmail.com>.
>1. Do we have to reindex all content again to use Solr 3.1 ?
>2. If we don't reindex all content are there any potential issues ? (I
>read somewhere that first commit would change the 1.4.1 format to 3.1.
>have the analyzer's behavior changed which warrants reindexing ?)
>3. Apart from deploying the new solr 3.1 war; Is it just enough to set
>"<luceneMatchVersion>LUCENE_31</luceneMatchVersion>" to get all the
>goodies and bug fixes of the LUCENE/SOLR 3.1 ?
HI Solr-3.1 version usage the the latest version of Lucene jars so if you
are planning to Upgrade then it is necessary to Re index all the content
with Solr3.1 version.
Not re-indexing will possibly cause of index corruption because newer
version of lucene will create indexes in Newer version which is backward
compatible for read only.
setting <luceneMatchVersion>LUCENE_31</luceneMatchVersion> is not enough
because it will not get the lucene 3.1 jar automatically.
-----
Thanx:
Grijesh
www.gettinhahead.co.in
--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-3-1-Upgrade-Reindex-necessary-tp2919679p2922645.html
Sent from the Solr - User mailing list archive at Nabble.com.