You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Webster Homer <we...@milliporesigma.com> on 2020/03/06 19:08:39 UTC

Upgrading Solrcloud indexes from 7.2 to 8.4.1

We are looking at upgrading our Solrcoud  instances from 7.2 to the most recent version of solr 8.4.1 at this time. The last time we upgraded a major solr release we were able to upgrade the index files to the newer version, this prevented us from having an outage. Subsequently we've reindexed all our collections. However the Solr documentation for 8.4.1 states that we need to be at Solr 7.3 or later to run the index upgrade.  https://lucene.apache.org/solr/guide/8_4/solr-upgrade-notes.html

So if we upgrade to 7.7,  and then move to 8.4.1  and run the index upgrade script just once?
I guess I'm confused about the 7.2 -> 8.* issue is it data related?

Regards,
Webster



This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith.



Click http://www.merckgroup.com/disclaimer to access the German, French, Spanish and Portuguese versions of this disclaimer.

Re: Upgrading Solrcloud indexes from 7.2 to 8.4.1

Posted by Erick Erickson <er...@gmail.com>.
When you say “reindexed”, how exactly was that done? Because if you didn’t start from an empty index, you will have to re-index from scratch to use 8x.

Starting with 6.x, a marker is written whenever a segment is created indicating what version was used. Whenever two or more segments are merged, the earliest marker is preserved. Then, starting with Solr 8x, if any segment has a marker more than one major revision old, Lucene will refuse to open the index.

So when you say “reindexed” and all you did was re-index every doc into an existing index, the 6x marker is preserved and you’ll be out of luck.

IndexUpgraderTool, BTW, merely does an optimize to one segment, which has it’s own downsides, see: https://lucidworks.com/post/segment-merging-deleted-documents-optimize-may-bad/ and the associated post for 7.5+.

It doesn’t (and can’t) make the index look just like the current index, so Lucene won’t try. Paraphrasing Robert Muir, Lucene stores an efficient, derived index rather than the raw data. So what’s in the index is y, where y=f(x). Since x may not be present, Lucene can’t recreate y when major versions change. 

Best,
Erick

> On Mar 6, 2020, at 11:08, Webster Homer <we...@milliporesigma.com> wrote:
> 
> We are looking at upgrading our Solrcoud  instances from 7.2 to the most recent version of solr 8.4.1 at this time. The last time we upgraded a major solr release we were able to upgrade the index files to the newer version, this prevented us from having an outage. Subsequently we've reindexed all our collections. However the Solr documentation for 8.4.1 states that we need to be at Solr 7.3 or later to run the index upgrade.  https://lucene.apache.org/solr/guide/8_4/solr-upgrade-notes.html
> 
> So if we upgrade to 7.7,  and then move to 8.4.1  and run the index upgrade script just once?
> I guess I'm confused about the 7.2 -> 8.* issue is it data related?
> 
> Regards,
> Webster
> 
> 
> 
> This message and any attachment are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person. If you have received this transmission in error, please notify the sender immediately and delete the message and any attachment from your system. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not accept liability for any omissions or errors in this message which may arise as a result of E-Mail-transmission or for damages resulting from any unauthorized changes of the content of this message and any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its subsidiaries do not guarantee that this message is free of viruses and does not accept liability for any damages caused by any virus transmitted therewith.
> 
> 
> 
> Click http://www.merckgroup.com/disclaimer to access the German, French, Spanish and Portuguese versions of this disclaimer.

Re: Upgrading Solrcloud indexes from 7.2 to 8.4.1

Posted by Dave <ha...@gmail.com>.
You best off doing a full reindex to a single solr cloud 8.x node and then when done start taking down 7.x nodes, upgrade them to 8.x and add them to the new cluster. upgrading indexes has so many potential issues, 

> On Mar 6, 2020, at 9:21 PM, lstusr 5u93n4 <ls...@gmail.com> wrote:
> 
> Hi Webster,
> 
> When we upgraded from 7.5 to 8.1 we ran into a very strange issue:
> https://lucene.472066.n3.nabble.com/Stored-field-values-don-t-update-after-7-gt-8-upgrade-td4442934.html
> 
> 
> We ended up having to do a full re-index to solve this issue, but if you're
> going to do this upgrade I would love to know if this issue shows up for
> you too. At the very least, I'd suggest doing some variant of the test
> outlined in that post, so you can be confident in your data integrity.
> 
> Kyle
> 
>> On Fri, 6 Mar 2020 at 14:08, Webster Homer <we...@milliporesigma.com>
>> wrote:
>> 
>> We are looking at upgrading our Solrcoud  instances from 7.2 to the most
>> recent version of solr 8.4.1 at this time. The last time we upgraded a
>> major solr release we were able to upgrade the index files to the newer
>> version, this prevented us from having an outage. Subsequently we've
>> reindexed all our collections. However the Solr documentation for 8.4.1
>> states that we need to be at Solr 7.3 or later to run the index upgrade.
>> https://lucene.apache.org/solr/guide/8_4/solr-upgrade-notes.html
>> 
>> So if we upgrade to 7.7,  and then move to 8.4.1  and run the index
>> upgrade script just once?
>> I guess I'm confused about the 7.2 -> 8.* issue is it data related?
>> 
>> Regards,
>> Webster
>> 
>> 
>> 
>> This message and any attachment are confidential and may be privileged or
>> otherwise protected from disclosure. If you are not the intended recipient,
>> you must not copy this message or attachment or disclose the contents to
>> any other person. If you have received this transmission in error, please
>> notify the sender immediately and delete the message and any attachment
>> from your system. Merck KGaA, Darmstadt, Germany and any of its
>> subsidiaries do not accept liability for any omissions or errors in this
>> message which may arise as a result of E-Mail-transmission or for damages
>> resulting from any unauthorized changes of the content of this message and
>> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
>> subsidiaries do not guarantee that this message is free of viruses and does
>> not accept liability for any damages caused by any virus transmitted
>> therewith.
>> 
>> 
>> 
>> Click http://www.merckgroup.com/disclaimer to access the German, French,
>> Spanish and Portuguese versions of this disclaimer.
>> 

Re: Upgrading Solrcloud indexes from 7.2 to 8.4.1

Posted by lstusr 5u93n4 <ls...@gmail.com>.
Hi Webster,

When we upgraded from 7.5 to 8.1 we ran into a very strange issue:
https://lucene.472066.n3.nabble.com/Stored-field-values-don-t-update-after-7-gt-8-upgrade-td4442934.html


We ended up having to do a full re-index to solve this issue, but if you're
going to do this upgrade I would love to know if this issue shows up for
you too. At the very least, I'd suggest doing some variant of the test
outlined in that post, so you can be confident in your data integrity.

Kyle

On Fri, 6 Mar 2020 at 14:08, Webster Homer <we...@milliporesigma.com>
wrote:

> We are looking at upgrading our Solrcoud  instances from 7.2 to the most
> recent version of solr 8.4.1 at this time. The last time we upgraded a
> major solr release we were able to upgrade the index files to the newer
> version, this prevented us from having an outage. Subsequently we've
> reindexed all our collections. However the Solr documentation for 8.4.1
> states that we need to be at Solr 7.3 or later to run the index upgrade.
> https://lucene.apache.org/solr/guide/8_4/solr-upgrade-notes.html
>
> So if we upgrade to 7.7,  and then move to 8.4.1  and run the index
> upgrade script just once?
> I guess I'm confused about the 7.2 -> 8.* issue is it data related?
>
> Regards,
> Webster
>
>
>
> This message and any attachment are confidential and may be privileged or
> otherwise protected from disclosure. If you are not the intended recipient,
> you must not copy this message or attachment or disclose the contents to
> any other person. If you have received this transmission in error, please
> notify the sender immediately and delete the message and any attachment
> from your system. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not accept liability for any omissions or errors in this
> message which may arise as a result of E-Mail-transmission or for damages
> resulting from any unauthorized changes of the content of this message and
> any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
> subsidiaries do not guarantee that this message is free of viruses and does
> not accept liability for any damages caused by any virus transmitted
> therewith.
>
>
>
> Click http://www.merckgroup.com/disclaimer to access the German, French,
> Spanish and Portuguese versions of this disclaimer.
>