migration from 5 to 8

["Stanonik, Ronald" <rs...@ucsd.edu.INVALID>]

We used the solr Upgrader and were able to migrate from 5 to 6 to 7.
For example
java -cp lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar org.apache.lucene.index.IndexUpgrader -delete-prior-commits -verbose $index

After each migration we used the CheckIndex to confirm and indeed it reported expected version.
For example
java -ea:org.apache.lucene... -cp lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar org.apache.lucene.index.CheckIndex

When we tried to upgrade from 7 to 8, that failed with a complaint that the index was still version 6 and updating could only be from the previious (ie 7) version.
Indeed running the 8 CheckIndex reported the index was version 6, even though the 7 CheckIndex reported version 7.

Oh well, so we're trying another approach.

Using the fieldtypes, fields, dynamicfileds, and copyfields API, we copy the schema from 5 to 8.

Then we extract the documents as json from 5 and update the json to 8.
For example
curl -X GET "$SRC/select?q=*%3A*&wt=json&indent=true&start=$start&rows=10" > $tmp 2>/dev/null
jq ".response.docs" <$tmp >$tmp2
curl -X POST -H 'Content-type:application/json' --data-binary @$tmp2 "$DST/update/json/docs?commit=true"

The documents are showing up in 8.  We still need to perform some validation, but are we overlooking some obvious reason, why this might fail?

Thanks,

Ron

Re: migration from 5 to 8

[Jan Høydahl <ja...@cominvent.com>]

Doing a full re-index into a clean solr 8 install is the recommended approach. So if you have all fields stored (or docValues) and manage to dump complete documents, then you should be all set.
You should be able to page through your entire index with ordinary &start= and &rows= params, but if you have a huge index, look into the more efficient cursorMark instead.

Jan

> 19. feb. 2022 kl. 22:12 skrev Stanonik, Ronald <rs...@ucsd.edu.INVALID>:
> 
> We used the solr Upgrader and were able to migrate from 5 to 6 to 7.
> For example
> java -cp lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar org.apache.lucene.index.IndexUpgrader -delete-prior-commits -verbose $index
> 
> After each migration we used the CheckIndex to confirm and indeed it reported expected version.
> For example
> java -ea:org.apache.lucene... -cp lucene-core-7.7.3.jar:lucene-backward-codecs-7.7.3.jar org.apache.lucene.index.CheckIndex
> 
> When we tried to upgrade from 7 to 8, that failed with a complaint that the index was still version 6 and updating could only be from the previious (ie 7) version.
> Indeed running the 8 CheckIndex reported the index was version 6, even though the 7 CheckIndex reported version 7.
> 
> Oh well, so we're trying another approach.
> 
> Using the fieldtypes, fields, dynamicfileds, and copyfields API, we copy the schema from 5 to 8.
> 
> Then we extract the documents as json from 5 and update the json to 8.
> For example
> curl -X GET "$SRC/select?q=*%3A*&wt=json&indent=true&start=$start&rows=10" > $tmp 2>/dev/null
> jq ".response.docs" <$tmp >$tmp2
> curl -X POST -H 'Content-type:application/json' --data-binary @$tmp2 "$DST/update/json/docs?commit=true"
> 
> The documents are showing up in 8.  We still need to perform some validation, but are we overlooking some obvious reason, why this might fail?
> 
> Thanks,
> 
> Ron
> 
>