You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Artjoms Laivins <Ar...@evry.com> on 2018/08/21 08:29:20 UTC
Index Upgrader tool
Hello,
We are running Solr cloud with 3 nodes v. 6.6.2
We started with version 5 so we have some old index that we need safely move over to v. 7 now.
New data comes in several times per day.
Our questions are:
Should we run IndexUpgrader tool on one slave node that is down or it is safe to run it while Solr is running and possible updates of the index are coming?
If yes, when we start it again will leader update this node with new data only or will it overwrite index?
Best Regards,
Artjoms Laivins
Re: Index Upgrader tool
Posted by Shawn Heisey <ap...@elyograg.org>.
On 8/24/2018 12:44 AM, damienk@gmail.com wrote:
> Shawn, Is it possible to run optimize on the live collection? For example,
> /solr/collection/update?commit=true&optimize=true
For all the reasons in the blog post that Erick referenced, we recommend
that you do not do this.
Something to note: That optimize operation is *EXACTLY* what
IndexUpgrader does. So when it comes right down to it, I wouldn't
recommend using IndexUpgrader either.
Solr 6 can read Solr 5 indexes directly, no "upgrade" required. But as
you've found, if you take a version 5 index and upgrade it to 6 (which
is really just an optimize/forceMerge), then try to upgrade it again to
version 7, it may not work.
The gist of all this is that I do not recommend using indexes from a
previous version AT ALL. You should build the indexes from scratch
using the new version.
Thanks,
Shawn
Re: Index Upgrader tool
Posted by Erick Erickson <er...@gmail.com>.
Yes, it's possible to run optimize on a live index. I wouldn't though, see:
https://lucidworks.com/2017/10/13/segment-merging-deleted-documents-optimize-may-bad/
Lucene has _never_ guaranteed proper functioning of an index created
with version X-2 with
version X. It hasn't been super-obvious, but we had a long discussion
about it on here:
https://issues.apache.org/jira/browse/LUCENE-8264
but here's the most succinct statement of why upgrading more than one
major version isn't
really possible from Robert Muir:
"I think the key issue here is Lucene is an index not a database.
Because it is a lossy index and does not retain all of the user's
data, its not possible to safely migrate some things automagically."
So Lucene labors heroically to maintain 1 version back-compat, but
that's all that's guaranteed.
So I'd _really_ recommend you reindex if at all possible.
Best,
Erick
Best,
Erick
On Thu, Aug 23, 2018 at 11:45 PM <da...@gmail.com> wrote:
>
> Shawn, Is it possible to run optimize on the live collection? For example,
> /solr/collection/update?commit=true&optimize=true
>
> On Wed, 22 Aug 2018 at 06:50, Shawn Heisey <ap...@elyograg.org> wrote:
>
> > On 8/21/2018 2:29 AM, Artjoms Laivins wrote:
> > > We are running Solr cloud with 3 nodes v. 6.6.2
> > > We started with version 5 so we have some old index that we need safely
> > move over to v. 7 now.
> > > New data comes in several times per day.
> > > Our questions are:
> > >
> > > Should we run IndexUpgrader tool on one slave node that is down or it is
> > safe to run it while Solr is running and possible updates of the index are
> > coming?
> > > If yes, when we start it again will leader update this node with new
> > data only or will it overwrite index?
> >
> > It might not be possible to upgrade two major versions like that, even
> > with IndexUpgrader. There is only a guarantee of reading an index
> > ORIGINALLY written by the previous major version.
> >
> > Even if it's possible to accomplish an upgrade, it is strongly
> > recommended that you index from scratch anyway.
> >
> > You cannot run IndexUpgrader while Solr has the index open. The index
> > must be completely closed. You cannot update an index while it is being
> > upgraded.
> >
> > Thanks,
> > Shawn
> >
> >
Re: Index Upgrader tool
Posted by da...@gmail.com.
Shawn, Is it possible to run optimize on the live collection? For example,
/solr/collection/update?commit=true&optimize=true
On Wed, 22 Aug 2018 at 06:50, Shawn Heisey <ap...@elyograg.org> wrote:
> On 8/21/2018 2:29 AM, Artjoms Laivins wrote:
> > We are running Solr cloud with 3 nodes v. 6.6.2
> > We started with version 5 so we have some old index that we need safely
> move over to v. 7 now.
> > New data comes in several times per day.
> > Our questions are:
> >
> > Should we run IndexUpgrader tool on one slave node that is down or it is
> safe to run it while Solr is running and possible updates of the index are
> coming?
> > If yes, when we start it again will leader update this node with new
> data only or will it overwrite index?
>
> It might not be possible to upgrade two major versions like that, even
> with IndexUpgrader. There is only a guarantee of reading an index
> ORIGINALLY written by the previous major version.
>
> Even if it's possible to accomplish an upgrade, it is strongly
> recommended that you index from scratch anyway.
>
> You cannot run IndexUpgrader while Solr has the index open. The index
> must be completely closed. You cannot update an index while it is being
> upgraded.
>
> Thanks,
> Shawn
>
>
Re: Index Upgrader tool
Posted by Shawn Heisey <ap...@elyograg.org>.
On 8/21/2018 2:29 AM, Artjoms Laivins wrote:
> We are running Solr cloud with 3 nodes v. 6.6.2
> We started with version 5 so we have some old index that we need safely move over to v. 7 now.
> New data comes in several times per day.
> Our questions are:
>
> Should we run IndexUpgrader tool on one slave node that is down or it is safe to run it while Solr is running and possible updates of the index are coming?
> If yes, when we start it again will leader update this node with new data only or will it overwrite index?
It might not be possible to upgrade two major versions like that, even
with IndexUpgrader. There is only a guarantee of reading an index
ORIGINALLY written by the previous major version.
Even if it's possible to accomplish an upgrade, it is strongly
recommended that you index from scratch anyway.
You cannot run IndexUpgrader while Solr has the index open. The index
must be completely closed. You cannot update an index while it is being
upgraded.
Thanks,
Shawn