You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Erick Erickson (JIRA)" <ji...@apache.org> on 2018/04/22 03:34:00 UTC
[jira] [Created] (LUCENE-8264) Allow an option to rewrite all
segments
Erick Erickson created LUCENE-8264:
--------------------------------------
Summary: Allow an option to rewrite all segments
Key: LUCENE-8264
URL: https://issues.apache.org/jira/browse/LUCENE-8264
Project: Lucene - Core
Issue Type: Improvement
Reporter: Erick Erickson
Assignee: Erick Erickson
For the background, see SOLR-12259.
There are several use-cases that would be much easier, especially during upgrades, if we could specify that all segments get rewritten.
One example: Upgrading 5x->6x->7x. When segments are merged, they're rewritten into the current format. However, there's no guarantee that a particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily be successful.
How many merge policies support this is an open question. I propose to start with TMP and raise other JIRAs as necessary for other merge policies.
So far the usual response has been "re-index from scratch", but that's increasingly difficult as systems get larger.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Created] (LUCENE-8264) Allow an option to rewrite
all segments
Posted by Yago Riveiro <ya...@gmail.com>.
Hi Erick,
“re-index from scratch” was always the main concern in every mayor update, our cluster has ~15T of data and re-index without downtime is always an epic task, not only in time but also in temporal resources to have all responsive.
It’s difficult for me justify to my boss that every time we hit a bug, we need to upgrade to a mayor version and do a full re-index.
If this issue can resolve some of the pains of upgrading to a mayor version, will be very welcome.
Regards
--
Yago Riveiro
On 22 Apr 2018 04:34 +0100, Erick Erickson (JIRA) <ji...@apache.org>, wrote:
> Erick Erickson created LUCENE-8264:
> --------------------------------------
>
> Summary: Allow an option to rewrite all segments
> Key: LUCENE-8264
> URL: https://issues.apache.org/jira/browse/LUCENE-8264
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Erick Erickson
> Assignee: Erick Erickson
>
>
> For the background, see SOLR-12259.
>
> There are several use-cases that would be much easier, especially during upgrades, if we could specify that all segments get rewritten.
>
> One example: Upgrading 5x->6x->7x. When segments are merged, they're rewritten into the current format. However, there's no guarantee that a particular segment _ever_ gets merged so the 6x-7x upgrade won't necessarily be successful.
>
> How many merge policies support this is an open question. I propose to start with TMP and raise other JIRAs as necessary for other merge policies.
>
> So far the usual response has been "re-index from scratch", but that's increasingly difficult as systems get larger.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>