[jira] [Commented] (LUCENE-9234) Keep write support for old codecs?

["Robert Muir (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LUCENE-9234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039424#comment-17039424 ] 

Robert Muir commented on LUCENE-9234:
-------------------------------------

I don't think we should do this. Having to write not just N but N-1 and support reads for those writes later is too much.
This seems to be a situation where the distributed systems are too lazy to have enough nodes or the correct logic.

{quote}
I understand this change would not come for free, especially when it comes to testing
{quote}

If this is the case, then perhaps offer a "concrete bargain" from the distributed systems side. Personally I assume they are just lazy, and trying to force the work on lucene (simply look at their tests for inconclusive proof of this!). So I would like to know what they would be willing to tradeoff in return. For example, solr tests running successfully in 5 minutes on my machine?

> Keep write support for old codecs?
> ----------------------------------
>
>                 Key: LUCENE-9234
>                 URL: https://issues.apache.org/jira/browse/LUCENE-9234
>             Project: Lucene - Core
>          Issue Type: Wish
>            Reporter: Adrien Grand
>            Priority: Minor
>
> Currenty we maintain read/write support for the latest codec in lucene/core, and read-only support for codecs of previous versions (up to \{N-1\}.0}) in  lucene/backward-codecs. We often keep write support in test-framework for testing purposes only.
> This raises challenges for Elasticsearch with regard to rolling upgrades: we have some users who index very large amounts of data on clusters that are quite large, so that rolling upgrades take significant time. Meanwhile, several indices may be created.
> Allocating indices when the cluster has nodes of different versions requires care as Lucene indices created on nodes with a newer version cannot be read by the nodes running the older version. It is possible to force primary replicas to be allocated on the older nodes, but this brings other problems like availability, uneven disk usage across nodes, or moving a lot of data around.
> If Lucene could write data using the minimum version that exists in the cluster, this would avoid this problem as the written data could be read by any node of the cluster. I understand this change would not come for free, especially when it comes to testing as we'd need to make sure that older Lucene versions can read indices created by this "compatibility mode".
> I'd be curious to understand whether this is a problem for Solr too, if not how this problem is being handled, and maybe whether there are other problems that you have encountered that would also benefit from the ability to write data with an older format.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org