[jira] [Commented] (SOLR-8119) Detect index corruption for all files on replication

["Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SOLR-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942562#comment-14942562 ] 

Shalin Shekhar Mangar commented on SOLR-8119:
---------------------------------------------

Note that there is some API mismatch in the way we replicate vs how we can validate the checksums e.g. FastInputStream vs IndexInput etc so a good amount of refactoring may be necessary.

> Detect index corruption for all files on replication
> ----------------------------------------------------
>
>                 Key: SOLR-8119
>                 URL: https://issues.apache.org/jira/browse/SOLR-8119
>             Project: Solr
>          Issue Type: Improvement
>          Components: replication (java)
>            Reporter: Shalin Shekhar Mangar
>              Labels: difficulty-medium, impact-high
>             Fix For: Trunk, 5.4
>
>
> Lucene writes checksums for large files but they aren't verified until a merge is necessary because it'd be too costly to go through the entire bytes. Only truncation of such files is checked during open. However, index replication is one activity that has to go through the entire file anyway so we can be more aggressive than Lucene in validating the checksum.
> I propose that we validate all files, large and small, during replication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org