[jira] [Resolved] (LUCENE-7113) OfflineSorter and BKD should verify checksums in their temp files

["Michael McCandless (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LUCENE-7113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless resolved LUCENE-7113.
----------------------------------------
    Resolution: Fixed

> OfflineSorter and BKD should verify checksums in their temp files
> -----------------------------------------------------------------
>
>                 Key: LUCENE-7113
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7113
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: master, 6.0
>
>         Attachments: LUCENE-7113.patch, LUCENE-7113.patch
>
>
> I am trying to index all 3.2 B points from the latest OpenStreetMaps export.
> My SSDs were not up to this, so I added a spinning magnets disk to beast2.
> But then I was hitting scary bug-like exceptions ({{ArrayIndexOutOfBoundsException}}) when indexing the first 2B points, and I finally checked dmesg and saw that my hard drive is dying.
> I think it's important that our temp file usages also validate checksums (like we do for all our index files, either at reader open or at merge or {{CheckIndex}}), so we can hopefully easily differentiate a bit-flipping IO system from a possible Lucene bug, in the future.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org