[jira] [Created] (FLINK-24460) Rocksdb Iterator Error Handling Improvement

["Victor Xu (Jira)" <ji...@apache.org>]

Victor Xu created FLINK-24460:
---------------------------------

             Summary: Rocksdb Iterator Error Handling Improvement
                 Key: FLINK-24460
                 URL: https://issues.apache.org/jira/browse/FLINK-24460
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / State Backends
    Affects Versions: 1.14.0
            Reporter: Victor Xu


In FLINK-9373, we introduced RocksIteratorWrapper which was a wrapper around RocksIterator to check the iterator status for all the methods. At that time, it was required because the iterator may pass the blocks or files it had difficulties in reading (because of IO errors, data corruptions, or other issues) and continue with the next available keys. *The status flag may not be OK, even if the iterator is valid.*

However, the above behaviour changed after [3810|https://github.com/facebook/rocksdb/pull/3810] was merged on May 17, 2018:

 *- If the iterator is valid, the status() is guaranteed to be OK;*

 *- If the iterator is not valid, there are two possibilities:*

    *1) We have reached the end of the data. And in this case, status() is OK;*

    *2) There is an error. In this case, status() is not OK;*

More information can be found here: https://github.com/facebook/rocksdb/wiki/Iterator#error-handling

Thus, it should be safe to proceed with other operations (e.g. seek, next, seekToFirst, seekToLast, seekForPrev, and prev) without checking status(). And we only need to check the status if the iterator is invalid. After the change, there will be less status() native calls and could theoretically improve performance.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)