You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Yu Li (Jira)" <ji...@apache.org> on 2019/12/10 07:15:00 UTC

[jira] [Resolved] (FLINK-14378) Cleanup rocksDB lib folder if fail to load library

     [ https://issues.apache.org/jira/browse/FLINK-14378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yu Li resolved FLINK-14378.
---------------------------
    Resolution: Fixed

Merged via: 0d9058c56e4f420d1c501e5aef981433a96650dd
(Thanks [~sewen] for help review and merge this)

> Cleanup rocksDB lib folder if fail to load library
> --------------------------------------------------
>
>                 Key: FLINK-14378
>                 URL: https://issues.apache.org/jira/browse/FLINK-14378
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / State Backends
>            Reporter: Yun Tang
>            Assignee: Yun Tang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.10.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> This improvement is inspired due to some of our machines need some time to load the rocksDB library. When some other unrecoverable exceptions continue to happen and the process to load library would be interrupted which cause the {{rocksdb-lib}} folder created but not cleaned up. As the job continues to failover, the {{rocksdb-lib}} folder would be created more and more. We even come across that machine was running out of inodes!
> Details could refer to current [implementation|https://github.com/apache/flink/blob/80b27a150026b7b5cb707bd9fa3e17f565bb8112/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackend.java#L860]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)