[jira] [Created] (FLINK-10954) Hardlink from files of previous local stored state might cross devices

["Yun Tang (JIRA)" <ji...@apache.org>]

Yun Tang created FLINK-10954:
--------------------------------

             Summary: Hardlink from files of previous local stored state might cross devices
                 Key: FLINK-10954
                 URL: https://issues.apache.org/jira/browse/FLINK-10954
             Project: Flink
          Issue Type: Bug
          Components: State Backends, Checkpointing
    Affects Versions: 1.6.2
            Reporter: Yun Tang
             Fix For: 1.6.3, 1.8.0, 1.7.1


Currently, local recovery's base directories is initialized from '{{io.tmp.dirs}}' if parameter '{{taskmanager.state.local.root-dirs}}' is not set. For Yarn environment, the tmp dirs is replaced by its '{{LOCAL_DIRS}}', which might consist of directories from different devices, such as /dump/1/nm-local-dir, /dump/2/nm-local-dir. The local directory for RocksDB is initialized from IOManager's spillingDirectories, which might located in different device from local recovery's folder. However, hard-link between different devices is not allowed, it will throw exception below:

{code:java}
java.nio.file.FileSystemException: target -> souce: Invalid cross-device link
{code}





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)