You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Jingsong Lee (Jira)" <ji...@apache.org> on 2020/06/04 12:57:00 UTC

[jira] [Created] (FLINK-18130) File name conflict for different jobs in filesystem/hive sink

Jingsong Lee created FLINK-18130:
------------------------------------

             Summary: File name conflict for different jobs in filesystem/hive sink
                 Key: FLINK-18130
                 URL: https://issues.apache.org/jira/browse/FLINK-18130
             Project: Flink
          Issue Type: Bug
          Components: Connectors / FileSystem, Connectors / Hive
            Reporter: Jingsong Lee
            Assignee: Jingsong Lee
             Fix For: 1.11.0


The sink of different jobs (or the same SQL runs multiple times) will produce the same file name, the rename will fail, and different file systems will produce different results:
 * HadoopFileSystem: ignore new data.
 * LocalFileSystem: overwrite old data.

We can add UUID to prefix, make sure there is no conflict between jobs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)