[jira] [Created] (FLINK-15098) CsvTableSink has no properties of a OVERWRITE mode to write, but the exception hint the user use it when a file or directory already exists

["xiaojin.wy (Jira)" <ji...@apache.org>]

xiaojin.wy created FLINK-15098:
----------------------------------

             Summary: CsvTableSink has no properties of a OVERWRITE mode to write, but the exception hint the user use it when a file or directory already exists
                 Key: FLINK-15098
                 URL: https://issues.apache.org/jira/browse/FLINK-15098
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / Client
    Affects Versions: 1.10.0
            Reporter: xiaojin.wy


When using sql-client to execute a sql , there will be a IOException just like the content below if the file or directory already exists. But there is no place for the user to set the overwrite mode. In another word, the hint tell users to use overwrite mode, but flink has not support it now.

 

 

The exception:

Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot initialize task 'HashAggregate(isMerge=[true], select=[Final_AVG(sum$0, count$1) AS avg_1]) -> SinkConversionToRow -> Map -> Sink: CsvTableSink(avg_1)': File or directory already exists. Existing files and directories are not overwritten in NO_OVERWRITE mode. Use OVERWRITE mode to overwrite existing files and directories.Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot initialize task 'HashAggregate(isMerge=[true], select=[Final_AVG(sum$0, count$1) AS avg_1]) -> SinkConversionToRow -> Map -> Sink: CsvTableSink(avg_1)': File or directory already exists. Existing files and directories are not overwritten in NO_OVERWRITE mode. Use OVERWRITE mode to overwrite existing files and directories. at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:216) at org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:243) at org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:215) at org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:203) at org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:119) at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:104) at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:278) at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:266) at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:98) at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:40) at org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:146) ... 10 moreCaused by: java.io.IOException: File or directory already exists. Existing files and directories are not overwritten in NO_OVERWRITE mode. Use OVERWRITE mode to overwrite existing files and directories. at org.apache.flink.core.fs.FileSystem.initOutPathDistFS(FileSystem.java:937) at org.apache.flink.api.common.io.FileOutputFormat.initializeGlobal(FileOutputFormat.java:286) at org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initializeOnMaster(InputOutputFormatVertex.java:100) at org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:212) ... 20 more

 

 

 

The properties that a csvTableSink support now:

connector.type, connector.property-version, connector.path, format.type, format.property-version, format.field-delimiter



--
This message was sent by Atlassian Jira
(v8.3.4#803005)