You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "ShenDa (Jira)" <ji...@apache.org> on 2019/12/11 02:22:00 UTC

[jira] [Created] (FLINK-15182) Add FileTableSink based on StreamingFileSink

ShenDa created FLINK-15182:
------------------------------

             Summary: Add FileTableSink based on StreamingFileSink
                 Key: FLINK-15182
                 URL: https://issues.apache.org/jira/browse/FLINK-15182
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / FileSystem
    Affects Versions: 1.9.0
            Reporter: ShenDa


Flink already has FileSystem connector that can sink data into specified file system, but it just writes data to a target path without bukcet and it can't configure file rolling policy neither.  We can use StreamingFileSink to avoid those shortcomings instead,  however, the StreamingFileSink is a SinkFunction that users have to implement their processing logic with DataStreamAPI. 

Therefore, we want to add a new FileTableSink based on StreamingFileSink. This will help us connect file system by using TableAPI&SQL. But according the two ways of StreamingFileSink to sinking data, bulk format and row format, it comes a question to us. Does the FileTableSink support bulk format and row format simultaneously? Or separating this sink into two sinks, BulkFileTableSink and RowFileTableSink, which writes data by using one format respectively.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)