You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Jun Zhang <82...@qq.com> on 2019/09/17 02:39:12 UTC

Add Bucket File System Table Sink

Hello everyone:

I am a user and fan of flink. I also want to join the flink community. I
contributed my first PR a few days ago. Can anyone help me to review my code?
If there is something wrong, hope I would be grateful if you can give some
advice.

  

This PR is mainly in the process of development, I use sql to read data from
kafka and then write to hdfs, I found that there is no suitable tablesink, I
found the document and found that File System Connector is only experimental
(<https://ci.apache.org/projects/flink/flink-docs-
release-1.9/dev/table/connect.html#file-system-connector>), so I wrote a
Bucket File System Table Sink that supports writing stream data. Hdfs, file
file system, data format supports json, csv, parquet, avro. Subsequently add
other format support, such as protobuf, thrift, etc.

  

In addition, I also added documentation, python api, units test, end-end-test,
sql-client, DDL, and compiled on travis.

  

the issue is <https://issues.apache.org/jira/browse/FLINK-12584>

thank you very much