You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yun Gao (Jira)" <ji...@apache.org> on 2020/05/09 07:34:00 UTC
[jira] [Updated] (FLINK-17589) Extend StreamingFileSink to Support
Streaming Hive Connector
[ https://issues.apache.org/jira/browse/FLINK-17589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yun Gao updated FLINK-17589:
----------------------------
Description:
According to the [Discussion on FLIP-115|[https://lists.apache.org/thread.html/rb1795428e481dbeaa1af2dcffed87b4804ed81d2af60256cd50032d7%40%3Cdev.flink.apache.org%3E |https://lists.apache.org/thread.html/rb1795428e481dbeaa1af2dcffed87b4804ed81d2af60256cd50032d7%40%3Cdev.flink.apache.org%3E]], we will use StreamingFileSink to support the streaming hive connector. This requires to make some extension to the current StreamingFileSink:
# Support path-based writer to write the specified Hadoop path.
# Provides listeners to collect the bucket state so that the Hive sink could decide when one bucket is terminated and then writing meta-info to Hive.
was:
According to the [ Discussion on FLIP-115 | [https://lists.apache.org/thread.html/rb1795428e481dbeaa1af2dcffed87b4804ed81d2af60256cd50032d7%40%3Cdev.flink.apache.org%3E |https://lists.apache.org/thread.html/rb1795428e481dbeaa1af2dcffed87b4804ed81d2af60256cd50032d7%40%3Cdev.flink.apache.org%3E]], we will use StreamingFileSink to support the streaming hive connector. This requires to make some extension to the current StreamingFileSink:
# Support path-based writer to write the specified Hadoop path.
# Provides listeners to collect the bucket state so that the Hive sink could decide when one bucket is terminated and then writing meta-info to Hive.
> Extend StreamingFileSink to Support Streaming Hive Connector
> ------------------------------------------------------------
>
> Key: FLINK-17589
> URL: https://issues.apache.org/jira/browse/FLINK-17589
> Project: Flink
> Issue Type: New Feature
> Components: Connectors / FileSystem
> Reporter: Yun Gao
> Assignee: Yun Gao
> Priority: Major
> Fix For: 1.11.0
>
>
> According to the [Discussion on FLIP-115|[https://lists.apache.org/thread.html/rb1795428e481dbeaa1af2dcffed87b4804ed81d2af60256cd50032d7%40%3Cdev.flink.apache.org%3E |https://lists.apache.org/thread.html/rb1795428e481dbeaa1af2dcffed87b4804ed81d2af60256cd50032d7%40%3Cdev.flink.apache.org%3E]], we will use StreamingFileSink to support the streaming hive connector. This requires to make some extension to the current StreamingFileSink:
> # Support path-based writer to write the specified Hadoop path.
> # Provides listeners to collect the bucket state so that the Hive sink could decide when one bucket is terminated and then writing meta-info to Hive.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)