You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@community.apache.org by "duheng (Jira)" <ji...@apache.org> on 2020/02/27 04:35:00 UTC
[jira] [Created] (COMDEV-360) RocketMQ Connect Hive
duheng created COMDEV-360:
-----------------------------
Summary: RocketMQ Connect Hive
Key: COMDEV-360
URL: https://issues.apache.org/jira/browse/COMDEV-360
Project: Community Development
Issue Type: Wish
Components: GSoC/Mentoring ideas
Reporter: duheng
h3. Content
The Hive sink connector allows you to export data from Apache RocketMQ topics to HDFS files in a variety of formats and integrates with Hive to make data immediately available for querying with HiveQL. The connector periodically polls data from RocketMQ and writes them to HDFS.
The data from each RocketMQ topic is partitioned by the provided partitioner and divided into chunks. Each chunk of data is represented as an HDFS file with topic, queueName, start and end offsets of this data chunk in the filename.
So, in this project, you need to implement a Hive sink connector based on OpenMessaging connect API, and run it on RocketMQ connect runtime.
You should learn before applying for this topic
Hive/Apache RocketMQ/Apache RocketMQ Connect/ OpenMessaging Connect API
h3. Mentor
[chenguangsheng@apache.org|mailto:chenguangsheng@apache.org], [vongosling@apache.org|mailto:vongosling@apache.org]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org