You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@community.apache.org by "Hongsheng Zhong (Jira)" <ji...@apache.org> on 2023/03/03 04:05:00 UTC
[jira] [Updated] (COMDEV-492) Apache ShardingSphere: Add ShardingSphere Kafka source connector
[ https://issues.apache.org/jira/browse/COMDEV-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hongsheng Zhong updated COMDEV-492:
-----------------------------------
Description:
h2. Apache ShardingSphere
Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.
{*}Page{*}: [https://shardingsphere.apache.org|https://shardingsphere.apache.org/]
{*}Github{*}: [https://github.com/apache/shardingsphere]
h2. Background
The community just added CDC (change data capture) [feature|https://github.com/apache/shardingsphere/issues/22500] recently. Change feed will be published in created network connection after logging in, then it could be consumed.
Since Kafka is popular distributed event streaming platform, it's useful to import change feed into Kafka for later processing.
h2. Task
# Familiar with ShardingSphere CDC client usage, create publication and subscribe change feed.
# Familiar with Kafka connector development, develop source connector, integrate with ShardingSphere CDC. Persist change feed to Kafka topics properly.
# Add unit test and E2E integration test.
h2. Relevant Skills
1. Java language
2. Basic knowledge of CDC and Kafka
3. Maven
h3. References
* [https://github.com/apache/shardingsphere/issues/22500]
* [https://kafka.apache.org/documentation/#connect_development]
* [https://github.com/apache/kafka/tree/trunk/connect/file/src]
* [https://github.com/confluentinc/kafka-connect-jdbc]
h3. Mentor
Hongsheng Zhong, PMC of Apache ShardingSphere, zhonghongsheng@apache.org
Xinze Guo, Committer of Apache ShardingSphere, azexin@apache.org
was:
h2. Apache ShardingSphere
Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.
{*}Page{*}: [https://shardingsphere.apache.org|https://shardingsphere.apache.org/]
{*}Github{*}: [https://github.com/apache/shardingsphere]
h2. Background
The community just added CDC (change data capture) [feature|https://github.com/apache/shardingsphere/issues/22500] recently. Change feed will be published in created network connection after logging in, then it could be consumed.
Since Kafka is popular distributed event streaming platform, it's useful to import change feed into Kafka for later processing.
h2. Task
# Familiar with ShardingSphere CDC client usage, create publication and subscribe change feed.
# Familiar with Kafka connector development, develop source connector, integrate with ShardingSphere CDC. Persist change feed to Kafka topics properly.
h2. Relevant Skills
1. Java language
2. Basic knowledge of CDC and Kafka
3. Maven
h3. References
* [https://github.com/apache/shardingsphere/issues/22500]
* [https://kafka.apache.org/documentation/#connect_development]
* [https://github.com/apache/kafka/tree/trunk/connect/file/src]
* [https://github.com/confluentinc/kafka-connect-jdbc]
h3. Mentor
Hongsheng Zhong, PMC of Apache ShardingSphere, [zhonghongsheng@apache.org|mailto:zhonghongsheng@apache.org]
Labels: ShardingSphere full-time gsoc2023 mentor (was: ShardingSphere gsoc2023 mentor)
Remaining Estimate: (was: 175h)
Original Estimate: (was: 175h)
> Apache ShardingSphere: Add ShardingSphere Kafka source connector
> ----------------------------------------------------------------
>
> Key: COMDEV-492
> URL: https://issues.apache.org/jira/browse/COMDEV-492
> Project: Community Development
> Issue Type: Improvement
> Components: GSoC/Mentoring ideas
> Reporter: Hongsheng Zhong
> Priority: Major
> Labels: ShardingSphere, full-time, gsoc2023, mentor
>
> h2. Apache ShardingSphere
> Apache ShardingSphere is positioned as a Database Plus, and aims at building a standard layer and ecosystem above heterogeneous databases. It focuses on how to reuse existing databases and their respective upper layer, rather than creating a new database. The goal is to minimize or eliminate the challenges caused by underlying databases fragmentation.
> {*}Page{*}: [https://shardingsphere.apache.org|https://shardingsphere.apache.org/]
> {*}Github{*}: [https://github.com/apache/shardingsphere]
> h2. Background
> The community just added CDC (change data capture) [feature|https://github.com/apache/shardingsphere/issues/22500] recently. Change feed will be published in created network connection after logging in, then it could be consumed.
> Since Kafka is popular distributed event streaming platform, it's useful to import change feed into Kafka for later processing.
> h2. Task
> # Familiar with ShardingSphere CDC client usage, create publication and subscribe change feed.
> # Familiar with Kafka connector development, develop source connector, integrate with ShardingSphere CDC. Persist change feed to Kafka topics properly.
> # Add unit test and E2E integration test.
> h2. Relevant Skills
> 1. Java language
> 2. Basic knowledge of CDC and Kafka
> 3. Maven
> h3. References
> * [https://github.com/apache/shardingsphere/issues/22500]
> * [https://kafka.apache.org/documentation/#connect_development]
> * [https://github.com/apache/kafka/tree/trunk/connect/file/src]
> * [https://github.com/confluentinc/kafka-connect-jdbc]
> h3. Mentor
> Hongsheng Zhong, PMC of Apache ShardingSphere, zhonghongsheng@apache.org
> Xinze Guo, Committer of Apache ShardingSphere, azexin@apache.org
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org