You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2022/05/09 08:22:00 UTC
[GitHub] [incubator-doris] xiebanggui777 opened a new issue, #9467: [Feature] When support debezium CDC type load or other database binlog load？
xiebanggui777 opened a new issue, #9467:
URL: https://github.com/apache/incubator-doris/issues/9467
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Description
我看到doris支持canal的binlog load，但是仅仅支持mysql的binlog，能否在后续版本规划其他数据库的binlog load，例如sqlserver CDC 。
据我所知debezium可以对接大多数的数据库cdc功能，且支持整个数据库同步，但是debezium输出到kakfa的json格式不太符合doris的json要求格式(需要doris这边用户配置Json Path,这种多重嵌套格式的json实在不知道如何指定)
I saw Doris support canal binlog load, but only support mysql binlog, whether in the subsequent version planning other database binlog load, such as sqlserver CDC.
As far as I know debezium can dock most function of the database to the CDC, and support the entire database synchronization, but output to the kakfa debezium json format doesn't conform to the requirements of the Doris's json format (need to Doris this way the user configuration "json Path", the multiple nested json really don't know how to specify the format)
### Use case
多种数据库的实时数据同步 CDC
整个数据库的同步
A variety of database real-time data synchronization to the CDC
The synchronization of the entire database
### Related issues
无
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [doris] uniorder commented on issue #9467: [Feature] When support debezium CDC type load or other database binlog load？
Posted by GitBox <gi...@apache.org>.
uniorder commented on issue #9467:
URL: https://github.com/apache/doris/issues/9467#issuecomment-1308423021
同问
其实在实际应用过程中，使用 Debezium（或类似组件）将监听到的数据发送到 Kafka，下游的Doris根据Kafka中的数据来进行同步可能比较合理。
因为针对数据库的CDC可能会有重复的使用场景，如果每个业务都去CDC，对目标数据源的压力也比较大。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org