You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "WindStill (via GitHub)" <gi...@apache.org> on 2024/09/19 03:28:39 UTC
[I] 能不能支持 RegexRouter 啊？ [doris-kafka-connector]
WindStill opened a new issue, #44:
URL: https://github.com/apache/doris-kafka-connector/issues/44
请问能不能支持 `org.apache.kafka.connect.transforms.RegexRouter` 啊？
我这边10来个MySQL库的几十张表需要同步到doris，如果用 `doris.topic2table.map` 属性的话，得手动写几十个topic与table的对应关系。
但是如果用之前同步到CrateDB的JDBC方式，只要加个 RegexRouter 的transform就可以映射topic与目标table了，如下：
```
"transforms.route.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.route.regex": "([^.]+)\\.([^.]+)\\.([^.]+)",
"transforms.route.replacement": "kafka_test.$3"
```
> debezuim 同步MySQL到kakfa的topic默认名称为 `dbserver.dbname.tablename`。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] 能不能支持 RegexRouter 啊？ [doris-kafka-connector]
Posted by "chuang-wang-pre (via GitHub)" <gi...@apache.org>.
chuang-wang-pre commented on issue #44:
URL: https://github.com/apache/doris-kafka-connector/issues/44#issuecomment-2567263529
Refer to the release note: https://github.com/apache/doris-kafka-connector/issues/60 , use the `'record.tablename.field'` configuration option, combined with some `Transforms` such as `'org.apache.kafka.connect.transforms.InsertField'`, it can easily achieve the effect you want. If it is possible to reduce the number of topics used in your data source step, you can even use only one topic to complete it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [I] 能不能支持 RegexRouter 啊？ [doris-kafka-connector]
Posted by "DongLiang-0 (via GitHub)" <gi...@apache.org>.
DongLiang-0 commented on issue #44:
URL: https://github.com/apache/doris-kafka-connector/issues/44#issuecomment-2367430086
In doris-kafka-connector, `doris.topic2table.map` is a required attribute, which is used to map topics to tables one-to-one.
`RegexRouter` can simplify the topic name, but `doris.topic2table.map` still needs to be filled in. It does not mean that doris.topic2table.map does not need to be filled in if RegexRouter is supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org