You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/03/03 09:34:59 UTC
[GitHub] [rocketmq-externals] Git-Yang opened a new issue #686: [rocketmq-connect] For replicator design ideas, seek advice.
Git-Yang opened a new issue #686:
URL: https://github.com/apache/rocketmq-externals/issues/686
After reading the code of rocketmq connect module, I have some questions to ask for advice.
1.The current usage of rocketmq-connect is that transferred to the middle tier by sourceTask, and then transferred to the target cluster by sinkTask. For rocketmq type data sources, can they be directly synchronized to the target cluster？
2.As for RocketMQConverter, I understand whether it is a special processing for rocketmq, which should not be implemented in rocketmq-connect or implemented in RocketMQConverter. Do you have any opinions on this part of the community?
3.I plan to implement the WorkerDirctTask of direct forwarding by encapsulating WokerSourceTask and WokerSinkTask. Is this method in line with the community development plan?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] odbozhou commented on issue #686: [rocketmq-connect] For replicator design ideas, seek advice.
Posted by GitBox <gi...@apache.org>.
odbozhou commented on issue #686:
URL: https://github.com/apache/rocketmq-externals/issues/686#issuecomment-903399125
> After reading the code of rocketmq connect module, I have some questions to ask for advice.
>
> 1.The current usage of rocketmq-connect is that transferred to the middle tier by sourceTask, and then transferred to the target cluster by sinkTask. For rocketmq type data sources, can they be directly synchronized to the target cluster？
>
> 2.As for RocketMQConverter, I understand whether it is a special processing for rocketmq, which should not be implemented in rocketmq-connect or implemented in RocketMQConverter. Do you have any opinions on this part of the community?
> 
>
> 3.I plan to implement the WorkerDirctTask of direct forwarding by encapsulating WokerSourceTask and WokerSinkTask. Is this method in line with the community development plan?
1. Your understanding is correct.
2. RocketMQConverter is the direct support of rocketmq source and target, and one less data transfer.
3. Of course what you said is also achievable to provide sink and source
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] vongosling commented on issue #686: [rocketmq-connect] For replicator design ideas, seek advice.
Posted by GitBox <gi...@apache.org>.
vongosling commented on issue #686:
URL: https://github.com/apache/rocketmq-externals/issues/686#issuecomment-806327819
The connector is a wider exchanger between any data source(including file system, event mesh, and so on). I would like to encourage you to enhance the replicator function in your case. while not implement an almost the same functionnality. Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] maixiaohai commented on issue #686: [rocketmq-connect] For replicator design ideas, seek advice.
Posted by GitBox <gi...@apache.org>.
maixiaohai commented on issue #686:
URL: https://github.com/apache/rocketmq-externals/issues/686#issuecomment-808640682
Agree with you. We have some changes at Connect Project related with Replicator, we may refactor it later. At the same time, we will submit other seperate prs.
Thanks for your reply in time~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] maixiaohai commented on issue #686: [rocketmq-connect] For replicator design ideas, seek advice.
Posted by GitBox <gi...@apache.org>.
maixiaohai commented on issue #686:
URL: https://github.com/apache/rocketmq-externals/issues/686#issuecomment-805877618
@vongosling @duhenglucky We planed to use **Replicator** and **Connector** in our company and found some design conflicts.
For example, the sink part of Connect is useless in Replicator job;
for order sync of Replicator, there will be some changes not only in Replicator but also in Connect.
So we want some suggestions from community about the design part and future development of Connect.
Thanks~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org