You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@dolphinscheduler.apache.org by Xuetong Zhu <zh...@163.com> on 2022/07/20 14:27:38 UTC
[PROPOSAL] Add new registry plugin based on RAFT consensus algorithm
Hi community:
In order to facilitate the deployment of DolphinScheduler, reduce external system dependencies, and the complexity of maintenance.
I proposal to implement a new registry using raft consensus algorithm[1],Replacing the registry of zookeeper.
I will use SOFA-JRAFT[2] lib implement the registry.
Registry store master and worker info, cluster data synchronization, leader master election and distributed lock manage.
I already add a GitHub Issue for my proposal, which you could see in https://github.com/apache/dolphinscheduler/issues/10874
Looking forward any feedback for this thread.
1. https://raft.github.io/ <https://raft.github.io/>
2. https://www.sofastack.tech/en/projects/sofa-jraft/overview/
Re: [PROPOSAL] Add new registry plugin based on RAFT consensus algorithm
Posted by Jiajie Zhong <zh...@gmail.com>.
Thanks xuetong for bringing this up, and I think it is good to add a
new registry plugin to DolphinScheduler, and I find out we just keep
moving forward according to the issue-10874. Looking forward to you
new add
On Wed, Jul 20, 2022 at 10:28 PM Xuetong Zhu <zh...@163.com> wrote:
>
> Hi community:
>
> In order to facilitate the deployment of DolphinScheduler, reduce external system dependencies, and the complexity of maintenance.
> I proposal to implement a new registry using raft consensus algorithm[1],Replacing the registry of zookeeper.
> I will use SOFA-JRAFT[2] lib implement the registry.
> Registry store master and worker info, cluster data synchronization, leader master election and distributed lock manage.
>
> I already add a GitHub Issue for my proposal, which you could see in https://github.com/apache/dolphinscheduler/issues/10874
> Looking forward any feedback for this thread.
>
>
> 1. https://raft.github.io/ <https://raft.github.io/>
> 2. https://www.sofastack.tech/en/projects/sofa-jraft/overview/
--
Best Wish
— Jiajie
Re: [PROPOSAL] Add new registry plugin based on RAFT consensus algorithm
Posted by Chufeng Gao <ch...@gmail.com>.
Hi Xuetong,
Thanks for working on this. With this feature, users will no more need to
set up and maintain a ZK cluster, which certainly saves budgets and
troubles. I'm looking forward to this.
*Best Regards,*
*Chufeng (Eric) Gao*
Xuetong Zhu <zh...@163.com> 于2022年7月20日周三 22:28写道：
> Hi community:
>
> In order to facilitate the deployment of DolphinScheduler, reduce external
> system dependencies, and the complexity of maintenance.
> I proposal to implement a new registry using raft consensus
> algorithm[1],Replacing the registry of zookeeper.
> I will use SOFA-JRAFT[2] lib implement the registry.
> Registry store master and worker info, cluster data synchronization,
> leader master election and distributed lock manage.
>
> I already add a GitHub Issue for my proposal, which you could see in
> https://github.com/apache/dolphinscheduler/issues/10874
> Looking forward any feedback for this thread.
>
>
> 1. https://raft.github.io/ <https://raft.github.io/>
> 2. https://www.sofastack.tech/en/projects/sofa-jraft/overview/