You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/10/21 06:27:51 UTC
[GitHub] [incubator-dolphinscheduler] CalvinKirs opened a new issue #3961: [Feature][SPI] register center
CalvinKirs opened a new issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961
After the code of the alert_plugin_design branch is merged into the dev branch, I will start the registration center (support zk, etcd, and at the same time, for transition, the selected components need to support distributed locks) SPI implementation, at that time, I will send an email to discuss the specific design
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] CalvinKirs edited a comment on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
CalvinKirs edited a comment on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-720069593
Registrar SPI, the user can specify the registration center method and specific related parameters in the configuration file. Zookeeper is used by default
todo list:
- [ ] 1 : Provide a unified registry Client related interface (org.apache.dolphinscheduler.spi.registry), @hermeshephaestus
Including registration, offline, update and other operations. The external access registration center uniformly calls the Client interface.
The registration center currently only provides a single registration center. We need to load the corresponding plug-in according to the user configuration. When the user does not specify it, zookeeper is used by default.
At the same time, in order to transition, we need to provide the implementation of distributed locks. (This is just for transition)
- [ ] 2: Regarding the implementation of ZK in the registration center. @AhahaGe
- [ ] 3:Alert Server supports the registration center. The alert server must share the same registry with the worker master, and multiple registries are not currently supported.
- [ ] 4: Implementation of Etcd in the registry @CalvinKirs
To realize this function, you need to vote for the related jars by mail. I have checked the related dependencies and they are all within the acceptable range of ASF.
注册中心SPI，用户可在配置文件指定注册中心的方式，以及具体的相关参数。默认使用Zookeeper
1：提供统一的注册中心Client相关接口(org.apache.dolphinscheduler.spi.registry)，
包含注册、下线、更新等操作。外部访问注册中心统一调用该Client接口。
注册中心目前只提供单注册中心，我们需要按照用户配置来加载相应的插件，当用户未指定，则默认使用zookeeper。
同时，为了过渡，我们需要提供分布式锁的实现。（这仅仅是为了过渡）
2:注册中心关于ZK的实现。
3:Alert Server支持注册中心。alert server 必须和worker master共用一个注册中心，暂不支持多注册中心。
3:注册中心关于Etcd的实现
该功能实现需要首先将相关依赖的jar进行邮件投票。我检查过相关依赖，都属ASF可以接受的范围。
具体任务需要出具体的设计，当前只是粗略的任务清单。
You can refer to the related code of aler spi: https://github.com/apache/incubator-dolphinscheduler/tree/alert_plugin_design
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] hermeshephaestus commented on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
hermeshephaestus commented on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-720246363
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] CalvinKirs commented on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-714847557
> Whether DS introduces the registry center function, we need to evaluate the value together.
> The service registry is essentially for decoupling service providers and service consumers.
Hi, the registry helps us maintain the online status of workers and masters in the cluster state. Service registration is one of its uses, but it is not only applicable to service registration and discovery in RPC. You can look at the server module of our project, using ZK as the registration center
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] hermeshephaestus commented on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
hermeshephaestus commented on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-714472182
Whether DS introduces the registry center function, we need to evaluate the value together.
The service registry is essentially for decoupling service providers and service consumers.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zhuangchong commented on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
zhuangchong commented on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-722344908
Good design!
I will claim task part 3! @CalvinKirs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] CalvinKirs edited a comment on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
CalvinKirs edited a comment on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-720069593
Registrar SPI, the user can specify the registration center method and specific related parameters in the configuration file. Zookeeper is used by default
todo list:
- [ ] 1 : Provide a unified registry Client related interface (org.apache.dolphinscheduler.spi.registry), @hermeshephaestus
Including registration, offline, update and other operations. The external access registration center uniformly calls the Client interface.
The registration center currently only provides a single registration center. We need to load the corresponding plug-in according to the user configuration. When the user does not specify it, zookeeper is used by default.
At the same time, in order to transition, we need to provide the implementation of distributed locks. (This is just for transition)
- [ ] 2: Regarding the implementation of ZK in the registration center. @AhahaGe
- [ ] 3:Alert Server supports the registration center. The alert server must share the same registry with the worker master, and multiple registries are not currently supported. @zhuangchong
- [ ] 4: Implementation of Etcd in the registry @CalvinKirs
To realize this function, you need to vote for the related jars by mail. I have checked the related dependencies and they are all within the acceptable range of ASF.
注册中心SPI，用户可在配置文件指定注册中心的方式，以及具体的相关参数。默认使用Zookeeper
1：提供统一的注册中心Client相关接口(org.apache.dolphinscheduler.spi.registry)，
包含注册、下线、更新等操作。外部访问注册中心统一调用该Client接口。
注册中心目前只提供单注册中心，我们需要按照用户配置来加载相应的插件，当用户未指定，则默认使用zookeeper。
同时，为了过渡，我们需要提供分布式锁的实现。（这仅仅是为了过渡）
2:注册中心关于ZK的实现。
3:Alert Server支持注册中心。alert server 必须和worker master共用一个注册中心，暂不支持多注册中心。
3:注册中心关于Etcd的实现
该功能实现需要首先将相关依赖的jar进行邮件投票。我检查过相关依赖，都属ASF可以接受的范围。
具体任务需要出具体的设计，当前只是粗略的任务清单。
You can refer to the related code of aler spi: https://github.com/apache/incubator-dolphinscheduler/tree/alert_plugin_design
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] AhahaGe commented on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
AhahaGe commented on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-721579588
Good design!
I will claim task part 2! @CalvinKirs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] CalvinKirs edited a comment on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
CalvinKirs edited a comment on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-720069593
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] hermeshephaestus removed a comment on issue #3961: [Feature][SPI] register center
Posted by GitBox <gi...@apache.org>.
hermeshephaestus removed a comment on issue #3961:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3961#issuecomment-720246363
Good design! I want to claim task part 1 and task part 3! @CalvinKirs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org