You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/07/20 10:17:15 UTC
[GitHub] [dolphinscheduler] wangmingzhiJohn opened a new issue #5866: [Question] Does DS support dynamic expansion/reduction?
wangmingzhiJohn opened a new issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866
On the index of DS website, I found this instruction called "Support dynamic expansion and reduction of master and worker".
However, in document of Expansion/Reduction, the method of achieving the goal needs to restart all services which seems like not dynamic at all.
So, my question is whether DS support dynamic way to expand or reduce nodes?
**Which version of DolphinScheduler:**
-[1.3.6]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883269735
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] dailidong closed issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
dailidong closed issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883896886
@wangmingzhiJohn The docs means you can use the stop-all.sh or start-all.sh to scaled-up or scaled-down the cluster.
In most case, we don't deploy all service on one machine, so you can directly use the shell. If you deploy multiple instances on one machine, you can manual start or stop the instance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-884041010
@wangmingzhiJohn Yes, and other registry center will be supported in the future, currently only zookeeper.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883269735
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangmingzhiJohn commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
wangmingzhiJohn commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-884056449
Or except zookeeper's ip:port which is necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] dailidong edited a comment on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
dailidong edited a comment on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883593402
> @wangmingzhiJohn When you add worker or master to the exist cluster, you don't need to restart the cluster.
Yes, when adding, only need to start the service on the new server, and only need to stop the service of the server to be stopped
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] dailidong commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883593402
> @wangmingzhiJohn When you add worker or master to the exist cluster, you don't need to restart the cluster.
Yes, when adding, only need to start the service on the new server, and only need to stop the service of the server to be stopped
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangmingzhiJohn commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
wangmingzhiJohn commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883933356
@ruanwenjun I can't get the point. You mean these scripts can scaled-up the cluster. For example, there is 2 workers in cluster and I need to add 1. In order to let the system know that a new worker is added, I need to add a new ip to install_config.conf. And the new config file should be consistent with that on other nodes(according to doc).
As I understand it, indeed, you just need to operate on the new server to add the node into cluster and restart service. But in the start-all.sh/stop-all.sh, it will ssh to every node to restart the service whatever master or worker. I think this behavior can not be called 'dynamic scaling' because it make all services perceptive to the expansion. They all will be restart not only the service on new node
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883420167
@wangmingzhiJohn When you add worker or master to the exist cluster, you don't need to restart the cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangmingzhiJohn commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
wangmingzhiJohn commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883844735
@dailidong @ruanwenjun So, according to documents, why we need to execute stop-all.sh/start-all.sh which may influence services on other nodes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangmingzhiJohn commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
wangmingzhiJohn commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-884053968
@ruanwenjun In another way, these specific ips in install_config.conf is not necessary if I setup every service on every node manually by dolphinscheduler-daemon.sh?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] wangmingzhiJohn commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
wangmingzhiJohn commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-884022478
@ruanwenjun So, actually, the exists node's service don't need to update their install_config.conf as they can know the new node by zookeeper automaticly, is that correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] ruanwenjun commented on issue #5866: [Question] Does DS support dynamic expansion/reduction?
Posted by GitBox <gi...@apache.org>.
ruanwenjun commented on issue #5866:
URL: https://github.com/apache/dolphinscheduler/issues/5866#issuecomment-883951988
@wangmingzhiJohn When you add a new worker or master, the new instance will register in registry center automatic, and to exist masters will receive the added event. The script just is a normal start, stop script.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org