You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "qin-sy (via GitHub)" <gi...@apache.org> on 2023/03/03 08:34:13 UTC

[GitHub] [dolphinscheduler] qin-sy opened a new issue, #13666: [Bug] [WorkerGroup] 通过3.1.4的helm部署时、设置values.yaml中work env中的WORKER_GROUPS和WORKER_GROUPS_0没有作用

qin-sy opened a new issue, #13666:
URL: https://github.com/apache/dolphinscheduler/issues/13666

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   当我在k8s中部署worker时不能将worker注册到指定的workergroup中。
   
   ### What you expected to happen
   
   期望当我修改WORKER_GROUPS中的分组时可以在启动时注册到我指定的workergroup中。
   当我在k8s中部署多个worker 的 StatefulSet 时，希望可以将不同的 StatefulSet 中的 POD 使用不同的分组管理。
   
   现在启动时只能注册到默认的default中。
   当我创建新的workergroup时，只能选择已经注册到default中的woker，
   当 StatefulSet 中的POD增加或者减少时，需要人工修改workergroup中的设置。
   
   
   ### How to reproduce
   
   使用[3.1.4版本](https://github.com/apache/dolphinscheduler/tree/3.1.4-release)的 helm 配置文件
   修改WORKER_GROUPS、WORKER_GROUPS_0中的分组后部署到k8s中后在画面中查看workgroup
   
   ### Anything else
   
   _No response_
   
   ### Version
   
   3.1.x
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] qin-sy commented on issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect

Posted by "qin-sy (via GitHub)" <gi...@apache.org>.
qin-sy commented on issue #13666:
URL: https://github.com/apache/dolphinscheduler/issues/13666#issuecomment-1459644570

   @EricGao888
   在3.1.0的版本中修改WORKER_GROUPS内容后可以自动创建指定分组。3.1.4版本不行。
   
   3.1.0: 
   ```diff
    env: 
   -  WORKER_GROUPS_0: default 
      WORKER_MAX_CPU_LOAD_AVG: "-1" 
      WORKER_RESERVED_MEMORY: "0.3" 
      WORKER_EXEC_THREADS: "100" 
      WORKER_HEARTBEAT_INTERVAL: "10s" 
      WORKER_HEART_ERROR_THRESHOLD: "5" 
      WORKER_HOST_WEIGHT: "100" 
   -  WORKER_GROUPS: "default" 
   +  WORKER_GROUPS: "another-works" 
   ```
   ![image](https://user-images.githubusercontent.com/126856011/223644675-6095637d-0e04-47b4-875a-deecae1b3090.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] Radeity commented on issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect

Posted by "Radeity (via GitHub)" <gi...@apache.org>.
Radeity commented on issue #13666:
URL: https://github.com/apache/dolphinscheduler/issues/13666#issuecomment-1463189666

   Hi, @qin-sy , we don't maintain worker group in registry since 3.1.4, you can config them in UI page by yourself and worker group info will be stored in db, relative issue: https://github.com/apache/dolphinscheduler/issues/12157


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] SbloodyS closed issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect

Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect
URL: https://github.com/apache/dolphinscheduler/issues/13666


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] EricGao888 commented on issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect

Posted by "EricGao888 (via GitHub)" <gi...@apache.org>.
EricGao888 commented on issue #13666:
URL: https://github.com/apache/dolphinscheduler/issues/13666#issuecomment-1458117048

   @qin-sy Hello, if u deploy DS through helm, the environment variable `WORKER_GROUPS` will take effect for you workers. Could u explain in details how you add more different `statefulsets`? Have u injected the environment variables defined in `values.yaml`?
   
   https://github.com/apache/dolphinscheduler/blob/30fd43e32ba2b156e0c68d8ce15cfbd56263ad62/deploy/kubernetes/dolphinscheduler/values.yaml#L384-L392
   
   https://github.com/apache/dolphinscheduler/blob/30fd43e32ba2b156e0c68d8ce15cfbd56263ad62/deploy/kubernetes/dolphinscheduler/templates/statefulset-dolphinscheduler-worker.yaml#L72
   
   BTW, could u plz use English for issue description in the future? Thx.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13666:
URL: https://github.com/apache/dolphinscheduler/issues/13666#issuecomment-1453161767

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   When I deploy a worker in k8s, I cannot register the worker to the specified workergroup.
   
   ### What you expected to happen
   
   It is expected that when I modify the grouping in WORKER_GROUPS, I can register to the workergroup I specified at startup.
   When I deploy multiple worker StatefulSets in k8s, I hope that the PODs in different StatefulSets can be managed in different groups.
   
   Now it can only be registered to the default default at startup.
   When I create a new workergroup, I can only choose wokers that have been registered in the default,
   When the POD in the StatefulSet increases or decreases, it is necessary to manually modify the settings in the workergroup.
   
   
   ### How to reproduce
   
   Use the helm configuration file for [version 3.1.4](https://github.com/apache/dolphinscheduler/tree/3.1.4-release)
   After modifying the groups in WORKER_GROUPS and WORKER_GROUPS_0 and deploying them to k8s, view the workgroup on the screen
   
   ### Anything else
   
   _No response_
   
   ### Version
   
   3.1.x
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13666:
URL: https://github.com/apache/dolphinscheduler/issues/13666#issuecomment-1453162077

   Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
   * In order for us to understand your request as soon as possible, please provide detailed information, version or pictures.
   * If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] Radeity commented on issue #13666: [Bug] [WorkerGroup] When deploying through 3.1.4 helm, setting WORKER_GROUPS and WORKER_GROUPS_0 in work env in values.yaml has no effect

Posted by "Radeity (via GitHub)" <gi...@apache.org>.
Radeity commented on issue #13666:
URL: https://github.com/apache/dolphinscheduler/issues/13666#issuecomment-1463190973

   > @qin-sy Hello, if u deploy DS through helm, the environment variable `WORKER_GROUPS` will take effect for you workers. Could u explain in details how you add more different `statefulsets`? Have u injected the environment variables defined in `values.yaml`?
   > 
   > https://github.com/apache/dolphinscheduler/blob/30fd43e32ba2b156e0c68d8ce15cfbd56263ad62/deploy/kubernetes/dolphinscheduler/values.yaml#L384-L392
   > 
   > https://github.com/apache/dolphinscheduler/blob/30fd43e32ba2b156e0c68d8ce15cfbd56263ad62/deploy/kubernetes/dolphinscheduler/templates/statefulset-dolphinscheduler-worker.yaml#L72
   > 
   > BTW, could u plz use English for issue description in the future? Thx.
   
   Hi, @EricGao888 , this configuration in values.yaml may be outdated, environment variable WORKER_GROUP is useless in the latest version, i can create a PR to remove them.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org