You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/01/11 12:30:15 UTC
[GitHub] [incubator-dolphinscheduler] zhaorongsheng opened a new issue #4419: [Question] the progress support k8s deployment
zhaorongsheng opened a new issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419
**Describe the question**
Hi, all.
Is there any progress about the plan supporting k8s deployment?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zhaorongsheng edited a comment on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
zhaorongsheng edited a comment on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-759149908
@chengshiwen
Yes, we want to deploy master&worker on k8s. Is there any way to do this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen edited a comment on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-757929499
@zhaorongsheng
In the latest dev branch, the kind types of all modules are as follows:
```
frontend: Deployment
api: Deployment
master: StatefulSet
worker: StatefulSet
alert: Deployment
```
Do you mean that the master and worker also support the `Deployment` type? Or just ask if K8s is supported?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen edited a comment on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-757929499
@zhaorongsheng
In the latest dev branch, the kind types of all modules are as follows:
frontend: Deployment
api: Deployment
master: StatefulSet
worker: StatefulSet
alert: Deployment
Do you mean that the master and worker also use the Deployment type?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] dailidong commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
dailidong commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-768875022
> Our company wants to replace oozie with ds this year and we are going to try using ds in a few days.
Did you see the doc : https://github.com/apache/incubator-dolphinscheduler/blob/dev/docker/kubernetes/dolphinscheduler/README.md ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-759171689
@zhaorongsheng
Yes, the conversion of StatefulSet to Deployment can usually be solved by mounting a PVC, but I will see specifically whether PVC is really needed.
I will finish this work and notify you in the next period of time, so please keep attention
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zhaorongsheng commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
zhaorongsheng commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-760150309
@chengshiwen thanks~ would u mind providing some detail documents about the deployment on k8s?
Our company wants to replace oozie with ds this year and we are going to try using ds in a few days.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] CalvinKirs closed issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
CalvinKirs closed issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zhaorongsheng commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
zhaorongsheng commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-770228411
> > Our company wants to replace oozie with ds this year and we are going to try using ds in a few days.
>
> Did you see the doc : https://github.com/apache/incubator-dolphinscheduler/blob/dev/docker/kubernetes/dolphinscheduler/README.md ?
thanks~ I will try this way later
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] CalvinKirs commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-774934686
close by #4529
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-757929499
@zhaorongsheng
In the latest dev branch, the kind types of all modules are as follows:
frontend: Deployment
api: Deployment
master: StatefulSet
worker: StatefulSet
alert: Deployment
Do you want the master and worker also use the Deployment type?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-774428803
@zhaorongsheng Hi, let you wait for a long time.
In the next 1.3.5-release, we adjust k8s deployment (this deployment is not the official **Controller** `**Deployment**`: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/):
- Remove `frontend`
- Support resource storage for `file:///`
- Fix task log disappear after recreating or upgrading helm release from #3298
- Fixing this bug requires a stable and unique network identifiers
- Only **StatefulSets** meets the requirements from [Official Document](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#using-statefulsets)
- Improve K8s a lot
```
api: Deployment
alert: Deployment
master: StatefulSet (contains logger)
worker: StatefulSet
```
Conclusion: K8s deployment is supported, but the **Controller Deployment** for master and worker will not be considering.
Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen edited a comment on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-774428803
@zhaorongsheng Hi, let you wait for a long time.
In the next 1.3.5-release, we adjust k8s deployment (this deployment is not the official **Controller** `**Deployment**`: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/):
- Remove `frontend`
- Support resource storage for `file:///`
- Fix task log disappear after recreating or upgrading helm release from #3298
- Fixing this bug requires a stable and unique network identifiers
- Only **StatefulSets** meets the requirements from [Official Document](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#using-statefulsets)
- Improve K8s a lot
```
api: Deployment
alert: Deployment
master: StatefulSet (contains logger)
worker: StatefulSet
```
Conclusion: K8s deployment is supported, but the **Controller Deployment** for master and worker will not be considering.
In the next few days, we will close this pr, if you have no questions.
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zhaorongsheng commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
zhaorongsheng commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-809039191
> @zhaorongsheng Hi, let you wait for a long time.
> In the next 1.3.5-release, we adjust k8s deployment (this deployment is not the official **Controller** `**Deployment**`: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/):
>
> * Remove `frontend`
> * Support resource storage for `file:///`
> * Fix task log disappear after recreating or upgrading helm release from #3298
>
> * Fixing this bug requires a stable and unique network identifiers
> * Only **StatefulSets** meets the requirements from [Official Document](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#using-statefulsets)
> * Improve K8s a lot
>
> ```
> api: Deployment
> alert: Deployment
> master: StatefulSet (contains logger)
> worker: StatefulSet
> ```
>
> Conclusion: K8s deployment is supported, but the **Controller Deployment** for master and worker will not be considering.
>
> In the next few days, we will close this pr, if you have no questions.
>
> Thanks!
tks, we will try it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] zhaorongsheng commented on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
zhaorongsheng commented on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-759149908
Yes, we want to deploy master&worker on k8s. Is there any way to do this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen edited a comment on issue #4419: [Question] the progress support k8s deployment
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #4419:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4419#issuecomment-757929499
@zhaorongsheng
In the latest dev branch, the kind types of all modules are as follows:
```
frontend: Deployment
api: Deployment
master: StatefulSet
worker: StatefulSet
alert: Deployment
```
Do you mean that the master and worker also support the `Deployment` type?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org