You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/04 20:12:35 UTC
[GitHub] [airflow] XD-DENG opened a new issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
XD-DENG opened a new issue #15193:
URL: https://github.com/apache/airflow/issues/15193
### Description
Currently command `kubernetes cleanup-pods` cleans up Pods that meet specific conditions in a given namespace.
Underlying code: https://github.com/apache/airflow/blob/2.0.1/airflow/cli/commands/kubernetes_command.py#L70
### Use case / motivation
The problem to me is: users may have other non-Airflow stuff running in this specific namespace, and they may want to have different strategy/logic for Pod cleaning-up for these non-Airflow pods.
We may want to have an additional boolean flag for this command, like `--only-airflow-pods`, so that users can decide if they only want to clean up Airflow-created Pods.
It should not be hard to identify Airflow-created pods, given the specific labels Airflow adds to the Pods it creates (https://github.com/apache/airflow/blob/2.0.1/airflow/kubernetes/pod_generator.py#L385)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on issue #15193:
URL: https://github.com/apache/airflow/issues/15193#issuecomment-813333133
That's my first thought as well, but I would like to hear the original author's opinion, like if there is any specific reason or not
This feature was added in https://github.com/apache/airflow/pull/11802 . @kaxil may you please share your inputs? Cheers!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #15193:
URL: https://github.com/apache/airflow/issues/15193#issuecomment-813371827
Hey, thanks for this issue @XD-DENG . I added this feature and we use it in Astronomer too -- to cleanup any hanging PODs and their sidecars. So yeah we can just restrict it to pod launched via Airflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on issue #15193:
URL: https://github.com/apache/airflow/issues/15193#issuecomment-813372813
Thanks everyone.
I think we have come to a consensus. So I will change `kind: feature` to `kind: bug` and work on this later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG closed issue #15193: CLI 'kubernetes cleanup-pods' should only clean up Airflow-created Pods
Posted by GitBox <gi...@apache.org>.
XD-DENG closed issue #15193:
URL: https://github.com/apache/airflow/issues/15193
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15193:
URL: https://github.com/apache/airflow/issues/15193#issuecomment-813363371
> in my opinion this is a bug and by default we should only manage pods created by Airflow i.e. we don't need extra option.
My thoughts exactly!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on issue #15193:
URL: https://github.com/apache/airflow/issues/15193#issuecomment-813297611
CC @kaxil @ashb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on issue #15193:
URL: https://github.com/apache/airflow/issues/15193#issuecomment-813093138
@dimberman @mik-laj may you please share your inputs on if this makes sense to you or not? Let me know if I overlooked anything. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #15193: CLI `kubernetes cleanup-pods` should allow "only clean up Pods created by Airflow"
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #15193:
URL: https://github.com/apache/airflow/issues/15193#issuecomment-813330915
in my opinion this is a bug and by default we should only manage pods created by Airflow i.e. we don't need extra option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org