You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/19 11:56:59 UTC
[GitHub] [airflow] jpyen opened a new issue #14167: KubernetesPodOperator Pods no unique names
jpyen opened a new issue #14167:
URL: https://github.com/apache/airflow/issues/14167
**Apache Airflow version**: 2.0.1
**Kubernetes version**: 1.18
**What happened**:
Tasks launched as pods inside a k8s cluster will not have unique names, if a pod_template is used.
**What you expected to happen**:
Tasks launched by KubernetesPodOperator should have unique pod-names. This works for tasks solely defined by the KubernetesPodOperator object. However, this does not apply when a 'pod_template_file' is used.
I was trying to define the pod_name additionally to the template_file by the KubernetesOperator attribute "name", but this is prohibited [here](https://github.com/apache/airflow/blob/9034f277ef935df98b63963c824ba71e0dcd92c7/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py#L413) , hence a non-unique pod-name based on the template file.
I don't see a good reason why pod names should be non unique when a pod_template is utilized.
**How to reproduce it**:
Create a new task with KubernetesPodOperator and make use of the 'pod_template_file' attribute.
**Anything else we need to know**:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Dr-Denzy commented on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
Dr-Denzy commented on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-790533319
@kaxil please assign to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kakarukeys commented on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
kakarukeys commented on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-817581351
Hi, I had the same problem.
It seems `KubernetesPodOperator`'s `name` argument cannot be used to overwrite `name` in the pod template.
and if I set `generateName` instead of `name`, I got the following errors:
```sh
[2021-04-12 06:19:35,044] {taskinstance.py:1455} ERROR - Missing the required parameter `name` when calling `delete_namespaced_pod`
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 485, in create_new_pod_for_operator
launcher.start_pod(self.pod, startup_timeout=self.startup_timeout_seconds)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 112, in start_pod
while self.pod_not_started(pod):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 182, in pod_not_started
state = self._task_status(self.read_pod(pod))
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 329, in wrapped_f
return self.call(f, *args, **kw)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 409, in call
do = self.iter(retry_state=retry_state)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 368, in iter
raise retry_exc.reraise()
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 186, in reraise
raise self.last_attempt.result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
raise self._exception
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 412, in call
result = fn(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 241, in read_pod
return self._client.read_namespaced_pod(pod.metadata.name, pod.metadata.namespace)
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 19078, in read_namespaced_pod
(data) = self.read_namespaced_pod_with_http_info(name, namespace, **kwargs) # noqa: E501
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 19120, in read_namespaced_pod_with_http_info
raise ValueError("Missing the required parameter `name` when calling `read_namespaced_pod`") # noqa: E501
ValueError: Missing the required parameter `name` when calling `read_namespaced_pod`
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1112, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1285, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1315, in _execute_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 339, in execute
final_state, _, result = self.create_new_pod_for_operator(labels, launcher)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 495, in create_new_pod_for_operator
launcher.delete_pod(self.pod)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 93, in delete_pod
self._client.delete_namespaced_pod(
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 10173, in delete_namespaced_pod
(data) = self.delete_namespaced_pod_with_http_info(name, namespace, **kwargs) # noqa: E501
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 10218, in delete_namespaced_pod_with_http_info
raise ValueError("Missing the required parameter `name` when calling `delete_namespaced_pod`") # noqa: E501
ValueError: Missing the required parameter `name` when calling `delete_namespaced_pod`
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-822410422
The issue itself is fixed but I agree with @jedcunningham the better solution would be to allow control of "randomization".
Can you create a new Github Issue for that one @jedcunningham ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aviau commented on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
aviau commented on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-790709875
Not sure, but maybe fixed by https://github.com/apache/airflow/commit/809b4f9b18c7040682e17879248d714f2664273d ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kakarukeys edited a comment on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
kakarukeys edited a comment on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-817581351
Hi, I had the same problem.
It seems `KubernetesPodOperator`'s `name` argument cannot be used to overwrite `name` in the pod template.
and if I set `generateName` instead of `name` in the pod template, I got the following errors:
```sh
[2021-04-12 06:19:35,044] {taskinstance.py:1455} ERROR - Missing the required parameter `name` when calling `delete_namespaced_pod`
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 485, in create_new_pod_for_operator
launcher.start_pod(self.pod, startup_timeout=self.startup_timeout_seconds)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 112, in start_pod
while self.pod_not_started(pod):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 182, in pod_not_started
state = self._task_status(self.read_pod(pod))
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 329, in wrapped_f
return self.call(f, *args, **kw)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 409, in call
do = self.iter(retry_state=retry_state)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 368, in iter
raise retry_exc.reraise()
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 186, in reraise
raise self.last_attempt.result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
raise self._exception
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 412, in call
result = fn(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 241, in read_pod
return self._client.read_namespaced_pod(pod.metadata.name, pod.metadata.namespace)
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 19078, in read_namespaced_pod
(data) = self.read_namespaced_pod_with_http_info(name, namespace, **kwargs) # noqa: E501
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 19120, in read_namespaced_pod_with_http_info
raise ValueError("Missing the required parameter `name` when calling `read_namespaced_pod`") # noqa: E501
ValueError: Missing the required parameter `name` when calling `read_namespaced_pod`
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1112, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1285, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1315, in _execute_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 339, in execute
final_state, _, result = self.create_new_pod_for_operator(labels, launcher)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 495, in create_new_pod_for_operator
launcher.delete_pod(self.pod)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 93, in delete_pod
self._client.delete_namespaced_pod(
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 10173, in delete_namespaced_pod
(data) = self.delete_namespaced_pod_with_http_info(name, namespace, **kwargs) # noqa: E501
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 10218, in delete_namespaced_pod_with_http_info
raise ValueError("Missing the required parameter `name` when calling `delete_namespaced_pod`") # noqa: E501
ValueError: Missing the required parameter `name` when calling `delete_namespaced_pod`
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-819723680
The latest provider should allow setting a `name` kwarg, however, the right fix for this is to decouple the randomization behavior from the way the name is set. We should add a new flag to control whether the name is randomized, defaulted to True.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aviau edited a comment on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
aviau edited a comment on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-783685448
Really important for us too! This makes pod_template_file completely unusable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aviau edited a comment on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
aviau edited a comment on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-783685448
Really important for us too! This makes `pod_template_file` completely unusable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-776503397
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #14167:
URL: https://github.com/apache/airflow/issues/14167
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aviau commented on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
aviau commented on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-783685448
Really important for us too!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kakarukeys edited a comment on issue #14167: KubernetesPodOperator Pods no unique names
Posted by GitBox <gi...@apache.org>.
kakarukeys edited a comment on issue #14167:
URL: https://github.com/apache/airflow/issues/14167#issuecomment-817581351
Hi, I had the same problem.
It seems `KubernetesPodOperator`'s `name` argument cannot be used to overwrite `name` in the pod template.
and if I set `generateName` instead of `name` in the pod template, I got the following errors:
```sh
[2021-04-12 06:19:35,044] {taskinstance.py:1455} ERROR - Missing the required parameter `name` when calling `delete_namespaced_pod`
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 485, in create_new_pod_for_operator
launcher.start_pod(self.pod, startup_timeout=self.startup_timeout_seconds)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 112, in start_pod
while self.pod_not_started(pod):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 182, in pod_not_started
state = self._task_status(self.read_pod(pod))
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 329, in wrapped_f
return self.call(f, *args, **kw)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 409, in call
do = self.iter(retry_state=retry_state)
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 368, in iter
raise retry_exc.reraise()
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 186, in reraise
raise self.last_attempt.result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
raise self._exception
File "/home/airflow/.local/lib/python3.8/site-packages/tenacity/__init__.py", line 412, in call
result = fn(*args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 241, in read_pod
return self._client.read_namespaced_pod(pod.metadata.name, pod.metadata.namespace)
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 19078, in read_namespaced_pod
(data) = self.read_namespaced_pod_with_http_info(name, namespace, **kwargs) # noqa: E501
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 19120, in read_namespaced_pod_with_http_info
raise ValueError("Missing the required parameter `name` when calling `read_namespaced_pod`") # noqa: E501
ValueError: Missing the required parameter `name` when calling `read_namespaced_pod`
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1112, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1285, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1315, in _execute_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 339, in execute
final_state, _, result = self.create_new_pod_for_operator(labels, launcher)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py", line 495, in create_new_pod_for_operator
launcher.delete_pod(self.pod)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/kubernetes/pod_launcher.py", line 93, in delete_pod
self._client.delete_namespaced_pod(
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 10173, in delete_namespaced_pod
(data) = self.delete_namespaced_pod_with_http_info(name, namespace, **kwargs) # noqa: E501
File "/home/airflow/.local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 10218, in delete_namespaced_pod_with_http_info
raise ValueError("Missing the required parameter `name` when calling `delete_namespaced_pod`") # noqa: E501
ValueError: Missing the required parameter `name` when calling `delete_namespaced_pod`
```
Airflow 2.0.1
DOKS 1.20.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org