[GitHub] [airflow] macdonald-keith-vmware opened a new issue #18380: KubernetesPodOperator not overwriting pod.metadata.name when using pod_template yaml file causing an exception

[GitBox <gi...@apache.org>]

macdonald-keith-vmware opened a new issue #18380:
URL: https://github.com/apache/airflow/issues/18380


   ### Apache Airflow version
   
   2.1.2
   
   ### Operating System
   
   Ubuntu 20.04.3 LTS
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other 3rd-party Helm chart
   
   ### Deployment details
   
   Helm, Bitnami charts, MicroK8 running on Ubuntu cluster.
   
   ### What happened
   
   When running multiple tasks that use the same pod-template yaml file, 
   
   [2021-09-20 15:50:02,605] {pod_launcher.py:95} ERROR - Exception when attempting to create Namespaced Pod: {
   ...
   File "/opt/bitnami/airflow/venv/lib/python3.8/site-packages/kubernetes/client/rest.py", line 231, in request
       raise ApiException(http_resp=r)
   kubernetes.client.rest.ApiException: (409)
   Reason: Conflict
   HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'Date': 'Mon, 20 Sep 2021 15:50:02 GMT', 'Content-Length': '204'})
   HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods \"k8s-small-pod\" already exists","reason":"AlreadyExists","details":{"name":"k8s-small-pod","kind":"pods"},"code":409}
   
   ### What you expected to happen
   
   The pod name in the template file should be overridden at runtime with a unique name.
   
   ### How to reproduce
   
   1. Create a pod template  small_pod.yaml
   `apiVersion: v1
   kind: Pod
   metadata:
     name: k8s-small-pod
   spec:
     containers:
       - name: small-pod
         imagePullPolicy: IfNotPresent
         args: []
         command: []
         env:
           - name: GIT_SSL_NO_VERIFY
             value: "true"
         envFrom: []
         image: dummy_image
         imagePullPolicy: IfNotPresent
         name: base
         ports: []
     restartPolicy: Never`
   2. Create two simple dags that use the same template
   `passing = KubernetesPodOperator(namespace='airflow',
                             image="python:3.6",
                             cmds=["python","-c"],
                             arguments=["print('hello world')"],
                             labels={"foo": "bar"},
                             name="passing-test",
                             task_id="passing-task",
                             pod_template_file="./templates/small_pod.yaml",
   						  is_delete_operator_pod=True,
                             get_logs=True,
                             dag=dag
                             )` 
   3. Run the dags at the same time
   
   ### Anything else
   
   Occurs every time both pod operators are executing at the same time.
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18380: KubernetesPodOperator not overwriting pod.metadata.name when using pod_template yaml file causing an exception

[GitBox <gi...@apache.org>]

boring-cyborg[bot] commented on issue #18380:
URL: https://github.com/apache/airflow/issues/18380#issuecomment-923088448


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #18380: KubernetesPodOperator not overwriting pod.metadata.name when using pod_template yaml file causing an exception

[GitBox <gi...@apache.org>]

eladkal commented on issue #18380:
URL: https://github.com/apache/airflow/issues/18380#issuecomment-923099550


   Related: https://github.com/apache/airflow/issues/15434
   
   cc @jedcunningham 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org