You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/21 17:39:18 UTC

[GitHub] [airflow] bpleines edited a comment on issue #15670: Pod overrides not working

bpleines edited a comment on issue #15670:
URL: https://github.com/apache/airflow/issues/15670#issuecomment-846125695


   Thanks for the example @kaxil. Understood if you want to keep the discussion for `2.0+` only, but wanted to note that I don't think this pod_override strategy works for `1.10.15`. I can see the scheduler pulling in the pod_override (logs attached), but the resources override are not realized on the resulting worker pod's container.
   
   ```
   from airflow import DAG
   from airflow.operators.python_operator import PythonOperator
   from datetime import timedelta
   from kubernetes.client import models as k8s
   import os
   import time
   
   from dag_utils import utils
   
   def test():
       print('Sleeping for 60')
       time.sleep(60)
   
   with DAG(
       'kubernetes_pod_override_example',
       default_args=utils.get_default_args(),
       description='Kubernetes pod_override Example',
       schedule_interval=timedelta(minutes=2),
       max_active_runs=1) as dag:
   
           pod_override = PythonOperator(
                task_id='pod_override',
                python_callable=test,
                executor_config={
                    "pod_override": k8s.V1PodSpec(
                        containers=[
                            k8s.V1Container(
                                name="base",
                                resources=k8s.V1ResourceRequirements(
                                    requests={'cpu': '4', 'memory': '8Gi'},
                                    limits={'cpu': '4', 'memory': '8Gi'}
                                )
                            )
                        ]
                    )
                }
           )
   
           pod_override
   ```
   [scheduler.log](https://github.com/apache/airflow/files/6524193/scheduler.log)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org