You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/21 17:39:18 UTC
[GitHub] [airflow] bpleines edited a comment on issue #15670: Pod overrides not working
bpleines edited a comment on issue #15670:
URL: https://github.com/apache/airflow/issues/15670#issuecomment-846125695
Thanks for the example @kaxil. Understood if you want to keep the discussion for `2.0+` only, but wanted to note that I don't think this pod_override strategy works for `1.10.15`. I can see the scheduler pulling in the pod_override (logs attached), but the resources override are not realized on the resulting worker pod's container.
```
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from datetime import timedelta
from kubernetes.client import models as k8s
import os
import time
from dag_utils import utils
def test():
print('Sleeping for 60')
time.sleep(60)
with DAG(
'kubernetes_pod_override_example',
default_args=utils.get_default_args(),
description='Kubernetes pod_override Example',
schedule_interval=timedelta(minutes=2),
max_active_runs=1) as dag:
pod_override = PythonOperator(
task_id='pod_override',
python_callable=test,
executor_config={
"pod_override": k8s.V1PodSpec(
containers=[
k8s.V1Container(
name="base",
resources=k8s.V1ResourceRequirements(
requests={'cpu': '4', 'memory': '8Gi'},
limits={'cpu': '4', 'memory': '8Gi'}
)
)
]
)
}
)
pod_override
```
[scheduler.log](https://github.com/apache/airflow/files/6524193/scheduler.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org