You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/23 20:04:12 UTC
[GitHub] [airflow] jedcunningham opened a new pull request #11796: Fix backwards compatibility with k8s executor_config resources
jedcunningham opened a new pull request #11796:
URL: https://github.com/apache/airflow/pull/11796
Ran across this issue when testing out the Airflow 2.0.0 alpha 1, specifically the backwards compatibility of resources via `executor_config`:
```
PythonOperator(
task_id="eat",
python_callable=eat,
executor_config={
"KubernetesExecutor": {"request_memory": "128Mi", "limit_memory": "128Mi",},
},
)
```
Results in:
```
/some/path/site-packages/airflow/kubernetes/pod_generator.py:202: DeprecationWarning: Using a dictionary for the executor_config is deprecated and will soon be removed.please use a `kubernetes.client.models.V1Pod` class with a "pod_
override" key instead.
category=DeprecationWarning)
[2020-10-23 15:21:06,806] {scheduler_job.py:1335} ERROR - Exception when executing SchedulerJob._run_scheduler_loop
Traceback (most recent call last):
File "/some/path/site-packages/airflow/jobs/scheduler_job.py", line 1316, in _execute
self._run_scheduler_loop()
File "/some/path/site-packages/airflow/jobs/scheduler_job.py", line 1391, in _run_scheduler_loop
self.executor.heartbeat()
File "/some/path/site-packages/airflow/executors/base_executor.py", line 156, in heartbeat
self.trigger_tasks(open_slots)
File "/some/path/site-packages/airflow/executors/base_executor.py", line 188, in trigger_tasks
executor_config=ti.executor_config)
File "/some/path/site-packages/airflow/executors/kubernetes_executor.py", line 612, in execute_async
kube_executor_config = PodGenerator.from_obj(executor_config)
File "/some/path/site-packages/airflow/kubernetes/pod_generator.py", line 203, in from_obj
return PodGenerator.from_legacy_obj(obj)
File "/some/path/site-packages/airflow/kubernetes/pod_generator.py", line 244, in from_legacy_obj
return PodGeneratorDeprecated(**namespaced).gen_pod()
TypeError: __init__() got an unexpected keyword argument 'request_memory'
```
I did look for test coverage for the backwards compatibility layer but came up empty. Did I miss it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman merged pull request #11796: Fix backwards compatibility with k8s executor_config resources
Posted by GitBox <gi...@apache.org>.
dimberman merged pull request #11796:
URL: https://github.com/apache/airflow/pull/11796
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org