You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/21 17:33:43 UTC
[GitHub] [airflow] subkanthi opened a new issue #21717: executor_config unpickled as str when using KubernetesExecutor
subkanthi opened a new issue #21717:
URL: https://github.com/apache/airflow/issues/21717
### Apache Airflow version
2.2.3 (latest released)
### What happened
```
start_task_executor_config = {
"pod_override": k8s.V1Pod(metadata=k8s.V1ObjectMeta(annotations={"test": "annotation"}))
}
@task(
executor_config=start_task_executor_config,
queue='kubernetes',
task_id='task_with_kubernetes_executor',
)
def task_with_template():
print_stuff()
```
When the k8s task is defined with the decorator and when executor_config is passed, it seems to be picked as `str`.
In pod_generator.py, it throws the following type error, because it gets unpicked as `str` and fails the `if isinstance(` checks
```
if isinstance(k8s_object, k8s.V1Pod):
return k8s_object
elif isinstance(k8s_legacy_object, dict):
warnings.warn(
'Using a dictionary for the executor_config is deprecated and will soon be removed.'
'please use a `kubernetes.client.models.V1Pod` class with a "pod_override" key'
' instead. ',
category=DeprecationWarning,
)
return PodGenerator.from_legacy_obj(obj)
else:
raise TypeError(
'Cannot convert a non-kubernetes.client.models.V1Pod object into a KubernetesExecutorConfig'
)
```
### What you expected to happen
_No response_
### How to reproduce
_No response_
### Operating System
Ubuntu
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on issue #21717: executor_config unpickled as str when using KubernetesExecutor
Posted by GitBox <gi...@apache.org>.
subkanthi commented on issue #21717:
URL: https://github.com/apache/airflow/issues/21717#issuecomment-1054522299
Just to add some context, I tested it by running scheduler and webserver in debug mode locally with MySQL and connecting to a kind cluster. Hopefully its not specific to M1 architecture.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #21717: executor_config unpickled as str when using KubernetesExecutor
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #21717:
URL: https://github.com/apache/airflow/issues/21717#issuecomment-1052744769
@dstandish @ephraimbuddy @dimberman @kaxil - maybe you can take a look - I am not too close to those parts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org