You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/10 17:01:02 UTC
[GitHub] [airflow] ybressler opened a new issue #22154: Ambiguity of format for template fields in Airflow Kubernetes Operator (airflow.providers.cncf.kubernetes.operators.kubernetes_pod
ybressler opened a new issue #22154:
URL: https://github.com/apache/airflow/issues/22154
### Apache Airflow Provider(s)
amazon
### Versions of Apache Airflow Providers
3.0.2 (https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/_modules/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.html)
### Apache Airflow version
2.2.4 (latest released)
### Operating System
NA
### Deployment
MWAA
### Deployment details
NA
### What happened
Documentation for [`airflow.providers.cncf.kubernetes.operators.kubernetes_pod`](https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/_api/airflow/providers/cncf/kubernetes/operators/kubernetes_pod/index.html#airflow.providers.cncf.kubernetes.operators.kubernetes_pod.KubernetesPodOperator) suggests that [`template_fields`](https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/_api/airflow/providers/cncf/kubernetes/operators/kubernetes_pod/index.html#airflow.providers.cncf.kubernetes.operators.kubernetes_pod.KubernetesPodOperator.template_fields) are a list of string values. This is accordance with Airflow's custom operator guides, to create a class attribute with a list of string values. However, when consulting the [source code](https://airflow.apache.org/docs/apache-airflow-providers-cncf-kubernetes/stable/_modules/airflow/providers/cncf/kubernetes/operators/kubernetes_pod.html#KubernetesPodOperator.template_fields), the value
s for `template_fields` are stored in a tuple:
```
template_fields: Sequence[str] = (
'image',
'cmds',
'arguments',
'env_vars',
'labels',
'config_file',
'pod_template_file',
'namespace',
)
```
This is problematic from a documentation standpoint, since it is unclear how to extend behaviors with ambiguous typing. For example, I will encounter a runtime error when extending template fields as such:
```
from airflow.providers.cncf.kubernetes.operators.kubernetes_pod import KubernetesPodOperator
class CustomKubernetesPodOperator(KubernetesPodOperator):
# The following will fail
template_fields = KubernetesPodOperator.template_fields + ["name"]
def __init__(self):
super.__init__()
```
My solution is to wrap the template_fields in a `list` constructor as such:
```
template_fields = list(KubernetesPodOperator.template_fields) + ["name"]
```
But this is not an elegant solution. Plus, documentation was not helpful. A solution would be to either clarify the documentation (that this is stored as a tuple default), or convert to storing as a list. (As typical with other Airflow operators.)
### What you expected to happen
_No response_
### How to reproduce
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #22154: Ambiguity of format for template fields in Airflow Kubernetes Operator (airflow.providers.cncf.kubernetes.operators.kubernetes_pod)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #22154:
URL: https://github.com/apache/airflow/issues/22154#issuecomment-1065394841
We should rewrite the documentation to say the attribute is only guaranteed to be a _sequence_, and maybe add some documentation to recommend using `*` instead. For your example, this is the safe syntax (without the overhead of creating a temporary list):
```python
template_fields = [*KubernetesPodOperator.template_fields, "name"]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #22154: Ambiguity of format for template fields in Airflow Kubernetes Operator (airflow.providers.cncf.kubernetes.operators.kubernetes_pod)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #22154:
URL: https://github.com/apache/airflow/issues/22154#issuecomment-1066021985
Done in readthedocs/sphinx-autoapi#330
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #22154: Ambiguity of format for template fields in Airflow Kubernetes Operator (airflow.providers.cncf.kubernetes.operators.kubernetes_pod)
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #22154:
URL: https://github.com/apache/airflow/issues/22154#issuecomment-1066018811
Hmm, this is weird. The documentation is automatically generated from AutoAPI, and I have no idea why it chooses to render the tuple as a list. Perhaps we need to raise this issue to AutoAPI. https://github.com/readthedocs/sphinx-autoapi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ybressler commented on issue #22154: Ambiguity of format for template fields in Airflow Kubernetes Operator (airflow.providers.cncf.kubernetes.operators.kubernetes_pod)
Posted by GitBox <gi...@apache.org>.
ybressler commented on issue #22154:
URL: https://github.com/apache/airflow/issues/22154#issuecomment-1066110921
Good solution. And yes, would be even more helpful for describing how to append to template fields with this unpacking approach. Thanks for being on top of this @uranusjr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org