You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/21 21:51:40 UTC

[GitHub] [airflow] dimon222 opened a new issue #10456: PodRuntimeInfoEnv is not working in 1.10.11

dimon222 opened a new issue #10456:
URL: https://github.com/apache/airflow/issues/10456

<!--

Welcome to Apache Airflow! For a smooth issue process, try to answer the following questions.
Don't worry if they're not all applicable; just try to include what you can :-)

If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>

Please delete these comment blocks before submitting the issue.

**Apache Airflow version**: 1.10.11

**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.17.0

**What happened**:
PodRuntimeInfoEnv object is not working in Airflow 1.10.11. It might be result of some recent (past 1 year?) refactoring of kubernetes code or so.

Exception thrown is this
```
Invalid value for `field_path`, must not be `None`
```

**What you expected to happen**:
It's able to initialize and produce respective KubernestPodOperator with it.

**How to reproduce it**:
Create PodRuntimeInfoEnv object, try to plug it in KubernetesPodOperator. Kubernetes Pod Operator never gets submitted and stacktrace pops.

It happens 100% of times. I know for sure that it worked perfectly fine on Airflow 1.10.9. So might be in 1.10.10 or 1.10.11 there was introduced regression.

After some investigation it seem to bring me to this place - https://github.com/apache/airflow/blob/44d4ae809c1e3784ff95b6a5e95113c3412e56b3/airflow/kubernetes/pod_runtime_info_env.py#L51
The object in there gets initialized like this:
```
field_ref=k8s.V1ObjectFieldSelector(
self.field_path
)
```

As you might see, field_path is first argument. However, if I look at official kubernetes-python code, the first argument in `V1ObjectFieldSelector` constructor is api_version.
https://github.com/kubernetes-client/python/blob/master/kubernetes/client/models/v1_object_field_selector.py#L45
Since its clearly not provided in here, we end up with exception related to field_path value:
https://github.com/kubernetes-client/python/blob/master/kubernetes/client/models/v1_object_field_selector.py#L103

Possible solutions could be:
1. Pass hardcoded api_version as well, this way ensuring that in case Kubernetes API changes, its remembered which version it complies to
```
field_ref=k8s.V1ObjectFieldSelector(
api_version='v1',
field_path=self.field_path
)
```
2. Instead of using positional arg, use kwarg in V1ObjectFieldSelector
```
field_ref=k8s.V1ObjectFieldSelector(
field_path=self.field_path
)
```

For people who might already be noticing issue right now the simplest approach would be inheritance that would overwrite or monkeypatch the `PodRuntimeInfoEnv ` class (specifically its method `to_k8s_client_obj`) and then use with it one of above mentioned solutions.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org