You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/07/24 09:43:46 UTC
[GitHub] [airflow] Zuchens opened a new issue #9966: Cutom kubernetes_label lost after upgrade to 1.10.11
Zuchens opened a new issue #9966:
URL: https://github.com/apache/airflow/issues/9966
**Apache Airflow version**: 1.10.11
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): 1.16
**Environment**:
- **Cloud provider or hardware configuration**: AKS
- **OS** (e.g. from /etc/os-release):
- **Kernel** (e.g. `uname -a`):
- **Install tools**: helm chart
- **Others**: KubernetesExecutor
**What happened**:
We have set up environment variable: AIRFLOW__KUBERNETES_LABELS__TYPE='airflow-task-pod' that should create label in airflow configuration.
After ugrading airflow from 1.10.10 to 1.10.11, custom kubernetes_label in config stopped appearing in the logs.
Previous sample from kibana:
```
"kubernetes": {
"labels": {
"try_number": "1",
"airflow-worker": "xxx",
"airflow_version": "1.10.10",
"dag_id": "xxx",
"kubernetes_executor": "True",
"task_id": "xxx",,
"execution_date": "2020-07-21T02_10_00_plus_00_00",
"type": "airflow-task-pod"
},
}
```
Current sample:
```
"kubernetes": {
"labels": {
"try_number": "1",
"airflow_version": "1.10.11",
"task_id": "xxx",
"dag_id": "xxx",
"kubernetes_pod_operator": "True",
"execution_date": "2020-07-21T0210000000-d4ef2bace"
}
}
```
**What you expected to happen**:
After upgrading logs should stay the same as in airflow 1.10.10.
**What do you think went wrong?**:
The label appears in airflow.configuration.as_dict().get('kubernetes_labels') the issue seems to be in PodGenerator.
PodGenerator.construct_pod() will ignore any labels from worker_configuration since they will always be replaced by dynamic_pod configuration labels.
**How to reproduce it**:
Here is a locally run snipet to show the behaviour:
```
import json
from airflow.kubernetes.pod_generator import PodGenerator
from kubernetes.client import V1ObjectMeta, V1Pod
pod = PodGenerator.construct_pod(
namespace="test",
worker_uuid="test",
pod_id="test",
dag_id="test",
task_id="test",
try_number=1,
date="23-07-2020",
command="test",
kube_executor_config=None,
worker_config=V1Pod(metadata=V1ObjectMeta(labels={"type": "airflow-task-pod"})))
print(json.dumps(pod.metadata.labels, indent=2))
```
If this is expected behaviour can you suggest me other way to handle custom labels? :)
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman closed issue #9966: Cutom kubernetes_label lost after upgrade to 1.10.11
Posted by GitBox <gi...@apache.org>.
dimberman closed issue #9966:
URL: https://github.com/apache/airflow/issues/9966
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman edited a comment on issue #9966: Cutom kubernetes_label lost after upgrade to 1.10.11
Posted by GitBox <gi...@apache.org>.
dimberman edited a comment on issue #9966:
URL: https://github.com/apache/airflow/issues/9966#issuecomment-668701195
Hi @Zuchens , thanks for bringing this to our attention and providing a test to reproduce. I have added a fix to this for 1.10.12.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman commented on issue #9966: Cutom kubernetes_label lost after upgrade to 1.10.11
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #9966:
URL: https://github.com/apache/airflow/issues/9966#issuecomment-670606146
fixed https://github.com/apache/airflow/pull/10084
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman commented on issue #9966: Cutom kubernetes_label lost after upgrade to 1.10.11
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #9966:
URL: https://github.com/apache/airflow/issues/9966#issuecomment-668701195
Hi @Zuchens , thanks for bringing this to our attention. I have added a fix to this for 1.10.12.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #9966: Cutom kubernetes_label lost after upgrade to 1.10.11
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #9966:
URL: https://github.com/apache/airflow/issues/9966#issuecomment-663500995
cc @dimberman
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org