You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "afusr (JIRA)" <ji...@apache.org> on 2019/01/21 09:32:00 UTC
[jira] [Updated] (AIRFLOW-3737) Kubernetes executor cannot handle
long dag/task names
[ https://issues.apache.org/jira/browse/AIRFLOW-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
afusr updated AIRFLOW-3737:
---------------------------
Description:
When using the Kubernetes Executor, if the combined DAG and Task name is longer than 63 characters the scheduler is unable to create a pod and stops processing further dags. For example the logs show the following:
{noformat}
[2019-01-21 08:07:05,337] {rest.py:219} DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"unable to parse requirement: invalid label value: \"very_long_dag_name.very_long_task_name\": must be no more than 63 characters","reason":"BadRequest","code":400}
[2019-01-21 08:07:05,389] {dag_processing.py:579} WARNING - Ending without manager process.
[2019-01-21 08:07:05,390] {jobs.py:1510} INFO - Exited execute loop
{noformat}
The error appears to be returned by the k8s api.
Perhaps related to the code here?
https://github.com/apache/airflow/blob/a7e369f082025fb089d909a6978106bb7cd24798/airflow/contrib/executors/kubernetes_executor.py#L499-L517
This was tested using 1.10.2RC2
was:
When using the Kubernetes Executor, if the combined DAG and Task name is longer than 63 characters the scheduler is unable to create a pod and stops processing further dags. For example the logs show the following:
{noformat}
[2019-01-21 08:07:05,337] {rest.py:219} DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"unable to parse requirement: invalid label value: \"very_long_dag_name.very_long_task_name\": must be no more than 63 characters","reason":"BadRequest","code":400}
[2019-01-21 08:07:05,389] {dag_processing.py:579} WARNING - Ending without manager process.
[2019-01-21 08:07:05,390] {jobs.py:1510} INFO - Exited execute loop
{noformat}
The error appears to be returned by the k8s api.
Perhaps related to the code here?
https://github.com/apache/airflow/blob/a7e369f082025fb089d909a6978106bb7cd24798/airflow/contrib/executors/kubernetes_executor.py#L459
This was tested using 1.10.2RC2
> Kubernetes executor cannot handle long dag/task names
> -----------------------------------------------------
>
> Key: AIRFLOW-3737
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3737
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.10.2
> Reporter: afusr
> Priority: Blocker
>
> When using the Kubernetes Executor, if the combined DAG and Task name is longer than 63 characters the scheduler is unable to create a pod and stops processing further dags. For example the logs show the following:
> {noformat}
> [2019-01-21 08:07:05,337] {rest.py:219} DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"unable to parse requirement: invalid label value: \"very_long_dag_name.very_long_task_name\": must be no more than 63 characters","reason":"BadRequest","code":400}
> [2019-01-21 08:07:05,389] {dag_processing.py:579} WARNING - Ending without manager process.
> [2019-01-21 08:07:05,390] {jobs.py:1510} INFO - Exited execute loop
> {noformat}
> The error appears to be returned by the k8s api.
> Perhaps related to the code here?
> https://github.com/apache/airflow/blob/a7e369f082025fb089d909a6978106bb7cd24798/airflow/contrib/executors/kubernetes_executor.py#L499-L517
> This was tested using 1.10.2RC2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)