[jira] [Created] (AIRFLOW-3760) Kubernetes Executor fails to launch pod

["Paul Bramhall (JIRA)" <ji...@apache.org>]

Paul Bramhall created AIRFLOW-3760:
--------------------------------------

             Summary: Kubernetes Executor fails to launch pod
                 Key: AIRFLOW-3760
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3760
             Project: Apache Airflow
          Issue Type: Bug
          Components: kubernetes
    Affects Versions: 1.10.2
            Reporter: Paul Bramhall


*Problem*

Since 1.10.2, Airflow fails to launch pods within kubernetes with the following error:
{code:java}
[2019-01-23 14:06:58,892] {rest.py:219} DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Pod in version \"v1\" cannot be handled as a Pod: v1.Pod.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Command: []string: decode slice: expect [ or n, but found \", error found in #10 byte of ...|ommand\": \"airflow ru|..., bigger context ...|.gcr.io/airflow:161\", \"command\": \"airflow run nested_dag_transfer_from_a_to_b|...","reason":"BadRequest","code":400}
{code}
This is due to how the airflow_command is passed through from within airflow/contrib/kubernetes/worker_configuration.py, specifically line 206:

 
{code:java}
cmds=airflow_command,
{code}
This needs to be a sequence in the pod yaml, such as

 
{code:java}
cmds=[airflow_command],
{code}
However, simply passing the command like this will not resolve the issue, and will result in the following error:

 
{code:java}
container_linux.go:247: starting container process caused "exec: \"airflow run dag_test start_task 2019-01-23T11:10:00+00:00 --local -sd /usr/local/org/app/dags/kubernetes.py\": stat airflow run dag_test start_task 2019-01-23T11:10:00+00:00 --local -sd /usr/local/org/app/dags/kubernetes.py: no such file or directory"
{code}
*Solution*

We need to ensure we pass the command as a list to kubernetes, so that this works as expected and the pod is launched:
{code:java}
cmds=airflow_command.split(" "),
{code}
 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)