You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/01/24 13:05:00 UTC
[jira] [Commented] (AIRFLOW-3760) Kubernetes Executor fails to
launch pod
[ https://issues.apache.org/jira/browse/AIRFLOW-3760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16751097#comment-16751097 ]
ASF GitHub Bot commented on AIRFLOW-3760:
-----------------------------------------
PaulW commented on pull request #4575: [AIRFLOW-3760] - Ensure that we pass the command to the Kubernetes Pod as a List
URL: https://github.com/apache/airflow/pull/4575
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Kubernetes Executor fails to launch pod
> ---------------------------------------
>
> Key: AIRFLOW-3760
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3760
> Project: Apache Airflow
> Issue Type: Bug
> Components: kubernetes
> Affects Versions: 1.10.2
> Reporter: Paul Bramhall
> Priority: Major
>
> *Problem*
> Since 1.10.2, Airflow fails to launch pods within kubernetes with the following error:
> {code:java}
> [2019-01-23 14:06:58,892] {rest.py:219} DEBUG - response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Pod in version \"v1\" cannot be handled as a Pod: v1.Pod.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Command: []string: decode slice: expect [ or n, but found \", error found in #10 byte of ...|ommand\": \"airflow ru|..., bigger context ...|.gcr.io/airflow:161\", \"command\": \"airflow run dag_test|...","reason":"BadRequest","code":400}{code}
> Yaml config sent to Kubernetes:
> {code:java}
> ...
> "spec": {
> "containers": [
> {
> "name": "base",
> "image": "eu.gcr.io/airflow:161",
> "command": "airflow run dag_test start_task 2019-01-23T11:15:00+00:00 --local -sd /usr/local/myorg/app/dags/kubernetes.py",
> "imagePullPolicy": "IfNotPresent",
> "args": [],
> ...
> {code}
> This is due to how the airflow_command is passed through from within airflow/contrib/kubernetes/worker_configuration.py, specifically line 206:
> {code:java}
> cmds=airflow_command,
> {code}
> This needs to be a list in the pod yaml, such as
> {code:java}
> cmds=[airflow_command],
> {code}
> However, simply passing the command like this will not resolve the issue, and will result in the following error:
> {code:java}
> container_linux.go:247: starting container process caused "exec: \"airflow run dag_test start_task 2019-01-23T11:10:00+00:00 --local -sd /usr/local/myorg/app/dags/kubernetes.py\": stat airflow run dag_test start_task 2019-01-23T11:10:00+00:00 --local -sd /usr/local/myorg/app/dags/kubernetes.py: no such file or directory"
> {code}
> *Solution*
> We need to ensure we pass the command as a list to kubernetes, so that this works as expected and the pod is launched:
> {code:java}
> cmds=airflow_command.split(" "),
> {code}
> Yaml config sent to Kubernetes:
> {code:java}
> ...
> "spec": {
> "containers": [
> {
> "name": "base",
> "image": "eu.gcr.io/airflow:161",
> "command": [
> "airflow",
> "run",
> "dag_test",
> "start_task",
> "2019-01-23T15:25:00+00:00",
> "--local",
> "-sd",
> "/usr/local/myorg/app/dags/kubernetes.py"
> ],
> "imagePullPolicy": "IfNotPresent",
> "args": [],
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)