[jira] [Updated] (AIRFLOW-5270) JSONDecodeError in DockerOperator when force_pull

["Matthew Petersen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew Petersen updated AIRFLOW-5270:
--------------------------------------
    Priority: Minor  (was: Major)

> JSONDecodeError in DockerOperator when force_pull
> -------------------------------------------------
>
>                 Key: AIRFLOW-5270
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5270
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: operators
>    Affects Versions: 1.10.3
>         Environment: MacOS Mojave
>            Reporter: Matthew Petersen
>            Assignee: Matthew Petersen
>            Priority: Minor
>
> Lines of the Docker API response usually end with "\r\n". We .strip() them to correct for this, but sometimes two lines are joined together, creating one line that is not valid JSON. The exact error we see is
> ```
> json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 55)
> ```
> And can be reproduced with the following code
> ```
> pip install docker
> from docker import APIClient
> import json
> cli = APIClient(base_url='unix://var/run/docker.sock')
> for l in cli.pull('hello-world', stream=*True*):
>     output = json.loads(l.decode('utf-8').strip()
> ```
> which is exactly what this line of `DockerOperator` does which causes it to crash
> [https://github.com/apache/airflow/blob/46885ccdbc8618f295b8f986da95c1abbb85bb02/airflow/operators/docker_operator.py#L259]
> The solution is to replace "\r\n" with "\n" before json parsing



--
This message was sent by Atlassian Jira
(v8.3.2#803003)