[jira] [Created] (AIRFLOW-5270) JSONDecodeError in DockerOperator when force_pull

["Matthew Petersen (Jira)" <ji...@apache.org>]

Matthew Petersen created AIRFLOW-5270:
-----------------------------------------

             Summary: JSONDecodeError in DockerOperator when force_pull
                 Key: AIRFLOW-5270
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5270
             Project: Apache Airflow
          Issue Type: Bug
          Components: operators
    Affects Versions: 1.10.3
         Environment: MacOS Mojave
            Reporter: Matthew Petersen
            Assignee: Matthew Petersen


Lines of the Docker API response usually end with "\r\n". We .strip() them to correct for this, but sometimes two lines are joined together, creating one line that is not valid JSON. The exact error we see is

```
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 55)
```

And can be reproduced with the following code

```

pip install docker


from docker import APIClient
import json

cli = APIClient(base_url='unix://var/run/docker.sock')

for l in cli.pull('hello-world', stream=*True*):
    output = json.loads(l.decode('utf-8').strip()
```

which is exactly what this line of `DockerOperator` does which causes it to crash
[https://github.com/apache/airflow/blob/46885ccdbc8618f295b8f986da95c1abbb85bb02/airflow/operators/docker_operator.py#L259]

The solution is to replace "\r\n" with "\n" before json parsing



--
This message was sent by Atlassian Jira
(v8.3.2#803003)