[jira] [Commented] (AIRFLOW-4363) Encounter JSON Decode Error when using docker operator

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16853639#comment-16853639 ] 

ASF GitHub Bot commented on AIRFLOW-4363:
-----------------------------------------

benbenbang commented on pull request #5137: [AIRFLOW-4363] Fix Json encoding error when retrieving `status` from cli in docker operator
URL: https://github.com/apache/airflow/pull/5137
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Encounter JSON Decode Error when using docker operator
> ------------------------------------------------------
>
>                 Key: AIRFLOW-4363
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4363
>             Project: Apache Airflow
>          Issue Type: Bug
>         Environment: - Mac OS 10.14
> - python 3.6.8
> - airflow 1.10.2
>            Reporter: Ben Chen
>            Assignee: Ben Chen
>            Priority: Blocker
>
> *[Description]* . 
> When using the docker_operator, I experienced some issue while using Mac OS 10.14.4. The error was json.JSONDecodeError. After my investigation about this error, I found that there are several messages for logging aren't well separated, for example it contains \n inside one single message which should be split into 2 to more different messages.
> *[Update]*  
> Confirmed that issue came from the implementation in airflow, issue cannot be solved by just passing `decode` to parameter in docker.pull method in docker api.
> *[Solution]* . 
>  For now, I use try-catch to run the original implementation, and in the exception part I split the message to list and then parse it. Looking for simpler solution to this non critical but still blocking point.
> *[Logs]*  
> {docker_operator.py:188}
> INFO - Starting docker container from image hello-world
> {docker_operator.py:202}
> INFO - Pulling docker image hello-world
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Pulling fs layer
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Downloading
> {docker_operator.py:207}
> INFO - Download complete
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Extracting
> {docker_operator.py:207}
> INFO - Pull complete
> {docker_operator.py:207}
> INFO - Digest: sha256:92695bc579f31df7a63da6922075d0666e565ceccad16b59c3374d2cf4e8e50e
> {docker_operator.py:207}
> INFO - Pulling from library/hello-world
> {docker_operator.py:207}
> INFO - Digest: sha256:1a67c1115b199aa9d964d5da5646917cbac2d5450c71a1deed7b1bfb79c2c82d
> {models.py:1788}
> ERROR - Extra data: line 2 column 1 (char 70)
>  Traceback (most recent call last):
>  line 1657, in _run_raw_task, result = task_copy.execute(context=context)
>  line 205, in execute output = json.loads(line)
>  line 354, in loads, return _default_decoder.decode(s)
>  line 342, in decode, raise JSONDecodeError("Extra data", s, end)
>  json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 70)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)