[jira] [Commented] (AIRFLOW-1780) Long Unicode Characters In BashOperator Output Cause Task Hang

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241208#comment-16241208 ] 

ASF subversion and git services commented on AIRFLOW-1780:
----------------------------------------------------------

Commit 1a7b63eb16ffa1cb97cb09f71997dcd39f28e645 in incubator-airflow's branch refs/heads/master from [~aoen]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=1a7b63e ]

[AIRFLOW-1780] Fix long output lines with unicode from hanging parent

Fix long task output lines with unicode from
hanging parent process. Tasks that create output
that gets piped into a file in the parent airflow
process would hang if they had long lines with
unicode characters.

Closes #2758 from aoen/ddavydov--
fix_unicode_output_string


> Long Unicode Characters In BashOperator Output Cause Task Hang
> --------------------------------------------------------------
>
>                 Key: AIRFLOW-1780
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1780
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Dan Davydov
>            Assignee: Dan Davydov
>
> Conditions to replicate:
> Create a DAG with a single BashOperator that logs ~10 long lines of text with at least one unicode character (e.g. cat a file with these contents)
> Use BashTaskRunner
> Log to a file
> Run a worker that picks up the task or an airflow run --local command
> Behavior:
> The BashOperator/cat command hangs
> Most likely this is due to a pipe issue, where the unicode characters are filling up the pipe and .format() is not able to process partial unicode characters. Interestingly removing one of the conditions above (e.g. logging to stdout instead of a file or having short lines doesn't cause the issue).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)