[jira] [Created] (AIRFLOW-1093) UnicodeEncodeError makes scheduler task freeze

["Oleksii Renov (JIRA)" <ji...@apache.org>]

Oleksii Renov created AIRFLOW-1093:
--------------------------------------

             Summary: UnicodeEncodeError makes scheduler task freeze
                 Key: AIRFLOW-1093
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1093
             Project: Apache Airflow
          Issue Type: Bug
          Components: scheduler
    Affects Versions: Airflow 1.8
         Environment:  Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-71-generic x86_64)
            Reporter: Oleksii Renov
            Assignee: Oleksii Renov
            Priority: Minor


The issue is pretty simple, but to identify it I spent good amount of time.

So the thing is that since Airflow v1.8 if output of BashOperator has non ascii symbols this tasks freezes. In my case BashOperator starts Rscript and in R i have some non ascii symbols in standard output. Again the task execution simply freezes.

Basically I already have a solution to this via removing all non ascii symbols before passing them to logging.info function. If it's ok I can prepare this one.

Here is an example of output if I run task by myself from airflow:

2017-04-03 19:48:02,413] {__init__.py:57} INFO - Using executor LocalExecutor
[2017-04-03 19:48:02,462] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2017-04-03 19:48:02,478] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into: /home/ubuntu/D/logs/local_m10_line/m10_line/2017-03-31T08:20:00
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/dist-packages/airflow/task_runner/base_task_runner.py", line 95, in _read_task_logs
    self.logger.info('Subtask: {}'.format(line.rstrip('\n')))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2018' in position 74: ordinal not in range(128)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)