You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Abhishek Sinha <ab...@infoworks.io> on 2018/11/09 14:39:30 UTC

Error for unicode characters in Bash script output

Hi,

I get the following error if a script inside bash operator gives out
unicode characters:

Traceback (most recent call last):
  File "/home/ec2-user/resources/python27/lib/python2.7/threading.py", line
810, in __bootstrap_inner
    self.run()
  File "/home/ec2-user/resources/python27/lib/python2.7/threading.py", line
763, in run
    self.__target(*self.__args, **self.__kwargs)
  File
"/home/ec2-user/resources/python27/lib/python2.7/site-packages/airflow/task_runner/base_task_runner.py",
line 95, in _read_task_logs
    self.logger.info('Subtask: {}'.format(line.rstrip('\n')))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in
position 572: ordinal not in range(128)


I saw in the BashOperator code that there is a way to override
output_encoding (default: UTF-8). However, in the base_task_runner.py code,
I see that the encoding is hardcoded to UTF-8. Am I missing something?

Airflow version: 1.8.2





Regards,

Abhishek

Re: Error for unicode characters in Bash script output

Posted by Abhishek Sinha <ab...@infoworks.io>.
The error logs in the thread below appeared in airflow-worker.err. Post
this error, none of the other DAG executes. It requires restarting Airflow
Worker & Scheduler to resume the run.

Airflow version: 1.8.2
Using Celery executor





Regards,

Abhishek | Infoworks.io | M: +91-9035191078

On 9 November 2018 at 6:39:30 AM, Abhishek Sinha (abhishek@infoworks.io)
wrote:

Hi,

I get the following error if a script inside bash operator gives out
unicode characters:

Traceback (most recent call last):
  File "/home/ec2-user/resources/python27/lib/python2.7/threading.py", line
810, in __bootstrap_inner
    self.run()
  File "/home/ec2-user/resources/python27/lib/python2.7/threading.py", line
763, in run
    self.__target(*self.__args, **self.__kwargs)
  File
"/home/ec2-user/resources/python27/lib/python2.7/site-packages/airflow/task_runner/base_task_runner.py",
line 95, in _read_task_logs
    self.logger.info('Subtask: {}'.format(line.rstrip('\n')))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in
position 572: ordinal not in range(128)


I saw in the BashOperator code that there is a way to override
output_encoding (default: UTF-8). However, in the base_task_runner.py code,
I see that the encoding is hardcoded to UTF-8. Am I missing something?

Airflow version: 1.8.2





Regards,

Abhishek