[jira] [Created] (AIRFLOW-668) Configuration parsing doesn't work properly with python 3

["Rafael Gomes Fernandes (JIRA)" <ji...@apache.org>]

Rafael Gomes Fernandes created AIRFLOW-668:
----------------------------------------------

             Summary: Configuration parsing doesn't work properly with python 3
                 Key: AIRFLOW-668
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-668
             Project: Apache Airflow
          Issue Type: Bug
         Environment:  Airflow version: v1.7.1.3
- Airflow components: webserver and scheduler with a postgres database and CeleryExecutor
- Python Version: 3.4.5
            Reporter: Rafael Gomes Fernandes


The problem is: if you use python3 and the '_cmd' on the config file airflow will not start due the error:
File "~/test/env/airflow3/lib/python3.4/site-packages/airflow/configuration.py", line 447, in _validate
    "sqlite" in self.get('core', 'sql_alchemy_conn')):
TypeError: 'str' does not support the buffer interface

To reproduce the problem change the following line on airflow.cfg:
sql_alchemy_conn_cmd = echo sqlite:////~/airflow/airflow.db

The solution is change the following run_command method's line on configuration.py:
command.split(), stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)

By setting the universal_newlines to true the file objects stdout and stderr are opened as text files and treated as string in python 2 and python 3 avoiding the error.

run_command with universal_newlines=True:

When using python 3 output type: <class 'str'> and no error.

When using python 2 output type: <type 'str'> and no error.


run_command as it is :
When using python 3 output type: <class 'bytes'> and TypeError.

When using python 2 output type: <type 'str'> and no error.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)