[jira] [Commented] (AIRFLOW-705) airflow.configuration.run_command output does not decode command output

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929172#comment-15929172 ] 

ASF subversion and git services commented on AIRFLOW-705:
---------------------------------------------------------

Commit 0bb6f2f458ddf305d77fc63d5e817c9b6eb22270 in incubator-airflow's branch refs/heads/master from [~thesquelched]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=0bb6f2f ]

[AIRFLOW-705][AIRFLOW-706] Fix run_command bugs

`run_command` currently uses `str.split` for shell
commands, but this
breaks commands like `echo "foo bar"`.  Instead,
use `shlex.split`,
which understands how to handle quotes.

In python3, subprocess functions return
stdout/stderr as `bytes`, not
`str`.  To fix py2/py3 compatibility,
`run_command` now decodes the
output so that it always returns a `str`.

Dear Airflow Maintainers,

Please accept this PR that addresses the following
issues:
-
https://issues.apache.org/jira/browse/AIRFLOW-705
-
https://issues.apache.org/jira/browse/AIRFLOW-706

Testing Done:
- Changed `sql_alchemy_conn` to
`sql_alchemy_conn_cmd` in test configuration to
exercise `run_command` in tests

Closes #2053 from thesquelched/AIRFLOW-706


> airflow.configuration.run_command output does not decode command output
> -----------------------------------------------------------------------
>
>                 Key: AIRFLOW-705
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-705
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: Airflow 1.7.1.3
>            Reporter: Scott Kruger
>
> In python3, command outputs (to {{stdout}} and {{stderr}}) are of type {{bytes}}, not {{str}}.  This causes the following exception when the {{sql_alchemy_conn_cmd}} option is used:
> {noformat}
> Traceback (most recent call last):
>   File "/usr/local/bin/airflow", line 4, in <module>
>     from airflow import configuration
>   File "/usr/local/lib/python3.5/site-packages/airflow/__init__.py", line 29, in <module>
>     from airflow import configuration as conf
>   File "/usr/local/lib/python3.5/site-packages/airflow/configuration.py", line 657, in <module>
>     conf.read(AIRFLOW_CONFIG)
>   File "/usr/local/lib/python3.5/site-packages/airflow/configuration.py", line 519, in read
>     self._validate()
>   File "/usr/local/lib/python3.5/site-packages/airflow/configuration.py", line 447, in _validate
>     "sqlite" in self.get('core', 'sql_alchemy_conn')):
> TypeError: a bytes-like object is required, not 'str'
> {noformat}
> For compatibility between python 2 and 3, {{airflow.configuration.run_command}} should decode {{stdout}} to produce a (unicode) string.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)