[jira] [Comment Edited] (AIRFLOW-130) SSHHook fails in OSX due to mktemp implementation

["Chris Riccomini (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289593#comment-15289593 ] 

Chris Riccomini edited comment on AIRFLOW-130 at 5/18/16 6:57 PM:
------------------------------------------------------------------

I checked my CentOX box, and it has:

{noformat}
  -p DIR, --tmpdir[=DIR]  interpret TEMPLATE relative to DIR; if DIR is not
                        specified, use $TMPDIR if set, else /tmp.  With
                        this option, TEMPLATE must not be an absolute name;
                        unlike with -t, TEMPLATE may contain slashes, but
                        mktemp creates only the final component
  -t                  interpret TEMPLATE as a single file name component,
                        relative to a directory: $TMPDIR, if set; else the
                        directory specified via -p; else /tmp [deprecated]
{noformat}


was (Author: criccomini):
I checked my CentOX box, and it has:

{noformat}
  -t                  interpret TEMPLATE as a single file name component,
                        relative to a directory: $TMPDIR, if set; else the
                        directory specified via -p; else /tmp [deprecated]
{noformat}

> SSHHook fails in OSX due to mktemp implementation
> -------------------------------------------------
>
>                 Key: AIRFLOW-130
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-130
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: hooks
>    Affects Versions: Airflow 1.7.0
>         Environment: Airflow components: 
> webserver and scheduler with a mysql database and LocalExecutor
> Relevant `airflow.cfg` settings:
> ssh_default connection correctly configured
> Python Version: 
> Python 2.7.11
> Operating System: 
> Darwin Kernel Version 15.4.0: Fri Feb 26 22:08:05 PST 2016; root:xnu-3248.40.184~3/RELEASE_X86_64 x86_64
> Python packages: 
> airflow (1.7.0)
> alembic (0.8.6)
> Babel (1.3)
> cffi (1.6.0)
> chartkick (0.4.2)
> croniter (0.3.12)
> cryptography (1.3.2)
> dill (0.2.5)
> enum34 (1.1.6)
> Flask (0.10.1)
> Flask-Admin (1.4.0)
> Flask-Cache (0.13.1)
> Flask-Login (0.2.11)
> Flask-WTF (0.12)
> future (0.15.2)
> gunicorn (19.3.0)
> idna (2.1)
> ipaddress (1.0.16)
> itsdangerous (0.24)
> Jinja2 (2.8)
> Mako (1.0.4)
> Markdown (2.6.6)
> MarkupSafe (0.23)
> mysqlclient (1.3.7)
> netaddr (0.7.18)
> numpy (1.11.0)
> pandas (0.18.1)
> pip (8.1.2)
> pyasn1 (0.1.9)
> pycparser (2.14)
> Pygments (2.1.3)
> python-dateutil (2.5.3)
> python-editor (1.0)
> pytz (2016.4)
> requests (2.10.0)
> setproctitle (1.1.10)
> setuptools (21.0.0)
> six (1.10.0)
> slackclient (1.0.0)
> SQLAlchemy (1.0.13)
> thrift (0.9.3)
> websocket-client (0.37.0)
> Werkzeug (0.11.9)
> wheel (0.29.0)
> WTForms (2.1)
>            Reporter: Raul Landa
>
> h3. Description of Issue
> * What did you expect to happen?
> In the {{__enter__}} method in the {{SSHTempFileContent}} class ({{ssh_execute_operator.py}}) a file should be created using {{mktemp}}
> * What happened instead?
> The call to mktemp fails, because the {{--tmpdir}} command line option is not supported in OSX
> (see https://developer.apple.com/legacy/library/documentation/Darwin/Reference/ManPages/man1/mktemp.1.html)
> * Stack trace, if appropriate:
> {noformat}
> [2016-05-18 17:39:31,735] {models.py:1144} ERROR - Failed to create remote temp file
> Traceback (most recent call last):
>   File "/Users/rlanda/.virtualenvs/datapath-airflow/bin/airflow", line 15, in <module>
>     args.func(args)
>   File "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/bin/cli.py", line 316, in test
>     ti.run(force=True, ignore_dependencies=True, test_mode=True)
>   File "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/models.py", line 1067, in run
>     result = task_copy.execute(context=context)
>   File "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/contrib/operators/ssh_execute_operator.py", line 116, in execute
>     self.task_id) as remote_file_path:
>   File "/Users/rlanda/.virtualenvs/datapath-airflow/lib/python2.7/site-packages/airflow/contrib/operators/ssh_execute_operator.py", line 52, in __enter__
>     raise AirflowException("Failed to create remote temp file")
> airflow.utils.AirflowException: Failed to create remote temp file
> {noformat}
> The stack trace is not very useful. However looking at the tuple returned by {{pmktemp.communicate()}} is helpful:
> {noformat}
> mktemp: illegal option -- -
> usage: mktemp [-d] [-q] [-t prefix] [-u] template ...
>        mktemp [-d] [-q] [-u] -t prefix 
> {noformat}
> h3. Reproducing the Issue
> Issue happens with normal usage of {{SSHHook}} when attempting to run commands on an OSX box.
> h3. Fix
> Changing {{--tmpdir}} by {{-t}} seems to fix the issue in OSX, but I do not have access to a linux box to check if it breaks there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)