[jira] [Created] (AIRFLOW-200) Hiding import errors / missing dependency on unicodecsv

["Brian Candler (JIRA)" <ji...@apache.org>]

Brian Candler created AIRFLOW-200:
-------------------------------------

             Summary: Hiding import errors / missing dependency on unicodecsv
                 Key: AIRFLOW-200
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-200
             Project: Apache Airflow
          Issue Type: Bug
          Components: core
    Affects Versions: Airflow 1.7.1.2
         Environment: ubuntu 14.04 (python 2.7), new virtualenv, pip install airflow
            Reporter: Brian Candler
            Priority: Minor


When running the quickstart instructions at http://pythonhosted.org/airflow/start.html inside a clean virtualenv:

ERROR [airflow.models.DagBag] Failed to import:
/home/brian/airflow/venv/local/lib/python2.7/site-packages/airflow/example_dags/example_twitter_dag.py
Traceback (most recent call last):
  File
"/home/brian/airflow/venv/local/lib/python2.7/site-packages/airflow/models.py",
line 247, in process_file
    m = imp.load_source(mod_name, filepath)
  File
"/home/brian/airflow/venv/local/lib/python2.7/site-packages/airflow/example_dags/example_twitter_dag.py",
line 26, in <module>
    from airflow.operators import BashOperator, HiveOperator,
PythonOperator
ImportError: cannot import name HiveOperator

Unfortunately that message doesn't help diagnose the problem, which is being hidden by auto-import magic.

It requires manually probing imports from the true source modules:

>>> from airflow.operators.hive_operator import HiveOperator
...
ImportError: cannot import name HiveCliHook

>>> from airflow.hooks.hive_hooks import HiveCliHook
...
ImportError: No module named unicodecsv

Aha. "pip install unicodecsv" fixes the error.

So I'd suggest two issues:

1. Add a packaging dependency on unicodecsv to fix this particular problem

2. Fix the auto-import magic so that it doesn't suppress these errors



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)