You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Jason Chen <ch...@gmail.com> on 2016/09/05 03:09:38 UTC
Name the DAG and the file for DAG ?
Hi airflow team,
I named my DAG file as something like "abc.def.v1.py" and the ID as
"abc.def.v1".
I noticed when I tried to view the codes from UI. It drops exceptions like
"...No module named unusual_prefix_abc.def.v1..." and details as below.
I am thinking it's because of the python hierarchy naming stuff for module.
Anyway to avoid this ? One way is to avoid "." in the naming.
Thanks. I am using airflow 1.7.1.3
-Jason
-------------------------------------------------------------------------------
Node: ip-10-128-131-164
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python2.7/site-packages/flask_admin/base.py",
line 68, in inner
return self._run_view(f, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/flask_admin/base.py",
line 367, in _run_view
return fn(self, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/flask_login.py", line
755, in decorated_view
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/airflow/www/views.py",
line 655, in code
m = importlib.import_module(dag.module_name)
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named unusual_prefix_abc.def.v1
Re: Name the DAG and the file for DAG ?
Posted by Jason Chen <ch...@gmail.com>.
Sure.
make sense.
Thanks.
On Mon, Sep 5, 2016 at 9:17 PM, Arthur Wiedmer <ar...@gmail.com>
wrote:
> Hi Jason,
>
> Naming things with dots is notoriously tricky in Python, and filenames
> with dots often lead to import errors. Because of the way dags and
> subdags are handled internally, '.'s (dots) are not supported in DAG
> names. May I recommend the use of underscores?
>
> Best,
> Arthur
>
> On Sun, Sep 4, 2016 at 8:09 PM, Jason Chen <ch...@gmail.com>
> wrote:
> > Hi airflow team,
> >
> > I named my DAG file as something like "abc.def.v1.py" and the ID as
> > "abc.def.v1".
> >
> > I noticed when I tried to view the codes from UI. It drops exceptions
> like
> > "...No module named unusual_prefix_abc.def.v1..." and details as below.
> >
> > I am thinking it's because of the python hierarchy naming stuff for
> module.
> > Anyway to avoid this ? One way is to avoid "." in the naming.
> > Thanks. I am using airflow 1.7.1.3
> >
> > -Jason
> >
> > ------------------------------------------------------------
> -------------------
> > Node: ip-10-128-131-164
> > ------------------------------------------------------------
> -------------------
> > Traceback (most recent call last):
> > File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> > 1817, in wsgi_app
> > response = self.full_dispatch_request()
> > File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> > 1477, in full_dispatch_request
> > rv = self.handle_user_exception(e)
> > File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> > 1381, in handle_user_exception
> > reraise(exc_type, exc_value, tb)
> > File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> > 1475, in full_dispatch_request
> > rv = self.dispatch_request()
> > File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> > 1461, in dispatch_request
> > return self.view_functions[rule.endpoint](**req.view_args)
> > File "/usr/local/lib/python2.7/site-packages/flask_admin/base.py",
> > line 68, in inner
> > return self._run_view(f, *args, **kwargs)
> > File "/usr/local/lib/python2.7/site-packages/flask_admin/base.py",
> > line 367, in _run_view
> > return fn(self, *args, **kwargs)
> > File "/usr/local/lib/python2.7/site-packages/flask_login.py", line
> > 755, in decorated_view
> > return func(*args, **kwargs)
> > File "/usr/local/lib/python2.7/site-packages/airflow/www/views.py",
> > line 655, in code
> > m = importlib.import_module(dag.module_name)
> > File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in
> import_module
> > __import__(name)
> > ImportError: No module named unusual_prefix_abc.def.v1
>
Re: Name the DAG and the file for DAG ?
Posted by Arthur Wiedmer <ar...@gmail.com>.
Hi Jason,
Naming things with dots is notoriously tricky in Python, and filenames
with dots often lead to import errors. Because of the way dags and
subdags are handled internally, '.'s (dots) are not supported in DAG
names. May I recommend the use of underscores?
Best,
Arthur
On Sun, Sep 4, 2016 at 8:09 PM, Jason Chen <ch...@gmail.com> wrote:
> Hi airflow team,
>
> I named my DAG file as something like "abc.def.v1.py" and the ID as
> "abc.def.v1".
>
> I noticed when I tried to view the codes from UI. It drops exceptions like
> "...No module named unusual_prefix_abc.def.v1..." and details as below.
>
> I am thinking it's because of the python hierarchy naming stuff for module.
> Anyway to avoid this ? One way is to avoid "." in the naming.
> Thanks. I am using airflow 1.7.1.3
>
> -Jason
>
> -------------------------------------------------------------------------------
> Node: ip-10-128-131-164
> -------------------------------------------------------------------------------
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> 1817, in wsgi_app
> response = self.full_dispatch_request()
> File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> 1477, in full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> 1381, in handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> 1475, in full_dispatch_request
> rv = self.dispatch_request()
> File "/usr/local/lib/python2.7/site-packages/flask/app.py", line
> 1461, in dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File "/usr/local/lib/python2.7/site-packages/flask_admin/base.py",
> line 68, in inner
> return self._run_view(f, *args, **kwargs)
> File "/usr/local/lib/python2.7/site-packages/flask_admin/base.py",
> line 367, in _run_view
> return fn(self, *args, **kwargs)
> File "/usr/local/lib/python2.7/site-packages/flask_login.py", line
> 755, in decorated_view
> return func(*args, **kwargs)
> File "/usr/local/lib/python2.7/site-packages/airflow/www/views.py",
> line 655, in code
> m = importlib.import_module(dag.module_name)
> File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
> __import__(name)
> ImportError: No module named unusual_prefix_abc.def.v1