You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Andrew Maguire <an...@gmail.com> on 2017/10/20 11:49:38 UTC
Import Deprecation Warnings
Hey,
I keep getting this warning and would like to try make it go away:
[2017-10-20 11:44:14,685] {{__init__.py:45}} INFO - Using executor LocalExecutor
[2017-10-20 11:44:14,735] {{models.py:186}} INFO - Filling up the
DagBag from /home/andrew_maguire/pmc-analytical-data-mart/airflow/dags
/usr/local/lib/python2.7/dist-packages/airflow/utils/helpers.py:351:
DeprecationWarning: Importing PythonOperator directly from <module
'airflow.operators' from
'/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>
has been deprecated. Please import from '<module 'airflow.operators'
from '/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>.[operator_module]'
instead. Support for direct imports will be dropped entirely in
Airflow 2.0.
DeprecationWarning)
[2017-10-20 11:44:19,887] {{cli.py:201}} INFO - Created <DagRun
ga_api_post_data @ 2017-10-04 00:02:00: manual__2017-10-04T00:02:00,
externally triggered: True>
My imports look like:
from airflow import DAG
from datetime import datetime, timedelta
from airflow.contrib.operators.bigquery_operator import BigQueryOperator
from airflow.contrib.operators.bigquery_check_operator import
BigQueryCheckOperator
from airflow.operators.bash_operator import BashOperator
import os
import jinja2
from airflow.operators.python_operator import PythonOperator,
BranchPythonOperator
from airflow.operators.dummy_operator import DummyOperator
from pprint import pprint
from pandas.io import gbq
from airflow.macros import ds_add , ds_format
from airflow.utils.dates import days_ago
It seems to be the PythonOperator it's complaining about.
Was reading AIRFLOW-31 <https://issues.apache.org/jira/browse/AIRFLOW-31> and
think i'm doing it the right way so unsure what import might be triggering
this warning or how i can resolve it.
Any ideas?
Cheers,
Andy
Re: Import Deprecation Warnings
Posted by Zack Lawson <zm...@gmail.com>.
Your imports look fine to me. Just a shot in the dark but are you sure it's
that DAG and not another one in your dag a folder? If I remember correctly,
when it loads up the dags it loads all of them regardless if you're doing a
command on just a single dag. I know when I was removing those imports it
was tricky to isolate the offending dag because of that.
On Fri, Oct 20, 2017 at 6:52 AM Andrew Maguire <an...@gmail.com>
wrote:
> Sorry - here is the full error message:
>
> andrew_maguire@airflow-server:~/pmc-analytical-data-mart$ airflow
> list_dags
> [2017-10-20 11:47:48,205] {{__init__.py:45}} INFO - Using executor
> LocalExecutor
> [2017-10-20 11:47:48,297] {{models.py:186}} INFO - Filling up the DagBag
> from /home/andrew_maguire/pmc-analytical-data-mart/airflow/dags
> /usr/local/lib/python2.7/dist-packages/airflow/utils/helpers.py:351:
> DeprecationWarning: Importing PythonOperator directly from <module
> 'airflow.operators' from
> '/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>
> has been deprecated. Please import from '<module 'airflow.operators' from
>
> '/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>.[operator_module]'
> instead. Support for direct imports will be dropped entirely in Airflow
> 2.0.
> DeprecationWarning)
>
>
> On Fri, Oct 20, 2017 at 12:49 PM Andrew Maguire <an...@gmail.com>
> wrote:
>
> > Hey,
> >
> > I keep getting this warning and would like to try make it go away:
> >
> > [2017-10-20 11:44:14,685] {{__init__.py:45}} INFO - Using executor
> LocalExecutor
> > [2017-10-20 11:44:14,735] {{models.py:186}} INFO - Filling up the DagBag
> from /home/andrew_maguire/pmc-analytical-data-mart/airflow/dags
> > /usr/local/lib/python2.7/dist-packages/airflow/utils/helpers.py:351:
> DeprecationWarning: Importing PythonOperator directly from <module
> 'airflow.operators' from
> '/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>
> has been deprecated. Please import from '<module 'airflow.operators' from
> '/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>.[operator_module]'
> instead. Support for direct imports will be dropped entirely in Airflow 2.0.
> > DeprecationWarning)
> > [2017-10-20 11:44:19,887] {{cli.py:201}} INFO - Created <DagRun
> ga_api_post_data @ 2017-10-04 00:02:00: manual__2017-10-04T00:02:00,
> externally triggered: True>
> >
> >
> > My imports look like:
> >
> > from airflow import DAG
> > from datetime import datetime, timedelta
> > from airflow.contrib.operators.bigquery_operator import BigQueryOperator
> > from airflow.contrib.operators.bigquery_check_operator import
> > BigQueryCheckOperator
> > from airflow.operators.bash_operator import BashOperator
> > import os
> > import jinja2
> > from airflow.operators.python_operator import PythonOperator,
> > BranchPythonOperator
> > from airflow.operators.dummy_operator import DummyOperator
> > from pprint import pprint
> > from pandas.io import gbq
> > from airflow.macros import ds_add , ds_format
> > from airflow.utils.dates import days_ago
> >
> > It seems to be the PythonOperator it's complaining about.
> >
> > Was reading AIRFLOW-31 <https://issues.apache.org/jira/browse/AIRFLOW-31>
> and
> > think i'm doing it the right way so unsure what import might be
> triggering
> > this warning or how i can resolve it.
> >
> > Any ideas?
> >
> > Cheers,
> > Andy
> >
>
Re: Import Deprecation Warnings
Posted by Andrew Maguire <an...@gmail.com>.
Sorry - here is the full error message:
andrew_maguire@airflow-server:~/pmc-analytical-data-mart$ airflow list_dags
[2017-10-20 11:47:48,205] {{__init__.py:45}} INFO - Using executor
LocalExecutor
[2017-10-20 11:47:48,297] {{models.py:186}} INFO - Filling up the DagBag
from /home/andrew_maguire/pmc-analytical-data-mart/airflow/dags
/usr/local/lib/python2.7/dist-packages/airflow/utils/helpers.py:351:
DeprecationWarning: Importing PythonOperator directly from <module
'airflow.operators' from
'/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>
has been deprecated. Please import from '<module 'airflow.operators' from
'/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>.[operator_module]'
instead. Support for direct imports will be dropped entirely in Airflow 2.0.
DeprecationWarning)
On Fri, Oct 20, 2017 at 12:49 PM Andrew Maguire <an...@gmail.com>
wrote:
> Hey,
>
> I keep getting this warning and would like to try make it go away:
>
> [2017-10-20 11:44:14,685] {{__init__.py:45}} INFO - Using executor LocalExecutor
> [2017-10-20 11:44:14,735] {{models.py:186}} INFO - Filling up the DagBag from /home/andrew_maguire/pmc-analytical-data-mart/airflow/dags
> /usr/local/lib/python2.7/dist-packages/airflow/utils/helpers.py:351: DeprecationWarning: Importing PythonOperator directly from <module 'airflow.operators' from '/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'> has been deprecated. Please import from '<module 'airflow.operators' from '/usr/local/lib/python2.7/dist-packages/airflow/operators/__init__.pyc'>.[operator_module]' instead. Support for direct imports will be dropped entirely in Airflow 2.0.
> DeprecationWarning)
> [2017-10-20 11:44:19,887] {{cli.py:201}} INFO - Created <DagRun ga_api_post_data @ 2017-10-04 00:02:00: manual__2017-10-04T00:02:00, externally triggered: True>
>
>
> My imports look like:
>
> from airflow import DAG
> from datetime import datetime, timedelta
> from airflow.contrib.operators.bigquery_operator import BigQueryOperator
> from airflow.contrib.operators.bigquery_check_operator import
> BigQueryCheckOperator
> from airflow.operators.bash_operator import BashOperator
> import os
> import jinja2
> from airflow.operators.python_operator import PythonOperator,
> BranchPythonOperator
> from airflow.operators.dummy_operator import DummyOperator
> from pprint import pprint
> from pandas.io import gbq
> from airflow.macros import ds_add , ds_format
> from airflow.utils.dates import days_ago
>
> It seems to be the PythonOperator it's complaining about.
>
> Was reading AIRFLOW-31 <https://issues.apache.org/jira/browse/AIRFLOW-31> and
> think i'm doing it the right way so unsure what import might be triggering
> this warning or how i can resolve it.
>
> Any ideas?
>
> Cheers,
> Andy
>