You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/16 11:03:34 UTC
[GitHub] [airflow] eladkal opened a new issue #14258: ImportChange should indicate about migration of Presto to providers
eladkal opened a new issue #14258:
URL: https://github.com/apache/airflow/issues/14258
**Use case / motivation**
Created the following `presto_dag.py`:
```
from airflow.hooks.presto_hook import PrestoHook
from airflow.operators.python_operator import PythonOperator
from airflow.contrib.hooks.salesforce_hook import SalesforceHook
from airflow.operators.bash_operator import BashOperator
from datetime import datetime
from airflow import DAG
def print(**context):
presto_hook = PrestoHook(presto_conn_id='presto_conn')
result = presto_hook.get_first("SELECT 1")
print(result)
sf_hook = SalesforceHook(conn_id="salesforce_conn")
result = sf_hook.make_query("SELECT 1")
print(result)
with DAG(
dag_id="presto",
default_args={'owner': 'airflow'},
start_date=datetime(2020, 12, 5),
schedule_interval=None,
) as dag:
t = PythonOperator(
task_id='run_presto_task',
python_callable=print,
provide_context=True,
)
t2 = BashOperator(
task_id='bash',
bash_command='echo 1'
)
```
This is the output with pip install apache-airflow-upgrade-check: (both 1.2.0 & 1.1.0)
```
Changes in import paths of hooks, operators, sensors and others
---------------------------------------------------------------
Many hooks, operators and other classes has been renamed and moved. Those changes were part of unifying names and imports paths as described in AIP-21.
The `contrib` folder has been replaced by `providers` directory and packages:
https://github.com/apache/airflow#backport-packages
Problems:
1. Using `airflow.contrib.hooks.salesforce_hook.SalesforceHook` should be replaced by `airflow.providers.salesforce.hooks.salesforce.SalesforceHook`. Affected file: /home/airflow/dags/presto_dag.py
```
This is the output with pip install apache-airflow-upgrade-check=='1.0.0'
```
Changes in import paths of hooks, operators, sensors and others
---------------------------------------------------------------
Many hooks, operators and other classes has been renamed and moved. Those changes were part of unifying names and imports paths as described in AIP-21.
The `contrib` folder has been replaced by `providers` directory and packages:
https://github.com/apache/airflow#backport-packages
Problems:
1. Using `airflow.operators.bash_operator.BashOperator` will be replaced by `airflow.operators.bash.BashOperator`. Affected file: /home/airflow/dags/presto_dag.py
2. Using `airflow.operators.python_operator.PythonOperator` will be replaced by `airflow.operators.python.PythonOperator`. Affected file: /home/airflow/dags/presto_dag.py
3. Using `airflow.hooks.presto_hook.PrestoHook` will be replaced by `airflow.providers.presto.hooks.presto.PrestoHook` and requires `presto` providers package. Affected file: /home/airflow/dags/presto_dag.py
4. Using `airflow.contrib.hooks.salesforce_hook.SalesforceHook` will be replaced by `airflow.providers.salesforce.hooks.salesforce.SalesforceHook` and requires `salesforce` providers package. Affected file: /home/airflow/dags/presto_dag.py
```
This happens due to https://github.com/apache/airflow/pull/13012
The problem is that it also suppress suggested change for `PrestoHook` which needs to be migrated to providers (the [PrestoHook](https://github.com/apache/airflow/blob/v2-0-stable/airflow/hooks/presto_hook.py#L23) in core imports from providers in Airflow 2)
@ashb WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14258: ImportChange should indicate about migration of Presto to providers
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14258:
URL: https://github.com/apache/airflow/issues/14258#issuecomment-785310671
Fixed by https://github.com/apache/airflow/pull/14344 and will be released in **upgrade check 1.3.0**
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #14258: ImportChange should indicate about migration of Presto to providers
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #14258:
URL: https://github.com/apache/airflow/issues/14258
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org