You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/01 10:57:26 UTC
[GitHub] [airflow] amaraliou opened a new issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
amaraliou opened a new issue #14541:
URL: https://github.com/apache/airflow/issues/14541
**Apache Airflow version**:
1.10.4
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
N/A
**Environment**:
- **Cloud provider or hardware configuration**: GCP
- **OS** (e.g. from /etc/os-release): Ubuntu 18.04.4 LTS
- **Kernel** (e.g. `uname -a`): N/A
- **Install tools**: pip
**What happened**:
Running `airflow upgrade_check` threw the following warnings:
```
1. Class <class 'airflow.hooks.http_hook.HttpHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
2. Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
3. Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
4. Class <class 'airflow.contrib.hooks.bigquery_hook.BigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
5. Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
6. Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
7. Class <class '_hooks.gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
8. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
9. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
10. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.BigQueryStandardHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
11. Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
12. Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
13. Class <class '_hooks.gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
14. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
15. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function run while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
16. Class <class 'unusual_prefix_9358643817187292af016429303a09afd4152fc5_gc_bigquery_hook.GCBigQueryHook'> incorrectly implements the function get_records while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
```
The custom hooks all inherit from `BigQueryHook`. Looking at the code, it inherits both `DbApiHook` and `GoogleCloudBaseHook` which inherits `BaseHook`. I also noticed that the same `BigQueryHook` implementation appears in the `2.0-stable` branch so this is probably a false positive. The warning (1) is already fixed by https://github.com/apache/airflow/pull/14344.
**What you expected to happen**:
No warnings
**How to reproduce it**: N/A
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-865002450
Closed by https://github.com/apache/airflow/pull/16543 .
Will cut 1.4.0rc2 today and start the vote to release it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] omar-binary commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
omar-binary commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-863786625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-864418011
PR: https://github.com/apache/airflow/pull/16543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-865002450
Closed by https://github.com/apache/airflow/pull/16543 .
Will cut 1.4.0rc2 today and start the vote to release it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] omar-binary edited a comment on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
omar-binary edited a comment on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-863915504
I can still see it in V1.3.0
Apache Airflow version:
1.10.15
```
airflow@d9b97fe4d7cc:/opt/airflow$ airflow upgrade_check --version
1.3.0
airflow@d9b97fe4d7cc:/opt/airflow$ airflow upgrade_check --config /opt/airflow/plugins/upgrade.yml
Using config file: /opt/airflow/plugins/upgrade.yml
====================================================================================== STATUS ======================================================================================
Check for latest versions of apache-airflow and checker...................................................................................................................SUCCESS
Remove airflow.AirflowMacroPlugin class...................................................................................................................................SUCCESS
Ensure users are not using custom metaclasses in custom operators.........................................................................................................SUCCESS
Chain between DAG and operator not allowed................................................................................................................................SUCCESS
Connection.conn_type is not nullable......................................................................................................................................SUCCESS
Custom Executors now require full path....................................................................................................................................SUCCESS
/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py:119: DeprecationWarning: This method will be deprecated. Please use `BigQueryHook.get_client` method
warnings.warn(
Hooks that run DB functions must inherit from DBApiHook...................................................................................................................FAIL
Fernet is enabled by default..............................................................................................................................................SUCCESS
GCP service account key deprecation.......................................................................................................................................SUCCESS
Unify hostname_callable option in core section............................................................................................................................SUCCESS
Changes in import paths of hooks, operators, sensors and others...........................................................................................................SUCCESS
Legacy UI is deprecated by default........................................................................................................................................SUCCESS
Logging configuration has been moved to new section.......................................................................................................................SUCCESS
Removal of Mesos Executor.................................................................................................................................................SUCCESS
No additional argument allowed in BaseOperator............................................................................................................................SUCCESS
SendGrid email uses old airflow.contrib module............................................................................................................................SUCCESS
Check Spark JDBC Operator default connection name.........................................................................................................................SUCCESS
Changes in import path of remote task handlers............................................................................................................................SUCCESS
Connection.conn_id is not unique..........................................................................................................................................SUCCESS
Use CustomSQLAInterface instead of SQLAInterface for custom data models...................................................................................................SUCCESS
Found 1 problem.
================================================================================= RECOMMENDATIONS ==================================================================================
Hooks that run DB functions must inherit from DBApiHook
-------------------------------------------------------
Hooks that run DB functions must inherit from DBApiHook instead of BaseHook
Problems:
1. Class <class 'airflow.providers.google.cloud.hooks.bigquery.BigQueryHook'> incorrectly implements the function get_pandas_df while inheriting from BaseHook. Please make this class inherit from airflow.hooks.db_api_hook.DbApiHook instead
airflow@d9b97fe4d7cc:/opt/airflow$
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-863890164
> any luck with this? 👆
What version of the upgrade-check are you using because I believe this was resolved in https://github.com/apache/airflow/pull/14344
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-863848293
@kaxil - maybe we should add this one as well to the last upgrade_check release? Sounds like pretty annoying warning for popular operators?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-869014842
fixed by https://github.com/apache/airflow/pull/16543 and released in 1.4.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #14541:
URL: https://github.com/apache/airflow/issues/14541
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14541: Airflow upgrade_check gives false positives on DbApi functions implementations check
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14541:
URL: https://github.com/apache/airflow/issues/14541#issuecomment-787857054
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org