You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Sunny-Island (via GitHub)" <gi...@apache.org> on 2023/07/20 07:13:52 UTC
[GitHub] [airflow] Sunny-Island opened a new issue, #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Sunny-Island opened a new issue, #32708:
URL: https://github.com/apache/airflow/issues/32708
### Apache Airflow version
2.6.3
### What happened
When I install airflow can set backend database, I set mysql as my backend. And I exec `airflow db upgrade`
It shows many warning info contains "WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version."
### What you think should happen instead
_No response_
### How to reproduce
mysql_config --version
8.0.34
mysql --version
mysql Ver 8.0.34 for Linux on x86_64 (MySQL Community Server - GPL)
setup airflow backend and run `airflow db upgrade`
### Operating System
CentOS 7
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1643400509
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Sunny-Island commented on issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "Sunny-Island (via GitHub)" <gi...@apache.org>.
Sunny-Island commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1645066623
When I run airflow standalone, there are too many :
scheduler | WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.
triggerer | WARNING: MYSQL_OPT_RECONNECT is deprecated and will be removed in a future version.
I google realted config but find nothing. @potiuk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade. [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
URL: https://github.com/apache/airflow/issues/32708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade. [airflow]
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1786224468
@bmarquesplanet @Sunny-Island good news, SQLAlchemy 1.4.50 was released 2 days ago and seems like this version fixed pre-ping behaviour for `mysqlclient` driver, see: https://docs.sqlalchemy.org/en/20/changelog/changelog_14.html#change-1.4.50-mysql
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade. [airflow]
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1777580166
> Guys, is there a workaround to suppress this warning at all?
No. The messages printed directly to stdout/stderr by `libmysqlclient`, until it doesn't changed in upstream libraries (mysqlclient / sqlalchemy) nothing we could do here.
Best solution for some one who only started with Airflow and doesn't have anything critical in production is use **Postgres** as DB Backend 🙄
Other potential solutions described here: https://github.com/apache/airflow/issues/32708#issuecomment-1722267498 and I can confirm that there are no such of warning messages with latest MariaDB client libraries (Note: this is experimental support)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Sunny-Island commented on issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "Sunny-Island (via GitHub)" <gi...@apache.org>.
Sunny-Island commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1643402013
To address this warning, you may consider using the OPT_RECONNECT option of MySQL Connector/C++ instead of MYSQL_OPT_RECONNECT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Sunny-Island commented on issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "Sunny-Island (via GitHub)" <gi...@apache.org>.
Sunny-Island commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1645068996
It seem that MYSQL_OPT_RECONNECT will be deprecated, but I dont know where to config, sqlalchemy or airflow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Sunny-Island commented on issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "Sunny-Island (via GitHub)" <gi...@apache.org>.
Sunny-Island commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1645169332
I solve this question by change diver from mysqlclient to mysql-connector-python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade. [airflow]
Posted by "bmarquesplanet (via GitHub)" <gi...@apache.org>.
bmarquesplanet commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1777386989
Guys, is there a workaround to suppress this warning at all? Tried some of the suggestions here https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#logging and the effect is applied only for airflow specific logging messages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Taragolis commented on issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "Taragolis (via GitHub)" <gi...@apache.org>.
Taragolis commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1722267498
I could confirm that this error persist and we could see it in CI tests for MySQL backend, and to be honest we do not have a lot of options here.
Spoiler Alert: It not affect our ARM image, only x86_64
Oracle deprecate this options in their C API https://dev.mysql.com/doc/c-api/8.0/en/c-api-auto-reconnect.html in 8.0.34 , which provided by `libmysqlclient21` which are used in our regular Airflow Images for x86_64. And after that warning started appear in the different places:
- [Ubuntu Launchpad about client](https://bugs.launchpad.net/ubuntu/+source/mysql-8.0/+bug/2031548)
- [Ruby](https://stackoverflow.com/questions/76931038/warning-mysql-opt-reconnect-is-deprecated)
- [Perl](https://stackoverflow.com/questions/76875569/warning-mysql-opt-reconnect-is-deprecated-and-will-be-removed-in-a-future-versi)
- And in `mysqlclient`, see [discussion](https://github.com/PyMySQL/mysqlclient/discussions/651). This [method called](https://github.com/sqlalchemy/sqlalchemy/blob/5132237f8ddaab1a8c6bfcaa03aca6487434ae95/lib/sqlalchemy/dialects/mysql/mysqldb.py#L170-L172) in SQLAclhemy's Pool if pre_ping activated
Possible options (I can miss something)
- Recommend set `[database] sql_alchemy_pool_pre_ping` to `False`, which might be a reason for other issues
- Switch to MariaDB client libraries -not sure if we can do this for x86_64
- Switch to Oracle's `mysql-connector-python` - Nor Airflow, nor SQLAlchemy test this driver
- Switch to PyMySQL - it pure python implementations which do not use C API, so it should be less performant. We never do not have tests against this library even if we have some parts in[ airflow codebase](https://github.com/apache/airflow/blob/ed552e294596b27288fb040abd1634cb77e419c9/airflow/settings.py#L387-L392)
- Close this issue as **won't fix**
cc: @potiuk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #32708:
URL: https://github.com/apache/airflow/issues/32708#issuecomment-1643970803
I beleve it's you who have to change it.
But there is nothing we can do about it because we do not set this option.
It likely comes from your configuration on version of driver you are using or sqlalchemy. You should investigate where it comes from (look at database section configuration where you can set your options or check in the documentation of sqlalchemy/drivers in the version that you are using adds this option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #32708: MYSQL_OPT_RECONNECT is deprecated. When exec airflow db upgrade.
URL: https://github.com/apache/airflow/issues/32708
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org