You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/13 16:01:51 UTC
[GitHub] [airflow] Giuzzilla opened a new issue #15349: Postgres connection URI incompatible with SQLAlchemy >= 1.4.0b1
Giuzzilla opened a new issue #15349:
URL: https://github.com/apache/airflow/issues/15349
**Apache Airflow version**: 2.0.1 (possibly any?)
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): N/A
**Environment**: Any
- **Cloud provider or hardware configuration**: Any
- **OS** (e.g. from /etc/os-release): Any
- **Kernel** (e.g. `uname -a`): Any
- **Install tools**: Any
- **Others**: Any
**What happened**:
From SQLAlchemy version 1.4.0b1, [SQLAlchemy removed the support for the `postgres` prefix](https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-3687655465c25a39b968b4f5f6e9170b) in URIs (used for example in `create_engine`), and now it supports only `postgresql`.
This affects all Airflow jobs that:
(A) Use SQLAlchemy version >= 1.4.0
(B) Obtain/use a Postgres URI from the `get_uri` method of a `PostgresHook` defined in the `DbApiHook` (`airflow/hooks/dbapi.py`).
This is because the `PostgresHook` (`airflow/providers/postgres/hooks/postgres.py`) class, defines a class attribute `conn_type = 'postgres'` instead of `postgresql`.
Right now my airflow jobs with the updated SQLAlchemy that are using SQLAlchemy `create_engine` with a postgres db are broken.
**What you expected to happen**:
That Airflow jobs using SQLAlchemy >= 1.4.0 and SQLAlchemy's create_engine with Postgres databases work correctly, and that `get_uri` generates a postgres URI starting with `postgresql://`.
Of course I could use an older SQLAlchemy version or manipulated the returned URI, but I guess it'd be nice if Airflow adapts to the updated prefix.
**How to reproduce it**:
Use the `get_uri` method of any `PostgresHook`.
**Anything else we need to know**:
N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #15349: Postgres connection URI incompatible with SQLAlchemy >= 1.4.0b1
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #15349:
URL: https://github.com/apache/airflow/issues/15349#issuecomment-818878101
Duplicate to #14811.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15349: Postgres connection URI incompatible with SQLAlchemy >= 1.4.0b1
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15349:
URL: https://github.com/apache/airflow/issues/15349#issuecomment-1031184477
Yep. confirmed (and it's being voted now) . Thanks for pointing it out. Closed by #21205 indeed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] Giuzzilla commented on issue #15349: Postgres connection URI incompatible with SQLAlchemy >= 1.4.0b1
Posted by GitBox <gi...@apache.org>.
Giuzzilla commented on issue #15349:
URL: https://github.com/apache/airflow/issues/15349#issuecomment-818951842
> Duplicate to #14811.
I saw #14811 before opening this but I thought it might be worth it to open a separate issue as it's a bit different, even if they are both caused by SQLAlchemy 1.4.
#14811 was fixed by pinning SQLAlchemy in Airflow requirements, but #14812 doesn't help here.
This is impacting jobs that are not sharing Airflow dependencies, e.g. custom docker images ran with the Kubernetes Pod Operator or Docker Operator, but which are relying on Airflow-generated URIs for connecting to postgresql.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #15349: Postgres connection URI incompatible with SQLAlchemy >= 1.4.0b1
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #15349:
URL: https://github.com/apache/airflow/issues/15349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #15349: Postgres connection URI incompatible with SQLAlchemy >= 1.4.0b1
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #15349:
URL: https://github.com/apache/airflow/issues/15349#issuecomment-1030925145
I think this issue is solved by https://github.com/apache/airflow/pull/21205
@potiuk can you confirm?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #15349: Postgres connection URI incompatible with SQLAlchemy >= 1.4.0b1
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #15349:
URL: https://github.com/apache/airflow/issues/15349#issuecomment-818852787
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org