You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "shahar1 (via GitHub)" <gi...@apache.org> on 2026/07/28 12:33:30 UTC
[PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
shahar1 opened a new pull request, #70619:
URL: https://github.com/apache/airflow/pull/70619
The amazon executor setup guides and the Celery `result_backend` example tell users to configure a `postgresql+psycopg://` (psycopg3) connection string. Both providers declare `apache-airflow>=2.11.0`, and Airflow 2.x and 3.0.x pin SQLAlchemy 1.4, which has no `postgresql+psycopg` dialect at all — a user following those instructions gets `sqlalchemy.exc.NoSuchModuleError` at startup, with nothing in the docs indicating the advice was version-specific.
The amazon guides are step-by-step deployment instructions where the connection string is pasted straight into an AWS console field, so the wrong value is especially easy to pick up.
This switches the documented value to `postgresql+psycopg2://`, which is valid on every Airflow version these providers support, and moves the version guidance into prose so the code block stays copy-pasteable. The three amazon executor guides had the same block duplicated inline; they now include a single shared block from `general.rst`, matching how the rest of those files already share content.
Verified with `breeze build-docs amazon celery` — both packages build successfully and all four affected pages render correctly.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 5)
Generated-by: Claude Code (Opus 5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #70619:
URL: https://github.com/apache/airflow/pull/70619#issuecomment-5108953869
> If this is for older Airflow, should this be filed to versioned branches instead of main?
It's not - providers currently (cc: @eladkal) support Airlfow 2.11+ from main - until we make a decision they don't
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 commented on PR #70619:
URL: https://github.com/apache/airflow/pull/70619#issuecomment-5150153997
> If specific providers will benfit from bumping min airflow version lets talk.
I don't think that specific providers will benefit from that - we'll have to bump to 3.2+ and handle dropping 2.11-3.1 just for a doc example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "Dev-iL (via GitHub)" <gi...@apache.org>.
Dev-iL commented on code in PR #70619:
URL: https://github.com/apache/airflow/pull/70619#discussion_r3694730930
##########
providers/amazon/docs/executors/general.rst:
##########
@@ -312,6 +312,21 @@ You will be prompted to enter the password if the connection is successful.
.. END DATABASE_CONNECTION
+.. BEGIN SQL_ALCHEMY_CONN
+
+.. code-block:: bash
+
+ postgresql+psycopg2://<username>:<password>@<endpoint>/<database_name>
+
+.. note::
+ ``psycopg2`` works on every Airflow version these executors support. From Airflow 3.2.0 you may use
+ ``postgresql+psycopg://`` instead, provided psycopg3 is installed. Airflow before 3.2.0 does not
Review Comment:
```suggestion
``postgresql+psycopg://`` instead, provided ``psycopg`` (v3) is installed. Airflow before 3.2.0 does not
```
I think it's better not to use `psycopg3` because this is not what users need to pip-install (the pypi entry looks like a placeholder last updated in 2013).
I think we can only mention "(v3)" the first time psycopg is mentioned in the document, no need to add that everywhere.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "shahar1 (via GitHub)" <gi...@apache.org>.
shahar1 merged PR #70619:
URL: https://github.com/apache/airflow/pull/70619
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on PR #70619:
URL: https://github.com/apache/airflow/pull/70619#issuecomment-5107712032
If this is for older Airflow, should this be filed to versioned branches instead of main?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #70619:
URL: https://github.com/apache/airflow/pull/70619#issuecomment-5123302519
> If specific providers will benfit from bumping min airflow version lets talk.
I think that's about 100 conversation fior you to have, But yeah, I am interesting to see how many extra conversations and failures to handle it will bring. 👀
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "Dev-iL (via GitHub)" <gi...@apache.org>.
Dev-iL commented on code in PR #70619:
URL: https://github.com/apache/airflow/pull/70619#discussion_r3694730930
##########
providers/amazon/docs/executors/general.rst:
##########
@@ -312,6 +312,21 @@ You will be prompted to enter the password if the connection is successful.
.. END DATABASE_CONNECTION
+.. BEGIN SQL_ALCHEMY_CONN
+
+.. code-block:: bash
+
+ postgresql+psycopg2://<username>:<password>@<endpoint>/<database_name>
+
+.. note::
+ ``psycopg2`` works on every Airflow version these executors support. From Airflow 3.2.0 you may use
+ ``postgresql+psycopg://`` instead, provided psycopg3 is installed. Airflow before 3.2.0 does not
Review Comment:
```suggestion
``postgresql+psycopg://`` instead, provided ``psycopg`` (v3) is installed. Airflow before 3.2.0 does not
```
I think it's better not to use `psycopg3` because this is not what users need to pip-install (the pypi entry looks like a placeholder last updated in 2013).
~~I think we can only mention "(v3)" the first time psycopg is mentioned in the document, no need to add that everywhere.~~ Disregard, I thought the repetitions were in the same file, but they aren't.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "Dev-iL (via GitHub)" <gi...@apache.org>.
Dev-iL commented on code in PR #70619:
URL: https://github.com/apache/airflow/pull/70619#discussion_r3681506300
##########
providers/amazon/docs/executors/general.rst:
##########
@@ -312,6 +312,24 @@ You will be prompted to enter the password if the connection is successful.
.. END DATABASE_CONNECTION
+.. BEGIN SQL_ALCHEMY_CONN
+
+.. code-block:: bash
+
+ postgresql+psycopg2://<username>:<password>@<endpoint>/<database_name>
+
+The ``psycopg2`` driver above is valid on every Airflow version these executors support. From Airflow
+3.4.0, a driverless URL resolves to ``psycopg`` (psycopg3) where psycopg3 is installed and to
+``psycopg2`` otherwise. You may instead
+give an explicit ``postgresql+psycopg://`` URL from Airflow 3.2.0, provided psycopg3 is installed —
+whether it is depends on your ``apache-airflow-providers-postgres`` version. Do not use
+``postgresql+psycopg://`` before Airflow 3.2.0 at all: those releases can run on SQLAlchemy 1.4, which
+has no ``postgresql+psycopg`` dialect, so Airflow fails to start with
+``sqlalchemy.exc.NoSuchModuleError``.
Review Comment:
Rephrased suggestion, feel free to adjust.
```suggestion
The ``psycopg2`` driver is valid on every Airflow version these executors support.
Support for the ``psycopg`` (v3) driver was added in Airflow 3.2.0. To use this driver, ensure it is installed, then specify ``postgresql+psycopg://``. Do not use ``postgresql+psycopg://`` before Airflow 3.2.0 at all: those releases typically run on SQLAlchemy 1.4, which has no ``postgresql+psycopg`` dialect, so Airflow will fail to start with ``sqlalchemy.exc.NoSuchModuleError``.
A driverless ``postgresql://`` URL resolves to ``psycopg2`` before Airflow 3.4.0, and to ``psycopg`` after.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Stop recommending psycopg3 connection strings on older Airflow [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #70619:
URL: https://github.com/apache/airflow/pull/70619#issuecomment-5109797066
If specific providers will benfit from bumping min airflow version lets talk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org