[airflow] branch main updated: Add information about keepalives for managed Postgres (#18850)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new f22c834  Add information about keepalives for managed Postgres (#18850)
f22c834 is described below

commit f22c834434fa7b15372558ea92b588b345fa2620
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Tue Oct 12 22:54:51 2021 +0200

    Add information about keepalives for managed Postgres (#18850)
    
    The managed Postgres DBs often kill the connection after
    some time of inactivity, so if you are using Airflow with those
    you need to configure keepalives.
    
    This PR adds description on how to do it. See #18846
---
 docs/apache-airflow/howto/set-up-database.rst | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/docs/apache-airflow/howto/set-up-database.rst b/docs/apache-airflow/howto/set-up-database.rst
index a23e4bc..932f15a 100644
--- a/docs/apache-airflow/howto/set-up-database.rst
+++ b/docs/apache-airflow/howto/set-up-database.rst
@@ -245,10 +245,35 @@ For more information regarding setup of the PostgresSQL connection, see `Postgre
 
    See also :ref:`Helm Chart production guide <production-guide:pgbouncer>`
 
+
+.. note::
+
+   For managed Postgres such as Redshift, Azure Postgresql, CloudSQL, Amazon RDS, you should use
+   ``keepalives_idle`` in the connection parameters and set it to less than the idle time because those
+   services will close idle connections after some time of inactivity (typically 300 seconds),
+   which results with error ``The error: psycopg2.operationalerror: SSL SYSCALL error: EOF detected``.
+   The ``keepalive`` settings can be changed via ``sql_alchemy_connect_args`` configuration parameter
+   :doc:`../configurations-ref` in ``[core]`` section. You can configure the args for example in your
+   local_settings.py and the ``sql_alchemy_connect_args`` should be a full import path to the dictionary
+   that stores the configuration parameters. You can read about
+   `Postgres Keepalives <https://www.postgresql.org/docs/current/libpq-connect.html>`_.
+   An example setup for ``keepalives`` that has been observe to fix the problem might be:
+
+   .. code-block:: python
+
+      keepalive_kwargs = {
+          "keepalives": 1,
+          "keepalives_idle": 30,
+          "keepalives_interval": 5,
+          "keepalives_count": 5,
+      }
+
+
 .. spelling::
 
      hba
 
+
 Setting up a MsSQL Database
 ---------------------------