Re: [PR] Make postgresql default engine args comply with SA 2.0 [airflow]

["Taragolis (via GitHub)" <gi...@apache.org>]

Taragolis commented on PR #38362:
URL: https://github.com/apache/airflow/pull/38362#issuecomment-2015433507

   And modified snippet, regular tests still can't run with SA20 😢 
   
   ```python
   from sqlalchemy import create_engine
   from importlib.metadata import version
   from airflow.settings import DEFAULT_ENGINE_ARGS
   
   SQL_ALCHEMY_CONN = "postgresql+psycopg2://postgres:airflow@127.0.0.1:25433/airflow"
   args = {
       "current": {
           "executemany_mode": "values",
           "executemany_values_page_size": 10000,
           "executemany_batch_page_size": 2000,
       },
       "from-airflow-settings": DEFAULT_ENGINE_ARGS.get("postgresql"),
       "not-set": {}
   }
   
   print(f"SQLAlchemy version: {version('sqlalchemy')}")
   for name, pg_default_args in args.items():
       print(f" {name} ".center(72, "="))
       print(f"Input: {pg_default_args}")
       try:
           engine = create_engine(SQL_ALCHEMY_CONN, **pg_default_args)
       except Exception as e:
           print(f"Error: {e}")
       else:
           print(f"executemany_mode: {getattr(engine.dialect, 'executemany_mode', 'NOT-EXISTS')}")
           print(f"executemany_batch_page_size: {getattr(engine.dialect, 'executemany_batch_page_size', 'NOT-EXISTS')}")
           print(f"insertmanyvalues_page_size: {getattr(engine.dialect, 'insertmanyvalues_page_size', 'NOT-EXISTS')}")
           print(f"executemany_values_page_size: {getattr(engine.dialect, 'executemany_values_page_size', 'NOT-EXISTS')}")
           engine.dispose()
   ```
   
   ```console
   SQLAlchemy version: 2.0.28
   =============================== current ================================
   Input: {'executemany_mode': 'values', 'executemany_values_page_size': 10000, 'executemany_batch_page_size': 2000}
   Error: Invalid value for 'executemany_mode': 'values'
   ======================== from-airflow-settings =========================
   Input: {'executemany_mode': 'values_plus_batch', 'insertmanyvalues_page_size': 10000, 'executemany_batch_page_size': 2000}
   executemany_mode: symbol('EXECUTEMANY_VALUES_PLUS_BATCH')
   executemany_batch_page_size: 2000
   insertmanyvalues_page_size: 10000
   executemany_values_page_size: NOT-EXISTS
   =============================== not-set ================================
   Input: {}
   executemany_mode: symbol('EXECUTEMANY_VALUES')
   executemany_batch_page_size: 100
   insertmanyvalues_page_size: 1000
   executemany_values_page_size: NOT-EXISTS
   ```
   
   ```console
   SQLAlchemy version: 1.4.52
   =============================== current ================================
   Input: {'executemany_mode': 'values', 'executemany_values_page_size': 10000, 'executemany_batch_page_size': 2000}
   executemany_mode: symbol('executemany_values_plus_batch')
   executemany_batch_page_size: 2000
   insertmanyvalues_page_size: NOT-EXISTS
   executemany_values_page_size: 10000
   ======================== from-airflow-settings =========================
   Input: {'executemany_mode': 'values_plus_batch', 'executemany_values_page_size': 10000, 'executemany_batch_page_size': 2000}
   executemany_mode: symbol('executemany_values_plus_batch')
   executemany_batch_page_size: 2000
   insertmanyvalues_page_size: NOT-EXISTS
   executemany_values_page_size: 10000
   =============================== not-set ================================
   Input: {}
   executemany_mode: symbol('executemany_values')
   executemany_batch_page_size: 100
   insertmanyvalues_page_size: NOT-EXISTS
   executemany_values_page_size: 1000
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org