[GitHub] [airflow] potiuk commented on issue #24124: hung database transactions on xcom table

[GitBox <gi...@apache.org>]

potiuk commented on issue #24124:
URL: https://github.com/apache/airflow/issues/24124#issuecomment-1145406959

   In aiflow 2.2 you should use run_id as index not execution date. This has changed in Airlfow 2.2 as part of [AIP-39](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval). Possibly you missed that in adjusting your queries. 
   
   Also In airflow 2.3.0 xcom and other tables were still heavily updated to include map_index.
   
   I propose you test it with the right selection query (using run_id) and also make sure that you adjust your DB queries when you migrate to latest Airflow 2.3 (2.3.2 is likely out tomorrow and test it there - DB in airflow is pretty much internal detail, so if you use any db queries, they might need adjustments.
   
   Also you might be happy to learn that in Airflow 2.3 there is the new "airflow db clean" command that you can likely make use of to replace your manual database retention. https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#clean. This might solve your "maintenance" issue for custom queries, because it will be always using the right queries to clean old data and you can implement your own retention schedule and criteria.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org