[GitHub] [airflow] takunnithan commented on issue #8494: TriggerDagOperator sometimes causes database to trigger UniqueViolation constraint

[GitBox <gi...@apache.org>]

takunnithan commented on issue #8494:
URL: https://github.com/apache/airflow/issues/8494#issuecomment-646824149


   Ran into the same issue with REST API, We were trying to trigger a dag via experimental API (sometimes more than 30 DAG runs at a time). 
   Tried setting the `replace_microseconds`  flag to `False` in the API, It doesn't solve the issue in my case. It just reduces the frequency of the issue. 
   
   I have a solution/Hack in mind, 
   
   Does it make sense to add a bool param in the experimental API, say `handle_duplicate_execution_date`. if set as true, it will handle the duplicate execution_date issue by increasing the `execution_date` , a few microseconds. 
   
   For eg: 
   
   
    id   |      dag_id      |        execution_date         
   -----+------------------+-------------------------------
    171 | TEST_DAG 	| 2020-06-19 12:56:06.171041+00
   
   
   if we encounter a unique constraint issue, we will simply increase the execution time by a few microseconds.
   
    id   |      dag_id      |        execution_date         
   -----+------------------+-------------------------------
    171 | TEST_DAG 	| 2020-06-19 12:56:06.171041+00
    172 | TEST_DAG 	| 2020-06-19 12:56:06.171051+00
   
   
   This can be done by adding `try... except` block here : https://github.com/apache/airflow/blob/05ea88869be6a62d312598b5b01095e16f0a16f8/airflow/api/common/experimental/trigger_dag.py#L86


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org