You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/08/03 17:04:20 UTC

[GitHub] [airflow] msumit opened a new pull request #10136: Handle IntegrityError while creating TIs

msumit opened a new pull request #10136:
URL: https://github.com/apache/airflow/pull/10136


   **Background**
   While doing a trigger_dag from UI, DagRun gets created first and then webserver starts creating TIs. Meanwhile, Scheduler also picks up the DagRun and starts creating the TIs, which results in IntegrityError as the Primary key constraint gets violated. This happens when a DAG has a good number of tasks (500+). 
   
   Also, changing the TIs array with a set for faster lookups. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on a change in pull request #10136: Handle IntegrityError while creating TIs

Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #10136:
URL: https://github.com/apache/airflow/pull/10136#discussion_r466970884



##########
File path: airflow/models/dagrun.py
##########
@@ -477,7 +478,14 @@ def verify_integrity(self, session=None):
                 task_instance_mutation_hook(ti)
                 session.add(ti)
 
-        session.commit()
+        try:
+            session.commit()
+        except IntegrityError as err:
+            self.log.info(str(err))
+            self.log.info(f'Hit IntegrityError while creating the TIs for '

Review comment:
       nit
   
   ```suggestion
               self.log.info('Hit IntegrityError while creating the TIs for '
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] msumit merged pull request #10136: Handle IntegrityError while creating TIs

Posted by GitBox <gi...@apache.org>.
msumit merged pull request #10136:
URL: https://github.com/apache/airflow/pull/10136


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] msumit commented on pull request #10136: Handle IntegrityError while creating TIs

Posted by GitBox <gi...@apache.org>.
msumit commented on pull request #10136:
URL: https://github.com/apache/airflow/pull/10136#issuecomment-670372083


   @ashb @kaxil @mik-laj @potiuk PTAL


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org