[GitHub] [airflow] KevinYang21 commented on a change in pull request #5194: [AIRFLOW-4419] Refine concurrency check in scheduler

[GitBox <gi...@apache.org>]

KevinYang21 commented on a change in pull request #5194: [AIRFLOW-4419] Refine concurrency check in scheduler
URL: https://github.com/apache/airflow/pull/5194#discussion_r279499723
 
 

 ##########
 File path: airflow/models/pool.py
 ##########
 @@ -32,9 +33,21 @@ class Pool(Base):
     slots = Column(Integer, default=0)
     description = Column(Text)
 
+    default_pool_name = 'not_pooled'
+
     def __repr__(self):
         return self.pool
 
+    @staticmethod
+    @provide_session
+    def default_pool_open_slots(session):
+        from airflow.models import TaskInstance as TI  # To avoid circular imports
+        total_slots = conf.getint('core', 'non_pooled_task_slot_count')
+        used_slots = session.query(func.count()).filter(
+            TI.pool == Pool.default_pool_name).filter(
+            TI.state.in_([State.RUNNING, State.QUEUED])).scalar()
+        return total_slots - used_slots
 
 Review comment:
   Yes we may have race issue and other cases causing the result to be negative( e.g. force triggered backfills). Right now the usage of this method does check the result using `<= 0` so even if we got negative returns we're fine. And logically I think it makes more sense to return negative values if we're overwhelming the pool. What do you think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services