Dynamic pool Allocation policy for a Task

[ra...@gmail.com, ra...@gmail.com]

Hi All,

Currently DAG tasks are assigned to a pool during DAG creation time so its kind of static pool assignment which is done at creation time and might lead to unused resources if there are multiple pools configured in the airflow.
So does it make sense to have support for pool allocation policy where a task can be assigned to some other pool if current pre-allocated pool resources are not available. There can be some limit on how much such slots can be used.
We have a use case where we have multiple pools which can better be utilised with some reallocation.

Thanks,
Raman Gupta