[GitHub] [airflow] darrenleeweber commented on issue #4068: [AIRFLOW-2310]: Add AWS Glue Job Compatibility to Airflow

[GitBox <gi...@apache.org>]

darrenleeweber commented on issue #4068: [AIRFLOW-2310]: Add AWS Glue Job Compatibility to Airflow
URL: https://github.com/apache/airflow/pull/4068#issuecomment-526224813
 
 
   - watching
   - 2c on operator and/or sensor:  a sensor has an option to be rescheduled for the next poll, so it can release a worker, whereas e.g. the batch operator ties up a worker while it polls for a job status (I recently contributed to that piece of code so it polls less frequently, but it still ties up a worker); ideally, it would be great if an operator that needs to poll for status could be rescheduled in the same way that a sensor can be rescheduled (not sure what that refactor looks like)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services