Re: [PR] feat: add `poll_now_notify` to `poll_loop` and `on_work_available` callback [datafusion-ballista]

["phillipleblanc (via GitHub)" <gi...@apache.org>]

phillipleblanc commented on code in PR #1893:
URL: https://github.com/apache/datafusion-ballista/pull/1893#discussion_r3611666503


##########
ballista/scheduler/src/config.rs:
##########
@@ -34,6 +34,16 @@ use datafusion_proto::physical_plan::PhysicalExtensionCodec;
 use std::fmt::Display;
 use std::sync::Arc;
 
+/// Callback invoked when new work becomes available for executors.
+///
+/// This is called after:
+/// - A job is submitted and tasks are ready to be scheduled
+/// - Tasks complete and new stages become runnable
+///
+/// This allows external systems to notify executors to poll immediately
+/// rather than waiting for their next poll interval.

Review Comment:
   Comment added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org