You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/19 23:20:47 UTC

[GitHub] [arrow-ballista] Ted-Jiang opened a new issue, #12: Make scheduler prefer assign the task with same jobId to same executor

Ted-Jiang opened a new issue, #12:
URL: https://github.com/apache/arrow-ballista/issues/12

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   In current code, each poll work request will scan all waiting task in the same name space.https://github.com/apache/arrow-datafusion/blob/d7ae8c2631b5ea86ac2328530f0a4745daaf0bda/ballista/rust/scheduler/src/state/mod.rs#L283
   
   ``
   plan1 -> shuffle wirte -> plan2
   ``
   
   Seems p1 and p2 run on the same executor may avoid sending intermediate results.
   **Describe the solution you'd like**
   1. executor request job with jobId 
   2. scheduler send same job task back
   
   
   **Describe alternatives you've considered**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org