You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/07/20 08:31:39 UTC

[GitHub] [arrow-ballista] liukun4515 opened a new issue, #90: Scheduler will lost the registered executor when restart it in the `push` mode

liukun4515 opened a new issue, #90:
URL: https://github.com/apache/arrow-ballista/issues/90

   **Describe the bug**
   When i restart the schedule, the schedule lost all the information of registered executor
   
   **To Reproduce**
   Start a scheduler with below config:
   ```
   scheduler_policy="PushStaged"
   ```
   Start a executor with below config:
   ```
   scheduler_port=50050
   scheduler_host="localhost"
   # PushStaged or PullStaged
   task_scheduling_policy="PushStaged"
   ```
   
   then kill the scheduler and restart the scheduler using the same config.
   
   And the scheduler will lost all registered executor in the memory.
   
   
   
   **Expected behavior**
   We should recover this data in memory after the scheduler restart.
   
   Solution:
   heartbeat with the registered information for the executor
   
   **Additional context**
   Add any other context about the problem here.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-ballista] thinkharderdev commented on issue #90: Scheduler will lost the registered executor when restart it in the `push` mode

Posted by GitBox <gi...@apache.org>.
thinkharderdev commented on issue #90:
URL: https://github.com/apache/arrow-ballista/issues/90#issuecomment-1193862531

   @liukun4515 Are you running in standalone mode? It should initialize any registered executors from the backend if you are using etcd as the state backend but in standalone mode the persistent state is stored in sled DB on disk (in a temp file). If we wanted to make standalone mode persist state across restarts then we would need to make the sled DB location a configurable path. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-ballista] mingmwang commented on issue #90: Scheduler will lost the registered executor when restart it in the `push` mode

Posted by GitBox <gi...@apache.org>.
mingmwang commented on issue #90:
URL: https://github.com/apache/arrow-ballista/issues/90#issuecomment-1214967822

   Is it still an issue?  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow-ballista] r4ntix commented on issue #90: Scheduler will lost the registered executor when restart it in the `push` mode

Posted by GitBox <gi...@apache.org>.
r4ntix commented on issue #90:
URL: https://github.com/apache/arrow-ballista/issues/90#issuecomment-1247999107

   @liukun4515 @thinkharderdev @mingmwang This can be fixed by specifying the `--sled-dir` parameter when starting the scheduler service.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org