You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "Deependra-Patel (via GitHub)" <gi...@apache.org> on 2023/08/08 10:48:41 UTC
[GitHub] [spark] Deependra-Patel opened a new pull request, #42391: [SPARK-44704][CORE] Cleanup files from host node after migration due to graceful decommissioning
Deependra-Patel opened a new pull request, #42391:
URL: https://github.com/apache/spark/pull/42391
### What changes were proposed in this pull request?
Deleting files on executor exit when migration is enabled
### Why are the changes needed?
1. This will release extra disk space on the node reducing the failures due to disk running out of space
2. If there is no shuffle data then YARN can decommission the node right away. Needs this other PR to be merged to expose this metric to YARN https://github.com/apache/spark/pull/42071
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Tested manually.
Will add unit tests once we get traction on this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-44704][CORE] Cleanup files from host node after migration due to graceful decommissioning [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #42391: [SPARK-44704][CORE] Cleanup files from host node after migration due to graceful decommissioning
URL: https://github.com/apache/spark/pull/42391
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-44704][CORE] Cleanup files from host node after migration due to graceful decommissioning [spark]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #42391:
URL: https://github.com/apache/spark/pull/42391#issuecomment-1815534229
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org