[GitHub] [spark] LuciferYang commented on pull request #39956: [SPARK-42389][CORE][TESTS] Ensure Live UI dir be cleaned up after test when `LIVE_UI_LOCAL_STORE_DIR` is configured

["LuciferYang (via GitHub)" <gi...@apache.org>]

LuciferYang commented on PR #39956:
URL: https://github.com/apache/spark/pull/39956#issuecomment-1424590303

   For example, `LauncherBackendSuite `
   
   https://github.com/apache/spark/blob/201a91bab0e3d540ee262bd73b597b137f3f987b/core/src/test/scala/org/apache/spark/launcher/LauncherBackendSuite.scala#L70-L72
   
   When I export `LIVE_UI_LOCAL_STORE_DIR` and test it locally, `handle.kill()` gives `AppStatusStore#cleanUpStorePath` no chance to execute, which will cause Live UI dirs to remain.
   
   This will not cause the test to fail, but will increase the disk usage. I didn't look for all similar cases, but I hope to add a protection mechanism to avoid this issue, this may friendly to developers.  At the same time, I added some logs to find out which cases did not actively clean up the Live UI dir.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org