[GitHub] [spark] mridulm commented on pull request #40412: [SPARK-42784] should still create subDir when the number of subDir in merge dir is less than conf

["mridulm (via GitHub)" <gi...@apache.org>]

mridulm commented on PR #40412:
URL: https://github.com/apache/spark/pull/40412#issuecomment-1549125020

   So to make sure I understand the issue here - we have a container started on a node, and got immediately killed (pre-empted/etc) - such that it was not able to complete creating `subDirsPerLocalDir` number of subdirectories (ended up create a few) ?
   
   If yes, very interesting corner case - will take a pass through the PR later this week, thanks for the contribution !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org