[GitHub] arunmahadevan commented on issue #23764: [SPARK-26825][SS] Fix temp checkpoint creation in cluster mode when default filesystem is not local.

[GitBox <gi...@apache.org>]

arunmahadevan commented on issue #23764: [SPARK-26825][SS] Fix temp checkpoint creation in cluster mode when default filesystem is not local.
URL: https://github.com/apache/spark/pull/23764#issuecomment-467183925
 
 
   @gaborgsomogyi thanks for the clarification, the `StreamExecution` apparently handles the deletion. However theres a slight behavior change here where earlier `Utils.CreateTempDir` would handle automatic deletion on shutdown and with the proposed one theres a possibility of the tmp dir being left over on shutdown. However I don't think its a big deal so I am ok with the proposed approach as long as we have validated creating the local dir via HDFS apis (the proposed one) does not have other differences with `Utils.CreateTempDir`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org