[GitHub] [spark] HeartSaVioR commented on issue #23764: [SPARK-26825][SS] Fix temp checkpoint creation in cluster mode when default filesystem is not local.

[GitBox <gi...@apache.org>]

HeartSaVioR commented on issue #23764: [SPARK-26825][SS] Fix temp checkpoint creation in cluster mode when default filesystem is not local.
URL: https://github.com/apache/spark/pull/23764#issuecomment-471792205
 
 
   We're only dealing with temporary checkpoint here, so while I can see what @jose-torres mentioned, it is unlikely to happen in happy case and I haven't seen cache miss from running query. That could still happen in bad case like if some executors are becoming bad.
   
   Btw, may be better to document somewhere that the checkpoint should be accessible from executor if query is stateful? I guess that's only thing where non-driver reads and writes into checkpoint directory, but I might be missing something.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org