[GitHub] [spark] attilapiros edited a comment on issue #24554: [SPARK-27622][Core] Avoiding the network when block manager fetches disk persisted RDD blocks from the same host

[GitBox <gi...@apache.org>]

attilapiros edited a comment on issue #24554: [SPARK-27622][Core] Avoiding the network when block manager fetches disk persisted RDD blocks from the same host
URL: https://github.com/apache/spark/pull/24554#issuecomment-491284321
 
 
   Thanks @squito! I will look into the problem regarding unpersist and file delete. On linux what I have found is already promising: https://linux.die.net/man/2/unlink :
   
   >If the name was the last link to a file but any processes still have the file open the file will remain in existence until the last file descriptor referring to it is closed.
   
   Of course I will test this and come back to you with the result. Moreover as I know on windows a file cannot be deleted if it is opened. But I really have to check the details.
   
   I have not tried this PR on a cluster but I will do that too.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org