[GitHub] [spark] ajithme commented on issue #24142: [SPARK-27194][core] Job failures when task attempts do not clean up spark-staging parquet files

[GitBox <gi...@apache.org>]

ajithme commented on issue #24142: [SPARK-27194][core] Job failures when task attempts do not clean up spark-staging parquet files
URL: https://github.com/apache/spark/pull/24142#issuecomment-474314089
 
 
   > > If a task failed without cleaning up its the files, the partially written file will still be there and moved to the final directory at the end.
   > 
   > Is this a problem or you fixed it as well?
   
   The fix is not changing the behavior of writing to temp location and then moving to final location. the fix deals only with file name ( even in temp/final location, the file names must not conflict)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org