[GitHub] [hadoop] ben-roling commented on issue #675: HADOOP-16085: use object version or etags to protect against inconsistent read after replace/overwrite

[GitBox <gi...@apache.org>]

ben-roling commented on issue #675: HADOOP-16085: use object version or etags to protect against inconsistent read after replace/overwrite
URL: https://github.com/apache/hadoop/pull/675#issuecomment-488696732
 
 
   > I've realised we need to handle the situation of "overwritten file with old file still found" in the input stream and rename operations
   
   Yeah, to be honest I had thought about that a little before but failed to come back to it.  In current state a read-after-overwrite might fail with RemoteFileChangedException when retries could possibly recover.  How frequently that is going to happen is difficult to guess.  With the code as-is, at least the silent inconsistent reads are prevented, albeit at the expense of some potential false-positive RemoteFileChangedException job failures.
   
   I'll start digging into the implementation of the retries though.
   
   > how about you create a new PR with the code squashed so we get this discussion preserved as is?
   
   Works for me.  I did that once before but didn't know whether there was another more preferred solution.  I'll go ahead and do that now with a rebase on trunk and squash to get rid of the new merge conflicts.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org