[GitHub] [hadoop] steveloughran commented on issue #1661: HADOOP-16484. S3A to warn or fail if S3Guard is disabled

[GitBox <gi...@apache.org>]

steveloughran commented on issue #1661: HADOOP-16484. S3A to warn or fail if S3Guard is disabled
URL: https://github.com/apache/hadoop/pull/1661#issuecomment-545050809
 
 
   
   Made my comments. Unlike Sid, I do believe the default should be to inform.
   
   1. AWS S3 is the main s3 store we deal with.
   1. If you don't use s3guard then intermittently things fail. rarely, but inevitably
   1. Sometimes the failures (new files not in list used during rename) miss data, and the result is corrupt -but people dont notice until later, when the support calls are "where is my data?"
   1. sometimes failures are immediate and there the support calls are "rename failed with error ..."
   1. either way, step 1 is trying to work out if s3guard is on
   1. which today we can only infer by looking for the absence of s3guard messages
   
   this fixes things so that we do get messages in the log, and users get warned. Everyone needs this.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org