[GitHub] [beam] ahmedabu98 commented on a diff in pull request #22138: Add support when writing to locked buckets by handling retentionPolicyNotMet error

[GitBox <gi...@apache.org>]

ahmedabu98 commented on code in PR #22138:
URL: https://github.com/apache/beam/pull/22138#discussion_r922299041


##########
sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtil.java:
##########
@@ -897,6 +898,35 @@ public void onFailure(GoogleJsonError e, HttpHeaders responseHeaders) throws IOE
         } else {
           throw new FileNotFoundException(from.toString());
         }
+      } else if (e.getCode() == 403
+          && e.getErrors().size() == 1
+          && e.getErrors().get(0).getReason().equals("retentionPolicyNotMet")) {
+        List<StorageObjectOrIOException> srcAndDestObjects = getObjects(Arrays.asList(from, to));
+        if (srcAndDestObjects
+            .get(0)

Review Comment:
   We get `retentionPolicyNotMet` error when we prematurely (1) try to overwrite an existing file or (2) try to delete an existing file: https://cloud.google.com/storage/docs/bucket-lock#retention-policy
   
   Deletions are handled in a separate `onFailure()` method [here](https://github.com/apache/beam/blob/209daedff3e60f86188bc9e2d3e7e331e283373b/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtil.java#L815-L843), so at this point I believe we can assume both source and destination exist.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org