[GitHub] [beam] damccorm opened a new issue, #20355: Ability to set S3 object ACL when uploading

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #20355:
URL: https://github.com/apache/beam/issues/20355

   I am using an AWS account to write to S3.  However, the bucket owner doesn't have the access to the objects I uploaded.  I'd like to set the object ACL to bucket-owner-full-control so that the bucket owner would have the full control over the objects I uploaded.
   
   Imported from Jira [BEAM-10850](https://issues.apache.org/jira/browse/BEAM-10850). Original Jira may contain additional context.
   Reported by: tun-chang.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] stefan-er commented on issue #20355: Ability to set S3 object ACL when uploading

[GitBox <gi...@apache.org>]

stefan-er commented on issue #20355:
URL: https://github.com/apache/beam/issues/20355#issuecomment-1178705671

   Hi we had the same issue and found a way to implement a solution which currently works fine for us.
   
   * first you have to extend `com.amazonaws.handlers.RequestHandler2`
   
   ```java
   import com.amazonaws.AmazonWebServiceRequest;
   import com.amazonaws.handlers.RequestHandler2;
   import com.amazonaws.services.s3.model.CannedAccessControlList;
   import com.amazonaws.services.s3.model.CopyObjectRequest;
   import com.amazonaws.services.s3.model.InitiateMultipartUploadRequest;
   
   public class AclRequestHandler extends RequestHandler2 {
     @Override
     public AmazonWebServiceRequest beforeExecution(AmazonWebServiceRequest request) {
       /*
        * this is the request which creates the file in the temp bucket
        */
       if (request instanceof InitiateMultipartUploadRequest) {
         ((InitiateMultipartUploadRequest) request)
             .setCannedACL(CannedAccessControlList.BucketOwnerFullControl);
       }
   
       /*
        * this is the request which copies the file from the temp bucket to the final bucket
        */
       if (request instanceof CopyObjectRequest) {
         ((CopyObjectRequest) request)
             .setCannedAccessControlList(CannedAccessControlList.BucketOwnerFullControl);
       }
   
       return request;
     }
   }
   ```
   
   * then you have to extend the `org.apache.beam.sdk.io.aws.s3.DefaultS3ClientBuilderFactory`
   
   ```java
   import com.amazonaws.services.s3.AmazonS3ClientBuilder;
   import org.apache.beam.sdk.io.aws.options.S3Options;
   import org.apache.beam.sdk.io.aws.s3.DefaultS3ClientBuilderFactory;
   
   public class PipelineS3ClientBuilderFactory extends DefaultS3ClientBuilderFactory {
     @Override
     public AmazonS3ClientBuilder createBuilder(S3Options s3Options) {
       return super.createBuilder(s3Options)
           .enableForceGlobalBucketAccess()
           .withRequestHandlers(new AclRequestHandler());
     }
   }
   ```
   
   * finally you have to set the S3ClientBuilder factory class to the `org.apache.beam.sdk.io.aws.options.S3Options` upon pipeline start/creation
   
   ```java
   options.setS3ClientFactoryClass(PipelineS3ClientBuilderFactory.class);
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org