You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/08 08:27:30 UTC
[GitHub] [beam] stefan-er commented on issue #20355: Ability to set S3 object ACL when uploading
stefan-er commented on issue #20355:
URL: https://github.com/apache/beam/issues/20355#issuecomment-1178705671
Hi we had the same issue and found a way to implement a solution which currently works fine for us.
* first you have to extend `com.amazonaws.handlers.RequestHandler2`
```java
import com.amazonaws.AmazonWebServiceRequest;
import com.amazonaws.handlers.RequestHandler2;
import com.amazonaws.services.s3.model.CannedAccessControlList;
import com.amazonaws.services.s3.model.CopyObjectRequest;
import com.amazonaws.services.s3.model.InitiateMultipartUploadRequest;
public class AclRequestHandler extends RequestHandler2 {
@Override
public AmazonWebServiceRequest beforeExecution(AmazonWebServiceRequest request) {
/*
* this is the request which creates the file in the temp bucket
*/
if (request instanceof InitiateMultipartUploadRequest) {
((InitiateMultipartUploadRequest) request)
.setCannedACL(CannedAccessControlList.BucketOwnerFullControl);
}
/*
* this is the request which copies the file from the temp bucket to the final bucket
*/
if (request instanceof CopyObjectRequest) {
((CopyObjectRequest) request)
.setCannedAccessControlList(CannedAccessControlList.BucketOwnerFullControl);
}
return request;
}
}
```
* then you have to extend the `org.apache.beam.sdk.io.aws.s3.DefaultS3ClientBuilderFactory`
```java
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import org.apache.beam.sdk.io.aws.options.S3Options;
import org.apache.beam.sdk.io.aws.s3.DefaultS3ClientBuilderFactory;
public class PipelineS3ClientBuilderFactory extends DefaultS3ClientBuilderFactory {
@Override
public AmazonS3ClientBuilder createBuilder(S3Options s3Options) {
return super.createBuilder(s3Options)
.enableForceGlobalBucketAccess()
.withRequestHandlers(new AclRequestHandler());
}
}
```
* finally you have to set the S3ClientBuilder factory class to the `org.apache.beam.sdk.io.aws.options.S3Options` upon pipeline start/creation
```java
options.setS3ClientFactoryClass(PipelineS3ClientBuilderFactory.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org