[I] Feature Request: Support ACL (and extra parameters) in put file Operation [arrow-rs-object-store]

["abdallahashraf22 (via GitHub)" <gi...@apache.org>]

abdallahashraf22 opened a new issue, #876:
URL: https://github.com/apache/arrow-rs-object-store/issues/876

   *please excuse my python only examples*
   
   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   Yes
   The current implementation doesn't support setting ACL policies on specific files on upload.
   it also doesn't support other parameters that could come be useful (like Metadata or other ExtraArgs)
   
   examples of the ACL settings:
   in boto3 of AWS
   https://docs.aws.amazon.com/boto3/latest/guide/s3-uploading-files.html#the-extraargs-parameter
   this allows the file to be set as public, so it's accessible using the CDN link or S3 origin link
   ```
   s3.upload_file(
       'FILE_NAME', 'amzn-s3-demo-bucket', 'OBJECT_NAME',
       ExtraArgs={'ACL': 'public-read'}
   )
   ```
   it's also available in the aws request syntax of put file
   https://docs.aws.amazon.com/botocore/latest/reference/services/s3/client/put_object.html
   <img width="1041" height="195" alt="Image" src="https://github.com/user-attachments/assets/18ec0a71-58dd-44ef-a60f-42bb17040736" />
   
   
   afaik this is also available in GCP as the "predefinedAcl"
   https://docs.cloud.google.com/storage/docs/json_api/v1/objects/insert
   https://docs.cloud.google.com/storage/docs/json_api/v1/objects/insert#parameters
   
   <img width="1141" height="524" alt="Image" src="https://github.com/user-attachments/assets/65a1da2b-1425-46c8-882e-f947f0769ef1" />
   
   it was requested in obstore (the library that I use) https://github.com/developmentseed/obstore/issues/601
   but since it's a wrapper around this library it couldn't be done
   
   currently I'm using digital Ocean and it's an aws-s3-like storage bucket, and it can use ACLs to set files to public read as well
   
   <img width="546" height="386" alt="Image" src="https://github.com/user-attachments/assets/80a8e690-b617-47ea-8e7f-6a83fbb539fa" />
   
   
   **Describe the solution you'd like**
   I would like the project to support a key like "extra information" where I can provide key: value pairs of what implementation specific options that needs to be added
   something like
   ```
   extra_options={
       "ACL": "public-read",
       "Metadata" :{
           "some_key": "some_value",
           "other_key": "other_value"
       },
       "SomeOtherPlatformSpecificKey": "some_value"
   }
   ```
   
   **Describe alternatives you've considered**
   alternative I considered is using the downstream library to upload, then setting the ACL using boto3
   ```
   store = S3Store(
       bucket=DO_BUCKET,
       endpoint=DO_ENDPOINT,
       access_key_id=DO_KEY,
       secret_access_key=DO_SECRET,
   )
   response = store.put(path=object_path, file=file, attributes={"Content-Type": content_type})
   async with s3_session.client(
               "s3", endpoint_url=DO_ENDPOINT
           ) as s3_client:
        response = await s3_client.put_object_acl(
                   Bucket=DO_BUCKET,
                   Key=object_path,
                   ACL="public-read",
               )
    # and for metadata it's worse
    await s3_client.copy_object(
       Bucket=DO_BUCKET,
       Key=key,
       CopySource={
           "Bucket": DO_BUCKET,
           "Key": object_path,
       },
       Metadata={
           "some_key": "some_value"
       },
       MetadataDirective="REPLACE",
   )
   
   
   **Additional context**
   no additional context, just that I know it opens hell's door, supporting some non specific set of keys, but it also opens up a lot of possibilities and supports a lot of extra use cases
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org