You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "collinoc (via GitHub)" <gi...@apache.org> on 2025/07/23 00:05:30 UTC
[I] Support implementation-specific attributes [arrow-rs-object-store]
collinoc opened a new issue, #446:
URL: https://github.com/apache/arrow-rs-object-store/issues/446
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
<!--
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*)
-->
User's should be able to specify implementation specific attributes when performing operations on objects within different object store implementations.
**Describe the solution you'd like**
<!--
A clear and concise description of what you want to happen.
-->
Expand the [`Attribute`](https://docs.rs/object_store/latest/object_store/enum.Attribute.html) enum to allow for a custom attribute to be added, similar to how the `Metadata` variant does for metadata attributes.
**Describe alternatives you've considered**
<!--
A clear and concise description of any alternative solutions or features you've considered.
-->
With AWS for example, you could use the [aws-sdk-s3](https://docs.rs/aws-sdk-s3/latest/aws_sdk_s3/) crate to perform a second operation on the object after its been uploaded, however this does not get the benefit of uploading and [applying an attribute within a single upload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_RequestSyntax) operation.
**Additional context**
<!--
Add any other context or screenshots about the feature request here.
-->
N/A
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support implementation-specific attributes [arrow-rs-object-store]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #446:
URL: https://github.com/apache/arrow-rs-object-store/issues/446#issuecomment-3141131734
> Though what do you think about creating separate issues for common individual attributes that are supported by all the cloud storage providers such as “storage class” and “object locking” (immutability locking in Azure)? At least that way the new corresponding enum variants would match well-defined common headers and keeps the quick swapability between cloud providers, rather than the generic unconstrained approach taken in [#447](https://github.com/apache/arrow-rs-object-store/pull/447).
I am not quite sure what this means or would entail
I filed a ticket to add a doc example
- https://github.com/apache/arrow-rs-object-store/issues/454
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support implementation-specific attributes [arrow-rs-object-store]
Posted by "collinoc (via GitHub)" <gi...@apache.org>.
collinoc commented on issue #446:
URL: https://github.com/apache/arrow-rs-object-store/issues/446#issuecomment-3141174243
Thanks for the new ticket @alamb
> I am not quite sure what this means or would entail
I just mean is it worth creating another ticket to add new `Attribute` variants `ObjectLockRetainUntilDate`, `ObjectLockMode`, `StorageClass`, and any other attributes supported by all of the cloud providers AWS, GCP, and Azure - instead of the catch-all variant I proposed in #447?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support implementation-specific attributes [arrow-rs-object-store]
Posted by "collinoc (via GitHub)" <gi...@apache.org>.
collinoc commented on issue #446:
URL: https://github.com/apache/arrow-rs-object-store/issues/446#issuecomment-3128952970
@alamb oh nice. Yeah that would be a better approach for general custom headers.
Though what do you think about creating separate issues for common individual attributes that are supported by all the cloud storage providers such as “storage class” and “object locking” (immutability locking in Azure)? At least that way the new corresponding enum variants would match well-defined common headers and keeps the quick swapability between cloud providers, rather than the generic unconstrained approach taken in #447.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support implementation-specific attributes [arrow-rs-object-store]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #446:
URL: https://github.com/apache/arrow-rs-object-store/issues/446#issuecomment-3127609690
Can you implement this using a custom client like https://docs.rs/object_store/latest/object_store/#customizing-http-clients
Unfortunately, it seems we are lacking a good doc example
But the basic idea is that you can provide your own wrapper around the lower level http client and inject whatever headers you want
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support implementation-specific attributes [arrow-rs-object-store]
Posted by "collinoc (via GitHub)" <gi...@apache.org>.
collinoc commented on issue #446:
URL: https://github.com/apache/arrow-rs-object-store/issues/446#issuecomment-3127278671
One alternative to this would be to expand `Attribute` only with attributes that have equivalents supported by all cloud storage providers, such as the ones mentioned in #447 ([x-amz-object-lock-mode](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#API_PutObject_RequestSyntax), [x-goog-storage-class](https://cloud.google.com/storage/docs/xml-api/reference-headers#xgoogstorageclass), etc.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org