You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/12/31 03:32:42 UTC
[GitHub] [arrow-rs] winding-lines opened a new issue, #3419: Derive Clone for the builders in object-store.
winding-lines opened a new issue, #3419:
URL: https://github.com/apache/arrow-rs/issues/3419
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
<!--
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
(This section helps Arrow developers understand the context and *why* for this feature, in addition to the *what*)
-->
I am using the object-store create in polars. The async code is generally contained in a small number of functions. I plan to pass around builders (eg AmazonS3Builder, MicrosoftAzureBuilder, GoogleCloudStorageBuilder) and instantiate an object-store based on the urls that are being passed in. I cannot currently use one builder more than one since they don't implement Clone.
Is there a reason why these structs are not implementing Clone?
**Describe the solution you'd like**
<!--
A clear and concise description of what you want to happen.
-->
Implement (most likely derive) Clone for AmazonS3Builder, MicrosoftAzureBuilder, GoogleCloudStorageBuilder and other object_store builders.
**Describe alternatives you've considered**
<!--
A clear and concise description of any alternative solutions or features you've considered.
-->
I can recreate the struct from their fields but then I will need to maintain the code on my side.
**Additional context**
<!--
Add any other context or screenshots about the feature request here.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold commented on issue #3419: Derive Clone for the builders in object-store.
Posted by GitBox <gi...@apache.org>.
tustvold commented on issue #3419:
URL: https://github.com/apache/arrow-rs/issues/3419#issuecomment-1368250855
> unfortunately the Builders contain Result and Error fields which are not clonable and there is no easy way to make them clonable.
This is a relatively recent change, added in #3327, and not yet released. We could tweak the implementation of this slightly to make this simpler. I'll have a play
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] tustvold closed issue #3419: Derive Clone for the builders in object-store.
Posted by GitBox <gi...@apache.org>.
tustvold closed issue #3419: Derive Clone for the builders in object-store.
URL: https://github.com/apache/arrow-rs/issues/3419
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Derive Clone for the builders in object-store. [arrow-rs]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #3419:
URL: https://github.com/apache/arrow-rs/issues/3419#issuecomment-2741685775
Migrating from arrow-rs issue [#3419](https://github.com/apache/arrow-rs/issues/3419)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #3419: Derive Clone for the builders in object-store.
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #3419:
URL: https://github.com/apache/arrow-rs/issues/3419#issuecomment-1368212235
I do not think there is any reason these structs can't implement clone. We would welcome a contribution to do so.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] alamb commented on issue #3419: Derive Clone for the builders in object-store.
Posted by GitBox <gi...@apache.org>.
alamb commented on issue #3419:
URL: https://github.com/apache/arrow-rs/issues/3419#issuecomment-1368448712
I think adding `Serialize` and `Deserialize` would be just fine, though perhaps we could do it as a follow on PR?
I think it would simply require adding
```
#[cfg_attr(feature = "serde", derive(Serialize, Deserialize))]
```
To each of the builders
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] winding-lines commented on issue #3419: Derive Clone for the builders in object-store.
Posted by GitBox <gi...@apache.org>.
winding-lines commented on issue #3419:
URL: https://github.com/apache/arrow-rs/issues/3419#issuecomment-1368239186
@alamb unfortunately the Builders contain `Result` and `Error` fields which are not clonable and there is no easy way to make them clonable.
One option that I see is do a slight refactoring of the code where the data from the user is saved in a separate Config object, before any fallible processing is done. Then we can clone that part.
Note that delta-rs is also going through some hoops here: https://github.com/delta-io/delta-rs/tree/main/rust/src/builder
Let me know what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] winding-lines commented on issue #3419: Derive Clone for the builders in object-store.
Posted by GitBox <gi...@apache.org>.
winding-lines commented on issue #3419:
URL: https://github.com/apache/arrow-rs/issues/3419#issuecomment-1368254114
Thanks @tustvold!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-rs] winding-lines commented on issue #3419: Derive Clone for the builders in object-store.
Posted by GitBox <gi...@apache.org>.
winding-lines commented on issue #3419:
URL: https://github.com/apache/arrow-rs/issues/3419#issuecomment-1368316840
@alamb and @tustvold Happy New Year 2023!
I tested @tustvold 's branch inside my `polars` PR and what would help me tremendously is an extension of this issue. In addition to Clone it would be super helpful to also derive `serde::Serializer` and `serde::Deserializer`. I tested this on @tustvold 's branch and it just works on top of the existing cleanup. This affects the 3 builders an 2 transitively used `struct`.
Would it be ok to increase the scope? Many thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org