You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "christianbudgetthuis (via GitHub)" <gi...@apache.org> on 2025/11/05 10:54:16 UTC
[I] Support upper-case configuration options in parse_url_opts [arrow-rs-object-store]
christianbudgetthuis opened a new issue, #529:
URL: https://github.com/apache/arrow-rs-object-store/issues/529
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
When using the from_env function of the object store builders, environmental variables are expected to be upper-case, as is coventional. They are then converted to lower-case to be able to parse them as configuration options (see https://github.com/apache/arrow-rs-object-store/blob/1b8ecc7f41eab2a3c2c373f1c2fe21bebe391727/src/azure/builder.rs#L517).
However, when using parse_url_opts the input is _not_ converted to lower-case (see https://github.com/apache/arrow-rs-object-store/blob/1b8ecc7f41eab2a3c2c373f1c2fe21bebe391727/src/parse.rs#L146) but, of course, still needs to be lower-case as this is what the parse function expects. The documentation even recommends passing std::env::vars() to the parse_url_opts function, which does not work as expected if you are using upper-case environmental variables.
**Describe the solution you'd like**
Make parse_url_opts work with uppercase env. var names, e.g. add to_ascii_lowercase in the builder macro.
**Describe alternatives you've considered**
Add a note in the documentation that the configuration options need to be lower-case for parse_url_opts and uppercase when using from_env() functions for concrete object stores.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support upper-case configuration options in parse_url_opts [arrow-rs-object-store]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #529:
URL: https://github.com/apache/arrow-rs-object-store/issues/529#issuecomment-3491660935
Why not just convert the options to lower case on input? The current design was chosen as it allows users to choose if they want case sensitivity or not
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support upper-case configuration options in parse_url_opts [arrow-rs-object-store]
Posted by "crepererum (via GitHub)" <gi...@apache.org>.
crepererum closed issue #529: Support upper-case configuration options in parse_url_opts
URL: https://github.com/apache/arrow-rs-object-store/issues/529
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support upper-case configuration options in parse_url_opts [arrow-rs-object-store]
Posted by "christianbudgetthuis (via GitHub)" <gi...@apache.org>.
christianbudgetthuis commented on issue #529:
URL: https://github.com/apache/arrow-rs-object-store/issues/529#issuecomment-3492051179
You are right, that is a possible workaround.
However, it seems a little confusing to me that the from_env() function in the builder does automatically convert to lower-case, while the parse_url_opts() function does not. But if that remains the case, I would suggest the documentation (https://docs.rs/object_store/latest/object_store/fn.parse_url_opts.html) to mention that the input here is case-sensitive and needs to be lower-case. Especially, since the builder documentation (https://docs.rs/object_store/latest/object_store/azure/struct.MicrosoftAzureBuilder.html#method.from_env), does state them in upper-case, and one could assume that both functions work alike.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Support upper-case configuration options in parse_url_opts [arrow-rs-object-store]
Posted by "criccomini (via GitHub)" <gi...@apache.org>.
criccomini commented on issue #529:
URL: https://github.com/apache/arrow-rs-object-store/issues/529#issuecomment-3702591100
Just came here to +1 this. Spent the better part of an hour debugging this for someone only to find it was because `AmazonS3Builder::from_env` works with upper case but resolve (parse_url_opts) does not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org