You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "Omega359 (via GitHub)" <gi...@apache.org> on 2024/01/15 15:50:14 UTC
[I] Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations [arrow-rs]
Omega359 opened a new issue, #5303:
URL: https://github.com/apache/arrow-rs/issues/5303
This is related to https://github.com/apache/arrow-datafusion/issues/5398 where functions for parsing string -> datetime and string -> nanos would be added to include a second 'format' parameter. The format would be a chrono format (https://docs.rs/chrono/latest/chrono/format/strftime/index.html) and would return an Err if the string couldn't be parsed with the provided format.
The proposed two new functions that would be added would be :
* ```pub fn string_to_datetime_formatted<T: TimeZone>(timezone: &T, s: &str, format: &str) -> Result<DateTime<T>, ArrowError>``` (mirrors the existing string_to_datetime function)
* ```pub fn string_to_timestamp_nanos_formatted(s: &str, format: &str) -> Result<i64, ArrowError>``` (mirrors the existing string_to_timestamp_nanos function)
In addition, for completeness all the existing Parser implementations in that file would be updated to include a 'parse_formatted' implementation where missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5303:
URL: https://github.com/apache/arrow-rs/issues/5303#issuecomment-1892515580
A few points:
* The current logic is quite highly optimised, we need to be careful not to introduce any performance regressions
* There are a couple of known bugs with the chrono format strings where the parser is overly permissive - https://github.com/chronotope/chrono/issues/332
* Timestamp parsing, especially timezones is very fiddly. I would request we keep code changes small, as it is very easy to subtly break things
Overall I am rather lukewarm on this functionality, but I will endeavour to review well tested and focused changes to this logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations [arrow-rs]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 closed issue #5303: Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations
URL: https://github.com/apache/arrow-rs/issues/5303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations [arrow-rs]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #5303:
URL: https://github.com/apache/arrow-rs/issues/5303#issuecomment-1896267431
Closing this issue with https://github.com/apache/arrow-datafusion/issues/5398 as an alternative.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #5303:
URL: https://github.com/apache/arrow-rs/issues/5303#issuecomment-1892594950
Incubating functionality in DataFusion before lifting it into arrow-rs has been a successful model in the past, and might be a quicker way to iterate on https://github.com/apache/arrow-datafusion/issues/5398 without having to wait on an arrow-rs releases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations [arrow-rs]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #5303:
URL: https://github.com/apache/arrow-rs/issues/5303#issuecomment-1892597897
Sounds good, I'll pull in my changes to DataFusion for now and we can close this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add support for providing Chrono formats when parsing strings -> datetime/nanos and Parser implementations [arrow-rs]
Posted by "Omega359 (via GitHub)" <gi...@apache.org>.
Omega359 commented on issue #5303:
URL: https://github.com/apache/arrow-rs/issues/5303#issuecomment-1892587163
> Overall I am rather lukewarm on this functionality, as it is by definition diverging from established standards, but I will endeavour to review well tested and focused changes to this logic that don't regress existing workloads.
Would you prefer this to not be in arrow-cast and rather be left in datafusion? I'm only adding it here to mirror the existing functionality - I have no desire to change any of the current implementation details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org