You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "niebayes (via GitHub)" <gi...@apache.org> on 2024/10/14 03:12:12 UTC
[I] Make casting from a timestamp without timezone to a timestamp with timezone optional [arrow-rs]
niebayes opened a new issue, #6555:
URL: https://github.com/apache/arrow-rs/issues/6555
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
In our time-series database project, we always interpret timestamps without a timezone as being in the UTC timezone. However, when we use the [arrow cast](https://docs.rs/arrow/latest/arrow/compute/fn.cast.html) function to convert a timestamp without a timezone to a specific timezone, the result isn't what we expect. This is because the cast function enforces an adjustment internally, which causes an unintended modification to the timestamp values during the cast process.
As specified in the [doc](https://docs.rs/arrow/latest/arrow/compute/fn.cast_with_options.html#casting-timestamps-without-timezone-to-timestamps-with-timezone) and in the comment of [codes](https://github.com/apache/arrow-rs/blob/9633f14f8d9e5b9b8fdf3514b7acb77217b30584/arrow-cast/src/cast/mod.rs#L1785-L1818), arrow cast will adjust the timestamp values if casting from a timestamp array without a timezone to a timestamp array with a timezone.
I propose to make such a behavior configurable and optional.
**Describe the solution you'd like**
<!--
A clear and concise description of what you want to happen.
-->
A sound solution may be add a boolean flag to [`CastOptions`](https://docs.rs/arrow/latest/arrow/compute/struct.CastOptions.html) to make the arrow cast function not perform such an adjustment. This boolean flag may be named `enable_timestamp_adjustment`.
**Describe alternatives you've considered**
<!--
A clear and concise description of any alternative solutions or features you've considered.
-->
**Additional context**
<!--
Add any other context or screenshots about the feature request here.
-->
#5827 stated that the behavior of the arrow cast function is somewhat confusing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make casting from a timestamp without timezone to a timestamp with timezone configurable [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold closed issue #6555: Make casting from a timestamp without timezone to a timestamp with timezone configurable
URL: https://github.com/apache/arrow-rs/issues/6555
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Make casting from a timestamp without timezone to a timestamp with timezone configurable [arrow-rs]
Posted by "tustvold (via GitHub)" <gi...@apache.org>.
tustvold commented on issue #6555:
URL: https://github.com/apache/arrow-rs/issues/6555#issuecomment-2410310762
Whilst perhaps not ideal, this behaviour can be achieved by first casting to a timestamp with a timezone of UTC and then casting to the desired target TZ.
That being said, if all timestamps are UTC, why not just store them as UTC timestamps?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org