You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/10/28 14:11:40 UTC
[GitHub] [arrow-datafusion] waitingkuo opened a new issue, #3997: change date_part return types to f64
waitingkuo opened a new issue, #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997
`date_part` currently returns `i32` which might lose some information, e.g.
```bash
❯ select date_part('second', timestamp '2000-01-01T00:00:00.1');
+--------------------------------------------------------+
| datepart(Utf8("second"),Utf8("2000-01-01T00:00:00.1")) |
+--------------------------------------------------------+
| 0 |
+--------------------------------------------------------+
1 row in set. Query took 0.000 seconds.
```
while postgresql returns double precision
```bash
willy=# select date_part('second', timestamp '2000-01-01T00:00:00.1');
date_part
-----------
0.1
(1 row)
```
is it recommended to follow postgresql's return type?
if it is preferred, perhaps we could do a separate pr to deal with this
change it to double precision
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
perhaps we could consider return decimals instead to align with extract to return decimal. postgres doc mentions this:
```
For historical reasons, the date_part function returns values of type double precision.
This can result in a loss of precision in certain uses. Using extract is recommended instead.
```
https://www.postgresql.org/docs/current/functions-datetime.html
**Additional context**
Add any other context or screenshots about the feature request here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] waitingkuo commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
waitingkuo commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1301758323
@comphead thank you. i think we need to wait for 27.0.0 . 26.0.0 is just fixed. It normally released 1 or 2 times a month
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1314566562
waiting for #4199
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1301649469
@waitingkuo the https://github.com/apache/arrow-rs/issues/2995 merged, should we wait for next arrow-rs version? How it usually happens?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1298858221
Related arrow-rs ticket https://github.com/apache/arrow-rs/issues/2995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
alamb closed issue #3997: change` date_part` return types to f64
URL: https://github.com/apache/arrow-datafusion/issues/3997
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] waitingkuo commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
waitingkuo commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1298824887
cc @alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1298705591
@waitingkuo the problem is `date_part` calls for `arrow::compute::kernels::temporal::second` which is i32, and they dont have millisecond support. Do you think its good idea to implement milliseconds by ourselves?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1298845170
@waitingkuo thanks for the details, let me try to add the ticket and PR to arrow-rs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] waitingkuo commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
waitingkuo commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1298824543
@comphead
`arrow-rs` uses `chrono-rs` to extract these date parts. Chono-rs does have [nanosecond](https://docs.rs/chrono/latest/chrono/trait.Timelike.html#tymethod.nanosecond) while arrow-rs doesn't have.
We could extend `arrow-rs` to support `nanosecond` and then get back to this issue.
I think adding nanosecond support in `arrow-rs` is a `good start issue`. If you're interested in, I'm happy to help you to figure out the codes to modify. Basically, you could follow the `second`'s logic to do so
https://github.com/apache/arrow-rs/blob/master/arrow/src/compute/kernels/temporal.rs#L723-L764
I could help to send a pr to arrow-rs as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] comphead commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1295975429
@waitingkuo may I try this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] waitingkuo commented on issue #3997: change` date_part` return types to f64
Posted by GitBox <gi...@apache.org>.
waitingkuo commented on issue #3997:
URL: https://github.com/apache/arrow-datafusion/issues/3997#issuecomment-1296151364
> @waitingkuo may I try this?
sure, please try it~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org