You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by GitBox <gi...@apache.org> on 2023/01/18 21:33:22 UTC

[GitHub] [arrow] jbrockmendel opened a new issue, #33769: ENH: support quantile for temporal dtypes

jbrockmendel opened a new issue, #33769:
URL: https://github.com/apache/arrow/issues/33769

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   cc @jorisvandenbossche 
   
   For some methods (e.g., dictionary_encode xref #15226, mode, min_max) it is straightforward to cast to integer, compute, then cast back.  For quantile I've found doing this breaks a bunch of pandas tests (or more accurately, fails to fix existing xfails).  I speculate that this has to do with lossiness in int->float->int conversions.
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jbrockmendel commented on issue #33769: [Python] support quantile for temporal dtypes

Posted by "jbrockmendel (via GitHub)" <gi...@apache.org>.
jbrockmendel commented on issue #33769:
URL: https://github.com/apache/arrow/issues/33769#issuecomment-1404421515

   confirmed that the numpy-backed versions have identical floating point errors.  closing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jbrockmendel closed issue #33769: [Python] support quantile for temporal dtypes

Posted by "jbrockmendel (via GitHub)" <gi...@apache.org>.
jbrockmendel closed issue #33769: [Python] support quantile for temporal dtypes
URL: https://github.com/apache/arrow/issues/33769


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jorisvandenbossche commented on issue #33769: [Python] support quantile for temporal dtypes

Posted by GitBox <gi...@apache.org>.
jorisvandenbossche commented on issue #33769:
URL: https://github.com/apache/arrow/issues/33769#issuecomment-1397094509

   I think also for that reason (quantile by default always returns floats), we might not want to add support for quantile to temporal types (not sure if we actually ever considered this). 
   
   There are interpolation methods (lowest, highest, nearest) that do preserve the input type, though. So maybe we could only allow those?
   
   But on your point of not matching pandas behaviour if you do the casting manually. I am not sure having this kernel in pyarrow would make any difference for that, since arrow would basically do the same under the hood and calculate the interpolated value based on the underlying integers.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org