You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Diego Argueta (JIRA)" <ji...@apache.org> on 2018/09/27 00:42:00 UTC
[jira] [Comment Edited] (ARROW-3329) [Python] Error casting
decimal(38, 4) to int64
[ https://issues.apache.org/jira/browse/ARROW-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629576#comment-16629576 ]
Diego Argueta edited comment on ARROW-3329 at 9/27/18 12:41 AM:
----------------------------------------------------------------
This isn't a bug, it's a mathematical issue. Let me explain:
To store a number with X digits of precision (in base 10) you need a number with at least {{log2(X)}} bits in it.
So 38 digits of precision in base 10 requires {{log2(38)}} bits to represent in binary. {{log2(38)}} is a little under 127, meaning you need your storage to be at least 127 bits long to store a {{decimal(38, 4)}} losslessly. The error message isn't exactly the most helpful, but it _is_ correct. There's no way to cast a decimal of that size to a 64-bit integer.
For future reference, a 64-bit integer can hold at most 19 digits of precision.
was (Author: yiannisliodakis):
This isn't a bug, it's a mathematical issue. Let me explain:
To store a number with X digits of precision (in base 10) you need a number with at least {{log2(X)}} bits in it.
So 38 digits of precision in base 10 requires {{log2(38)}} bits to represent in binary. {{log2(38)}} is a little under 127, meaning you need your storage to be at least 127 bits long to store a {{decimal(38, 4)}} losslessly. The error message isn't exactly the most helpful, but it _is_ correct. There's no way to cast a decimal of that size to a 64-bit integer.
> [Python] Error casting decimal(38, 4) to int64
> ----------------------------------------------
>
> Key: ARROW-3329
> URL: https://issues.apache.org/jira/browse/ARROW-3329
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Environment: Python version : 3.6.5
> Pyarrow version : 0.10.0
> Reporter: Kavita Sheth
> Priority: Major
>
> Git issue LInk : https://github.com/apache/arrow/issues/2627
> I want to cast pyarrow table column from decimal(38,4) to int64.
> col.cast(pa.int64())
> Error:
> File "pyarrow/table.pxi", line 443, in pyarrow.lib.Column.cast
> File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
> pyarrow.lib.ArrowNotImplementedError: No cast implemented from decimal(38, 4) to int64
> Python version : 3.6.5
> Pyarrow version : 0.10.0
> is it not implemented yet or I am not using it correctly? If not implemented yet, then any work around to cast columns?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)