[jira] [Commented] (ARROW-2432) [Python] from_pandas fails when converting decimals if have None values

["Antoine Pitrou (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431331#comment-16431331 ] 

Antoine Pitrou commented on ARROW-2432:
---------------------------------------

Ow. For some reason it seems we have various code conversion paths depend on which API is called :-/

{code:python}
>>> data = [decimal.Decimal('3.14'), None]
>>> pa.array(data, type=pa.decimal128(12, 4))
<pyarrow.lib.Decimal128Array object at 0x7fcb67a3fbd8>
[
  Decimal('3.1400'),
  NA
]
>>> pa.array(data, type=pa.decimal128(12, 4), from_pandas=True)
<pyarrow.lib.Decimal128Array object at 0x7fcb67a3fb38>
[
  Decimal('3.1400'),
  NA
]
>>> pa.Array.from_pandas(data, type=pa.decimal128(12, 4))
<pyarrow.lib.Decimal128Array object at 0x7fcb67a3f818>
[
  Decimal('3.1400'),
  NA
]
>>> pa.Array.from_pandas(pd.Series(data), type=pa.decimal128(12, 4))
Traceback (most recent call last):
  File "<ipython-input-10-1bf1dadbec70>", line 1, in <module>
    pa.Array.from_pandas(pd.Series(data), type=pa.decimal128(12, 4))
  File "array.pxi", line 383, in pyarrow.lib.Array.from_pandas
  File "array.pxi", line 177, in pyarrow.lib.array
  File "error.pxi", line 77, in pyarrow.lib.check_status
  File "error.pxi", line 77, in pyarrow.lib.check_status
ArrowInvalid: /home/antoine/arrow/cpp/src/arrow/python/numpy_to_arrow.cc:1702 code: converter.Convert()
Error converting from Python objects to Decimal: Got Python object of type NoneType but can only handle these types: decimal.Decimal

{code}

> [Python] from_pandas fails when converting decimals if have None values
> -----------------------------------------------------------------------
>
>                 Key: ARROW-2432
>                 URL: https://issues.apache.org/jira/browse/ARROW-2432
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.9.0
>            Reporter: Bryan Cutler
>            Assignee: Bryan Cutler
>            Priority: Major
>
> Using from_pandas to convert decimals fails if encounters a value of {{None}}. For example:
> {code:java}
> In [1]: import pyarrow as pa
> ...: import pandas as pd
> ...: from decimal import Decimal
> ...:
> In [2]: s_dec = pd.Series([Decimal('3.14'), None])
> In [3]: pa.Array.from_pandas(s_dec, type=pa.decimal128(3, 2))
> ---------------------------------------------------------------------------
> ArrowInvalid Traceback (most recent call last)
> <ipython-input-3-2da56007a0da> in <module>()
> ----> 1 pa.Array.from_pandas(s_dec, type=pa.decimal128(3, 2))
> array.pxi in pyarrow.lib.Array.from_pandas()
> array.pxi in pyarrow.lib.array()
> error.pxi in pyarrow.lib.check_status()
> error.pxi in pyarrow.lib.check_status()
> ArrowInvalid: Error converting from Python objects to Decimal: Got Python object of type NoneType but can only handle these types: decimal.Decimal
> {code}
> The above error is raised when specifying decimal type. When no type is specified, a seg fault happens.
> This previously worked in 0.8.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)