[jira] [Commented] (ARROW-6325) [Python] wrong conversion of DataFrame with boolean values

["Joris Van den Bossche (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-6325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913572#comment-16913572 ] 

Joris Van den Bossche commented on ARROW-6325:
----------------------------------------------

So when converting an array of the column gives also wrong values, but when taking a copy of the values, not:

{code}
In [39]: pa.array(np.asarray(df['a']))                                                                                                                                                                             
Out[39]: 
<pyarrow.lib.BooleanArray object at 0x7fd08a8a2c50>
[
  true,
  false,
  false
]

In [40]: pa.array(np.array(df['a']))                                                                                                                                                                               
Out[40]: 
<pyarrow.lib.BooleanArray object at 0x7fd08a88c990>
[
  true,
  true,
  true
]
{code}

So it probably has to do with in the first case the values being a view on a 2D array ?



> [Python] wrong conversion of DataFrame with boolean values
> ----------------------------------------------------------
>
>                 Key: ARROW-6325
>                 URL: https://issues.apache.org/jira/browse/ARROW-6325
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.14.1
>            Reporter: Joris Van den Bossche
>            Priority: Major
>             Fix For: 0.15.0
>
>
> From https://github.com/pandas-dev/pandas/issues/28090
> {code}
> In [19]: df = pd.DataFrame(np.ones((3, 2), dtype=bool), columns=['a', 'b']) 
> In [20]: df  
> Out[20]: 
>       a     b
> 0  True  True
> 1  True  True
> 2  True  True
> In [21]: table = pa.table(df) 
> In [23]: table.column(0)
> Out[23]: 
> <pyarrow.lib.ChunkedArray object at 0x7fd08a96e090>
> [
>   [
>     true,
>     false,
>     false,
>   ]
> ]
> {code}
> The resulting table has False values while the original DataFrame had only true values. 
> It seems this has to do with the fact that it are multiple columns, as with a single column it converts correctly.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)