[jira] [Resolved] (ARROW-2476) [Python/Question] Maximum length of an Array created from ndarray

["Wes McKinney (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney resolved ARROW-2476.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 0.12.0

This was clarified in the format documentation

> [Python/Question] Maximum length of an Array created from ndarray
> -----------------------------------------------------------------
>
>                 Key: ARROW-2476
>                 URL: https://issues.apache.org/jira/browse/ARROW-2476
>             Project: Apache Arrow
>          Issue Type: Improvement
>            Reporter: Krisztian Szucs
>            Priority: Minor
>             Fix For: 0.12.0
>
>
> So the format [describes|https://github.com/apache/arrow/blob/master/format/Layout.md#array-lengths] that an array max length is 2^31 - 1, however the following python snippet creates a 2**32 length arrow array:
> {code:python}
> a = np.ones((2**32,), dtype='int8')
> A = pa.Array.from_pandas(a)
> type(A)
> {code}
> {code}pyarrow.lib.Int8Array{code}
> Based the layout specification I'd expect a ChunkedArray of three Int8Array's with lengths:
> [2^31 - 1, 2^31 - 1, 2] or should raise an exception?
> If it's the expectation is there any documentation for it?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)