You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Antoine Pitrou (JIRA)" <ji...@apache.org> on 2019/06/03 12:36:00 UTC
[jira] [Updated] (ARROW-2476) [Python/Question] Maximum length of
an Array created from ndarray
[ https://issues.apache.org/jira/browse/ARROW-2476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Antoine Pitrou updated ARROW-2476:
----------------------------------
Component/s: Python
> [Python/Question] Maximum length of an Array created from ndarray
> -----------------------------------------------------------------
>
> Key: ARROW-2476
> URL: https://issues.apache.org/jira/browse/ARROW-2476
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Krisztian Szucs
> Priority: Minor
> Fix For: 0.12.0
>
>
> So the format [describes|https://github.com/apache/arrow/blob/master/format/Layout.md#array-lengths] that an array max length is 2^31 - 1, however the following python snippet creates a 2**32 length arrow array:
> {code:python}
> a = np.ones((2**32,), dtype='int8')
> A = pa.Array.from_pandas(a)
> type(A)
> {code}
> {code}pyarrow.lib.Int8Array{code}
> Based the layout specification I'd expect a ChunkedArray of three Int8Array's with lengths:
> [2^31 - 1, 2^31 - 1, 2] or should raise an exception?
> If it's the expectation is there any documentation for it?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)