[jira] [Commented] (ARROW-2457) garrow_array_builder_append_values() won't work for large arrays

["Rok Mihevc (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17659484#comment-17659484 ] 

Rok Mihevc commented on ARROW-2457:
-----------------------------------

This issue has been migrated to [issue #18613|https://github.com/apache/arrow/issues/18613] on GitHub. Please see the [migration documentation|https://github.com/apache/arrow/issues/14542] for further details.

> garrow_array_builder_append_values() won't work for large arrays
> ----------------------------------------------------------------
>
>                 Key: ARROW-2457
>                 URL: https://issues.apache.org/jira/browse/ARROW-2457
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, GLib
>    Affects Versions: 0.8.0, 0.9.0
>            Reporter: Haralampos Gavriilidis
>            Assignee: Kouhei Sutou
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 0.10.0
>
>
> I am using garrow_array_builder_append_values() to transform a native C array to an Arrow array, without calling arrow_array_builder_append multiple times. When calling garrow_array_builder_append_values() in array-builder.cpp with following signature:
> {code:java}
> garrow_array_builder_append_values(GArrowArrayBuilder *builder,
> const VALUE *values,
> gint64 values_length,
> const gboolean *is_valids,
> gint64 is_valids_length,
> GError **error,
> const gchar *context)
> {code}
> it will fail for large arrays. This is probably happening because the is_valids array is copied to the valid_bytes array (of different type), for which the memory is allocated on the stack, and not on the heap, like shown on the snippet below:
> {code:java}
> uint8_t valid_bytes[is_valids_length];
> for (gint64 i = 0; i < is_valids_length; ++i){ 
>   valid_bytes[i] = is_valids[i]; 
> }
> {code}
>  A way to avoid this problem would be to allocate memory for the valid_bytes array using malloc() or something similar. Is this behavior intended, maybe because no large arrays should be handed over to that function, or it is rather a bug?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)