You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by "Haralampos Gavriilidis (JIRA)" <ji...@apache.org> on 2018/04/13 15:25:00 UTC
[jira] [Created] (ARROW-2457) garrow_array_builder_append_values()
won't work for large arrays
Haralampos Gavriilidis created ARROW-2457:
---------------------------------------------
Summary: garrow_array_builder_append_values() won't work for large arrays
Key: ARROW-2457
URL: https://issues.apache.org/jira/browse/ARROW-2457
Project: Apache Arrow
Issue Type: Bug
Components: C, C++, GLib
Affects Versions: 0.9.0, 0.8.0
Reporter: Haralampos Gavriilidis
I am using garrow_array_builder_append_values() to transform a native C array to an Arrow array, without calling arrow_array_builder_append multiple times. When calling garrow_array_builder_append_values() in array-builder.cpp with following signature:
{code:java}
garrow_array_builder_append_values(GArrowArrayBuilder *builder,
const VALUE *values,
gint64 values_length,
const gboolean *is_valids,
gint64 is_valids_length,
GError **error,
const gchar *context)
{code}
it will fail for large arrays. This is probably happening because the is_valids array is copied to the valid_bytes array (of different type), for which the memory is allocated on the stack, and not on the heap, like shown on the snippet below:
{code:java}
uint8_t valid_bytes[is_valids_length];
for (gint64 i = 0; i < is_valids_length; ++i){
valid_bytes[i] = is_valids[i];
}
{code}
A way to avoid this problem would be to allocate memory for the valid_bytes array using malloc() or something similar. Is this behavior intended, maybe because no large arrays should be handed over to that function, or it is rather a bug?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)