You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by "Wes McKinney (JIRA)" <ji...@apache.org> on 2016/12/05 23:43:58 UTC
[jira] [Commented] (ARROW-400) [Java] ArrowWriter writes length 0
for Struct types
[ https://issues.apache.org/jira/browse/ARROW-400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15723757#comment-15723757 ]
Wes McKinney commented on ARROW-400:
------------------------------------
Here's what the integration test showed after that change:
{code}
Error message: Invalid: Record batch 0 did not match
JSON:
list_nullable:
-- is_valid: [true, false, true, false, true, true, false]
-- offsets: [0, 3, 3, 5, 5, 7, 10, 10]
-- values: [520810962, null, null, null, -384659144, null, null, -432193908, null, 941790011]
struct_nullable:
-- is_valid: [false, true, false, true, true, false, false]
-- child 0 type: int32 values: [244698405, -1126222074, 1691075042, 153394561, 1656915567, 838293194, null]
-- child 1 type: string values: [null, "RKx30xD", "x6PTAOM", null, null, "UFVSfrh", "h8LGlkE"]
Arrow:
list_nullable:
-- is_valid: [true, false, true, false, true, true, false]
-- offsets: [0, 3, 3, 5, 5, 7, 10, 10]
-- values: [520810962, null, null, null, -384659144, null, null, -432193908, null, 941790011]
struct_nullable:
-- is_valid: [false, true, false, true, true, false, false]
-- child 0 type: int32 values: [244698405, -1126222074, 1691075042, 153394561, 1656915567, 838293194, null]
-- child 1 type: string values: [null, "", "", null, null, "", ""]
{code}
I looked at the buffers for the 2nd child, and all the offsets are 0 and the data buffer is length 0, odd:
{code}
(gdb) p metadata
$1 = {page = 0, offset = 176, length = 0}
{code}
> [Java] ArrowWriter writes length 0 for Struct types
> ---------------------------------------------------
>
> Key: ARROW-400
> URL: https://issues.apache.org/jira/browse/ARROW-400
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java - Vectors
> Reporter: Wes McKinney
> Priority: Blocker
> Attachments: struct_error.json
>
>
> Observed in integration testing. In a length-7 struct, the field metadata for the struct node has length 0 in the flatbuffer. From C++ gdb:
> {code}
> (gdb) p field_meta
> $1 = {length = 0, null_count = 0}
> {code}
> See attached example json file
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)