You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Vincent Trumpff (Jira)" <ji...@apache.org> on 2021/04/30 13:29:00 UTC

[jira] [Comment Edited] (ARROW-12223) ArrayData buffers are inconsistent accross implementations

    [ https://issues.apache.org/jira/browse/ARROW-12223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17337386#comment-17337386 ] 

Vincent Trumpff edited comment on ARROW-12223 at 4/30/21, 1:28 PM:
-------------------------------------------------------------------

Reopening as we still have difference between C++ and JavaScript unless I missed something?

 

Edit:

If the community do agree with the point, I would be happy to propose a PR on C++ and JavaScript sources.


was (Author: vincent-tr):
Reopening as we still have difference between C++ and JavaScript unless I missed something?

> ArrayData buffers are inconsistent accross implementations
> ----------------------------------------------------------
>
>                 Key: ARROW-12223
>                 URL: https://issues.apache.org/jira/browse/ARROW-12223
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: C++, JavaScript, Rust
>            Reporter: Vincent Trumpff
>            Priority: Major
>
> ArrayData implementations seems to share close structure fields accross languages, but their usage is not consistent accross implementation.
>  
> Example using ListArray's offsets buffer, in C++, Rust and JavaScript implementation:
>  - C++: offset's buffer is the second buffer (validity bitmap is first buffer, and buffers are laid in a type-dependant way) https://github.com/apache/arrow/blob/master/cpp/src/arrow/array/array_nested.cc#L189
>  - Rust: offset's buffer is the first buffer (validity bitmap is not part of the collection, and buffers are laid in a type-dependant way) https://github.com/apache/arrow/blob/master/rust/arrow/src/array/array_list.rs#L235
>  - JavaScript: offset's buffer is the first buffer (they have fixed position) [https://github.com/apache/arrow/blob/8e43f23dcc6a9e630516228f110c48b64d13cec6/js/src/data.ts#L125]
>  
> Note that we have the same inconsistency for validity and data buffers.
>  
> This is important in my project because I would like to transport buffers list accross technologies, and ArrayData seemed the easiest structure to transport.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)