[jira] [Closed] (ARROW-5916) [C++] Allow RecordBatch.length to be less than array lengths

["Wes McKinney (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney closed ARROW-5916.
-------------------------------
    Resolution: Later

We didn't reach a conclusion on this so closing for now

> [C++] Allow RecordBatch.length to be less than array lengths
> ------------------------------------------------------------
>
>                 Key: ARROW-5916
>                 URL: https://issues.apache.org/jira/browse/ARROW-5916
>             Project: Apache Arrow
>          Issue Type: New Feature
>          Components: C++
>            Reporter: John Muehlhausen
>            Priority: Minor
>              Labels: pull-request-available
>         Attachments: test.arrow_ipc
>
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> 0.13 ignored RecordBatch.length.  0.14 requires that RecordBatch.length and array length be equal.  As per [https://lists.apache.org/thread.html/2692dd8fe09c92aa313bded2f4c2d4240b9ef75a8604ec214eb02571@%3Cdev.arrow.apache.org%3E] , we discussed changing this so that RecordBatch.length can be [0,array length].
>  If RecordBatch.length is less than array length, the reader should ignore the portion of the array(s) beyond RecordBatch.length.  This will allow partially populated batches to be read in scenarios identified in the above discussion.
> {code:c++}
>   Status GetFieldMetadata(int field_index, ArrayData* out) {
>     auto nodes = metadata_->nodes();
>     // pop off a field
>     if (field_index >= static_cast<int>(nodes->size())) {
>       return Status::Invalid("Ran out of field metadata, likely malformed");
>     }
>     const flatbuf::FieldNode* node = nodes->Get(field_index);
> *    //out->length = node->length();*
> *    out->length = metadata_->length();*
>     out->null_count = node->null_count();
>     out->offset = 0;
>     return Status::OK();
>   }
> {code}
> Attached is a test IPC File containing a batch with length 1, array length 3.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)