[jira] [Updated] (ARROW-264) Create an Arrow File format

["Antoine Pitrou (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Antoine Pitrou updated ARROW-264:
---------------------------------
    Component/s: Format

> Create an Arrow File format
> ---------------------------
>
>                 Key: ARROW-264
>                 URL: https://issues.apache.org/jira/browse/ARROW-264
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Format
>            Reporter: Julien Le Dem
>            Assignee: Julien Le Dem
>            Priority: Major
>             Fix For: 0.1.0
>
>
> File layout:
> (DictionaryBatch, RecordBatch, Schema as defined in Message.fbs)
> {noformat}
> MAGIC:   ARROW1
> (
> DictionaryBatch:  DictionaryBatch Header (FlatBuffer)
> DictionaryBatch: DictionaryBatch Body (buffers concatenated)
> )*
> (
> RecordBacth: RecordBatch Header (FlatBuffer)
> RecordBacth: RecordBatch Body (buffers concatenated)
> )+
> Footer: Flatbuffer
> Footer length: int (4 bytes unsigned LE)
> MAGIC: ARROW1
> {noformat}
> Footer definition:
> {noformat}
> table Footer {
>   schema: org.apache.arrow.flatbuf.Schema;
>   dictionaries: [ Block ];
>   recordBatches: [ Block ];
> }
> struct Block {
>   offset: long;
>   metaDataLength: int;
>   bodyLength: long;
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)