[jira] [Created] (ARROW-264) Create an Arrow File format

["Julien Le Dem (JIRA)" <ji...@apache.org>]

Julien Le Dem created ARROW-264:
-----------------------------------

             Summary: Create an Arrow File format
                 Key: ARROW-264
                 URL: https://issues.apache.org/jira/browse/ARROW-264
             Project: Apache Arrow
          Issue Type: Improvement
            Reporter: Julien Le Dem
            Assignee: Julien Le Dem


File layout:
(DictionaryBatch, RecordBatch, Schema as defined in Message.fbs)
{noformat}
MAGIC:   ARROW1
(
DictionaryBatch:  DictionaryBatch Header (FlatBuffer)
DictionaryBatch: DictionaryBatch Body (buffers concatenated)
)*
(
RecordBacth: RecordBatch Header (FlatBuffer)
RecordBacth: RecordBatch Body (buffers concatenated)
)+
Footer: Flatbuffer
Footer length: int (4 bytes unsigned LE)
MAGIC: ARROW1
{noformat}
Footer definition:
{noformat}
table Footer {
  schema: org.apache.arrow.flatbuf.Schema;
  dictionaries: [ Block ];
  recordBatches: [ Block ];
}
struct Block {
  offset: long;
  metaDataLength: int;
  bodyLength: long;
}
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)