[jira] [Updated] (ARROW-3164) [Java] Port Row Set abstraction from Drill to Arrow

["Wes McKinney (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ARROW-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wes McKinney updated ARROW-3164:
--------------------------------
    Summary: [Java] Port Row Set abstraction from Drill to Arrow  (was: Port Row Set abstraction from Drill to Arrow)

> [Java] Port Row Set abstraction from Drill to Arrow
> ---------------------------------------------------
>
>                 Key: ARROW-3164
>                 URL: https://issues.apache.org/jira/browse/ARROW-3164
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Java
>            Reporter: Paul Rogers
>            Priority: Major
>
> Arrow is a great way to exchange data between systems. Somewhere in the process, however, data must be load into, and read out of the Arrow vectors.
> Arrow evolved from Apache Drill. The Drill project created a "Row Set" abstraction that:
> * Provides a simple way to define the schema for a set of batches.
> * Loads data into vectors from row-oriented inputs.
> * Reads data out of vectors in row-oriented output.
> * Controls memory consumed by the record batch when loading data into vectors.
> * Ensures maximum usage of the allocated vector space when loading data Into vectors.
> * Optionally handles projection when reading data from an input file into a set of vectors.
> * Optionally handles data conversion from input to vector formats.
> This mechanism is handy for any Java developer who produces or consumes Arrow vectors.
> Detailed information is available in [this wiki|https://github.com/paul-rogers/arrow/wiki], including a more detailed description of the motivation for this project, and an analysis of the work required to do the Drill-to-Arrow port.
> The code is in Java simply because Drill is written in Java. The same mechanisms can be ported to other languages if useful. Those ports would be separate future projects.
> The code will be placed in a new Java module which can be imported by projects that wish to use the code. Changes may be needed to expose items from the {{vector}} module; we'll tackle those issues if/when they occur.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)