[jira] [Created] (ARROW-3949) parquet cpp - improve examples

["Rajeshwar Agrawal (JIRA)" <ji...@apache.org>]

Rajeshwar Agrawal created ARROW-3949:
----------------------------------------

             Summary: parquet cpp - improve examples
                 Key: ARROW-3949
                 URL: https://issues.apache.org/jira/browse/ARROW-3949
             Project: Apache Arrow
          Issue Type: Improvement
          Components: C++
            Reporter: Rajeshwar Agrawal


It would be a great help to have examples of using parquet arrow high-level API for the following 2 cases
 * Storing nested data types (storing nested data types is touted as major merit of parquet, so I think this case should be included as an example). Ideally, an example of how to use StructArray nested with several primities types, list types and other struct type would cover every case of nested hierarchy of complex data representations
 * Buffered or Batched writes to parquet file. Parquet is meant to be used for large amounts of data. The current examples store all of the data as in arrow data structures, before writing to parquet file. Would be great to include an example of batched writes, which is helpful in most use cases of parquet. The current example creates a {{arrow::Table}}, which needs to be filled with {{arrow::Array}}(s) of entire data. Ideally, an example which generates some data in several {{arrow::Array}}(s), and then stores (appends) them as a new Row Group (or Column Chunk) in an existing (new) parquet file (writer), using {{NewRowGroup}} and {{WriteColumnChunk}} functions, thus demonstrating a lower memory footprint for writing a parquet file



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)