You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Wes McKinney (Jira)" <ji...@apache.org> on 2019/08/23 21:26:00 UTC

[jira] [Created] (PARQUET-1642) [C++] Provide for readahead-buffering in column readers

Wes McKinney created PARQUET-1642:
-------------------------------------

             Summary: [C++] Provide for readahead-buffering in column readers
                 Key: PARQUET-1642
                 URL: https://issues.apache.org/jira/browse/PARQUET-1642
             Project: Parquet
          Issue Type: Improvement
          Components: parquet-cpp
            Reporter: Wes McKinney


Currently we support two modes of reading a column chunk:

* Read entire chunk into memory before beginning to deserialize
* Use {{arrow::io::BufferedInputStream}} to buffer reads, where IO calls are triggered when a buffer is consumed

One downside to the latter case is that the IO may sit idle while deserialization is happening. It might be preferable to allow "lookahead buffering" (possibly using "ReadaheadSpooler") so that data will continue to be requested in the background



--
This message was sent by Atlassian Jira
(v8.3.2#803003)