You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Wes McKinney (Jira)" <ji...@apache.org> on 2019/08/23 21:26:00 UTC
[jira] [Created] (PARQUET-1642) [C++] Provide for
readahead-buffering in column readers
Wes McKinney created PARQUET-1642:
-------------------------------------
Summary: [C++] Provide for readahead-buffering in column readers
Key: PARQUET-1642
URL: https://issues.apache.org/jira/browse/PARQUET-1642
Project: Parquet
Issue Type: Improvement
Components: parquet-cpp
Reporter: Wes McKinney
Currently we support two modes of reading a column chunk:
* Read entire chunk into memory before beginning to deserialize
* Use {{arrow::io::BufferedInputStream}} to buffer reads, where IO calls are triggered when a buffer is consumed
One downside to the latter case is that the IO may sit idle while deserialization is happening. It might be preferable to allow "lookahead buffering" (possibly using "ReadaheadSpooler") so that data will continue to be requested in the background
--
This message was sent by Atlassian Jira
(v8.3.2#803003)