You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by "Spiro Michaylov (JIRA)" <ji...@apache.org> on 2016/02/14 23:10:18 UTC
[jira] [Created] (PARQUET-531) Can't read past first page in a
column
Spiro Michaylov created PARQUET-531:
---------------------------------------
Summary: Can't read past first page in a column
Key: PARQUET-531
URL: https://issues.apache.org/jira/browse/PARQUET-531
Project: Parquet
Issue Type: Bug
Components: parquet-cpp
Environment: Ubuntu Linux 14.04 (no obvious platform dependence), Parquet file created by Apache Spark 1.5.0 on the same platform.
Reporter: Spiro Michaylov
Building the code as of 2/14/2015 and adding the obvious three lines of code to serialized-page.cc to enable the newly added CompressionCodec::GZIP:
{code}
case parquet::CompressionCodec::GZIP:
decompressor_.reset(new GZipCodec());
break;
{code}
I try to run the parquet_reader example on the column I'm about to attach, which was created by Apache Spark 1.5.0. It works surprisingly well until it hits the end of the first page, where it dies with
{quote}
Parquet error: Value was non-null, but has not been buffered
{quote}
I realize you may be reluctant to look at this because (a) the GZip support is new and (b) I had to modify the code to enable it, but actually things seem to decompress just fine (congratulations: this is awesome!): looking at the problem in the debugger and tracing through a bit it seems to me like the buffering is a bit screwed up in general -- some kind of confusion between the buffering at the Scanner and Reader levels. I can reproduce the problem by reading through just a single column too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)