You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Ivan Sadikov <iv...@gmail.com> on 2018/02/11 23:40:38 UTC
Help with Parquet record materialiser
Hello,
Are there any docs or blogs on how parquet record materialiser works? It
would be great, if someone could explain me what the input is and how
record materialiser traverses scheme and reconstructs a record.
I understand how API is structured and how to use it, I am more after help
with the content of RecordReaderImplementation.java or an equivalent in
parquet-cpp. Would appreciate an explanation of init and read methods.
Thank you in advance!
Cheers,
Ivan
Re: Help with Parquet record materialiser
Posted by Ivan Sadikov <iv...@gmail.com>.
Thank you very much, Ryan!
Will check it out right away!
Cheers,
Ivan
On Tue, 13 Feb 2018 at 7:25 AM, Ryan Blue <rb...@netflix.com.invalid> wrote:
> Ivan,
>
> I was working on a more straightforward implementation recently that you
> might find helpful. Here's the basic code for building types of records for
> (repetition, definition, value) triples:
>
>
> https://github.com/Netflix/iceberg/blob/parquet-value-readers/parquet/src/main/java/com/netflix/iceberg/parquet/ParquetValueReaders.java#L142
>
> rb
>
> On Sun, Feb 11, 2018 at 3:40 PM, Ivan Sadikov <iv...@gmail.com>
> wrote:
>
> > Hello,
> >
> > Are there any docs or blogs on how parquet record materialiser works? It
> > would be great, if someone could explain me what the input is and how
> > record materialiser traverses scheme and reconstructs a record.
> >
> > I understand how API is structured and how to use it, I am more after
> help
> > with the content of RecordReaderImplementation.java or an equivalent in
> > parquet-cpp. Would appreciate an explanation of init and read methods.
> >
> > Thank you in advance!
> >
> >
> > Cheers,
> >
> > Ivan
> >
>
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>
Re: Help with Parquet record materialiser
Posted by Ryan Blue <rb...@netflix.com.INVALID>.
Ivan,
I was working on a more straightforward implementation recently that you
might find helpful. Here's the basic code for building types of records for
(repetition, definition, value) triples:
https://github.com/Netflix/iceberg/blob/parquet-value-readers/parquet/src/main/java/com/netflix/iceberg/parquet/ParquetValueReaders.java#L142
rb
On Sun, Feb 11, 2018 at 3:40 PM, Ivan Sadikov <iv...@gmail.com>
wrote:
> Hello,
>
> Are there any docs or blogs on how parquet record materialiser works? It
> would be great, if someone could explain me what the input is and how
> record materialiser traverses scheme and reconstructs a record.
>
> I understand how API is structured and how to use it, I am more after help
> with the content of RecordReaderImplementation.java or an equivalent in
> parquet-cpp. Would appreciate an explanation of init and read methods.
>
> Thank you in advance!
>
>
> Cheers,
>
> Ivan
>
--
Ryan Blue
Software Engineer
Netflix