Key-Value metadata at RowGroup level in C++ parquet library

[rahul challapalli <ch...@gmail.com>]

Team,

I am looking for ways to store more metadata (key-value style) at the row
group level using the parquet-cpp library. However the RowGroupMetaData
class defined in [1] doesn't provide anything relevant for what I am
looking. Does the existing library support something like this?

Also I observed that the below code returns file level KeyValueMetadata
while I was expecting KeyValueMetadata at the row-group level. Kindly
correct me if I am doing something wrong.

 arrow::FileReader->ReadRowGroup(index, rg);
(*rg)->schema()->metadata()

[1]
https://github.com/apache/parquet-cpp/blob/master/src/parquet/file/metadata.h

- Rahul

Re: Key-Value metadata at RowGroup level in C++ parquet library

[Wes McKinney <we...@gmail.com>]

hi Rahul,

the key value metadata is only supported at the file/schema level and
at the column chunk (i.e. each column in a row group) level:

https://github.com/apache/parquet-cpp/blob/master/src/parquet/parquet.thrift#L530

We should add an accessor for the column chunk key-value metadata to
parquet::ColumnChunkMetaData:

https://github.com/apache/parquet-cpp/blob/master/src/parquet/file/metadata.h#L89

I opened https://issues.apache.org/jira/browse/PARQUET-1107 -- are you
up to submit a patch?

Thanks
Wes

On Wed, Sep 20, 2017 at 12:59 AM, rahul challapalli
<ch...@gmail.com> wrote:
> Team,
>
> I am looking for ways to store more metadata (key-value style) at the row
> group level using the parquet-cpp library. However the RowGroupMetaData
> class defined in [1] doesn't provide anything relevant for what I am
> looking. Does the existing library support something like this?
>
> Also I observed that the below code returns file level KeyValueMetadata
> while I was expecting KeyValueMetadata at the row-group level. Kindly
> correct me if I am doing something wrong.
>
>  arrow::FileReader->ReadRowGroup(index, rg);
> (*rg)->schema()->metadata()
>
> [1]
> https://github.com/apache/parquet-cpp/blob/master/src/parquet/file/metadata.h
>
> - Rahul