You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Xinli shang <sh...@uber.com.INVALID> on 2022/03/23 16:55:43 UTC
Parquet sync meeting notes 3/23/2022
Attendee (Jorge from Munin Data), Gidon, Huaxin, Vinoo, Xinli
1.
Cell level encryption
1.
Formal design
<https://docs.google.com/document/d/1PUonl9i_fVlRhUmqEmWBQJ8zesX7mlvnu3ubemT11rk/edit>
is sent out
2.
We choose the 2nd option - splitting columns because it doesn’t need
specification change
3.
Implementation is going on
4.
Create a feature branch for review
2.
Column resolution by ID (pr
<https://github.com/apache/parquet-mr/pull/950>)
1.
The ‘field_id’ in the schema is used.
2.
Uniqueness might not guarantee to be used by column resolution with
ID. We might need a place to remember a flag that this Parquet file is
column id resolvable
3.
Concat tool might be a problem
4.
Need some help from Iceberg
3.
Parquet writer for Iceberg (Adding a new constructor)
1.
A diff will be sent out soon
4.
New website (link <https://parquet.staged.apache.org/>)
1.
Looks good, will make it formal
--
Xinli Shang
VP Apache Parquet PMC Chair, Tech Lead Manager at Uber Data Infra