Parquet sync meeting notes Jan 26, 2021

[Xinli shang <sh...@uber.com.INVALID>]

Hi all,

Attendees: Xinli Shang, Gábor Szádovszky, Gidon Gershinsky, Jason
Alterkruse


Topics:

   1.

   Parquet column encryption
   1.

      The ported code in Presto will be open-sourced soon.
      2.

      Crypto factory could be changed when new use cases come in(Gidon).
      2.

   Parquet-mr 1.12.0 release
   1.

      The higher version of Avro only has unit test diff. We should go
      ahead to release.
      2.

      Check with Spark community for the Spark Avro issue - Xinli
      3.

      There are still a lot of open PRs. We want owners to indicate should
      those PRs be included in 1.12.0.
      4.

      Will issue an RC0 release soon - Gabor
      3.

   Core features definition
   1.

      Send emails to all PMCs - Gabor
      4.

   Deprecation of summary files
   1.

      There is a discussion in the Iceberg community for summary files.
      Those files could be still useful in some cases (Jason).
      2.

      At least in Parquet 1.12.0, we will leave it as is while the
      discussion is still going on. Maybe we can add a switch to turn
it on/off.


Please let me know if you have any questions.

Xinli Shang | Tech Lead Manager @ Uber Data Infra