[GitHub] [parquet-format] mapleFU commented on a diff in pull request #197: PARQUET-2261: Proposal for unencoded/uncompressed statistics

["mapleFU (via GitHub)" <gi...@apache.org>]

mapleFU commented on code in PR #197:
URL: https://github.com/apache/parquet-format/pull/197#discussion_r1169441253


##########
src/main/thrift/parquet.thrift:
##########
@@ -190,6 +190,60 @@ enum FieldRepetitionType {
   /** The field is repeated and can contain 0 or more values */
   REPEATED = 2;
 }
+ /**
+   * Tracks a histogram of repetition and definition levels for either a page or column chunk. 
+   *
+   * This is useful for:
+   *   1. Estimating the size of the data when materialized in memory 
+   *   2. For filter push-down on nulls at various levels of nested structures and 
+   *      list lengths.
+   */ 
+ struct RepetitionDefinitionLevelHistogram {
+   /** 
+     * When present there is expected to be one element corresponding to each repetition (i.e. size=max repetition_level+1) 
+     * where each element represents the number of time the repetition level was observed in the data.
+     *
+     * This value should not be written if max_repetition_level is 0.
+     **/
+   1: optional list<i64> repetition_level_histogram;
+   /**
+    * Same as repetition_level_histogram except for definition levels.
+    *
+    * This value should not be written when max_definition_level is 0. 
+    **/ 
+   2: optional list<i64> definition_level_histogram;
+ }
+/**

Review Comment:
   Should we add a white space between `RepetitionDefinitionLevelHistogram` and comment here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@parquet.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org