[GitHub] [incubator-iceberg] jun-ma-0 opened a new issue #978: lower_bounds and upper_bounds are not collected for columns in map type

[GitBox <gi...@apache.org>]

jun-ma-0 opened a new issue #978:
URL: https://github.com/apache/incubator-iceberg/issues/978


   In DataFile, stats are collected for each column to help with file level filtering. Seeing lower_bounds and upper_bounds are not collected for key and value column in map type. Here is an example.
   For schema with one id column and one map type column:
   `"schema" : {
       "type" : "struct",
       "fields" : [ {
         "id" : 1,
         "name" : "id",
         "required" : true,
         "type" : "long"
       }, {
         "id" : 2,
         "name" : "data",
         "required" : false,
         "type" : {
           "type" : "map",
           "key-id" : 3,
           "key" : "string",
           "value-id" : 4,
           "value" : "string",
           "value-required" : false
         }
       } ]
     }`
   This is one data file in the table. lower_bound & upper_bound is only collected for the id column:
   `{
   	"status":1,"snapshot_id":{"long":2910119422438955588},"data_file":{
   	"file_path":"/var/folders/h7/9k2dv061505_9hwcmfx7l7nh0000gn/T/junit6407630582688821701/parquet/test/data/00000-0-9b3ef17d-ac0d-4094-8775-0ceaa40c013b-00000.parquet",
   	"file_format":"PARQUET",
   	"partition":{},
   	"record_count":50,
   	"file_size_in_bytes":22507,
   	"block_size_in_bytes":67108864,
   	"file_ordinal":null,
   	"sort_columns":null,
   	"column_sizes":{"array":[{"key":1,"value":476},{"key":3,"value":10618},{"key":4,"value":10574}]},
   	"value_counts":{"array":[{"key":1,"value":50},{"key":3,"value":510},{"key":4,"value":510}]},
   	"null_value_counts":{"array":[{"key":1,"value":0},{"key":3,"value":2},{"key":4,"value":23}]},
   	"lower_bounds":{"array":[{"key":1,"value":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000€"}]},
   	"upper_bounds":{"array":[{"key":1,"value":"ÿÿÿÿÿÿÿ"}]},
   	"key_metadata":null,"split_offsets":{"array":[4]}}
   }`


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org