[GitHub] [incubator-pinot] siddharthteotia commented on pull request #5709: Store column min/max value into segment metadata

[GitBox <gi...@apache.org>]

siddharthteotia commented on pull request #5709:
URL: https://github.com/apache/incubator-pinot/pull/5709#issuecomment-659762915


   So to be on the same page, we are doing the following:
   
   1. Until now, the segment load code generated ColumnMinMaxValue **only for dictionary encoded columns.** This was then persisted in PropertiesConfiguration and made available to ColumnMetatadata and used for pruning. The default min-max value generator mode was for TIME column until now and this PR changes it to ALL. May be we should have a mode like DIMENSIONS ?
   
   2. This PR adds the support for generating the min-max value for **both raw and dictionary encoded columns** during segment generation and persisting in SegmentMetadata. The generator mode used here is ALL?
   
   3. Now that going forward we will have min-max available in SegmentMetadata, we detect that in segment load code and return. 
   
   @Jackie-Jiang , would be good to add/edit anything I may have missed. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org