[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10891: [WIP]: ensure min/max value generation in the segment metadata.

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on code in PR #10891:
URL: https://github.com/apache/pinot/pull/10891#discussion_r1232863110


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -570,12 +570,14 @@ public static void addColumnMetadataInfo(PropertiesConfiguration properties, Str
   public static void addColumnMinMaxValueInfo(PropertiesConfiguration properties, String column, String minValue,
       String maxValue) {
     if (isValidPropertyValue(minValue)) {
-      properties.setProperty(getKeyFor(column, MIN_VALUE), minValue);
+      properties.setProperty(getKeyFor(column, MIN_VALUE),
+          minValue.substring(0, Math.min(minValue.length(), METADATA_PROPERTY_LENGTH_LIMIT)));
     } else {
       properties.setProperty(getKeyFor(column, MIN_MAX_VALUE_INVALID), true);
     }
     if (isValidPropertyValue(maxValue)) {
-      properties.setProperty(getKeyFor(column, MAX_VALUE), maxValue);
+      properties.setProperty(getKeyFor(column, MAX_VALUE),
+          maxValue.substring(0, Math.min(maxValue.length(), METADATA_PROPERTY_LENGTH_LIMIT)));

Review Comment:
   We cannot directly use substring as the max value because it is smaller than the actual max value and can cause false positive pruning. We want to pick a string that is larger than the actual max value



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org