You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "somandal (via GitHub)" <gi...@apache.org> on 2025/10/16 19:18:08 UTC

[PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

somandal opened a new pull request, #17028:
URL: https://github.com/apache/pinot/pull/17028

   This PR adds the following support:
   
   - Support to skip adding star-tree indexes if adding them fails due to invalid config. The segment build can still go through
   - Support to handle rolling back to the existing star-tree index if update fails
   
   This PR won't skip star-tree index creation in all cases, as we need to correctly handle rollback to remove extra files created, or rollback to the previous state. If these fail, this can leave the segment in an inconsistent state
   
   Testing:
   Added unit tests, did some testing with `HybridQuickstart` as well (and checked the segment directories to ensure that the correct changes take effect)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2445608719


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   I think the `build()` method is currently only used by the minion path / external segment creation whereas `buildByColumn()` is currently only used by the realtime segment converter. The star-tree index creation is done within the `handlePostCreation()` method which is called in both the paths so I think we do need to avoid using minion metrics here. IMO for now, we could simply pass in an `InstanceType` enum indicating whether it's being run on a server or a minion and hard code it to minion for `build()` and server for `buildByColumn()`. WDYT?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2441439707


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2461722364


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2441393659


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -175,28 +185,44 @@ public void build()
     LOGGER.info("Starting building {} star-trees with configs: {} using {} builder", numStarTrees, _builderConfigs,
         _buildMode);
 
-    try (StarTreeIndexCombiner indexCombiner = new StarTreeIndexCombiner(
-        new File(_segmentDirectory, StarTreeV2Constants.INDEX_FILE_NAME))) {
+    File starTreeV2IndexFile = new File(_segmentDirectory, StarTreeV2Constants.INDEX_FILE_NAME);
+    try (StarTreeIndexCombiner indexCombiner = new StarTreeIndexCombiner(starTreeV2IndexFile)) {
       File starTreeIndexDir = new File(_segmentDirectory, StarTreeV2Constants.STAR_TREE_TEMP_DIR);
       FileUtils.forceMkdir(starTreeIndexDir);
       _metadataProperties.addProperty(MetadataKey.STAR_TREE_COUNT, numStarTrees);
       List<List<Pair<IndexKey, IndexValue>>> indexMaps = new ArrayList<>(numStarTrees);
 
       // Build all star-trees
-      for (int i = 0; i < numStarTrees; i++) {
-        StarTreeV2BuilderConfig builderConfig = _builderConfigs.get(i);
-        Configuration metadataProperties = _metadataProperties.subset(MetadataKey.getStarTreePrefix(i));
-        if (_separator != null && handleExistingStarTreeAddition(starTreeIndexDir, metadataProperties, builderConfig)) {
-          // Used existing tree
-          LOGGER.info("Reused existing star-tree: {}", builderConfig.toString());
-          reusedStarTrees++;
-        } else {
-          try (SingleTreeBuilder singleTreeBuilder = getSingleTreeBuilder(builderConfig, starTreeIndexDir, _segment,
-              metadataProperties, _buildMode)) {
-            singleTreeBuilder.build();
+      try {
+        for (int i = 0; i < numStarTrees; i++) {
+          StarTreeV2BuilderConfig builderConfig = _builderConfigs.get(i);
+          Configuration metadataProperties = _metadataProperties.subset(MetadataKey.getStarTreePrefix(i));
+          if (_separator != null && handleExistingStarTreeAddition(starTreeIndexDir, metadataProperties,
+              builderConfig)) {
+            // Used existing tree
+            LOGGER.info("Reused existing star-tree: {}", builderConfig.toString());
+            reusedStarTrees++;
+          } else {
+            try (SingleTreeBuilder singleTreeBuilder = getSingleTreeBuilder(builderConfig, starTreeIndexDir, _segment,
+                metadataProperties, _buildMode)) {
+              singleTreeBuilder.build();
+            }
           }
+          indexMaps.add(indexCombiner.combine(builderConfig, starTreeIndexDir));
+        }
+      } catch (Exception e) {
+        // Clean-up some of the files created before throwing exception back to caller
+        // No need to undo changes to _metadataProperties as changes weren't saved to the file
+        LOGGER.error("Failed to build star-trees, cleaning up the index file and temp directory if they exist");
+        _starTreeCreationFailed = true;
+        if (starTreeV2IndexFile.exists()) {
+          FileUtils.forceDelete(starTreeV2IndexFile);
+        }
+        if (starTreeIndexDir.exists()) {
+          FileUtils.forceDelete(starTreeIndexDir);
         }
-        indexMaps.add(indexCombiner.combine(builderConfig, starTreeIndexDir));
+        _metadataProperties.clearProperty(MetadataKey.STAR_TREE_COUNT);
+        throw e;
       }

Review Comment:
   we'd fail segment build by throwing an exception prior to this. so even though the segment is inconsistent, it wouldn't technically be built or available to query



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2445856594


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   i actually do see the `build()` method called by the realtime segment conversion path (`RealtimeSegmentConverter`):
   
   ```
     /**
      * Common method to build segment with the provided record reader
      */
     private void buildSegmentWithReader(SegmentIndexCreationDriverImpl driver, SegmentGeneratorConfig genConfig,
         RecordReader recordReader, int[] sortedDocIds, boolean useCompactedReader,
         @Nullable ThreadSafeMutableRoaringBitmap validDocIdsSnapshot)
         throws Exception {
       RealtimeSegmentSegmentCreationDataSource dataSource;
       if (useCompactedReader) {
         // For compacted readers, use the constructor that takes sortedDocIds and pass the validDocIds snapshot
         dataSource = new RealtimeSegmentSegmentCreationDataSource(_realtimeSegmentImpl, recordReader, sortedDocIds,
             validDocIdsSnapshot);
       } else {
         // For regular readers, use the original constructor
         dataSource =
             new RealtimeSegmentSegmentCreationDataSource(_realtimeSegmentImpl, (PinotSegmentRecordReader) recordReader);
       }
       driver.init(genConfig, dataSource, TransformPipeline.getPassThroughPipeline(_tableName)); // initializes reader
   
       if (!_enableColumnMajor) {
         driver.build();
       } else {
         //buildByColumn uses validDocIds to skip invalid record while indexing each column. We pass the validDocIds
         // only if we are using compacted reader.
         driver.buildByColumn(_realtimeSegmentImpl, useCompactedReader ? validDocIdsSnapshot : null);
       }
     }
   ```
   
   so I do think we need to pass some context on whether this is called from minion code path or server 😅 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2445863423


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   Oops good catch, I missed that one.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2462181715


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {
+          if (_starTreeCreationFailed) {
+            LOGGER.error("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+                _segmentDirectory,
+                false);
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+                _segmentDirectory, false);
+
+            // Copy back the older star-tree related metadata
+            Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+            while (keys.hasNext()) {
+              String key = keys.next();
+              Object value = _existingStarTreeMetadata.getProperty(key);
+              _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+            }
+            CommonsConfigurationUtils.saveToFile(_metadataProperties,
+                new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+          }
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);

Review Comment:
   Hm there's only two call-sites really for `MultipleTreesBuilder`, so IMO it might just be cleaner and more readable to _not_ use the try-with-resources pattern. Instead we can call the `close` method manually in a finally block, where the exception can propagate freely.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2467077851


##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,397 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.reflect.Field;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.V1Constants;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.segment.spi.index.startree.StarTreeV2Constants;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {
+    // This test verifies that when build() fails and close() also fails,
+    // the close() exception is added as a suppressed exception to the build() exception
+
+    // Build a test segment with star-tree
+    buildTestSegment();
+
+    // Create a MultipleTreesBuilder with invalid config to force build() to fail
+    List<StarTreeV2BuilderConfig> builderConfigs = createInvalidBuilderConfigs();
+
+    MultipleTreesBuilder builder = null;
+    Exception finalException = null;
+
+    try {
+      builder = new MultipleTreesBuilder(builderConfigs, INDEX_DIR, MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+      // Manually set up the scenario for close() to fail
+      // First, create the separator temp directory and existing metadata
+      Field separatorTempDirField = MultipleTreesBuilder.class.getDeclaredField("_separatorTempDir");
+      separatorTempDirField.setAccessible(true);
+
+      File segmentDir = INDEX_DIR.listFiles()[0];
+      File manualSeparatorTempDir = new File(segmentDir, StarTreeV2Constants.EXISTING_STAR_TREE_TEMP_DIR);
+      FileUtils.forceMkdir(manualSeparatorTempDir);
+      separatorTempDirField.set(builder, manualSeparatorTempDir);
+
+      // Create temp files to simulate existing star-tree files that need restoration
+      File tempIndexFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME);
+      File tempMapFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME);
+      FileUtils.touch(tempIndexFile);
+      FileUtils.touch(tempMapFile);
+
+      // Create existing star-tree metadata using reflection
+      Field existingStarTreeMetadataField = MultipleTreesBuilder.class.getDeclaredField("_existingStarTreeMetadata");
+      existingStarTreeMetadataField.setAccessible(true);
+      PropertiesConfiguration existingMetadata = new PropertiesConfiguration();
+      existingMetadata.setProperty("test.key", "test.value");
+      existingStarTreeMetadataField.set(builder, existingMetadata);
+
+      // Make the metadata file read-only to force close() restoration to fail
+      File metadataFile = new File(segmentDir, V1Constants.MetadataKeys.METADATA_FILE_NAME);
+      if (metadataFile.exists()) {
+        metadataFile.setReadOnly();
+      }
+
+      // Use try-with-resources just like SegmentPreProcessor does
+      // This will automatically call close() if build() throws an exception
+      try (MultipleTreesBuilder autoCloseBuilder = builder) {
+        // This should fail due to invalid config, setting _starTreeCreationFailed = true
+        autoCloseBuilder.build();
+        fail("Expected build() to throw an exception due to invalid config");
+      } catch (Exception e) {
+        finalException = e;
+        // Verify that _starTreeCreationFailed was set to true
+        Field starTreeCreationFailedField = MultipleTreesBuilder.class.getDeclaredField("_starTreeCreationFailed");

Review Comment:
   redid the test, removed all reflection APIs and moved to mocking instead



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2467075586


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {
+          if (_starTreeCreationFailed) {
+            LOGGER.error("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+                _segmentDirectory,
+                false);
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+                _segmentDirectory, false);
+
+            // Copy back the older star-tree related metadata
+            Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+            while (keys.hasNext()) {
+              String key = keys.next();
+              Object value = _existingStarTreeMetadata.getProperty(key);
+              _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+            }
+            CommonsConfigurationUtils.saveToFile(_metadataProperties,
+                new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+          }
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2467345327


##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,207 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.env.CommonsConfigurationUtils;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.mockito.ArgumentMatchers.any;
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {
+    // This test verifies that when build() fails and close() also fails,
+
+    // Build a test segment with star-tree
+    buildTestSegment();
+
+    Exception buildException = null;
+    Exception closeException = null;
+
+    // Build the star-tree index with a good configuration and ensure it passes. This will ensure that the correct
+    // close clean-up path is called
+    List<StarTreeV2BuilderConfig> builderConfigsValid = createBuilderConfigs();
+    MultipleTreesBuilder builder = new MultipleTreesBuilder(builderConfigsValid, INDEX_DIR,
+        MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+    try {
+      builder.build();
+    } catch (Exception e) {
+      fail("Building the first time with valid configs should pass", e);
+    } finally {
+      builder.close();
+    }
+
+    // Create a MultipleTreesBuilder with invalid config to force build() to fail
+    List<StarTreeV2BuilderConfig> builderConfigsInvalid = createInvalidBuilderConfigs();
+    builder = new MultipleTreesBuilder(builderConfigsInvalid, INDEX_DIR, MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+    // Mock the CommonsConfigurationUtils to emulate failure during close
+    try (MockedStatic<CommonsConfigurationUtils> mockedStatic = Mockito.mockStatic(CommonsConfigurationUtils.class)) {
+      try {
+        // This should fail due to invalid config
+        builder.build();
+        fail("Expected build() to throw an exception due to invalid config");
+      } catch (Exception e) {
+        buildException = e;
+      } finally {
+        try {
+          // Mock the static method to always throw RuntimeException on any input to force a close() failure
+          mockedStatic.when(() -> CommonsConfigurationUtils.saveToFile(any(PropertiesConfiguration.class),
+              any(File.class))).thenThrow(new RuntimeException("Simulated failure"));
+          builder.close();
+          fail("Closing the builder should fail since the build should have failed");
+        } catch (Exception e) {
+          closeException = e;
+        }
+      }
+    }
+
+    // Verify that the build exception occurred
+    assertNotNull(buildException, "Expected an exception from try-catch (build)");
+
+    // Verify that the close exception occurred
+    assertNotNull(closeException, "Expected an exception from try-catch-finally (close in finally)");

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2467344987


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -240,8 +266,33 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   }
 
   @Override
-  public void close() {
+  public void close()
+      throws IOException {
     if (_separatorTempDir != null) {
+      if (_starTreeCreationFailed) {
+        try {
+          LOGGER.info("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+          FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+              _segmentDirectory,
+              false);
+          FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+              _segmentDirectory, false);
+
+          // Copy back the older star-tree related metadata
+          Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+          while (keys.hasNext()) {
+            String key = keys.next();
+            Object value = _existingStarTreeMetadata.getProperty(key);
+            _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+          }
+          CommonsConfigurationUtils.saveToFile(_metadataProperties,
+              new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);
+          throw e;

Review Comment:
   good point, added the cleanup before throwing



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,207 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.env.CommonsConfigurationUtils;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.mockito.ArgumentMatchers.any;
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2441388493


##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
         // Process should complete without throwing exception, but star-tree should not be updated
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Test 2: Try to update existing star-tree index with invalid functionColumnPair (SUM__*)
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Test 1: Adding a new star-tree index with invalid functionColumnPair (SUM__*)
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Verify that the original star-tree index still exists and hasn't changed
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed
+    segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> currentStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(currentStarTreeMetadataList);
+    assertFalse(currentStarTreeMetadataList.isEmpty());
+    assertEquals(currentStarTreeMetadataList.size(), 1);
+
+    // Verify the metadata hasn't changed (original startree should still be there)
+    assertEquals(currentStarTreeMetadataList.get(0).getDimensionsSplitOrder(),
+        starTreeMetadataList.get(0).getDimensionsSplitOrder());
+    assertEquals(currentStarTreeMetadataList.get(0).getFunctionColumnPairs(),
+        starTreeMetadataList.get(0).getFunctionColumnPairs());
+  }
+
+  @Test
+  public void testStarTreeCreationWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 3: Adding a new startree index with valid functionColumnPair (COUNT__*)

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Test 3: Adding a new star-tree index with valid functionColumnPair (COUNT__*)
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed
+    segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> currentStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(currentStarTreeMetadataList);
+    assertFalse(currentStarTreeMetadataList.isEmpty());
+    assertEquals(currentStarTreeMetadataList.size(), 1);
+
+    // Verify the metadata hasn't changed (original startree should still be there)
+    assertEquals(currentStarTreeMetadataList.get(0).getDimensionsSplitOrder(),
+        starTreeMetadataList.get(0).getDimensionsSplitOrder());
+    assertEquals(currentStarTreeMetadataList.get(0).getFunctionColumnPairs(),
+        starTreeMetadataList.get(0).getFunctionColumnPairs());
+  }
+
+  @Test
+  public void testStarTreeCreationWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 3: Adding a new startree index with valid functionColumnPair (COUNT__*)
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created successfully

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Verify that star-tree index was created successfully
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed
+    segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> currentStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(currentStarTreeMetadataList);
+    assertFalse(currentStarTreeMetadataList.isEmpty());
+    assertEquals(currentStarTreeMetadataList.size(), 1);
+
+    // Verify the metadata hasn't changed (original startree should still be there)

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Verify the metadata hasn't changed (original star-tree should still be there)
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed
+    segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> currentStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(currentStarTreeMetadataList);
+    assertFalse(currentStarTreeMetadataList.isEmpty());
+    assertEquals(currentStarTreeMetadataList.size(), 1);
+
+    // Verify the metadata hasn't changed (original startree should still be there)
+    assertEquals(currentStarTreeMetadataList.get(0).getDimensionsSplitOrder(),
+        starTreeMetadataList.get(0).getDimensionsSplitOrder());
+    assertEquals(currentStarTreeMetadataList.get(0).getFunctionColumnPairs(),
+        starTreeMetadataList.get(0).getFunctionColumnPairs());
+  }
+
+  @Test
+  public void testStarTreeCreationWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 3: Adding a new startree index with valid functionColumnPair (COUNT__*)
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created successfully
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    StarTreeV2Metadata starTreeMetadata = starTreeMetadataList.get(0);
+    assertEquals(starTreeMetadata.getDimensionsSplitOrder(), List.of("stringCol"));
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().size(), 1);
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().iterator().next().toColumnName(), "count__*");
+  }
+
+  @Test
+  public void testStarTreeUpdateWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a startree index with SUM function

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // First, create a star-tree index with SUM function
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed
+    segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> currentStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(currentStarTreeMetadataList);
+    assertFalse(currentStarTreeMetadataList.isEmpty());
+    assertEquals(currentStarTreeMetadataList.size(), 1);
+
+    // Verify the metadata hasn't changed (original startree should still be there)
+    assertEquals(currentStarTreeMetadataList.get(0).getDimensionsSplitOrder(),
+        starTreeMetadataList.get(0).getDimensionsSplitOrder());
+    assertEquals(currentStarTreeMetadataList.get(0).getFunctionColumnPairs(),
+        starTreeMetadataList.get(0).getFunctionColumnPairs());
+  }
+
+  @Test
+  public void testStarTreeCreationWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 3: Adding a new startree index with valid functionColumnPair (COUNT__*)
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created successfully
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    StarTreeV2Metadata starTreeMetadata = starTreeMetadataList.get(0);
+    assertEquals(starTreeMetadata.getDimensionsSplitOrder(), List.of("stringCol"));
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().size(), 1);
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().iterator().next().toColumnName(), "count__*");
+  }
+
+  @Test
+  public void testStarTreeUpdateWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a startree index with SUM function
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig originalStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__longCol"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(originalStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that original startree index was created

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Verify that original star-tree index was created
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed
+    segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> currentStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(currentStarTreeMetadataList);
+    assertFalse(currentStarTreeMetadataList.isEmpty());
+    assertEquals(currentStarTreeMetadataList.size(), 1);
+
+    // Verify the metadata hasn't changed (original startree should still be there)
+    assertEquals(currentStarTreeMetadataList.get(0).getDimensionsSplitOrder(),
+        starTreeMetadataList.get(0).getDimensionsSplitOrder());
+    assertEquals(currentStarTreeMetadataList.get(0).getFunctionColumnPairs(),
+        starTreeMetadataList.get(0).getFunctionColumnPairs());
+  }
+
+  @Test
+  public void testStarTreeCreationWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 3: Adding a new startree index with valid functionColumnPair (COUNT__*)
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created successfully
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    StarTreeV2Metadata starTreeMetadata = starTreeMetadataList.get(0);
+    assertEquals(starTreeMetadata.getDimensionsSplitOrder(), List.of("stringCol"));
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().size(), 1);
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().iterator().next().toColumnName(), "count__*");
+  }
+
+  @Test
+  public void testStarTreeUpdateWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a startree index with SUM function
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig originalStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__longCol"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(originalStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that original startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> originalStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(originalStarTreeMetadataList);
+    assertFalse(originalStarTreeMetadataList.isEmpty());
+    assertEquals(originalStarTreeMetadataList.size(), 1);
+    assertEquals(originalStarTreeMetadataList.get(0).getFunctionColumnPairs().iterator().next().toColumnName(),
+        "sum__longCol");
+
+    // Test 4: Update existing startree index with valid functionColumnPair (COUNT__*)
+    StarTreeIndexConfig updatedStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__longCol", "COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(updatedStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was updated successfully

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Verify that star-tree index was updated successfully
   ```



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/segment/index/loader/SegmentPreProcessorTest.java:
##########
@@ -1723,6 +1725,214 @@ public void testStarTreeCreationWithDictionaryChanges()
     }
   }
 
+  @Test
+  public void testStarTreeCreationWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 1: Adding a new startree index with invalid functionColumnPair (SUM__*)
+    // This should fail during config creation but segment build should be fine
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to new startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be created
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that no startree index was created due to invalid config
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    assertNull(segmentMetadata.getStarTreeV2MetadataList());
+  }
+
+  @Test
+  public void testStarTreeUpdateWithInvalidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a valid startree index
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    // Test 2: Try to update existing startree index with invalid functionColumnPair (SUM__*)
+    StarTreeIndexConfig invalidStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*", "SUM__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(invalidStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      // Should need processing due to changed startree config
+      assertTrue(processor.needProcess());
+      // Process should complete without throwing exception, but startree should not be updated
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that the original startree index still exists and hasn't changed
+    segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> currentStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(currentStarTreeMetadataList);
+    assertFalse(currentStarTreeMetadataList.isEmpty());
+    assertEquals(currentStarTreeMetadataList.size(), 1);
+
+    // Verify the metadata hasn't changed (original startree should still be there)
+    assertEquals(currentStarTreeMetadataList.get(0).getDimensionsSplitOrder(),
+        starTreeMetadataList.get(0).getDimensionsSplitOrder());
+    assertEquals(currentStarTreeMetadataList.get(0).getFunctionColumnPairs(),
+        starTreeMetadataList.get(0).getFunctionColumnPairs());
+  }
+
+  @Test
+  public void testStarTreeCreationWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // Test 3: Adding a new startree index with valid functionColumnPair (COUNT__*)
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig validStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("COUNT__*"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(validStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that startree index was created successfully
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> starTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(starTreeMetadataList);
+    assertFalse(starTreeMetadataList.isEmpty());
+    assertEquals(starTreeMetadataList.size(), 1);
+
+    StarTreeV2Metadata starTreeMetadata = starTreeMetadataList.get(0);
+    assertEquals(starTreeMetadata.getDimensionsSplitOrder(), List.of("stringCol"));
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().size(), 1);
+    assertEquals(starTreeMetadata.getFunctionColumnPairs().iterator().next().toColumnName(), "count__*");
+  }
+
+  @Test
+  public void testStarTreeUpdateWithValidFunctionColumnPair()
+      throws Exception {
+    // Build the sample segment
+    String[] stringValues = {"A", "C", "B", "C", "D", "E", "E", "E"};
+    long[] longValues = {2, 1, 2, 3, 4, 5, 3, 2};
+    TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE).setTableName("testTable").build();
+    Schema schema = new Schema.SchemaBuilder().addSingleValueDimension("stringCol", FieldSpec.DataType.STRING)
+        .addMetric("longCol", DataType.LONG)
+        .build();
+
+    // Build good segment, no need for preprocess
+    buildTestSegment(tableConfig, schema, stringValues, longValues);
+
+    // First, create a startree index with SUM function
+    IndexingConfig indexingConfig = tableConfig.getIndexingConfig();
+    indexingConfig.setEnableDynamicStarTreeCreation(true);
+    StarTreeIndexConfig originalStarTreeIndexConfig =
+        new StarTreeIndexConfig(List.of("stringCol"), null, List.of("SUM__longCol"), null, 1000);
+    indexingConfig.setStarTreeIndexConfigs(List.of(originalStarTreeIndexConfig));
+
+    try (SegmentDirectory segmentDirectory = new SegmentLocalFSDirectory(INDEX_DIR, ReadMode.mmap);
+        SegmentPreProcessor processor = new SegmentPreProcessor(segmentDirectory,
+            new IndexLoadingConfig(tableConfig, schema))) {
+      assertTrue(processor.needProcess());
+      processor.process(SEGMENT_OPERATIONS_THROTTLER);
+    }
+
+    // Verify that original startree index was created
+    SegmentMetadataImpl segmentMetadata = new SegmentMetadataImpl(INDEX_DIR);
+    List<StarTreeV2Metadata> originalStarTreeMetadataList = segmentMetadata.getStarTreeV2MetadataList();
+    assertNotNull(originalStarTreeMetadataList);
+    assertFalse(originalStarTreeMetadataList.isEmpty());
+    assertEquals(originalStarTreeMetadataList.size(), 1);
+    assertEquals(originalStarTreeMetadataList.get(0).getFunctionColumnPairs().iterator().next().toColumnName(),
+        "sum__longCol");
+
+    // Test 4: Update existing startree index with valid functionColumnPair (COUNT__*)

Review Comment:
   Corrected spelling of 'startree' to 'star-tree' in comment for consistency with other comments in the file.
   ```suggestion
       // Test 4: Update existing star-tree index with valid functionColumnPair (COUNT__*)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2441394944


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   yeah i'm confused about this too, but see that the `build()` was already updating MinionMetrics:
   
   ```
       MinionMetrics metrics = MinionMetrics.get();
       String tableNameWithType = _config.getTableConfig().getTableName();
       if (_incompleteRowsFound > 0) {
         metrics.addMeteredTableValue(tableNameWithType, MinionMeter.TRANSFORMATION_ERROR_COUNT, _incompleteRowsFound);
       }
       if (_skippedRowsFound > 0) {
         metrics.addMeteredTableValue(tableNameWithType, MinionMeter.DROPPED_RECORD_COUNT, _skippedRowsFound);
       }
       if (_sanitizedRowsFound > 0) {
         metrics.addMeteredTableValue(tableNameWithType, MinionMeter.CORRUPTED_RECORD_COUNT, _sanitizedRowsFound);
       }
   ```
   
   Perhaps this should be fixed, but don't think clear context is available here to differentiate between build due to realtime conversion vs. minion. Any suggestions? happy to fix this (or can fix as a follow-up)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2453616695


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   I've opened a separate PR to address this: https://github.com/apache/pinot/pull/17057
   Will rebase this one with that change once it is in. PTAL, thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2445856594


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   i actually do see the `build()` method called by the realtime segment conversion path (`RealtimeSegmentConverter`:
   
   ```
     /**
      * Common method to build segment with the provided record reader
      */
     private void buildSegmentWithReader(SegmentIndexCreationDriverImpl driver, SegmentGeneratorConfig genConfig,
         RecordReader recordReader, int[] sortedDocIds, boolean useCompactedReader,
         @Nullable ThreadSafeMutableRoaringBitmap validDocIdsSnapshot)
         throws Exception {
       RealtimeSegmentSegmentCreationDataSource dataSource;
       if (useCompactedReader) {
         // For compacted readers, use the constructor that takes sortedDocIds and pass the validDocIds snapshot
         dataSource = new RealtimeSegmentSegmentCreationDataSource(_realtimeSegmentImpl, recordReader, sortedDocIds,
             validDocIdsSnapshot);
       } else {
         // For regular readers, use the original constructor
         dataSource =
             new RealtimeSegmentSegmentCreationDataSource(_realtimeSegmentImpl, (PinotSegmentRecordReader) recordReader);
       }
       driver.init(genConfig, dataSource, TransformPipeline.getPassThroughPipeline(_tableName)); // initializes reader
   
       if (!_enableColumnMajor) {
         driver.build();
       } else {
         //buildByColumn uses validDocIds to skip invalid record while indexing each column. We pass the validDocIds
         // only if we are using compacted reader.
         driver.buildByColumn(_realtimeSegmentImpl, useCompactedReader ? validDocIdsSnapshot : null);
       }
     }
   ```
   
   so I do think we need to pass some context on whether this is called from minion code path or server 😅 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2467076449


##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,397 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.reflect.Field;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.V1Constants;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.segment.spi.index.startree.StarTreeV2Constants;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {
+    // This test verifies that when build() fails and close() also fails,
+    // the close() exception is added as a suppressed exception to the build() exception
+
+    // Build a test segment with star-tree
+    buildTestSegment();
+
+    // Create a MultipleTreesBuilder with invalid config to force build() to fail
+    List<StarTreeV2BuilderConfig> builderConfigs = createInvalidBuilderConfigs();
+
+    MultipleTreesBuilder builder = null;
+    Exception finalException = null;
+
+    try {
+      builder = new MultipleTreesBuilder(builderConfigs, INDEX_DIR, MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+      // Manually set up the scenario for close() to fail
+      // First, create the separator temp directory and existing metadata
+      Field separatorTempDirField = MultipleTreesBuilder.class.getDeclaredField("_separatorTempDir");
+      separatorTempDirField.setAccessible(true);
+
+      File segmentDir = INDEX_DIR.listFiles()[0];
+      File manualSeparatorTempDir = new File(segmentDir, StarTreeV2Constants.EXISTING_STAR_TREE_TEMP_DIR);
+      FileUtils.forceMkdir(manualSeparatorTempDir);
+      separatorTempDirField.set(builder, manualSeparatorTempDir);
+
+      // Create temp files to simulate existing star-tree files that need restoration
+      File tempIndexFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME);
+      File tempMapFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME);
+      FileUtils.touch(tempIndexFile);
+      FileUtils.touch(tempMapFile);
+
+      // Create existing star-tree metadata using reflection
+      Field existingStarTreeMetadataField = MultipleTreesBuilder.class.getDeclaredField("_existingStarTreeMetadata");
+      existingStarTreeMetadataField.setAccessible(true);
+      PropertiesConfiguration existingMetadata = new PropertiesConfiguration();
+      existingMetadata.setProperty("test.key", "test.value");
+      existingStarTreeMetadataField.set(builder, existingMetadata);
+
+      // Make the metadata file read-only to force close() restoration to fail
+      File metadataFile = new File(segmentDir, V1Constants.MetadataKeys.METADATA_FILE_NAME);
+      if (metadataFile.exists()) {
+        metadataFile.setReadOnly();
+      }
+
+      // Use try-with-resources just like SegmentPreProcessor does
+      // This will automatically call close() if build() throws an exception
+      try (MultipleTreesBuilder autoCloseBuilder = builder) {
+        // This should fail due to invalid config, setting _starTreeCreationFailed = true
+        autoCloseBuilder.build();
+        fail("Expected build() to throw an exception due to invalid config");
+      } catch (Exception e) {
+        finalException = e;
+        // Verify that _starTreeCreationFailed was set to true
+        Field starTreeCreationFailedField = MultipleTreesBuilder.class.getDeclaredField("_starTreeCreationFailed");
+        starTreeCreationFailedField.setAccessible(true);
+        assertTrue((Boolean) starTreeCreationFailedField.get(builder),
+                   "_starTreeCreationFailed should be true after build() fails");
+      } finally {
+        // Restore write permissions for cleanup
+        if (metadataFile.exists()) {
+          metadataFile.setWritable(true);
+        }
+      }
+    } finally {
+      // Ensure builder is closed even if test fails
+      if (builder != null) {
+        try {
+          // Restore permissions first
+          File segmentDir = INDEX_DIR.listFiles()[0];
+          File metadataFile = new File(segmentDir, V1Constants.MetadataKeys.METADATA_FILE_NAME);
+          if (metadataFile.exists()) {
+            metadataFile.setWritable(true);
+          }
+          builder.close();
+        } catch (Exception e) {
+          // Ignore cleanup exceptions
+        }
+      }
+    }
+
+    // Verify that the final exception occurred (this includes both build and close exceptions)
+    assertNotNull(finalException, "Expected an exception from try-with-resources (build + close)");
+
+    // The key test: verify that finalException has buildException as a suppressed exception
+    // OR that finalException is the same as buildException with suppressed exceptions added
+    Throwable[] suppressedExceptions = finalException.getSuppressed();
+
+    // The close() exception should have suppressed exceptions
+    assertTrue(suppressedExceptions.length > 0,
+               "Expected close() exception to have suppressed exceptions, but found none. "
+                   + "Final exception: " + finalException.getMessage());
+
+    // Verify the suppressed exception is related to the cleanup failure
+    boolean foundCleanupException = false;
+    for (Throwable suppressed : suppressedExceptions) {
+      if (suppressed.getMessage().contains("Could not reset") || suppressed.getMessage().contains("Permission denied")
+          || suppressed.getMessage().contains("Unable to save") || suppressed instanceof IOException) {
+        foundCleanupException = true;
+        break;
+      }
+    }
+
+    assertTrue(foundCleanupException,
+               "Expected to find cleanup-related suppressed exception, but found: "
+                   + Arrays.toString(suppressedExceptions));
+
+    // This scenario should trigger the re-throw condition in SegmentPreProcessor.processStarTrees()
+    // because finalException.getSuppressed().length > 0
+    System.out.println("SUCCESS: Exception with " + suppressedExceptions.length

Review Comment:
   oops, yes had added for testing, removed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal merged PR #17028:
URL: https://github.com/apache/pinot/pull/17028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2441396848


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {
+          if (_starTreeCreationFailed) {
+            LOGGER.error("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+                _segmentDirectory,
+                false);
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+                _segmentDirectory, false);
+
+            // Copy back the older star-tree related metadata
+            Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+            while (keys.hasNext()) {
+              String key = keys.next();
+              Object value = _existingStarTreeMetadata.getProperty(key);
+              _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+            }
+            CommonsConfigurationUtils.saveToFile(_metadataProperties,
+                new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+          }
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);

Review Comment:
   what clean-up can we do? you mean redo the steps in the exception handling too?
   how about we throw the exception if it fails at this stage - that way segment build fails and we can't query this segment and won't see inconsistencies?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2445615091


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {
+          if (_starTreeCreationFailed) {
+            LOGGER.error("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+                _segmentDirectory,
+                false);
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+                _segmentDirectory, false);
+
+            // Copy back the older star-tree related metadata
+            Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+            while (keys.hasNext()) {
+              String key = keys.next();
+              Object value = _existingStarTreeMetadata.getProperty(key);
+              _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+            }
+            CommonsConfigurationUtils.saveToFile(_metadataProperties,
+                new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+          }
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);

Review Comment:
   > how about we throw the exception if it fails at this stage - that way segment build fails and we can't query this segment and won't see inconsistencies?
   
   Yeah that sounds like a better idea actually 👍 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2461724311


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {
+          if (_starTreeCreationFailed) {
+            LOGGER.error("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+                _segmentDirectory,
+                false);
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+                _segmentDirectory, false);
+
+            // Copy back the older star-tree related metadata
+            Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+            while (keys.hasNext()) {
+              String key = keys.next();
+              Object value = _existingStarTreeMetadata.getProperty(key);
+              _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+            }
+            CommonsConfigurationUtils.saveToFile(_metadataProperties,
+                new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+          }
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);

Review Comment:
   done - exception handling gets a bit weird when using try-with-resources and we throw exception from close() when an earlier exception was thrown - basically need to check suppressed exceptions list in this scenario. let me know your thoughts on this 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2462191222


##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,397 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.reflect.Field;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.V1Constants;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.segment.spi.index.startree.StarTreeV2Constants;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {
+    // This test verifies that when build() fails and close() also fails,
+    // the close() exception is added as a suppressed exception to the build() exception
+
+    // Build a test segment with star-tree
+    buildTestSegment();
+
+    // Create a MultipleTreesBuilder with invalid config to force build() to fail
+    List<StarTreeV2BuilderConfig> builderConfigs = createInvalidBuilderConfigs();
+
+    MultipleTreesBuilder builder = null;
+    Exception finalException = null;
+
+    try {
+      builder = new MultipleTreesBuilder(builderConfigs, INDEX_DIR, MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+      // Manually set up the scenario for close() to fail
+      // First, create the separator temp directory and existing metadata
+      Field separatorTempDirField = MultipleTreesBuilder.class.getDeclaredField("_separatorTempDir");
+      separatorTempDirField.setAccessible(true);
+
+      File segmentDir = INDEX_DIR.listFiles()[0];
+      File manualSeparatorTempDir = new File(segmentDir, StarTreeV2Constants.EXISTING_STAR_TREE_TEMP_DIR);
+      FileUtils.forceMkdir(manualSeparatorTempDir);
+      separatorTempDirField.set(builder, manualSeparatorTempDir);
+
+      // Create temp files to simulate existing star-tree files that need restoration
+      File tempIndexFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME);
+      File tempMapFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME);
+      FileUtils.touch(tempIndexFile);
+      FileUtils.touch(tempMapFile);
+
+      // Create existing star-tree metadata using reflection
+      Field existingStarTreeMetadataField = MultipleTreesBuilder.class.getDeclaredField("_existingStarTreeMetadata");
+      existingStarTreeMetadataField.setAccessible(true);
+      PropertiesConfiguration existingMetadata = new PropertiesConfiguration();
+      existingMetadata.setProperty("test.key", "test.value");
+      existingStarTreeMetadataField.set(builder, existingMetadata);
+
+      // Make the metadata file read-only to force close() restoration to fail
+      File metadataFile = new File(segmentDir, V1Constants.MetadataKeys.METADATA_FILE_NAME);
+      if (metadataFile.exists()) {
+        metadataFile.setReadOnly();
+      }
+
+      // Use try-with-resources just like SegmentPreProcessor does
+      // This will automatically call close() if build() throws an exception
+      try (MultipleTreesBuilder autoCloseBuilder = builder) {
+        // This should fail due to invalid config, setting _starTreeCreationFailed = true
+        autoCloseBuilder.build();
+        fail("Expected build() to throw an exception due to invalid config");
+      } catch (Exception e) {
+        finalException = e;
+        // Verify that _starTreeCreationFailed was set to true
+        Field starTreeCreationFailedField = MultipleTreesBuilder.class.getDeclaredField("_starTreeCreationFailed");
+        starTreeCreationFailedField.setAccessible(true);
+        assertTrue((Boolean) starTreeCreationFailedField.get(builder),
+                   "_starTreeCreationFailed should be true after build() fails");
+      } finally {
+        // Restore write permissions for cleanup
+        if (metadataFile.exists()) {
+          metadataFile.setWritable(true);
+        }
+      }
+    } finally {
+      // Ensure builder is closed even if test fails
+      if (builder != null) {
+        try {
+          // Restore permissions first
+          File segmentDir = INDEX_DIR.listFiles()[0];
+          File metadataFile = new File(segmentDir, V1Constants.MetadataKeys.METADATA_FILE_NAME);
+          if (metadataFile.exists()) {
+            metadataFile.setWritable(true);
+          }
+          builder.close();
+        } catch (Exception e) {
+          // Ignore cleanup exceptions
+        }
+      }
+    }
+
+    // Verify that the final exception occurred (this includes both build and close exceptions)
+    assertNotNull(finalException, "Expected an exception from try-with-resources (build + close)");
+
+    // The key test: verify that finalException has buildException as a suppressed exception
+    // OR that finalException is the same as buildException with suppressed exceptions added
+    Throwable[] suppressedExceptions = finalException.getSuppressed();
+
+    // The close() exception should have suppressed exceptions
+    assertTrue(suppressedExceptions.length > 0,
+               "Expected close() exception to have suppressed exceptions, but found none. "
+                   + "Final exception: " + finalException.getMessage());
+
+    // Verify the suppressed exception is related to the cleanup failure
+    boolean foundCleanupException = false;
+    for (Throwable suppressed : suppressedExceptions) {
+      if (suppressed.getMessage().contains("Could not reset") || suppressed.getMessage().contains("Permission denied")
+          || suppressed.getMessage().contains("Unable to save") || suppressed instanceof IOException) {
+        foundCleanupException = true;
+        break;
+      }
+    }
+
+    assertTrue(foundCleanupException,
+               "Expected to find cleanup-related suppressed exception, but found: "
+                   + Arrays.toString(suppressedExceptions));
+
+    // This scenario should trigger the re-throw condition in SegmentPreProcessor.processStarTrees()
+    // because finalException.getSuppressed().length > 0
+    System.out.println("SUCCESS: Exception with " + suppressedExceptions.length

Review Comment:
   I'm guessing this wasn't mean to be left in?



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,397 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.reflect.Field;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.V1Constants;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.segment.spi.index.startree.StarTreeV2Constants;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {
+    // This test verifies that when build() fails and close() also fails,
+    // the close() exception is added as a suppressed exception to the build() exception
+
+    // Build a test segment with star-tree
+    buildTestSegment();
+
+    // Create a MultipleTreesBuilder with invalid config to force build() to fail
+    List<StarTreeV2BuilderConfig> builderConfigs = createInvalidBuilderConfigs();
+
+    MultipleTreesBuilder builder = null;
+    Exception finalException = null;
+
+    try {
+      builder = new MultipleTreesBuilder(builderConfigs, INDEX_DIR, MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+      // Manually set up the scenario for close() to fail
+      // First, create the separator temp directory and existing metadata
+      Field separatorTempDirField = MultipleTreesBuilder.class.getDeclaredField("_separatorTempDir");
+      separatorTempDirField.setAccessible(true);
+
+      File segmentDir = INDEX_DIR.listFiles()[0];
+      File manualSeparatorTempDir = new File(segmentDir, StarTreeV2Constants.EXISTING_STAR_TREE_TEMP_DIR);
+      FileUtils.forceMkdir(manualSeparatorTempDir);
+      separatorTempDirField.set(builder, manualSeparatorTempDir);
+
+      // Create temp files to simulate existing star-tree files that need restoration
+      File tempIndexFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME);
+      File tempMapFile = new File(manualSeparatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME);
+      FileUtils.touch(tempIndexFile);
+      FileUtils.touch(tempMapFile);
+
+      // Create existing star-tree metadata using reflection
+      Field existingStarTreeMetadataField = MultipleTreesBuilder.class.getDeclaredField("_existingStarTreeMetadata");
+      existingStarTreeMetadataField.setAccessible(true);
+      PropertiesConfiguration existingMetadata = new PropertiesConfiguration();
+      existingMetadata.setProperty("test.key", "test.value");
+      existingStarTreeMetadataField.set(builder, existingMetadata);
+
+      // Make the metadata file read-only to force close() restoration to fail
+      File metadataFile = new File(segmentDir, V1Constants.MetadataKeys.METADATA_FILE_NAME);
+      if (metadataFile.exists()) {
+        metadataFile.setReadOnly();
+      }
+
+      // Use try-with-resources just like SegmentPreProcessor does
+      // This will automatically call close() if build() throws an exception
+      try (MultipleTreesBuilder autoCloseBuilder = builder) {
+        // This should fail due to invalid config, setting _starTreeCreationFailed = true
+        autoCloseBuilder.build();
+        fail("Expected build() to throw an exception due to invalid config");
+      } catch (Exception e) {
+        finalException = e;
+        // Verify that _starTreeCreationFailed was set to true
+        Field starTreeCreationFailedField = MultipleTreesBuilder.class.getDeclaredField("_starTreeCreationFailed");

Review Comment:
   I'd prefer not using reflection APIs directly wherever avoidable -- could we use partial mocks instead or else some test-only methods?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2467172926


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -578,10 +578,26 @@ private void buildStarTreeV2IfNecessary(File indexDir)
     if (CollectionUtils.isNotEmpty(starTreeIndexConfigs) || enableDefaultStarTree) {
       MultipleTreesBuilder.BuildMode buildMode =
           _config.isOnHeap() ? MultipleTreesBuilder.BuildMode.ON_HEAP : MultipleTreesBuilder.BuildMode.OFF_HEAP;
-      try (
-          MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
-              buildMode)) {
+      MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
+          buildMode);
+      try {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        if (_instanceType == InstanceType.MINION) {
+          MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        } else {
+          ServerMetrics.get().addMeteredTableValue(tableNameWithType, ServerMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        }
+      } finally {
+        try {
+          builder.close();
+        } catch (Exception e) {
+          LOGGER.error("Closing builder threw an exception, potentially leaving the star-tree index in an "
+              + "inconsistent state, throwing exception", e);
+          throw e;

Review Comment:
   Do we really need this? We're already logging the error in the builder's close method. Feels a bit odd to have an additional try-catch + throw in the finally block for a runtime exception.



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,207 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.env.CommonsConfigurationUtils;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.mockito.ArgumentMatchers.any;
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {
+    // This test verifies that when build() fails and close() also fails,
+
+    // Build a test segment with star-tree
+    buildTestSegment();
+
+    Exception buildException = null;
+    Exception closeException = null;
+
+    // Build the star-tree index with a good configuration and ensure it passes. This will ensure that the correct
+    // close clean-up path is called
+    List<StarTreeV2BuilderConfig> builderConfigsValid = createBuilderConfigs();
+    MultipleTreesBuilder builder = new MultipleTreesBuilder(builderConfigsValid, INDEX_DIR,
+        MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+    try {
+      builder.build();
+    } catch (Exception e) {
+      fail("Building the first time with valid configs should pass", e);
+    } finally {
+      builder.close();
+    }
+
+    // Create a MultipleTreesBuilder with invalid config to force build() to fail
+    List<StarTreeV2BuilderConfig> builderConfigsInvalid = createInvalidBuilderConfigs();
+    builder = new MultipleTreesBuilder(builderConfigsInvalid, INDEX_DIR, MultipleTreesBuilder.BuildMode.OFF_HEAP);
+
+    // Mock the CommonsConfigurationUtils to emulate failure during close
+    try (MockedStatic<CommonsConfigurationUtils> mockedStatic = Mockito.mockStatic(CommonsConfigurationUtils.class)) {
+      try {
+        // This should fail due to invalid config
+        builder.build();
+        fail("Expected build() to throw an exception due to invalid config");
+      } catch (Exception e) {
+        buildException = e;
+      } finally {
+        try {
+          // Mock the static method to always throw RuntimeException on any input to force a close() failure
+          mockedStatic.when(() -> CommonsConfigurationUtils.saveToFile(any(PropertiesConfiguration.class),
+              any(File.class))).thenThrow(new RuntimeException("Simulated failure"));
+          builder.close();
+          fail("Closing the builder should fail since the build should have failed");
+        } catch (Exception e) {
+          closeException = e;
+        }
+      }
+    }
+
+    // Verify that the build exception occurred
+    assertNotNull(buildException, "Expected an exception from try-catch (build)");
+
+    // Verify that the close exception occurred
+    assertNotNull(closeException, "Expected an exception from try-catch-finally (close in finally)");

Review Comment:
   nit: I think it might be more readable to use the canonical `assertThrows` pattern instead of try-catch-fail-assert?



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -240,8 +266,33 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   }
 
   @Override
-  public void close() {
+  public void close()
+      throws IOException {
     if (_separatorTempDir != null) {
+      if (_starTreeCreationFailed) {
+        try {
+          LOGGER.info("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+          FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+              _segmentDirectory,
+              false);
+          FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+              _segmentDirectory, false);
+
+          // Copy back the older star-tree related metadata
+          Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+          while (keys.hasNext()) {
+            String key = keys.next();
+            Object value = _existingStarTreeMetadata.getProperty(key);
+            _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+          }
+          CommonsConfigurationUtils.saveToFile(_metadataProperties,
+              new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);
+          throw e;

Review Comment:
   Do we not need to delete the separator temp dir and call `IndexSegment::destroy` here?



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -578,10 +578,26 @@ private void buildStarTreeV2IfNecessary(File indexDir)
     if (CollectionUtils.isNotEmpty(starTreeIndexConfigs) || enableDefaultStarTree) {
       MultipleTreesBuilder.BuildMode buildMode =
           _config.isOnHeap() ? MultipleTreesBuilder.BuildMode.ON_HEAP : MultipleTreesBuilder.BuildMode.OFF_HEAP;
-      try (
-          MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
-              buildMode)) {
+      MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
+          buildMode);
+      try {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        if (_instanceType == InstanceType.MINION) {
+          MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        } else {
+          ServerMetrics.get().addMeteredTableValue(tableNameWithType, ServerMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        }
+      } finally {
+        try {
+          builder.close();

Review Comment:
   Might be worth adding a one liner comment on why we aren't using the try-with-resources pattern, because some IDEs flag this as a problem.



##########
pinot-segment-local/src/test/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilderCloseTest.java:
##########
@@ -0,0 +1,207 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.segment.local.startree.v2.builder;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.List;
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.local.startree.StarTreeBuilderUtils;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.StarTreeIndexConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.env.CommonsConfigurationUtils;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.mockito.MockedStatic;
+import org.mockito.Mockito;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.Test;
+
+import static org.mockito.ArgumentMatchers.any;
+import static org.testng.Assert.*;
+
+/**
+ * Unit test for MultipleTreesBuilder.close() method to verify exception handling
+ * when cleanup operations fail.
+ */
+public class MultipleTreesBuilderCloseTest {
+  private static final File TEMP_DIR = new File(FileUtils.getTempDirectory(), "MultipleTreesBuilderCloseTest");
+  private static final File INDEX_DIR = new File(TEMP_DIR, "testSegment");
+
+  @BeforeMethod
+  public void setUp() throws Exception {
+    FileUtils.deleteQuietly(TEMP_DIR);
+    FileUtils.forceMkdir(TEMP_DIR);
+  }
+
+  @AfterMethod
+  public void tearDown() {
+    FileUtils.deleteQuietly(TEMP_DIR);
+  }
+
+  @Test
+  public void testBuildFailureThenCloseFailureWithSuppressedException() throws Exception {

Review Comment:
   nit: there's no suppressed exceptions anymore I suppose?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2467344276


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -578,10 +578,26 @@ private void buildStarTreeV2IfNecessary(File indexDir)
     if (CollectionUtils.isNotEmpty(starTreeIndexConfigs) || enableDefaultStarTree) {
       MultipleTreesBuilder.BuildMode buildMode =
           _config.isOnHeap() ? MultipleTreesBuilder.BuildMode.ON_HEAP : MultipleTreesBuilder.BuildMode.OFF_HEAP;
-      try (
-          MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
-              buildMode)) {
+      MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
+          buildMode);
+      try {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        if (_instanceType == InstanceType.MINION) {
+          MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        } else {
+          ServerMetrics.get().addMeteredTableValue(tableNameWithType, ServerMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        }
+      } finally {
+        try {
+          builder.close();
+        } catch (Exception e) {
+          LOGGER.error("Closing builder threw an exception, potentially leaving the star-tree index in an "
+              + "inconsistent state, throwing exception", e);
+          throw e;

Review Comment:
   good point, done



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -578,10 +578,26 @@ private void buildStarTreeV2IfNecessary(File indexDir)
     if (CollectionUtils.isNotEmpty(starTreeIndexConfigs) || enableDefaultStarTree) {
       MultipleTreesBuilder.BuildMode buildMode =
           _config.isOnHeap() ? MultipleTreesBuilder.BuildMode.ON_HEAP : MultipleTreesBuilder.BuildMode.OFF_HEAP;
-      try (
-          MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
-              buildMode)) {
+      MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
+          buildMode);
+      try {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        if (_instanceType == InstanceType.MINION) {
+          MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        } else {
+          ServerMetrics.get().addMeteredTableValue(tableNameWithType, ServerMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);
+        }
+      } finally {
+        try {
+          builder.close();

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2437439477


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   Why are we using minion metrics here? Isn't this segment creation driver also used when converting mutable / consuming segments to immutable ones for realtime tables? 



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -175,28 +185,44 @@ public void build()
     LOGGER.info("Starting building {} star-trees with configs: {} using {} builder", numStarTrees, _builderConfigs,
         _buildMode);
 
-    try (StarTreeIndexCombiner indexCombiner = new StarTreeIndexCombiner(
-        new File(_segmentDirectory, StarTreeV2Constants.INDEX_FILE_NAME))) {
+    File starTreeV2IndexFile = new File(_segmentDirectory, StarTreeV2Constants.INDEX_FILE_NAME);
+    try (StarTreeIndexCombiner indexCombiner = new StarTreeIndexCombiner(starTreeV2IndexFile)) {
       File starTreeIndexDir = new File(_segmentDirectory, StarTreeV2Constants.STAR_TREE_TEMP_DIR);
       FileUtils.forceMkdir(starTreeIndexDir);
       _metadataProperties.addProperty(MetadataKey.STAR_TREE_COUNT, numStarTrees);
       List<List<Pair<IndexKey, IndexValue>>> indexMaps = new ArrayList<>(numStarTrees);
 
       // Build all star-trees
-      for (int i = 0; i < numStarTrees; i++) {
-        StarTreeV2BuilderConfig builderConfig = _builderConfigs.get(i);
-        Configuration metadataProperties = _metadataProperties.subset(MetadataKey.getStarTreePrefix(i));
-        if (_separator != null && handleExistingStarTreeAddition(starTreeIndexDir, metadataProperties, builderConfig)) {
-          // Used existing tree
-          LOGGER.info("Reused existing star-tree: {}", builderConfig.toString());
-          reusedStarTrees++;
-        } else {
-          try (SingleTreeBuilder singleTreeBuilder = getSingleTreeBuilder(builderConfig, starTreeIndexDir, _segment,
-              metadataProperties, _buildMode)) {
-            singleTreeBuilder.build();
+      try {
+        for (int i = 0; i < numStarTrees; i++) {
+          StarTreeV2BuilderConfig builderConfig = _builderConfigs.get(i);
+          Configuration metadataProperties = _metadataProperties.subset(MetadataKey.getStarTreePrefix(i));
+          if (_separator != null && handleExistingStarTreeAddition(starTreeIndexDir, metadataProperties,
+              builderConfig)) {
+            // Used existing tree
+            LOGGER.info("Reused existing star-tree: {}", builderConfig.toString());
+            reusedStarTrees++;
+          } else {
+            try (SingleTreeBuilder singleTreeBuilder = getSingleTreeBuilder(builderConfig, starTreeIndexDir, _segment,
+                metadataProperties, _buildMode)) {
+              singleTreeBuilder.build();
+            }
           }
+          indexMaps.add(indexCombiner.combine(builderConfig, starTreeIndexDir));
+        }
+      } catch (Exception e) {
+        // Clean-up some of the files created before throwing exception back to caller
+        // No need to undo changes to _metadataProperties as changes weren't saved to the file
+        LOGGER.error("Failed to build star-trees, cleaning up the index file and temp directory if they exist");
+        _starTreeCreationFailed = true;
+        if (starTreeV2IndexFile.exists()) {
+          FileUtils.forceDelete(starTreeV2IndexFile);
+        }
+        if (starTreeIndexDir.exists()) {
+          FileUtils.forceDelete(starTreeIndexDir);
         }
-        indexMaps.add(indexCombiner.combine(builderConfig, starTreeIndexDir));
+        _metadataProperties.clearProperty(MetadataKey.STAR_TREE_COUNT);
+        throw e;
       }

Review Comment:
   Just so I understand correctly - before these changes, any failure here could leave the startree index file in an inconsistent state?



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {

Review Comment:
   This looks a little strange - can we move the outer try to after the new try block (i.e., just enclosing the separator tmp directory deletion step)?



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/startree/v2/builder/MultipleTreesBuilder.java:
##########
@@ -243,6 +269,28 @@ private static SingleTreeBuilder getSingleTreeBuilder(StarTreeV2BuilderConfig bu
   public void close() {
     if (_separatorTempDir != null) {
       try {
+        try {
+          if (_starTreeCreationFailed) {
+            LOGGER.error("Star-tree index creation failed, trying to reset the older star-tree index and metadata");
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_FILE_NAME),
+                _segmentDirectory,
+                false);
+            FileUtils.moveFileToDirectory(new File(_separatorTempDir, StarTreeV2Constants.INDEX_MAP_FILE_NAME),
+                _segmentDirectory, false);
+
+            // Copy back the older star-tree related metadata
+            Iterator<String> keys = _existingStarTreeMetadata.getKeys();
+            while (keys.hasNext()) {
+              String key = keys.next();
+              Object value = _existingStarTreeMetadata.getProperty(key);
+              _metadataProperties.addProperty(MetadataKey.STAR_TREE_PREFIX + key, value);
+            }
+            CommonsConfigurationUtils.saveToFile(_metadataProperties,
+                new File(_segmentDirectory, V1Constants.MetadataKeys.METADATA_FILE_NAME));
+          }
+        } catch (Exception e) {
+          LOGGER.error("Could not reset the star-tree index state to the previous one", e);

Review Comment:
   Should we cleanup everything in this case instead of leaving things in a potentially inconsistent state?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "somandal (via GitHub)" <gi...@apache.org>.
somandal commented on code in PR #17028:
URL: https://github.com/apache/pinot/pull/17028#discussion_r2445860066


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -542,6 +541,10 @@ private void buildStarTreeV2IfNecessary(File indexDir)
           MultipleTreesBuilder builder = new MultipleTreesBuilder(starTreeIndexConfigs, enableDefaultStarTree, indexDir,
               buildMode)) {
         builder.build();
+      } catch (Exception e) {
+        String tableNameWithType = _config.getTableConfig().getTableName();
+        LOGGER.error("Failed to build star-tree index for table: {}, skipping", tableNameWithType, e);
+        MinionMetrics.get().addMeteredTableValue(tableNameWithType, MinionMeter.STAR_TREE_INDEX_BUILD_FAILURES, 1);

Review Comment:
   let me explore passing in the context from both code paths



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Don't fail segment build if star-tree index build fails, skip it instead and add rollback support for existing star-tree indexes [pinot]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #17028:
URL: https://github.com/apache/pinot/pull/17028#issuecomment-3412951734

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/17028?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :x: Patch coverage is `46.93878%` with `26 lines` in your changes missing coverage. Please review.
   :white_check_mark: Project coverage is 56.28%. Comparing base ([`9d32f37`](https://app.codecov.io/gh/apache/pinot/commit/9d32f376d839330fbec98d48738ee0d82de122e7?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4cde66d`](https://app.codecov.io/gh/apache/pinot/commit/4cde66d098ae16bb280d62b6019c0b6b0ccb3319?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/pinot/pull/17028?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [...ocal/startree/v2/builder/MultipleTreesBuilder.java](https://app.codecov.io/gh/apache/pinot/pull/17028?src=pr&el=tree&filepath=pinot-segment-local%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsegment%2Flocal%2Fstartree%2Fv2%2Fbuilder%2FMultipleTreesBuilder.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zdGFydHJlZS92Mi9idWlsZGVyL011bHRpcGxlVHJlZXNCdWlsZGVyLmphdmE=) | 42.10% | [19 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/17028?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [...ocal/segment/index/loader/SegmentPreProcessor.java](https://app.codecov.io/gh/apache/pinot/pull/17028?src=pr&el=tree&filepath=pinot-segment-local%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fsegment%2Flocal%2Fsegment%2Findex%2Floader%2FSegmentPreProcessor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zZWdtZW50L2luZGV4L2xvYWRlci9TZWdtZW50UHJlUHJvY2Vzc29yLmphdmE=) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/17028?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   > :exclamation:  There is a different number of reports uploaded between BASE (9d32f37) and HEAD (4cde66d). Click for more details.
   > 
   > <details><summary>HEAD has 36 uploads less than BASE</summary>
   >
   >| Flag | BASE (9d32f37) | HEAD (4cde66d) |
   >|------|------|------|
   >|java-21|5|1|
   >|unittests1|2|1|
   >|unittests|4|1|
   >|temurin|10|1|
   >|java-11|5|0|
   >|unittests2|2|0|
   >|integration|6|0|
   >|integration2|2|0|
   >|integration1|2|0|
   >|custom-integration1|2|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #17028      +/-   ##
   ============================================
   - Coverage     63.51%   56.28%   -7.24%     
   + Complexity     1419      701     -718     
   ============================================
     Files          3082     2416     -666     
     Lines        181844   134795   -47049     
     Branches      27916    21534    -6382     
   ============================================
   - Hits         115500    75864   -39636     
   + Misses        57458    52597    -4861     
   + Partials       8886     6334    -2552     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration1](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration2](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-11](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-21](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.28% <46.93%> (-7.17%)` | :arrow_down: |
   | [temurin](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.28% <46.93%> (-7.24%)` | :arrow_down: |
   | [unittests](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.28% <46.93%> (-7.24%)` | :arrow_down: |
   | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.28% <46.93%> (-0.05%)` | :arrow_down: |
   | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/17028/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/17028?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org