[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #9851: Merge new columns in existing record with default merge strategy

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on code in PR #9851:
URL: https://github.com/apache/pinot/pull/9851#discussion_r1034162841


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/PartialUpsertHandler.java:
##########
@@ -33,18 +34,25 @@
 public class PartialUpsertHandler {
   // _column2Mergers maintains the mapping of merge strategies per columns.
   private final Map<String, PartialUpsertMerger> _column2Mergers = new HashMap<>();
+  private final UpsertConfig.Strategy _defaultPartialUpsertStrategy;

Review Comment:
   Store a default merger (`PartialUpsertMergerFactory.getMerger(defaultPartialUpsertStrategy)`) instead of the strategy to avoid per record creation of the merger



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/PartialUpsertHandler.java:
##########
@@ -33,18 +34,25 @@
 public class PartialUpsertHandler {
   // _column2Mergers maintains the mapping of merge strategies per columns.
   private final Map<String, PartialUpsertMerger> _column2Mergers = new HashMap<>();
+  private final UpsertConfig.Strategy _defaultPartialUpsertStrategy;
+  private final String _comparisonColumn;
+  private final List<String> _primaryKeyColumns;
 
   public PartialUpsertHandler(Schema schema, Map<String, UpsertConfig.Strategy> partialUpsertStrategies,
       UpsertConfig.Strategy defaultPartialUpsertStrategy, String comparisonColumn) {
+    _defaultPartialUpsertStrategy = defaultPartialUpsertStrategy;
+    _comparisonColumn = comparisonColumn;
+    _primaryKeyColumns = schema.getPrimaryKeyColumns();
+
     for (Map.Entry<String, UpsertConfig.Strategy> entry : partialUpsertStrategies.entrySet()) {
       _column2Mergers.put(entry.getKey(), PartialUpsertMergerFactory.getMerger(entry.getValue()));
     }
     // For all physical columns (including date time columns) except for primary key columns and comparison column.
     // If no comparison column is configured, use main time column as the comparison time.
     for (String columnName : schema.getPhysicalColumnNames()) {
-      if (!schema.getPrimaryKeyColumns().contains(columnName) && !_column2Mergers.containsKey(columnName)
-          && !comparisonColumn.equals(columnName)) {
-        _column2Mergers.put(columnName, PartialUpsertMergerFactory.getMerger(defaultPartialUpsertStrategy));
+      if (!_primaryKeyColumns.contains(columnName) && !_column2Mergers.containsKey(columnName)
+          && !_comparisonColumn.equals(columnName)) {
+        _column2Mergers.put(columnName, PartialUpsertMergerFactory.getMerger(_defaultPartialUpsertStrategy));
       }
     }

Review Comment:
   This part is no longer needed. We apply default strategy for all columns not explicitly configured



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org