[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7906: Add global strategy for partial upsert

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on a change in pull request #7906:
URL: https://github.com/apache/pinot/pull/7906#discussion_r770164262



##########
File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/UpsertConfig.java
##########
@@ -53,21 +53,27 @@
   @JsonPropertyDescription("Partial update strategies.")
   private final Map<String, Strategy> _partialUpsertStrategies;
 
+  @JsonPropertyDescription("global upsert strategy")
+  private final Strategy _globalUpsertStrategy;
+
   @JsonPropertyDescription("Column for upsert comparison, default to time column")
   private final String _comparisonColumn;
 
   @JsonCreator
   public UpsertConfig(@JsonProperty(value = "mode", required = true) Mode mode,
       @JsonProperty("partialUpsertStrategies") @Nullable Map<String, Strategy> partialUpsertStrategies,
+      @JsonProperty("globalUpsertStrategy") @Nullable Strategy globalUpsertStrategy,
       @JsonProperty("comparisonColumn") @Nullable String comparisonColumn,
       @JsonProperty("hashFunction") @Nullable HashFunction hashFunction) {
     Preconditions.checkArgument(mode != null, "Upsert mode must be configured");
     _mode = mode;
 
     if (mode == Mode.PARTIAL) {
       _partialUpsertStrategies = partialUpsertStrategies != null ? partialUpsertStrategies : new HashMap<>();
+      _globalUpsertStrategy = globalUpsertStrategy != null ? globalUpsertStrategy : Strategy.OVERWRITE;
     } else {
       _partialUpsertStrategies = null;
+      _globalUpsertStrategy = Strategy.OVERWRITE;

Review comment:
       Probably set to `null` for full upsert?

##########
File path: pinot-common/src/test/java/org/apache/pinot/common/utils/config/TableConfigSerDeTest.java
##########
@@ -251,7 +249,8 @@ public void testSerDe()
     {
       // with upsert config
       UpsertConfig upsertConfig =
-          new UpsertConfig(UpsertConfig.Mode.FULL, null, "comparison", UpsertConfig.HashFunction.NONE);
+          new UpsertConfig(UpsertConfig.Mode.FULL, null, UpsertConfig.Strategy.OVERWRITE, "comparison",

Review comment:
       For `FULL` upsert, pass in `null` as the default strategy as that does not apply? Same for other places

##########
File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/UpsertConfig.java
##########
@@ -53,21 +53,27 @@
   @JsonPropertyDescription("Partial update strategies.")
   private final Map<String, Strategy> _partialUpsertStrategies;
 
+  @JsonPropertyDescription("global upsert strategy")
+  private final Strategy _globalUpsertStrategy;
+
   @JsonPropertyDescription("Column for upsert comparison, default to time column")
   private final String _comparisonColumn;
 
   @JsonCreator
   public UpsertConfig(@JsonProperty(value = "mode", required = true) Mode mode,
       @JsonProperty("partialUpsertStrategies") @Nullable Map<String, Strategy> partialUpsertStrategies,
+      @JsonProperty("globalUpsertStrategy") @Nullable Strategy globalUpsertStrategy,
       @JsonProperty("comparisonColumn") @Nullable String comparisonColumn,
       @JsonProperty("hashFunction") @Nullable HashFunction hashFunction) {
     Preconditions.checkArgument(mode != null, "Upsert mode must be configured");
     _mode = mode;
 
     if (mode == Mode.PARTIAL) {
       _partialUpsertStrategies = partialUpsertStrategies != null ? partialUpsertStrategies : new HashMap<>();
+      _globalUpsertStrategy = globalUpsertStrategy != null ? globalUpsertStrategy : Strategy.OVERWRITE;

Review comment:
       This is changing the existing behavior. I do agree `OVERWRITE` makes more sense to partial upsert, but not sure if we want to introduce backward incompatibility here

##########
File path: pinot-spi/src/main/java/org/apache/pinot/spi/config/table/UpsertConfig.java
##########
@@ -53,21 +53,27 @@
   @JsonPropertyDescription("Partial update strategies.")
   private final Map<String, Strategy> _partialUpsertStrategies;
 
+  @JsonPropertyDescription("global upsert strategy")
+  private final Strategy _globalUpsertStrategy;

Review comment:
       Probably more clear if rename to `_defaultPartialUpsertStrategy`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org