[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10927: [POC] improve disk read for partial upsert handler

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on code in PR #10927:
URL: https://github.com/apache/pinot/pull/10927#discussion_r1235963196


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java:
##########
@@ -243,6 +249,42 @@ protected GenericRow doUpdateRecord(GenericRow record, RecordInfo recordInfo) {
     }
   }
 
+  protected GenericRow doOverwriteUpdateRecord(GenericRow record, RecordInfo recordInfo) {

Review Comment:
   We can always use this algorithm (loop over each field, only read value when required)



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/upsert/ConcurrentMapPartitionUpsertMetadataManager.java:
##########
@@ -219,6 +221,10 @@ protected void doAddRecord(MutableSegment segment, RecordInfo recordInfo) {
   @Override
   protected GenericRow doUpdateRecord(GenericRow record, RecordInfo recordInfo) {
     assert _partialUpsertHandler != null;
+    // For Overwrite merger, it respects the incoming values unless it's null. We can use this to optimize disk read.
+    if (UpsertConfig.Strategy.OVERWRITE.equals(_partialUpsertHandler.getDefaultPartialUpsertMerger())) {

Review Comment:
   Are you comparing a merger with an enum?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org