[GitHub] [pinot] zhtaoxiang commented on a diff in pull request #9457: track progress from within segment processor framework

[GitBox <gi...@apache.org>]

zhtaoxiang commented on code in PR #9457:
URL: https://github.com/apache/pinot/pull/9457#discussion_r980460488


##########
pinot-core/src/main/java/org/apache/pinot/core/segment/processing/mapper/SegmentMapper.java:
##########
@@ -105,8 +108,12 @@ public SegmentMapper(List<RecordReader> recordReaders, SegmentProcessorConfig pr
    */
   public Map<String, GenericRowFileManager> map()
       throws Exception {
+    Consumer<Object> observer = _processorConfig.getProgressObserver();
+    int totalCount = _recordReaders.size();
+    int count = 1;
     GenericRow reuse = new GenericRow();
     for (RecordReader recordReader : _recordReaders) {
+      observer.accept(String.format("Doing map phase on data from RecordReader: %d/%d", count++, totalCount));

Review Comment:
   It seems to me that it's hard to get the meaning of those two numbers. Maybe change it to `RecordReader %d (%d in total)` or something similar?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org