[GitHub] [pinot] claudeyj commented on a diff in pull request #10445: Replace HashMap with LinkedHashMap to avoid flaky tests

["claudeyj (via GitHub)" <gi...@apache.org>]

claudeyj commented on code in PR #10445:
URL: https://github.com/apache/pinot/pull/10445#discussion_r1141624029


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentIndexCreationDriverImpl.java:
##########
@@ -175,7 +175,7 @@ public void init(SegmentGeneratorConfig config, SegmentCreationDataSource dataSo
 
     // Initialize index creation
     _segmentIndexCreationInfo = new SegmentIndexCreationInfo();
-    _indexCreationInfoMap = new HashMap<>();
+    _indexCreationInfoMap = new LinkedHashMap<>();

Review Comment:
   Thanks for your comment. 
   > Is the only goal to fix flaky tests?
   The main goal is to fix flaky test. But I also have concern that the non-determinism of CRC computing will affect the functional correctness for some corner cases. I can image the CRC verification could fail for the same segments in different environments, which is not desired.
   
   > HashMap and LinkedHashMap have different performance on different operations. Changing HashMap to LinkedHashMap may slow down the index creation. This may be a problem it the index creation is slowed down too much. Did you try to compare the performance and can you share the results if you did?
   I didn't do that myself but there are some developers doing profiling experiments to do general comparison, see https://stackoverflow.com/questions/12998568/hashmap-vs-linkedhashmap-performance-in-iteration-over-values. To my understanding the performance is similar but I'm not sure what's the actual scenario where the Map will take effect and what's the dominant factor for performance. If you can provide more hints I may run some profiling experiments to verify that.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org