[GitHub] [pinot] jackjlli commented on a diff in pull request #8846: Reduce the heap memory usage for segment creation

[GitBox <gi...@apache.org>]

jackjlli commented on code in PR #8846:
URL: https://github.com/apache/pinot/pull/8846#discussion_r891664722


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/stats/BigDecimalColumnPreIndexStatsCollector.java:
##########
@@ -113,8 +111,11 @@ public int getCardinality() {
 
   @Override
   public void seal() {
-    _sortedValues = _values.toArray(new BigDecimal[0]);
-    Arrays.sort(_sortedValues);
-    _sealed = true;
+    if (!_sealed) {
+      _sortedValues = _values.toArray(new BigDecimal[0]);
+      _values = null;

Review Comment:
   Should we consider invoking `_values.clear()` method before setting it `null`? Same for all the other stats collectors.



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/stats/LongColumnPreIndexStatsCollector.java:
##########
@@ -57,10 +56,10 @@ public void collect(Object entry) {
     }
   }
 
-  void addressSorted(long entry) {
-    if (_isSorted) {
+  private void addressSorted(long entry) {

Review Comment:
   I saw some of the `addressSorted` method use `private` but some don't. It'd be good to unify them.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org