You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/11/09 23:04:20 UTC
[GitHub] [pinot] klsince commented on a diff in pull request #9309: Introduce Segment AssignmentStrategy Interface
klsince commented on code in PR #9309:
URL: https://github.com/apache/pinot/pull/9309#discussion_r1018496195
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/RealtimeSegmentAssignment.java:
##########
@@ -182,9 +185,21 @@ public Map<String, Map<String, String>> rebalanceTable(Map<String, Map<String, S
boolean bootstrap =
config.getBoolean(RebalanceConfigConstants.BOOTSTRAP, RebalanceConfigConstants.DEFAULT_BOOTSTRAP);
+ // TODO: remove this check after we also refactor consuming segments assignment strategy
+ // See https://github.com/apache/pinot/issues/9047
+ SegmentAssignmentStrategy segmentAssignmentStrategy = null;
+ if (completedInstancePartitions != null) {
Review Comment:
hi @GSharayu , I'm testing locally with RealtimeQuickStart but customized a little bit, and I came into NPE issue like below. The tableConfig has some tierConfigs, and SegmentRelocator is enabled to run periodically to move segments around based on data age as set in tierConfigs. As suggested above, I feel it'd be safer to init segmentAssignmentStrategy with a default policy.
```
2022/11/09 14:43:09.474 INFO [RealtimeSegmentAssignment] [async-task-thread-1] Rebalancing tier: myTier for table: airlineStats_REALTIME with bootstrap: false, instance partitions: {"instancePartitionsName":"airlineStats
_myTier","partitionToInstancesMap":{"0_0":["Server_192.168.0.106_7000"]}}
2022/11/09 14:43:09.475 WARN [TableRebalancer] [async-task-thread-1] Caught exception while calculating target assignment for table: airlineStats_REALTIME, aborting the rebalance
java.lang.NullPointerException: null
at org.apache.pinot.controller.helix.core.assignment.segment.BaseSegmentAssignment.reassignSegments(BaseSegmentAssignment.java:158) ~[classes/:?]
at org.apache.pinot.controller.helix.core.assignment.segment.BaseSegmentAssignment.rebalanceTiers(BaseSegmentAssignment.java:129) ~[classes/:?]
at org.apache.pinot.controller.helix.core.assignment.segment.RealtimeSegmentAssignment.rebalanceTable(RealtimeSegmentAssignment.java:201) ~[classes/:?]
at org.apache.pinot.controller.helix.core.rebalance.TableRebalancer.rebalance(TableRebalancer.java:219) ~[classes/:?]
at org.apache.pinot.controller.helix.core.relocation.SegmentRelocator.lambda$processTable$0(SegmentRelocator.java:137) ~[classes/:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
```
cc @npawar for awareness ^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org