[GitHub] [pinot] sajjad-moradi commented on a diff in pull request #8877: Emit metrics if there's no consuming segment for a partition

[GitBox <gi...@apache.org>]

sajjad-moradi commented on code in PR #8877:
URL: https://github.com/apache/pinot/pull/8877#discussion_r896232278


##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/realtime/PinotLLCRealtimeSegmentManager.java:
##########
@@ -874,6 +871,8 @@ public void ensureAllPartitionsConsuming(TableConfig tableConfig, PartitionLevel
     HelixHelper.updateIdealState(_helixManager, realtimeTableName, idealState -> {
       assert idealState != null;
       if (idealState.isEnabled()) {
+        new MissingConsumingSegmentFinder(realtimeTableName, _propertyStore, _controllerMetrics)

Review Comment:
   The reason it was at the beginning of the validation manager was to avoid stream connection issues. One root cause for not having consuming segments in ideal state is that stream client can throw exception during stream metadata for creating new consuming segment either in 1) segment commit or in 2) validation manager for detecting new partition groups. By just looking at ideal state, we can detect for how long we haven't had a consuming segment for a partition.
   
   Anyway I moved this SegmentStatusChecker job per Subbu's suggestion, so it runs more frequently.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org