[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #4714: Remove the segment missing error message when acquiring segments

[GitBox <gi...@apache.org>]

jackjlli commented on a change in pull request #4714: Remove the segment missing error message when acquiring segments
URL: https://github.com/apache/incubator-pinot/pull/4714#discussion_r335824226
 
 

 ##########
 File path: pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java
 ##########
 @@ -126,8 +126,25 @@ public DataTable processQuery(ServerQueryRequest queryRequest, ExecutorService e
 
     TableDataManager tableDataManager = _instanceDataManager.getTableDataManager(tableNameWithType);
     Preconditions.checkState(tableDataManager != null, "Failed to find data manager for table: " + tableNameWithType);
-    List<SegmentDataManager> segmentDataManagers = tableDataManager.acquireSegments(queryRequest.getSegmentsToQuery());
-    int numSegmentsQueried = segmentDataManagers.size();
+
+    List<String> segmentsToQuery = queryRequest.getSegmentsToQuery();
+    List<SegmentDataManager> segmentDataManagers = tableDataManager.acquireSegments(segmentsToQuery);
+
+    // When segment is removed from the IdealState:
+    // 1. Controller schedules a state transition to server to turn segment OFFLINE
+    // 2. Server gets the state transition, removes the segment data manager and update its CurrentState
+    // 3. Controller gathers the CurrentState and update the ExternalView
+    // 4. Broker watches ExternalView change and updates the routing table to stop querying the segment
+    //
+    // After step 2 but before step 4, segment will be missing on server side
+    // TODO: Change broker to watch both IdealState and ExternalView to not query the removed segments
+    int numSegmentsQueried = segmentsToQuery.size();
+    int numSegmentsAcquired = segmentDataManagers.size();
+    if (numSegmentsQueried > numSegmentsAcquired) {
+      _serverMetrics.addMeteredTableValue(tableNameWithType, ServerMeter.NUM_MISSING_SEGMENTS,
 
 Review comment:
   Should we log the missing segment error message here?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org