[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8485: Manual tracing

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on code in PR #8485:
URL: https://github.com/apache/pinot/pull/8485#discussion_r848842171


##########
pinot-core/src/main/java/org/apache/pinot/core/common/DataFetcher.java:
##########
@@ -415,10 +421,12 @@ private ForwardIndexReaderContext getReaderContext() {
     }
 
     void readDictIds(int[] docIds, int length, int[] dictIdBuffer) {
+      Tracing.activeRecording().setInputDataType(_dataType, _singleValue);

Review Comment:
   Should we check if the recording is enabled before setting the data type? In case some implementations do implement this method, but have `_enabled` set dynamically



##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java:
##########
@@ -131,6 +131,11 @@ public DataTable processQuery(ServerQueryRequest queryRequest, ExecutorService e
     QueryContext queryContext = queryRequest.getQueryContext();
     LOGGER.debug("Incoming request Id: {}, query: {}", requestId, queryContext);
 
+    boolean enableTrace = queryRequest.isEnableTrace();
+    if (enableTrace) {
+      Tracing.getTracer().register(requestId);

Review Comment:
   (MAJOR) Register trace here can cause leak because there is no guarantee trace is unregistered



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/LuceneFSTIndexReader.java:
##########
@@ -61,19 +65,27 @@ public MutableRoaringBitmap getDocIds(String searchQuery) {
 
   @Override
   public ImmutableRoaringBitmap getDictIds(String searchQuery) {
-    try {
+    try (InvocationScope span = Tracing.getTracer().createScope(LuceneFSTIndexReader.class)) {
       MutableRoaringBitmap dictIds = new MutableRoaringBitmap();
       List<Long> matchingIds = RegexpMatcher.regexMatch(searchQuery, _readFST);
       for (Long matchingId : matchingIds) {
         dictIds.add(matchingId.intValue());
       }
+      record(span, matchingIds.size());

Review Comment:
   This is matching values instead of matching documents



##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/realtime/impl/json/MutableJsonIndexImpl.java:
##########
@@ -291,6 +297,19 @@ private RoaringBitmap getMatchingFlattenedDocIds(Predicate predicate) {
     }
   }
 
+  private void record(ImmutableRoaringBitmap bitmap, FilterContext filter) {
+    InvocationRecording recording = Tracing.activeRecording();
+    if (recording.isEnabled()) {
+      recording.setColumnCardinality(_postingListMap.size());
+      recording.setFilter(FilterType.INDEX, describeJsonPredicate(filter.getPredicate()));
+      recording.setNumDocsMatchingAfterFilter(bitmap.getCardinality());
+    }
+  }
+
+  private static String describeJsonPredicate(Predicate predicate) {
+    return predicate == null ? "?" : predicate.getLhs().getIdentifier() + ":" + predicate.getType() + "?";

Review Comment:
   Is this complete? This will return something like `myCol:JSON_MATCH?`



##########
pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPruner.java:
##########
@@ -41,9 +43,12 @@ default List<IndexSegment> prune(List<IndexSegment> segments, QueryContext query
       return segments;
     }
     List<IndexSegment> selectedSegments = new ArrayList<>(segments.size());
-    for (IndexSegment segment : segments) {
-      if (!prune(segment, query)) {
-        selectedSegments.add(segment);
+    try (InvocationScope scope = Tracing.getTracer().createScope(getClass())) {

Review Comment:
   Suggest moving this trace into the `SegmentPrunerService` because this whole method might be overridden



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org