You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "cryptoe (via GitHub)" <gi...@apache.org> on 2023/02/10 12:56:20 UTC

[GitHub] [druid] cryptoe opened a new pull request, #13790: Better logging to MSQ worker task

cryptoe opened a new pull request, #13790:
URL: https://github.com/apache/druid/pull/13790

   <!-- Thanks for trying to help us make Apache Druid be the best it can be! Please fill out as much of the following information as is possible (where relevant, and remove it when irrelevant) to help make the intention and scope of this PR clear in order to ease review. -->
   
   <!-- Please read the doc for contribution (https://github.com/apache/druid/blob/master/CONTRIBUTING.md) before making this PR. Also, once you open a PR, please _avoid using force pushes and rebasing_ since these make it difficult for reviewers to see what you've changed in response to their reviews. See [the 'If your pull request shows conflicts with master' section](https://github.com/apache/druid/blob/master/CONTRIBUTING.md#if-your-pull-request-shows-conflicts-with-master) for more details. -->
   
   Adding more logs statements to the worker implementation which makes it easier to debug.
   
   
   
   <!--
   In each section, please describe design decisions made, including:
    - Choice of algorithms
    - Behavioral aspects. What configuration values are acceptable? How are corner cases and error conditions handled, such as when there are insufficient resources?
    - Class organization and design (how the logic is split between classes, inheritance, composition, design patterns)
    - Method organization and design (how the logic is split between methods, parameters and return types)
    - Naming (class, method, API, configuration, HTTP endpoint, names of emitted metrics)
   -->
   
   
   <!-- It's good to describe an alternative design (or mention an alternative name) for every design (or naming) decision point and compare the alternatives with the designs that you've implemented (or the names you've chosen) to highlight the advantages of the chosen designs and names. -->
   
   <!-- If there was a discussion of the design of the feature implemented in this PR elsewhere (e. g. a "Proposal" issue, any other issue, or a thread in the development mailing list), link to that discussion from this PR description and explain what have changed in your final design compared to your original proposal or the consensus version in the end of the discussion. If something hasn't changed since the original discussion, you can omit a detailed discussion of those aspects of the design here, perhaps apart from brief mentioning for the sake of readability of this PR description. -->
   
   <!-- Some of the aspects mentioned above may be omitted for simple and small changes. -->
   
   <hr>
   
   ##### Key changed/added classes in this PR
    * `WorkerImpl`
    * `WorkerStageKernel`
   
   
   <hr>
   
   <!-- Check the items by putting "x" in the brackets for the done things. Not all of these items apply to every PR. Remove the items which are not done or not relevant to the PR. None of the items from the checklist below are strictly necessary, but it would be very helpful if you at least self-review the PR. -->
   
   This PR has:
   
   - [x] been self-reviewed.
   - [ ] been tested in a test Druid cluster.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


[GitHub] [druid] cryptoe commented on a diff in pull request #13790: Better logging for MSQ worker task

Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe commented on code in PR #13790:
URL: https://github.com/apache/druid/pull/13790#discussion_r1105285582


##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -582,12 +588,14 @@ public void postCleanupStage(final StageId stageId)
   @Override
   public void postFinish()
   {
+    log.info("Finish received");
     kernelManipulationQueue.add(KernelHolder::setDone);
   }
 
   @Override
   public ClusterByStatisticsSnapshot fetchStatisticsSnapshot(StageId stageId)
   {
+    log.info("Fetching statistics for stage:[%d]", stageId.getStageNumber());

Review Comment:
   Each worker can only have one taskID and each log is generally task log is generally pulled from the druid console. 
   In the case, people are using popular log agg tools, then they should add a dimension taskId to the "log line" via the constructs provided by the tool, "looking at file name in case of druid to figure out the taskId" is one such approach.
   Hence I am not pushing taskId everywhere which just bloats the message. 
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


[GitHub] [druid] cryptoe commented on pull request #13790: Better logging for MSQ worker task

Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe commented on PR #13790:
URL: https://github.com/apache/druid/pull/13790#issuecomment-1438545445

   @paul-rogers This PR is waiting on review/approval on you. Let me know if there are more things that are needed to be addressed. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


[GitHub] [druid] cryptoe commented on a diff in pull request #13790: Better logging for MSQ worker task

Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe commented on code in PR #13790:
URL: https://github.com/apache/druid/pull/13790#discussion_r1105287001


##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -520,6 +525,7 @@ public InputStream readChannel(
   @Override
   public void postWorkOrder(final WorkOrder workOrder)
   {
+    log.info("Got work order for stage[%d]", workOrder.getStageNumber());

Review Comment:
   In MSQ, in most places we are using this `[]` convention. I would hate to change it partially. Hence, if we decide to move away from using no brackets for numbers, then we should do it as part of another pr which touches all MSQ code. 
    



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


[GitHub] [druid] paul-rogers commented on a diff in pull request #13790: Better logging for MSQ worker task

Posted by "paul-rogers (via GitHub)" <gi...@apache.org>.
paul-rogers commented on code in PR #13790:
URL: https://github.com/apache/druid/pull/13790#discussion_r1103876331


##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -331,8 +331,12 @@ public Optional<MSQErrorReport> runTask(final Closer closer) throws Exception
         final StageDefinition stageDefinition = kernel.getStageDefinition();
 
         if (kernel.getPhase() == WorkerStagePhase.NEW) {
-          log.debug("New work order: %s", context.jsonMapper().writeValueAsString(kernel.getWorkOrder()));
 
+          if (log.isDebugEnabled()) {
+            log.debug("Processing work order: %s", context.jsonMapper().writeValueAsString(kernel.getWorkOrder()));
+          } else {
+            log.info("Processing work order for stage[%d]", kernel.getStageDefinition().getStageNumber());

Review Comment:
   Would recommend first emitting the info-level general comment. If debug is enabled, emit the work order as well. Else when searching for this message, one has to know what log level was set to know which message to search for.



##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -520,6 +525,7 @@ public InputStream readChannel(
   @Override
   public void postWorkOrder(final WorkOrder workOrder)
   {
+    log.info("Got work order for stage[%d]", workOrder.getStageNumber());

Review Comment:
   Nit: better to use standard English spacing: a space between "stage" and the bracket: `stage [%d]`.
   
   The brackets are really only needed for items that can contain spaces, so we know what is the message and what is the value. But, this is a number. so `state %d`.



##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -603,6 +611,11 @@ public ClusterByStatisticsSnapshot fetchStatisticsSnapshot(StageId stageId)
   @Override
   public ClusterByStatisticsSnapshot fetchStatisticsSnapshotForTimeChunk(StageId stageId, long timeChunk)
   {
+    log.debug(
+        "Fetching statistics for stage:[%d] with timechunk:[%d]",

Review Comment:
   Edit as above `stage %d, time chunk %d`



##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -444,6 +448,7 @@ public Optional<MSQErrorReport> runTask(final Closer closer) throws Exception
   @Override
   public void stopGracefully()
   {
+    log.info("Stopping gracefully");

Review Comment:
   Does the logger emit the task ID? If not, it might help to say what is stopping gracefully.



##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -582,12 +588,14 @@ public void postCleanupStage(final StageId stageId)
   @Override
   public void postFinish()
   {
+    log.info("Finish received");
     kernelManipulationQueue.add(KernelHolder::setDone);
   }
 
   @Override
   public ClusterByStatisticsSnapshot fetchStatisticsSnapshot(StageId stageId)
   {
+    log.info("Fetching statistics for stage:[%d]", stageId.getStageNumber());

Review Comment:
   As above. Inconsistent use of colon. No colon is needed here: `stage %d`.
   
   Does the logger automagically fill in the task ID? If not, would be good to include that so we can find events for a specific task.



##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/kernel/worker/WorkerStageKernel.java:
##########
@@ -210,6 +212,12 @@ private void assertPreshuffleStatisticsNeeded()
   private void transitionTo(final WorkerStagePhase newPhase)
   {
     if (newPhase.canTransitionFrom(phase)) {
+      log.info(

Review Comment:
   Same comments as above.



##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/WorkerImpl.java:
##########
@@ -582,12 +588,14 @@ public void postCleanupStage(final StageId stageId)
   @Override
   public void postFinish()
   {
+    log.info("Finish received");

Review Comment:
   Finish received for what task?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


[GitHub] [druid] cryptoe merged pull request #13790: Better logging for MSQ worker task

Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe merged PR #13790:
URL: https://github.com/apache/druid/pull/13790


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org