You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/07/03 16:57:24 UTC
[GitHub] [rocketmq-externals] StyleTang opened a new pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
StyleTang opened a new pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744
## What is the purpose of the change
Add RocketMQ message trace timeline Graph UI.
We can find detail information here [#743](https://github.com/apache/rocketmq-externals/issues/743)
This is the initial version of Message Trace UI, any suggestions are welcome😀
## Brief changelog
This pull request is mainly for message trace UI, the console project have introduced Echarts before, so we keep using Echarts as our framework to implement the message trace timeline UI.
The timeline UI definition is as follows:
* xAxis is the duration( Since retry will make the duration very long, we show the first 10 seconds by default)
* yAxis shows the performance of each group (1 producer group and >=0 subscription group)
* The first line is the producer message trace information.
* Everything below the first line is consumer message trace information. (if there are multiple subscription groups then multiple lines in parallel)
* message trace node label is costTime (in ms), hover information contains
```
costTime
status
beginTimeStamp
endTimeStamp
clientHost
storeHost
retryTimes
```
* If the consumer trace status is fail, it will turn red, otherwise it will be green
The change log in this pull request are as follows:
- [x] Backend
- [x] Add viewMessageTraceGraph method (aggregate MessageTraceView list)
- [x] MessageTraceView add filed requestId and retryTimes
- [x] Add Unit Test MessageTraceServiceImplTest
- [x] Add lombok dependency
- [x] Frontend
- [x] Implement message trace graph UI
- [x] produce message trace
- [x] consume message trace
- [x] consumer retry message (error message display)
- [x] multiple subscriptions support
- [x] Upgrade echarts to 5.1.2
- [x] Message trace page can fetch message id from routeParams
## Verifying this change
### Test Cases
#### 1 Multiple subscription groups consume succeed
#### 2 Multiple subscription groups consume succeed (with retry)
### Unit Test
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.
- [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
- [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
- [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
- [ ] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test` to make sure integration-test pass.
- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
[[rocketmq-console] Adjust the order of task state management.](https://github.com/apache/rocketmq-externals/issues/702)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] vongosling commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
vongosling commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663741680
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/TraceNode.java
##########
@@ -0,0 +1,30 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
Need to update...
##########
File path: rocketmq-console/pom.xml
##########
@@ -264,6 +269,22 @@
<artifactId>findbugs-maven-plugin</artifactId>
<version>3.0.4</version>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.6.2</version>
+ <configuration>
+ <source>1.8</source>
Review comment:
You could remove configuration parts while just leave annotationProcessorPaths here. refer to the properties in the pom of RocketMQ.
##########
File path: rocketmq-console/src/test/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImplTest.java
##########
@@ -0,0 +1,137 @@
+package org.apache.rocketmq.console.service.impl;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import lombok.SneakyThrows;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.console.config.RMQConfigure;
+import org.apache.rocketmq.console.model.MessageTraceView;
+import org.apache.rocketmq.console.model.trace.MessageTraceGraph;
+import org.apache.rocketmq.console.util.JsonUtil;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.assertj.core.util.Lists;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.MockitoAnnotations;
+import org.mockito.junit.MockitoJUnitRunner;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
Review comment:
License ...
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/MessageTraceGraph.java
##########
@@ -0,0 +1,29 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
License should be before package parts.
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -47,8 +79,99 @@
messageTraceViews.addAll(messageTraceView);
}
return messageTraceViews;
- } catch (Exception err) {
+ }
+ catch (Exception err) {
Review comment:
Checkstyle will pass here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663834041
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -47,8 +79,99 @@
messageTraceViews.addAll(messageTraceView);
}
return messageTraceViews;
- } catch (Exception err) {
+ }
+ catch (Exception err) {
Review comment:
Done. (Yes, it passed the check)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r667088448
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -51,4 +83,94 @@
throw Throwables.propagate(err);
}
}
+
+ @Override
+ public MessageTraceGraph queryMessageTraceGraph(String key) {
+ List<MessageTraceView> messageTraceViews = queryMessageTraceKey(key);
+ return buildMessageTraceGraph(messageTraceViews);
+ }
+
+ private MessageTraceGraph buildMessageTraceGraph(List<MessageTraceView> messageTraceViews) {
+ MessageTraceGraph messageTraceGraph = new MessageTraceGraph();
+ messageTraceGraph.setMessageTraceViews(messageTraceViews);
+ if (CollectionUtils.isEmpty(messageTraceViews)) {
+ return messageTraceGraph;
+ }
+ ProducerNode producerNode = null;
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap = Maps.newHashMap();
+ for (MessageTraceView messageTraceView : messageTraceViews) {
+ switch (TraceType.valueOf(messageTraceView.getMsgType())) {
+ case Pub:
+ producerNode = buildMessageRoot(messageTraceView);
+ break;
+ case SubBefore:
+ case SubAfter:
+ putIntoMessageTraceViewGroupMap(messageTraceView, messageTraceViewGroupMap);
+ break;
+ default:
+ break;
+ }
+ }
+ messageTraceGraph.setProducerNode(producerNode);
+ messageTraceGraph.setSubscriptionNodeList(buildSubscriptionNodeList(messageTraceViewGroupMap));
+ return messageTraceGraph;
+ }
+
+ private List<SubscriptionNode> buildSubscriptionNodeList(
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ List<SubscriptionNode> subscriptionNodeList = new ArrayList<>(messageTraceViewGroupMap.size());
+ for (Map.Entry<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> groupTraceView : messageTraceViewGroupMap
+ .entrySet()) {
+ SubscriptionNode subscriptionNode = new SubscriptionNode();
+ subscriptionNode.setSubscriptionGroup(groupTraceView.getKey());
+ List<TraceNode> consumeNodeList = Lists.newArrayList();
+ subscriptionNode.setConsumeNodeList(consumeNodeList);
+ subscriptionNodeList.add(subscriptionNode);
+ for (Map.Entry<String, Pair<MessageTraceView, MessageTraceView>> requestIdTracePair : groupTraceView
+ .getValue().entrySet()) {
+ MessageTraceView subBeforeTrace = requestIdTracePair.getValue().getObject1();
+ MessageTraceView subAfterTrace = requestIdTracePair.getValue().getObject2();
+ TraceNode consumeNode = new TraceNode();
+ consumeNode.setRequestId(requestIdTracePair.getKey());
+ consumeNode.setStoreHost(subBeforeTrace.getStoreHost());
+ consumeNode.setClientHost(subBeforeTrace.getClientHost());
+ consumeNode.setRetryTimes(subBeforeTrace.getRetryTimes());
+ consumeNode.setBeginTimeStamp(subBeforeTrace.getTimeStamp());
+ consumeNode.setCostTime(subAfterTrace.getCostTime());
+ consumeNode.setEndTimeStamp(subAfterTrace.getTimeStamp());
+ consumeNode.setStatus(subAfterTrace.getStatus());
+ consumeNodeList.add(consumeNode);
+ }
+ }
+ return subscriptionNodeList;
+ }
+
+ private void putIntoMessageTraceViewGroupMap(MessageTraceView messageTraceView,
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ Map<String, Pair<MessageTraceView, MessageTraceView>> requestIdTraceMap = messageTraceViewGroupMap
+ .computeIfAbsent(messageTraceView.getGroupName(), (o) -> new HashMap<>(2));
Review comment:
@yuz10 @vongosling
Sorry for late response, busy with other things recently.
As **[yuz10](https://github.com/yuz10)** mentioned, the **timeStamp** and **consumeGroup** have been deleted by this commit.
It is fine for deleting SubAfter consumeGroup because we can find consume group from **SubBefore traceContext** by **messageId** and **requestId**
SubAfter timestamp means when consumeMessage(final List<MessageExt> msgs,
final ConsumeConcurrentlyContext context) is end, this value not equals to timestamp+costTime and we can't find it anywhere. What is the purpose of deleting timestamp here, can we add it back?
In my CR I already fixed the consumeGroup issue, but the end timestamp is calculate by timestamp+costTime, maybe it is not reasonable .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] vongosling commented on pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
vongosling commented on pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#issuecomment-876956034
@StyleTang would you like to continue to followup the feedback for reviewer suggestions :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r667816635
##########
File path: rocketmq-console/src/main/resources/static/src/messageTrace.js
##########
@@ -52,73 +61,293 @@ module.controller('messageTraceController', ['$scope', 'ngDialog', '$http','Noti
url: "message/queryMessageByTopicAndKey.query",
params: {
topic: $scope.selectedTopic,
- key:$scope.key
+ key: $scope.key
}
}).success(function (resp) {
if (resp.status == 0) {
console.log(resp);
$scope.queryMessageByTopicAndKeyResult = resp.data;
console.log($scope.queryMessageByTopicAndKeyResult);
- }else {
+ } else {
Notification.error({message: resp.errMsg, delay: 2000});
}
});
};
- $scope.queryMessageByMessageId = function (messageId,topic) {
+ $scope.queryMessageByMessageId = function (messageId, topic) {
$http({
method: "GET",
url: "messageTrace/viewMessage.query",
params: {
msgId: messageId,
- topic:topic
+ topic: topic
}
}).success(function (resp) {
if (resp.status == 0) {
console.log(resp);
$scope.queryMessageByMessageIdResult = resp.data;
console.log($scope.queryMessageByTopicAndKeyResult);
- }else {
+ } else {
Notification.error({message: resp.errMsg, delay: 2000});
}
});
};
- $scope.queryMessageTraceByMessageId = function (messageId,topic) {
+ $scope.queryMessageTraceByMessageId = function (messageId, topic) {
$http({
method: "GET",
- url: "messageTrace/viewMessageTraceDetail.query",
+ url: "messageTrace/viewMessageTraceGraph.query",
params: {
msgId: messageId,
- topic:topic
+ topic: topic
}
}).success(function (resp) {
if (resp.status == 0) {
console.log(resp);
ngDialog.open({
template: 'messageTraceDetailViewDialog',
controller: 'messageTraceDetailViewDialogController',
- data:resp.data
+ data: resp.data
});
- }else {
+ } else {
Notification.error({message: resp.errMsg, delay: 2000});
}
});
};
}]);
-module.controller('messageTraceDetailViewDialogController',['$scope', 'ngDialog', '$http','Notification', function ($scope, ngDialog, $http,Notification) {
+module.controller('messageTraceDetailViewDialogController', ['$scope', '$timeout', 'ngDialog', '$http', 'Notification', function ($scope, $timeout, ngDialog, $http, Notification) {
+ $scope.displayGraph = false;
+ $scope.graphButtonName = 'Show Graph';
+ $scope.displayMessageTraceGraph = function (messageTraceGraph) {
+ let dom = document.getElementById("messageTraceGraph");
+ $scope.messageTraceGraph = echarts.init(dom);
+ let option;
+ let data = [];
+ let dataZoomEnd = 100;
+ let startTime = Number.MAX_VALUE;
+ let endTime = 0;
+ let messageGroups = [];
+ if (messageTraceGraph.producerNode) {
+ startTime = +messageTraceGraph.producerNode.traceNode.beginTimeStamp;
+ endTime = +messageTraceGraph.producerNode.traceNode.endTimeStamp;
+ } else {
+ messageTraceGraph.subscriptionNodeList.forEach(subscriptionNode => {
+ subscriptionNode.consumeNodeList.forEach(consumeNode => {
+ startTime = Math.min(startTime, consumeNode.beginTimeStamp);
+ })
+ })
+ }
+
+ function buildNodeColor(traceNode, index) {
+ let nodeColor = SUCCESS_COLOR;
+ if (traceNode.transactionState) {
+ switch (traceNode.transactionState) {
+ case 'COMMIT_MESSAGE':
+ return TRANSACTION_COMMIT_COLOR;
+ case 'ROLLBACK_MESSAGE':
+ return TRANSACTION_ROLLBACK_COLOR;
+ case 'UNKNOW':
+ return TRANSACTION_UNKNOWN_COLOR;
+ default:
+ return ERROR_COLOR;
+ }
+ }
+ if (traceNode.status !== 'success') {
+ nodeColor = ERROR_COLOR;
+ }
+ if (index === messageGroups.length - 1) {
+ nodeColor = PRODUCER_COLOR;
+ }
+ return nodeColor;
+ }
+
+ function formatXAxisTime(value) {
+ let duration = Math.max(0, value - startTime);
+ if (duration < 1000)
+ return duration + 'ms';
+ duration /= 1000;
+ if (duration < 60)
+ return duration + 's';
+ duration /= 60;
+ if (duration < 60)
+ return duration + 'm';
+ duration /= 60;
+ return duration + 'h';
+ }
+
+ function buildTraceInfo(itemName, itemValue) {
+ if (itemValue) {
+ return `${itemName}: ${itemValue}<br />`
+ }
+ return "";
+ }
+
+ function formatNodeToolTip(params) {
+ let traceNode = params.data.traceData.traceNode;
+ return `
+ costTime: ${traceNode.costTime}ms<br />
+ status: ${traceNode.status}<br />
+ beginTimeStamp: ${new moment(traceNode.beginTimeStamp).format(TIME_FORMAT_PATTERN)}<br />
+ endTimeStamp: ${new moment(traceNode.endTimeStamp).format(TIME_FORMAT_PATTERN)}<br />
+ clientHost: ${traceNode.clientHost}<br />
+ storeHost: ${traceNode.storeHost}<br />
+ retryTimes: ${traceNode.retryTimes}<br />
+ ${buildTraceInfo('msgType', traceNode.msgType)}
+ ${buildTraceInfo('transactionId', traceNode.transactionId)}
+ ${buildTraceInfo('transactionState', traceNode.transactionState)}
+ ${buildTraceInfo('fromTransactionCheck', traceNode.fromTransactionCheck)}
+ `;
+ }
+
+ function addTraceData(traceNode, index) {
+ data.push({
+ value: [
+ index,
+ traceNode.beginTimeStamp,
+ traceNode.endTimeStamp,
+ traceNode.costTime
+ ],
+ itemStyle: {
+ normal: {
+ color: buildNodeColor(traceNode, index),
+ opacity: 1
+ }
+ },
+ traceData: {
+ traceNode: traceNode
+ }
+ });
+ endTime = Math.max(traceNode.endTimeStamp, endTime);
+ }
+
+ messageTraceGraph.subscriptionNodeList.forEach(item => {
+ messageGroups.push(item.subscriptionGroup)
+ })
+ messageTraceGraph.subscriptionNodeList.forEach((subscriptionNode, index) => {
+ subscriptionNode.consumeNodeList.forEach(traceNode => addTraceData(traceNode, index))
+ })
+ if (messageTraceGraph.producerNode) {
+ messageGroups.push(messageTraceGraph.producerNode.groupName)
+ let producerNodeIndex = messageGroups.length - 1;
+ addTraceData(messageTraceGraph.producerNode.traceNode, producerNodeIndex);
+ messageTraceGraph.producerNode.transactionNodeList.forEach(transactionNode => {
+ transactionNode.beginTimeStamp = Math.max(messageTraceGraph.producerNode.traceNode.endTimeStamp,
+ transactionNode.endTimeStamp - transactionCheckCostTime);
+ addTraceData(transactionNode, producerNodeIndex)
+ endTime = Math.max(endTime, transactionNode.endTimeStamp);
+ })
+ }
+
+ let totalDuration = endTime - startTime;
+ if (totalDuration > DEFAULT_DISPLAY_DURATION) {
+ dataZoomEnd = DEFAULT_DISPLAY_DURATION / totalDuration * 100
+ }
+
+ function renderItem(params, api) {
+ let messageGroup = api.value(0);
+ let start = api.coord([api.value(1), messageGroup]);
+ let end = api.coord([api.value(2), messageGroup]);
+ let height = api.size([0, 1])[1] * 0.6;
+
+ let rectShape = echarts.graphic.clipRectByRect({
+ x: start[0],
+ y: start[1] - height / 2,
+ width: end[0] - start[0],
Review comment:
Yes, you are right. If the the end timestamp is equals to the begin timestamp, the width will be 0, no trace node will show in the graph.
It is a good idea to use minimum width 1, I will fix it later, thanks for your suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] yuz10 commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
yuz10 commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r667496582
##########
File path: rocketmq-console/src/main/resources/static/src/messageTrace.js
##########
@@ -52,73 +61,293 @@ module.controller('messageTraceController', ['$scope', 'ngDialog', '$http','Noti
url: "message/queryMessageByTopicAndKey.query",
params: {
topic: $scope.selectedTopic,
- key:$scope.key
+ key: $scope.key
}
}).success(function (resp) {
if (resp.status == 0) {
console.log(resp);
$scope.queryMessageByTopicAndKeyResult = resp.data;
console.log($scope.queryMessageByTopicAndKeyResult);
- }else {
+ } else {
Notification.error({message: resp.errMsg, delay: 2000});
}
});
};
- $scope.queryMessageByMessageId = function (messageId,topic) {
+ $scope.queryMessageByMessageId = function (messageId, topic) {
$http({
method: "GET",
url: "messageTrace/viewMessage.query",
params: {
msgId: messageId,
- topic:topic
+ topic: topic
}
}).success(function (resp) {
if (resp.status == 0) {
console.log(resp);
$scope.queryMessageByMessageIdResult = resp.data;
console.log($scope.queryMessageByTopicAndKeyResult);
- }else {
+ } else {
Notification.error({message: resp.errMsg, delay: 2000});
}
});
};
- $scope.queryMessageTraceByMessageId = function (messageId,topic) {
+ $scope.queryMessageTraceByMessageId = function (messageId, topic) {
$http({
method: "GET",
- url: "messageTrace/viewMessageTraceDetail.query",
+ url: "messageTrace/viewMessageTraceGraph.query",
params: {
msgId: messageId,
- topic:topic
+ topic: topic
}
}).success(function (resp) {
if (resp.status == 0) {
console.log(resp);
ngDialog.open({
template: 'messageTraceDetailViewDialog',
controller: 'messageTraceDetailViewDialogController',
- data:resp.data
+ data: resp.data
});
- }else {
+ } else {
Notification.error({message: resp.errMsg, delay: 2000});
}
});
};
}]);
-module.controller('messageTraceDetailViewDialogController',['$scope', 'ngDialog', '$http','Notification', function ($scope, ngDialog, $http,Notification) {
+module.controller('messageTraceDetailViewDialogController', ['$scope', '$timeout', 'ngDialog', '$http', 'Notification', function ($scope, $timeout, ngDialog, $http, Notification) {
+ $scope.displayGraph = false;
+ $scope.graphButtonName = 'Show Graph';
+ $scope.displayMessageTraceGraph = function (messageTraceGraph) {
+ let dom = document.getElementById("messageTraceGraph");
+ $scope.messageTraceGraph = echarts.init(dom);
+ let option;
+ let data = [];
+ let dataZoomEnd = 100;
+ let startTime = Number.MAX_VALUE;
+ let endTime = 0;
+ let messageGroups = [];
+ if (messageTraceGraph.producerNode) {
+ startTime = +messageTraceGraph.producerNode.traceNode.beginTimeStamp;
+ endTime = +messageTraceGraph.producerNode.traceNode.endTimeStamp;
+ } else {
+ messageTraceGraph.subscriptionNodeList.forEach(subscriptionNode => {
+ subscriptionNode.consumeNodeList.forEach(consumeNode => {
+ startTime = Math.min(startTime, consumeNode.beginTimeStamp);
+ })
+ })
+ }
+
+ function buildNodeColor(traceNode, index) {
+ let nodeColor = SUCCESS_COLOR;
+ if (traceNode.transactionState) {
+ switch (traceNode.transactionState) {
+ case 'COMMIT_MESSAGE':
+ return TRANSACTION_COMMIT_COLOR;
+ case 'ROLLBACK_MESSAGE':
+ return TRANSACTION_ROLLBACK_COLOR;
+ case 'UNKNOW':
+ return TRANSACTION_UNKNOWN_COLOR;
+ default:
+ return ERROR_COLOR;
+ }
+ }
+ if (traceNode.status !== 'success') {
+ nodeColor = ERROR_COLOR;
+ }
+ if (index === messageGroups.length - 1) {
+ nodeColor = PRODUCER_COLOR;
+ }
+ return nodeColor;
+ }
+
+ function formatXAxisTime(value) {
+ let duration = Math.max(0, value - startTime);
+ if (duration < 1000)
+ return duration + 'ms';
+ duration /= 1000;
+ if (duration < 60)
+ return duration + 's';
+ duration /= 60;
+ if (duration < 60)
+ return duration + 'm';
+ duration /= 60;
+ return duration + 'h';
+ }
+
+ function buildTraceInfo(itemName, itemValue) {
+ if (itemValue) {
+ return `${itemName}: ${itemValue}<br />`
+ }
+ return "";
+ }
+
+ function formatNodeToolTip(params) {
+ let traceNode = params.data.traceData.traceNode;
+ return `
+ costTime: ${traceNode.costTime}ms<br />
+ status: ${traceNode.status}<br />
+ beginTimeStamp: ${new moment(traceNode.beginTimeStamp).format(TIME_FORMAT_PATTERN)}<br />
+ endTimeStamp: ${new moment(traceNode.endTimeStamp).format(TIME_FORMAT_PATTERN)}<br />
+ clientHost: ${traceNode.clientHost}<br />
+ storeHost: ${traceNode.storeHost}<br />
+ retryTimes: ${traceNode.retryTimes}<br />
+ ${buildTraceInfo('msgType', traceNode.msgType)}
+ ${buildTraceInfo('transactionId', traceNode.transactionId)}
+ ${buildTraceInfo('transactionState', traceNode.transactionState)}
+ ${buildTraceInfo('fromTransactionCheck', traceNode.fromTransactionCheck)}
+ `;
+ }
+
+ function addTraceData(traceNode, index) {
+ data.push({
+ value: [
+ index,
+ traceNode.beginTimeStamp,
+ traceNode.endTimeStamp,
+ traceNode.costTime
+ ],
+ itemStyle: {
+ normal: {
+ color: buildNodeColor(traceNode, index),
+ opacity: 1
+ }
+ },
+ traceData: {
+ traceNode: traceNode
+ }
+ });
+ endTime = Math.max(traceNode.endTimeStamp, endTime);
+ }
+
+ messageTraceGraph.subscriptionNodeList.forEach(item => {
+ messageGroups.push(item.subscriptionGroup)
+ })
+ messageTraceGraph.subscriptionNodeList.forEach((subscriptionNode, index) => {
+ subscriptionNode.consumeNodeList.forEach(traceNode => addTraceData(traceNode, index))
+ })
+ if (messageTraceGraph.producerNode) {
+ messageGroups.push(messageTraceGraph.producerNode.groupName)
+ let producerNodeIndex = messageGroups.length - 1;
+ addTraceData(messageTraceGraph.producerNode.traceNode, producerNodeIndex);
+ messageTraceGraph.producerNode.transactionNodeList.forEach(transactionNode => {
+ transactionNode.beginTimeStamp = Math.max(messageTraceGraph.producerNode.traceNode.endTimeStamp,
+ transactionNode.endTimeStamp - transactionCheckCostTime);
+ addTraceData(transactionNode, producerNodeIndex)
+ endTime = Math.max(endTime, transactionNode.endTimeStamp);
+ })
+ }
+
+ let totalDuration = endTime - startTime;
+ if (totalDuration > DEFAULT_DISPLAY_DURATION) {
+ dataZoomEnd = DEFAULT_DISPLAY_DURATION / totalDuration * 100
+ }
+
+ function renderItem(params, api) {
+ let messageGroup = api.value(0);
+ let start = api.coord([api.value(1), messageGroup]);
+ let end = api.coord([api.value(2), messageGroup]);
+ let height = api.size([0, 1])[1] * 0.6;
+
+ let rectShape = echarts.graphic.clipRectByRect({
+ x: start[0],
+ y: start[1] - height / 2,
+ width: end[0] - start[0],
Review comment:
The trace don't show if cost time is 0, for example, the graph is:
if change this line to: ` width: Math.max(end[0] - start[0], 1),`
the graph is
you can see the consum trace shows in the second gragh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] vongosling commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
vongosling commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663741680
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/TraceNode.java
##########
@@ -0,0 +1,30 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
Need to update...
##########
File path: rocketmq-console/pom.xml
##########
@@ -264,6 +269,22 @@
<artifactId>findbugs-maven-plugin</artifactId>
<version>3.0.4</version>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.6.2</version>
+ <configuration>
+ <source>1.8</source>
Review comment:
You could remove configuration parts while just leave annotationProcessorPaths here. refer to the properties in the pom of RocketMQ.
##########
File path: rocketmq-console/src/test/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImplTest.java
##########
@@ -0,0 +1,137 @@
+package org.apache.rocketmq.console.service.impl;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import lombok.SneakyThrows;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.console.config.RMQConfigure;
+import org.apache.rocketmq.console.model.MessageTraceView;
+import org.apache.rocketmq.console.model.trace.MessageTraceGraph;
+import org.apache.rocketmq.console.util.JsonUtil;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.assertj.core.util.Lists;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.MockitoAnnotations;
+import org.mockito.junit.MockitoJUnitRunner;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
Review comment:
License ...
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/MessageTraceGraph.java
##########
@@ -0,0 +1,29 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
License should be before package parts.
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -47,8 +79,99 @@
messageTraceViews.addAll(messageTraceView);
}
return messageTraceViews;
- } catch (Exception err) {
+ }
+ catch (Exception err) {
Review comment:
Checkstyle will pass here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r667088710
##########
File path: rocketmq-console/src/test/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImplTest.java
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.rocketmq.console.service.impl;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import lombok.SneakyThrows;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.console.config.RMQConfigure;
+import org.apache.rocketmq.console.model.MessageTraceView;
+import org.apache.rocketmq.console.model.trace.MessageTraceGraph;
+import org.apache.rocketmq.console.util.JsonUtil;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.assertj.core.util.Lists;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.MockitoAnnotations;
+import org.mockito.junit.MockitoJUnitRunner;
+
+@RunWith(MockitoJUnitRunner.class)
Review comment:
@RunWith(MockitoJUnitRunner.class) already removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] yuz10 commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
yuz10 commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r664154949
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -51,4 +83,94 @@
throw Throwables.propagate(err);
}
}
+
+ @Override
+ public MessageTraceGraph queryMessageTraceGraph(String key) {
+ List<MessageTraceView> messageTraceViews = queryMessageTraceKey(key);
+ return buildMessageTraceGraph(messageTraceViews);
+ }
+
+ private MessageTraceGraph buildMessageTraceGraph(List<MessageTraceView> messageTraceViews) {
+ MessageTraceGraph messageTraceGraph = new MessageTraceGraph();
+ messageTraceGraph.setMessageTraceViews(messageTraceViews);
+ if (CollectionUtils.isEmpty(messageTraceViews)) {
+ return messageTraceGraph;
+ }
+ ProducerNode producerNode = null;
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap = Maps.newHashMap();
Review comment:
complicated definition could be defined as class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663834816
##########
File path: rocketmq-console/pom.xml
##########
@@ -264,6 +269,22 @@
<artifactId>findbugs-maven-plugin</artifactId>
<version>3.0.4</version>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.6.2</version>
+ <configuration>
+ <source>1.8</source>
Review comment:
Duplicate maven-compiler-plugin has been removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663834894
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/TraceNode.java
##########
@@ -0,0 +1,30 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
Done
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/MessageTraceGraph.java
##########
@@ -0,0 +1,29 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
Done
##########
File path: rocketmq-console/src/test/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImplTest.java
##########
@@ -0,0 +1,137 @@
+package org.apache.rocketmq.console.service.impl;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import lombok.SneakyThrows;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.console.config.RMQConfigure;
+import org.apache.rocketmq.console.model.MessageTraceView;
+import org.apache.rocketmq.console.model.trace.MessageTraceGraph;
+import org.apache.rocketmq.console.util.JsonUtil;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.assertj.core.util.Lists;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.MockitoAnnotations;
+import org.mockito.junit.MockitoJUnitRunner;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] vongosling merged pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
vongosling merged pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] yuz10 commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
yuz10 commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r664153766
##########
File path: rocketmq-console/src/test/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImplTest.java
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.rocketmq.console.service.impl;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import lombok.SneakyThrows;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.console.config.RMQConfigure;
+import org.apache.rocketmq.console.model.MessageTraceView;
+import org.apache.rocketmq.console.model.trace.MessageTraceGraph;
+import org.apache.rocketmq.console.util.JsonUtil;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.assertj.core.util.Lists;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.MockitoAnnotations;
+import org.mockito.junit.MockitoJUnitRunner;
+
+@RunWith(MockitoJUnitRunner.class)
Review comment:
run this test fails in my computer, MockitoJUnitRunner init fail, it.s fine if remove @RunWith(MockitoJUnitRunner.class)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] yuz10 commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
yuz10 commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663995671
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -51,4 +83,94 @@
throw Throwables.propagate(err);
}
}
+
+ @Override
+ public MessageTraceGraph queryMessageTraceGraph(String key) {
+ List<MessageTraceView> messageTraceViews = queryMessageTraceKey(key);
+ return buildMessageTraceGraph(messageTraceViews);
+ }
+
+ private MessageTraceGraph buildMessageTraceGraph(List<MessageTraceView> messageTraceViews) {
+ MessageTraceGraph messageTraceGraph = new MessageTraceGraph();
+ messageTraceGraph.setMessageTraceViews(messageTraceViews);
+ if (CollectionUtils.isEmpty(messageTraceViews)) {
+ return messageTraceGraph;
+ }
+ ProducerNode producerNode = null;
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap = Maps.newHashMap();
+ for (MessageTraceView messageTraceView : messageTraceViews) {
+ switch (TraceType.valueOf(messageTraceView.getMsgType())) {
+ case Pub:
+ producerNode = buildMessageRoot(messageTraceView);
+ break;
+ case SubBefore:
+ case SubAfter:
+ putIntoMessageTraceViewGroupMap(messageTraceView, messageTraceViewGroupMap);
+ break;
+ default:
+ break;
+ }
+ }
+ messageTraceGraph.setProducerNode(producerNode);
+ messageTraceGraph.setSubscriptionNodeList(buildSubscriptionNodeList(messageTraceViewGroupMap));
+ return messageTraceGraph;
+ }
+
+ private List<SubscriptionNode> buildSubscriptionNodeList(
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ List<SubscriptionNode> subscriptionNodeList = new ArrayList<>(messageTraceViewGroupMap.size());
+ for (Map.Entry<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> groupTraceView : messageTraceViewGroupMap
+ .entrySet()) {
+ SubscriptionNode subscriptionNode = new SubscriptionNode();
+ subscriptionNode.setSubscriptionGroup(groupTraceView.getKey());
+ List<TraceNode> consumeNodeList = Lists.newArrayList();
+ subscriptionNode.setConsumeNodeList(consumeNodeList);
+ subscriptionNodeList.add(subscriptionNode);
+ for (Map.Entry<String, Pair<MessageTraceView, MessageTraceView>> requestIdTracePair : groupTraceView
+ .getValue().entrySet()) {
+ MessageTraceView subBeforeTrace = requestIdTracePair.getValue().getObject1();
+ MessageTraceView subAfterTrace = requestIdTracePair.getValue().getObject2();
+ TraceNode consumeNode = new TraceNode();
+ consumeNode.setRequestId(requestIdTracePair.getKey());
+ consumeNode.setStoreHost(subBeforeTrace.getStoreHost());
+ consumeNode.setClientHost(subBeforeTrace.getClientHost());
+ consumeNode.setRetryTimes(subBeforeTrace.getRetryTimes());
+ consumeNode.setBeginTimeStamp(subBeforeTrace.getTimeStamp());
+ consumeNode.setCostTime(subAfterTrace.getCostTime());
+ consumeNode.setEndTimeStamp(subAfterTrace.getTimeStamp());
+ consumeNode.setStatus(subAfterTrace.getStatus());
+ consumeNodeList.add(consumeNode);
+ }
+ }
+ return subscriptionNodeList;
+ }
+
+ private void putIntoMessageTraceViewGroupMap(MessageTraceView messageTraceView,
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ Map<String, Pair<MessageTraceView, MessageTraceView>> requestIdTraceMap = messageTraceViewGroupMap
+ .computeIfAbsent(messageTraceView.getGroupName(), (o) -> new HashMap<>(2));
Review comment:
the newest commit of rocketmq has removed groupname field of SubAfter trace, so you will not be able to match the pair in later releases. see https://github.com/apache/rocketmq/pull/3005/files
this will result null exception at: org.apache.rocketmq.console.service.impl.MessageTraceServiceImpl.buildSubscriptionNodeList(MessageTraceServiceImpl.java:135)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r667090575
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -51,4 +83,94 @@
throw Throwables.propagate(err);
}
}
+
+ @Override
+ public MessageTraceGraph queryMessageTraceGraph(String key) {
+ List<MessageTraceView> messageTraceViews = queryMessageTraceKey(key);
+ return buildMessageTraceGraph(messageTraceViews);
+ }
+
+ private MessageTraceGraph buildMessageTraceGraph(List<MessageTraceView> messageTraceViews) {
+ MessageTraceGraph messageTraceGraph = new MessageTraceGraph();
+ messageTraceGraph.setMessageTraceViews(messageTraceViews);
+ if (CollectionUtils.isEmpty(messageTraceViews)) {
+ return messageTraceGraph;
+ }
+ ProducerNode producerNode = null;
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap = Maps.newHashMap();
Review comment:
Yes it is too complicated, already use requestIdTracePairMap instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] yuz10 commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
yuz10 commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663995671
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -51,4 +83,94 @@
throw Throwables.propagate(err);
}
}
+
+ @Override
+ public MessageTraceGraph queryMessageTraceGraph(String key) {
+ List<MessageTraceView> messageTraceViews = queryMessageTraceKey(key);
+ return buildMessageTraceGraph(messageTraceViews);
+ }
+
+ private MessageTraceGraph buildMessageTraceGraph(List<MessageTraceView> messageTraceViews) {
+ MessageTraceGraph messageTraceGraph = new MessageTraceGraph();
+ messageTraceGraph.setMessageTraceViews(messageTraceViews);
+ if (CollectionUtils.isEmpty(messageTraceViews)) {
+ return messageTraceGraph;
+ }
+ ProducerNode producerNode = null;
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap = Maps.newHashMap();
+ for (MessageTraceView messageTraceView : messageTraceViews) {
+ switch (TraceType.valueOf(messageTraceView.getMsgType())) {
+ case Pub:
+ producerNode = buildMessageRoot(messageTraceView);
+ break;
+ case SubBefore:
+ case SubAfter:
+ putIntoMessageTraceViewGroupMap(messageTraceView, messageTraceViewGroupMap);
+ break;
+ default:
+ break;
+ }
+ }
+ messageTraceGraph.setProducerNode(producerNode);
+ messageTraceGraph.setSubscriptionNodeList(buildSubscriptionNodeList(messageTraceViewGroupMap));
+ return messageTraceGraph;
+ }
+
+ private List<SubscriptionNode> buildSubscriptionNodeList(
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ List<SubscriptionNode> subscriptionNodeList = new ArrayList<>(messageTraceViewGroupMap.size());
+ for (Map.Entry<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> groupTraceView : messageTraceViewGroupMap
+ .entrySet()) {
+ SubscriptionNode subscriptionNode = new SubscriptionNode();
+ subscriptionNode.setSubscriptionGroup(groupTraceView.getKey());
+ List<TraceNode> consumeNodeList = Lists.newArrayList();
+ subscriptionNode.setConsumeNodeList(consumeNodeList);
+ subscriptionNodeList.add(subscriptionNode);
+ for (Map.Entry<String, Pair<MessageTraceView, MessageTraceView>> requestIdTracePair : groupTraceView
+ .getValue().entrySet()) {
+ MessageTraceView subBeforeTrace = requestIdTracePair.getValue().getObject1();
+ MessageTraceView subAfterTrace = requestIdTracePair.getValue().getObject2();
+ TraceNode consumeNode = new TraceNode();
+ consumeNode.setRequestId(requestIdTracePair.getKey());
+ consumeNode.setStoreHost(subBeforeTrace.getStoreHost());
+ consumeNode.setClientHost(subBeforeTrace.getClientHost());
+ consumeNode.setRetryTimes(subBeforeTrace.getRetryTimes());
+ consumeNode.setBeginTimeStamp(subBeforeTrace.getTimeStamp());
+ consumeNode.setCostTime(subAfterTrace.getCostTime());
+ consumeNode.setEndTimeStamp(subAfterTrace.getTimeStamp());
+ consumeNode.setStatus(subAfterTrace.getStatus());
+ consumeNodeList.add(consumeNode);
+ }
+ }
+ return subscriptionNodeList;
+ }
+
+ private void putIntoMessageTraceViewGroupMap(MessageTraceView messageTraceView,
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ Map<String, Pair<MessageTraceView, MessageTraceView>> requestIdTraceMap = messageTraceViewGroupMap
+ .computeIfAbsent(messageTraceView.getGroupName(), (o) -> new HashMap<>(2));
Review comment:
the newest commit of rocketmq has removed groupname field of SubAfter trace, so you will not be able to match the pair in later releases. see https://github.com/apache/rocketmq/pull/3005/files
this will result null exception at: org.apache.rocketmq.console.service.impl.MessageTraceServiceImpl.buildSubscriptionNodeList(MessageTraceServiceImpl.java:135)
##########
File path: rocketmq-console/src/test/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImplTest.java
##########
@@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.rocketmq.console.service.impl;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import lombok.SneakyThrows;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.console.config.RMQConfigure;
+import org.apache.rocketmq.console.model.MessageTraceView;
+import org.apache.rocketmq.console.model.trace.MessageTraceGraph;
+import org.apache.rocketmq.console.util.JsonUtil;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.assertj.core.util.Lists;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.MockitoAnnotations;
+import org.mockito.junit.MockitoJUnitRunner;
+
+@RunWith(MockitoJUnitRunner.class)
Review comment:
run this test fails in my computer, MockitoJUnitRunner init fail, it.s fine if remove @RunWith(MockitoJUnitRunner.class)
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -51,4 +83,94 @@
throw Throwables.propagate(err);
}
}
+
+ @Override
+ public MessageTraceGraph queryMessageTraceGraph(String key) {
+ List<MessageTraceView> messageTraceViews = queryMessageTraceKey(key);
+ return buildMessageTraceGraph(messageTraceViews);
+ }
+
+ private MessageTraceGraph buildMessageTraceGraph(List<MessageTraceView> messageTraceViews) {
+ MessageTraceGraph messageTraceGraph = new MessageTraceGraph();
+ messageTraceGraph.setMessageTraceViews(messageTraceViews);
+ if (CollectionUtils.isEmpty(messageTraceViews)) {
+ return messageTraceGraph;
+ }
+ ProducerNode producerNode = null;
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap = Maps.newHashMap();
Review comment:
complicated definition could be defined as class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#issuecomment-877801383
Commit ca96adb contains some of the improvements for the trace UI.
* Implement Features
* support transaction message trace
* support only producer trace enabled case
* support only consumer trace enabled case
* add(fix) trace type (message type)
* use rocketmq.version = 4.9.0
Test evidence are as follows:
1. when only consumer is trace enabled
2. when only producer is trace enabled
3. executeLocalTransaction fail but checkLocalTransaction succeed
(we can not get the costTime and beginTime of the transaction message trace, for now I just use 50ms to draw the graph, this can be optimized in the future)
Please help review when you are available.
(No other new features will be added to this PR, if there is, I will open a new PR)
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] StyleTang commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
StyleTang commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r663834041
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -47,8 +79,99 @@
messageTraceViews.addAll(messageTraceView);
}
return messageTraceViews;
- } catch (Exception err) {
+ }
+ catch (Exception err) {
Review comment:
Done. (Yes, it passed the check)
##########
File path: rocketmq-console/pom.xml
##########
@@ -264,6 +269,22 @@
<artifactId>findbugs-maven-plugin</artifactId>
<version>3.0.4</version>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>3.6.2</version>
+ <configuration>
+ <source>1.8</source>
Review comment:
Duplicate maven-compiler-plugin has been removed.
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/TraceNode.java
##########
@@ -0,0 +1,30 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
Done
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/model/trace/MessageTraceGraph.java
##########
@@ -0,0 +1,29 @@
+package org.apache.rocketmq.console.model.trace;/*
Review comment:
Done
##########
File path: rocketmq-console/src/test/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImplTest.java
##########
@@ -0,0 +1,137 @@
+package org.apache.rocketmq.console.service.impl;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.List;
+import lombok.SneakyThrows;
+import org.apache.rocketmq.client.QueryResult;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.apache.rocketmq.console.config.RMQConfigure;
+import org.apache.rocketmq.console.model.MessageTraceView;
+import org.apache.rocketmq.console.model.trace.MessageTraceGraph;
+import org.apache.rocketmq.console.util.JsonUtil;
+import org.apache.rocketmq.tools.admin.MQAdminExt;
+import org.assertj.core.util.Lists;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.runner.RunWith;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+import org.mockito.Mockito;
+import org.mockito.MockitoAnnotations;
+import org.mockito.junit.MockitoJUnitRunner;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-externals] vongosling commented on a change in pull request #744: [ISSUE #743] Implement RocketMQ message trace UI
Posted by GitBox <gi...@apache.org>.
vongosling commented on a change in pull request #744:
URL: https://github.com/apache/rocketmq-externals/pull/744#discussion_r666709626
##########
File path: rocketmq-console/src/main/java/org/apache/rocketmq/console/service/impl/MessageTraceServiceImpl.java
##########
@@ -51,4 +83,94 @@
throw Throwables.propagate(err);
}
}
+
+ @Override
+ public MessageTraceGraph queryMessageTraceGraph(String key) {
+ List<MessageTraceView> messageTraceViews = queryMessageTraceKey(key);
+ return buildMessageTraceGraph(messageTraceViews);
+ }
+
+ private MessageTraceGraph buildMessageTraceGraph(List<MessageTraceView> messageTraceViews) {
+ MessageTraceGraph messageTraceGraph = new MessageTraceGraph();
+ messageTraceGraph.setMessageTraceViews(messageTraceViews);
+ if (CollectionUtils.isEmpty(messageTraceViews)) {
+ return messageTraceGraph;
+ }
+ ProducerNode producerNode = null;
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap = Maps.newHashMap();
+ for (MessageTraceView messageTraceView : messageTraceViews) {
+ switch (TraceType.valueOf(messageTraceView.getMsgType())) {
+ case Pub:
+ producerNode = buildMessageRoot(messageTraceView);
+ break;
+ case SubBefore:
+ case SubAfter:
+ putIntoMessageTraceViewGroupMap(messageTraceView, messageTraceViewGroupMap);
+ break;
+ default:
+ break;
+ }
+ }
+ messageTraceGraph.setProducerNode(producerNode);
+ messageTraceGraph.setSubscriptionNodeList(buildSubscriptionNodeList(messageTraceViewGroupMap));
+ return messageTraceGraph;
+ }
+
+ private List<SubscriptionNode> buildSubscriptionNodeList(
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ List<SubscriptionNode> subscriptionNodeList = new ArrayList<>(messageTraceViewGroupMap.size());
+ for (Map.Entry<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> groupTraceView : messageTraceViewGroupMap
+ .entrySet()) {
+ SubscriptionNode subscriptionNode = new SubscriptionNode();
+ subscriptionNode.setSubscriptionGroup(groupTraceView.getKey());
+ List<TraceNode> consumeNodeList = Lists.newArrayList();
+ subscriptionNode.setConsumeNodeList(consumeNodeList);
+ subscriptionNodeList.add(subscriptionNode);
+ for (Map.Entry<String, Pair<MessageTraceView, MessageTraceView>> requestIdTracePair : groupTraceView
+ .getValue().entrySet()) {
+ MessageTraceView subBeforeTrace = requestIdTracePair.getValue().getObject1();
+ MessageTraceView subAfterTrace = requestIdTracePair.getValue().getObject2();
+ TraceNode consumeNode = new TraceNode();
+ consumeNode.setRequestId(requestIdTracePair.getKey());
+ consumeNode.setStoreHost(subBeforeTrace.getStoreHost());
+ consumeNode.setClientHost(subBeforeTrace.getClientHost());
+ consumeNode.setRetryTimes(subBeforeTrace.getRetryTimes());
+ consumeNode.setBeginTimeStamp(subBeforeTrace.getTimeStamp());
+ consumeNode.setCostTime(subAfterTrace.getCostTime());
+ consumeNode.setEndTimeStamp(subAfterTrace.getTimeStamp());
+ consumeNode.setStatus(subAfterTrace.getStatus());
+ consumeNodeList.add(consumeNode);
+ }
+ }
+ return subscriptionNodeList;
+ }
+
+ private void putIntoMessageTraceViewGroupMap(MessageTraceView messageTraceView,
+ Map<String, Map<String, Pair<MessageTraceView, MessageTraceView>>> messageTraceViewGroupMap) {
+ Map<String, Pair<MessageTraceView, MessageTraceView>> requestIdTraceMap = messageTraceViewGroupMap
+ .computeIfAbsent(messageTraceView.getGroupName(), (o) -> new HashMap<>(2));
Review comment:
@StyleTang any update for here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org