You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/05/24 15:08:45 UTC

[GitHub] [rocketmq] lizhimins opened a new pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

lizhimins opened a new pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943


   **Make sure set the target branch to `develop`**
   
   ## What is the purpose of the change
   
   Fix #2735
   
   ## Brief changelog
   
   XX
   
   ## Verifying this change
   
   XXXX
   
   Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.
   
   - [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue. 
   - [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
   - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
   - [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
   - [x] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test`  to make sure integration-test pass.
   - [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] coveralls edited a comment on pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#issuecomment-847120088


   
   [![Coverage Status](https://coveralls.io/builds/39990558/badge)](https://coveralls.io/builds/39990558)
   
   Coverage decreased (-0.02%) to 53.981% when pulling **c34796bd06321560c88deab8bfa7a1ab560ddeb7 on lizhimins:develop** into **952772fa1bc48bb27b5e0ee586b2b086c2ef611a on apache:develop**.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] codecov-commenter edited a comment on pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#issuecomment-847877903


   # [Codecov](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > :exclamation: No coverage uploaded for pull request base (`develop@7028af6`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#section-missing-base-commit).
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/rocketmq/pull/2943/graphs/tree.svg?width=650&height=150&src=pr&token=4w0sxP1wZv&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@            Coverage Diff             @@
   ##             develop    #2943   +/-   ##
   ==========================================
     Coverage           ?   47.80%           
     Complexity         ?     4534           
   ==========================================
     Files              ?      552           
     Lines              ?    36541           
     Branches           ?     4838           
   ==========================================
     Hits               ?    17469           
     Misses             ?    16843           
     Partials           ?     2229           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7028af6...c34796b](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] codecov-commenter commented on pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#issuecomment-847877903


   # [Codecov](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > :exclamation: No coverage uploaded for pull request base (`develop@7028af6`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#section-missing-base-commit).
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/rocketmq/pull/2943/graphs/tree.svg?width=650&height=150&src=pr&token=4w0sxP1wZv&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   
   ```diff
   @@            Coverage Diff             @@
   ##             develop    #2943   +/-   ##
   ==========================================
     Coverage           ?   47.80%           
     Complexity         ?     4534           
   ==========================================
     Files              ?      552           
     Lines              ?    36541           
     Branches           ?     4838           
   ==========================================
     Hits               ?    17469           
     Misses             ?    16843           
     Partials           ?     2229           
   ```
   
   
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [7028af6...c34796b](https://codecov.io/gh/apache/rocketmq/pull/2943?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] lizhimins commented on a change in pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
lizhimins commented on a change in pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#discussion_r638772601



##########
File path: tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExt.java
##########
@@ -128,12 +128,18 @@ public MessageExt viewMessage(
     }
 
     @Override
-    public QueryResult queryMessage(String topic, String key, int maxNum, long begin,
-        long end) throws MQClientException,
-        InterruptedException {
+    public QueryResult queryMessage(String topic, String key, int maxNum, long begin, long end)
+        throws MQClientException, InterruptedException {
+
         return defaultMQAdminExtImpl.queryMessage(topic, key, maxNum, begin, end);
     }
 
+    public QueryResult queryMessageByUniqueKey(String topic, String key, int maxNum, long begin, long end)

Review comment:
       ok, I will fix it




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] coveralls commented on pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#issuecomment-847120088


   
   [![Coverage Status](https://coveralls.io/builds/39952149/badge)](https://coveralls.io/builds/39952149)
   
   Coverage increased (+0.06%) to 51.882% when pulling **672f581953673e4e47ef7153677937ddbe50e6c4 on lizhimins:develop** into **aaa92a2e53d773c7f1d9e9f25e41709f6948fa83 on apache:master**.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] duhenglucky merged pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
duhenglucky merged pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] lizhimins commented on a change in pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
lizhimins commented on a change in pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#discussion_r638771510



##########
File path: tools/src/main/java/org/apache/rocketmq/tools/command/message/QueryMsgByUniqueKeySubCommand.java
##########
@@ -57,82 +58,40 @@ private DefaultMQAdminExt createMQAdminExt(RPCHook rpcHook) throws SubCommandExc
         }
     }
 
-    public static void queryById(final DefaultMQAdminExt admin, final String topic,
-        final String msgId) throws MQClientException,
-        RemotingException, MQBrokerException, InterruptedException, IOException {
-        MessageExt msg = admin.viewMessage(topic, msgId);
-
-        String bodyTmpFilePath = createBodyFile(msg);
-
-        System.out.printf("%-20s %s%n",
-            "Topic:",
-            msg.getTopic()
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Tags:",
-            "[" + msg.getTags() + "]"
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Keys:",
-            "[" + msg.getKeys() + "]"
-        );
-
-        System.out.printf("%-20s %d%n",
-            "Queue ID:",
-            msg.getQueueId()
-        );
-
-        System.out.printf("%-20s %d%n",
-            "Queue Offset:",
-            msg.getQueueOffset()
-        );
-
-        System.out.printf("%-20s %d%n",
-            "CommitLog Offset:",
-            msg.getCommitLogOffset()
-        );
-
-        System.out.printf("%-20s %d%n",
-            "Reconsume Times:",
-            msg.getReconsumeTimes()
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Born Timestamp:",
-            UtilAll.timeMillisToHumanString2(msg.getBornTimestamp())
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Store Timestamp:",
-            UtilAll.timeMillisToHumanString2(msg.getStoreTimestamp())
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Born Host:",
-            RemotingHelper.parseSocketAddressAddr(msg.getBornHost())
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Store Host:",
-            RemotingHelper.parseSocketAddressAddr(msg.getStoreHost())
-        );
-
-        System.out.printf("%-20s %d%n",
-            "System Flag:",
-            msg.getSysFlag()
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Properties:",
-            msg.getProperties() != null ? msg.getProperties().toString() : ""
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Message Body Path:",
-            bodyTmpFilePath
-        );
+    public static void queryById(final DefaultMQAdminExt admin, final String topic, final String msgId,
+                                 final boolean showAll) throws MQClientException,
+            RemotingException, MQBrokerException, InterruptedException, IOException {
+
+        QueryResult queryResult = admin.queryMessageByUniqueKey(topic, msgId, 32, 0, Long.MAX_VALUE);
+        assert queryResult != null;
+        List<MessageExt> list = queryResult.getMessageList();
+        list.sort((o1, o2) -> (int) (o1.getStoreTimestamp() - o2.getStoreTimestamp()));
+        for (int i = 0; i < (showAll ? list.size() : 1); i++) {

Review comment:
       queryMessageByUniqueKey will not return null




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] yuz10 commented on a change in pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
yuz10 commented on a change in pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#discussion_r638399550



##########
File path: tools/src/main/java/org/apache/rocketmq/tools/command/message/QueryMsgByUniqueKeySubCommand.java
##########
@@ -57,82 +58,40 @@ private DefaultMQAdminExt createMQAdminExt(RPCHook rpcHook) throws SubCommandExc
         }
     }
 
-    public static void queryById(final DefaultMQAdminExt admin, final String topic,
-        final String msgId) throws MQClientException,
-        RemotingException, MQBrokerException, InterruptedException, IOException {
-        MessageExt msg = admin.viewMessage(topic, msgId);
-
-        String bodyTmpFilePath = createBodyFile(msg);
-
-        System.out.printf("%-20s %s%n",
-            "Topic:",
-            msg.getTopic()
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Tags:",
-            "[" + msg.getTags() + "]"
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Keys:",
-            "[" + msg.getKeys() + "]"
-        );
-
-        System.out.printf("%-20s %d%n",
-            "Queue ID:",
-            msg.getQueueId()
-        );
-
-        System.out.printf("%-20s %d%n",
-            "Queue Offset:",
-            msg.getQueueOffset()
-        );
-
-        System.out.printf("%-20s %d%n",
-            "CommitLog Offset:",
-            msg.getCommitLogOffset()
-        );
-
-        System.out.printf("%-20s %d%n",
-            "Reconsume Times:",
-            msg.getReconsumeTimes()
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Born Timestamp:",
-            UtilAll.timeMillisToHumanString2(msg.getBornTimestamp())
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Store Timestamp:",
-            UtilAll.timeMillisToHumanString2(msg.getStoreTimestamp())
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Born Host:",
-            RemotingHelper.parseSocketAddressAddr(msg.getBornHost())
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Store Host:",
-            RemotingHelper.parseSocketAddressAddr(msg.getStoreHost())
-        );
-
-        System.out.printf("%-20s %d%n",
-            "System Flag:",
-            msg.getSysFlag()
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Properties:",
-            msg.getProperties() != null ? msg.getProperties().toString() : ""
-        );
-
-        System.out.printf("%-20s %s%n",
-            "Message Body Path:",
-            bodyTmpFilePath
-        );
+    public static void queryById(final DefaultMQAdminExt admin, final String topic, final String msgId,
+                                 final boolean showAll) throws MQClientException,
+            RemotingException, MQBrokerException, InterruptedException, IOException {
+
+        QueryResult queryResult = admin.queryMessageByUniqueKey(topic, msgId, 32, 0, Long.MAX_VALUE);
+        assert queryResult != null;
+        List<MessageExt> list = queryResult.getMessageList();
+        list.sort((o1, o2) -> (int) (o1.getStoreTimestamp() - o2.getStoreTimestamp()));
+        for (int i = 0; i < (showAll ? list.size() : 1); i++) {

Review comment:
       if list is empty, and showAll is false, `list.get(i)` will throw exeption 

##########
File path: tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExt.java
##########
@@ -128,12 +128,18 @@ public MessageExt viewMessage(
     }
 
     @Override
-    public QueryResult queryMessage(String topic, String key, int maxNum, long begin,
-        long end) throws MQClientException,
-        InterruptedException {
+    public QueryResult queryMessage(String topic, String key, int maxNum, long begin, long end)
+        throws MQClientException, InterruptedException {
+
         return defaultMQAdminExtImpl.queryMessage(topic, key, maxNum, begin, end);
     }
 
+    public QueryResult queryMessageByUniqueKey(String topic, String key, int maxNum, long begin, long end)

Review comment:
       The function name, queryMessageByUniqueKey is inconsistent with queryMessageByUniqKey, I suggest keep the same




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [rocketmq] coveralls edited a comment on pull request #2943: [ISSUE #2735] QueryMsgByUniqueKey tool should return all messages with same unique key

Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #2943:
URL: https://github.com/apache/rocketmq/pull/2943#issuecomment-847120088


   
   [![Coverage Status](https://coveralls.io/builds/39952313/badge)](https://coveralls.io/builds/39952313)
   
   Coverage decreased (-0.06%) to 51.761% when pulling **672f581953673e4e47ef7153677937ddbe50e6c4 on lizhimins:develop** into **aaa92a2e53d773c7f1d9e9f25e41709f6948fa83 on apache:master**.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org