You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "lzkisok (via GitHub)" <gi...@apache.org> on 2023/10/25 12:24:50 UTC

[I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

lzkisok opened a new issue, #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1782223913

   ![8a02f52e-cad0-4185-8727-134b47f241f0](https://github.com/apache/rocketmq-mqtt/assets/24490175/25300017-269d-4b7c-838f-ba0a0147bd75)
   如上图，rmq5.1.4 当消息并发时，这里取了相同的offset
   ![img_v2_08eb737f-4333-453c-858e-8cc5bf71d7ag](https://github.com/apache/rocketmq-mqtt/assets/24490175/7223d39a-9bb8-4fea-96d8-825052ccbe7f)
   如上图，测试rmq5.1.4 的测试结果可以看到确实是相同


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1782243031

   ![企业微信截图_5869c60d-1567-4de2-bb0b-d38c9f19afa1](https://github.com/apache/rocketmq-mqtt/assets/24490175/1c75817a-a9ec-40aa-afee-2a98c8116d66)
   这是rockemq的代码，这里这个topicQueueKey 是topic+queneid，当一个消息被发送到两个quene的时候，defaultMessageStore.assignOffset(msg);这行代码没有被锁住


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1781039005

   ![企业微信截图_3ae8bd29-1cbb-4f6b-809f-f72b9b4c9ea2](https://github.com/apache/rocketmq-mqtt/assets/24490175/e6f422f2-3591-4159-9653-a2c4391e870b)
   如上图，pull到两条消息，offset相同
   ![企业微信截图_74373a44-2ae8-42ee-afe5-582587d30f1e](https://github.com/apache/rocketmq-mqtt/assets/24490175/df82263e-c841-4036-9954-c2cf5ac736ca)
   如上图，addSendingMessages 丢失一条消息


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1784762308

   嗯嗯，好的，7511也是我提的，我们看代码找到了一个暂时的解决方式，将同一个LMQ的消息，始终发送到同一个MessageQuene中，让topic+queneid的锁能生效
   ![image](https://github.com/apache/rocketmq-mqtt/assets/24490175/e3a7a3a0-a6b3-4f80-aef5-dcaff46e4cc4)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "DongyuanPan (via GitHub)" <gi...@apache.org>.
DongyuanPan commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780471713

   给一下rocketmq和rocketmq的版本，以及复现的步骤


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780653347

   Thread t1 = new Thread(new Runnable(){
               @Override
               public void run() {
                   try {
                       sendMessage(1);
                   } catch (MQBrokerException e) {
                       throw new RuntimeException(e);
                   } catch (RemotingException e) {
                       throw new RuntimeException(e);
                   } catch (InterruptedException e) {
                       throw new RuntimeException(e);
                   } catch (MQClientException e) {
                       throw new RuntimeException(e);
                   }
               }
           });
   
           Thread t2 = new Thread(new Runnable(){
               @Override
               public void run() {
                   try {
                       sendMessage(2);
                   } catch (MQBrokerException e) {
                       throw new RuntimeException(e);
                   } catch (RemotingException e) {
                       throw new RuntimeException(e);
                   } catch (InterruptedException e) {
                       throw new RuntimeException(e);
                   } catch (MQClientException e) {
                       throw new RuntimeException(e);
                   }
               }
           });
   
           Thread t3 = new Thread(new Runnable(){
               @Override
               public void run() {
                   try {
                       sendMessage(3);
                   } catch (MQBrokerException e) {
                       throw new RuntimeException(e);
                   } catch (RemotingException e) {
                       throw new RuntimeException(e);
                   } catch (InterruptedException e) {
                       throw new RuntimeException(e);
                   } catch (MQClientException e) {
                       throw new RuntimeException(e);
                   }
               }
           });
   
           Thread t4 = new Thread(new Runnable(){
               @Override
               public void run() {
                   try {
                       sendMessage(4);
                   } catch (MQBrokerException e) {
                       throw new RuntimeException(e);
                   } catch (RemotingException e) {
                       throw new RuntimeException(e);
                   } catch (InterruptedException e) {
                       throw new RuntimeException(e);
                   } catch (MQClientException e) {
                       throw new RuntimeException(e);
                   }
               }
           });
   
           Thread t5 = new Thread(new Runnable(){
               @Override
               public void run() {
                   try {
                       sendMessage(5);
                   } catch (MQBrokerException e) {
                       throw new RuntimeException(e);
                   } catch (RemotingException e) {
                       throw new RuntimeException(e);
                   } catch (InterruptedException e) {
                       throw new RuntimeException(e);
                   } catch (MQClientException e) {
                       throw new RuntimeException(e);
                   }
               }
           });
           t1.start();
           t2.start();
           t3.start();
           t4.start();
           t5.start();
           
           这是测试用的RocketMQProducer中main方法的发送mq的代码


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1779162773

   或者异步发送5条消息，MqttConsumer 会丢消息，麻烦官方确认下


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780486431

   目前观察到的原因：
   从rocketmq一次pull到两条消息，两条消息的INNER_MULTI_QUEUE_OFFSET 相同， 导致了这个问题


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780630936

   ![企业微信截图_5905f1b2-4e79-4314-bb91-7e8b9a715acd](https://github.com/apache/rocketmq-mqtt/assets/24490175/2ae50385-5726-443a-88b3-064244fa5b97)
   ![image](https://github.com/apache/rocketmq-mqtt/assets/24490175/0a5ee4a2-f400-47f2-9f65-2ef74756e263)
   
   ![企业微信截图_a9c191a5-3817-462e-b794-ba66ce19a942](https://github.com/apache/rocketmq-mqtt/assets/24490175/32907e44-a11a-41a9-ab42-2dabfd205834)
   ![企业微信截图_0d522e3d-7945-4376-9498-d44b07b86651](https://github.com/apache/rocketmq-mqtt/assets/24490175/52f52a53-9b0b-4c3d-98e7-3511b3c7fd0d)
   
   如图，我也用了develop分支的最新代码部署cs，rmq5.1.4版本，比较容易复现，多试几次，很容易复现


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "DongyuanPan (via GitHub)" <gi...@apache.org>.
DongyuanPan commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1788466427

   > 能私信下吗？给个私信方式，咋们交流交流
   
   https://github.com/apache/rocketmq-mqtt/issues/64 这里面有SIG的群，可以扫码进群。
   也欢迎私戳我主页邮箱


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1779163414

   多试几次，必现


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1782666863

   能私信下吗？给个私信方式，咋们交流交流


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "DongyuanPan (via GitHub)" <gi...@apache.org>.
DongyuanPan commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1788461430

   > 嗯嗯，好的，7511也是我提的，我们看代码找到了一个暂时的解决方式，将同一个LMQ的消息，始终发送到同一个MessageQuene中，让topic+queneid的锁能生效 ![image](https://user-images.githubusercontent.com/24490175/278995066-e3a7a3a0-a6b3-4f80-aef5-dcaff46e4cc4.png)
   
   嗯嗯，这样可以避免这个问题，目前我提交了一个修改的版本 https://github.com/apache/rocketmq/pull/7525


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "DongyuanPan (via GitHub)" <gi...@apache.org>.
DongyuanPan commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1788462099

   > > 嗯嗯，好的，7511也是我提的，我们看代码找到了一个暂时的解决方式，将同一个LMQ的消息，始终发送到同一个MessageQuene中，让topic+queneid的锁能生效 ![image](https://user-images.githubusercontent.com/24490175/278995066-e3a7a3a0-a6b3-4f80-aef5-dcaff46e4cc4.png)
   > 
   > 嗯嗯，这样可以简单暂时避免这个问题。
   目前我提交了一个修改的版本，解决这个并发问题，可以CR一下 [apache/rocketmq#7525](https://github.com/apache/rocketmq/pull/7525)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780430843

   进过测试发现，addSendingMessages 导致两条lmq消息offset相同，addSendingMessages 只add进一条，hashcode和equals是根据offset判断的


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780485102

   rocktmq 5.1.4版本，rocketmq-mqtt 1.0.1 版本
   1，项目demo改RocketMQProducer 改成5个子线程同步发送5条mq消息
   2，MqttConsumer可以观察到没有完整收到5条消息，消息丢失了
   3，如果MqttConsumer完整收到5条消息，重复1，2步骤，继续观察，比较容易复现


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "DongyuanPan (via GitHub)" <gi...@apache.org>.
DongyuanPan commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780507473

   > rocktmq 5.1.4版本，rocketmq-mqtt 1.0.1 版本 1，项目demo改RocketMQProducer 改成5个子线程同步发送5条mq消息 2，MqttConsumer可以观察到没有完整收到5条消息，消息丢失了 3，如果MqttConsumer完整收到5条消息，重复1，2步骤，继续观察，比较容易复现
   
   我用的develop分支的最新代码，没有复现哈


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "lzkisok (via GitHub)" <gi...@apache.org>.
lzkisok commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1780686198

   ![企业微信截图_9d8aaffd-c167-4780-a4bf-cb9358fbacbb](https://github.com/apache/rocketmq-mqtt/assets/24490175/867094e4-b6b4-4e1e-98e6-1148ce0fe131)
   这个是cs 一次拉了两条消息，两条消息的INNER_MULTI_QUEUE_OFFSET 相同，导致addSendingMessages只能成功一次


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] 开启5个子线程发送5条MQ消息，MqttConsumer 只能收到4条或者更少，必现。 [rocketmq-mqtt]

Posted by "DongyuanPan (via GitHub)" <gi...@apache.org>.
DongyuanPan commented on issue #215:
URL: https://github.com/apache/rocketmq-mqtt/issues/215#issuecomment-1784545860

   > rmq5.1.4版本，比较容易复现，多试几次，很容易复现
   
   是的，这个版本的RMQ复现了，应该是一个BUG，这个到RMQ的ISSUE讨论吧 
   
   https://github.com/apache/rocketmq/issues/7511


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org