You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2021/10/27 03:53:52 UTC
[GitHub] [rocketmq] Cczzzz opened a new issue #3434: BUG ： Consumers start slowly
Cczzzz opened a new issue #3434:
URL: https://github.com/apache/rocketmq/issues/3434
It takes a long time for consumers to start to consume. max waiting time is ClientConfig#heartbeatBrokerInterval = 1000*30
(30s).
and in broker broker.log,you will find :
2021-10-27 11:29:03 WARN PullMessageThread_35 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:03 WARN PullMessageThread_49 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:03 WARN PullMessageThread_62 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:03 WARN PullMessageThread_75 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:04 WARN PullMessageThread_33 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:04 WARN PullMessageThread_44 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:04 WARN PullMessageThread_55 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:04 WARN PullMessageThread_106 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:04 WARN PullMessageThread_102 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:05 WARN PullMessageThread_37 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:05 WARN PullMessageThread_109 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:05 WARN PullMessageThread_1 - The broker's subscription is not latest, group: test_order *
2021-10-27 11:29:05 WARN PullMessageThread_4 - The broker's subscription is not latest, group: test_order *
原因：
broker 中会比较 subscription 的版本号，这个版本号由客户端心跳上报。理论broker 的版本号和客户端是一致的。
在进行rebalance 时，如果rebalance 发生了变化（第一次启动会触发），就会更新本地的版本号，然后主动更新到broker。
RebalancePushImpl#83
// notify broker
this.getmQClientFactory().sendHeartbeatToAllBrokerWithLock();
理论上这样 客户端和broker 的版本号还是一样的。
但是在实际情况中，rebalance 完成后这时有可能心跳定时任务也在进行，这会导致获取 获取 lockHeartbeat 锁失败。无法发送心跳。此时 broker 的版本号会小于客户端版本号。直到 下次心跳定时任务
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Cczzzz commented on issue #3434: BUG ： Consumers start slowly
Posted by GitBox <gi...@apache.org>.
Cczzzz commented on issue #3434:
URL: https://github.com/apache/rocketmq/issues/3434#issuecomment-952546173
这个问题在 非新启动的时候应该也会，比如当 消费者重负载时，如果本机的系统时间大于其他同group 消费者，同时发送心跳失败，就得等到下次心跳才能正常消费
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] panzhi33 edited a comment on issue #3434: BUG ： Consumers start slowly
Posted by GitBox <gi...@apache.org>.
panzhi33 edited a comment on issue #3434:
URL: https://github.com/apache/rocketmq/issues/3434#issuecomment-952620879
> 修复方式 修改 MQClientInstance#sendHeartbeatToAllBrokerWithLock 方法 ，返回值改成 boolean ，表示是否正常发送了心跳任务。如果发送失败，进行定时重试
>
> 或者 tryLock 改成 tryLock(long time, TimeUnit unit) throws InterruptedException; 允许传入最大的等待时间。rebalance 线程可以被堵塞吗？
可以用tryLock(long time, TimeUnit unit)来优化
MQClientInstance#sendHeartbeatToAllBrokerWithLock 如果发生broker掉线，在下次心跳期间， 就会一直失败定时重试
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] caigy commented on issue #3434: BUG ： Consumers start slowly
Posted by GitBox <gi...@apache.org>.
caigy commented on issue #3434:
URL: https://github.com/apache/rocketmq/issues/3434#issuecomment-1039803890
If heartbeat time is short, would it be more simple to use `tryLock(long time, TimeUnit unit)` to wait for lock releasing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] Cczzzz commented on issue #3434: BUG ： Consumers start slowly
Posted by GitBox <gi...@apache.org>.
Cczzzz commented on issue #3434:
URL: https://github.com/apache/rocketmq/issues/3434#issuecomment-952516089
修复方式
修改 MQClientInstance#sendHeartbeatToAllBrokerWithLock 方法 ，返回值改成 boolean ，表示是否正常发送了心跳任务。如果发送失败，进行定时重试
或者
tryLock
改成
tryLock(long time, TimeUnit unit) throws InterruptedException;
允许传入最大的等待时间。rebalance 线程可以被堵塞吗？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] caigy commented on issue #3434: BUG ： Consumers start slowly
Posted by GitBox <gi...@apache.org>.
caigy commented on issue #3434:
URL: https://github.com/apache/rocketmq/issues/3434#issuecomment-1039803890
If heartbeat time is short, would it be more simple to use `tryLock(long time, TimeUnit unit)` to wait for lock releasing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq] panzhi33 commented on issue #3434: BUG ： Consumers start slowly
Posted by GitBox <gi...@apache.org>.
panzhi33 commented on issue #3434:
URL: https://github.com/apache/rocketmq/issues/3434#issuecomment-952620879
> 修复方式 修改 MQClientInstance#sendHeartbeatToAllBrokerWithLock 方法 ，返回值改成 boolean ，表示是否正常发送了心跳任务。如果发送失败，进行定时重试
>
> 或者 tryLock 改成 tryLock(long time, TimeUnit unit) throws InterruptedException; 允许传入最大的等待时间。rebalance 线程可以被堵塞吗？
这个问题出现的概率很小，我觉得这不是一个bug，只是延迟了，并不是没解决。
可以用tryLock(long time, TimeUnit unit)来优化
MQClientInstance#sendHeartbeatToAllBrokerWithLock 如果发生broker掉线，在下次心跳期间， 就会一直失败定时重试
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org