You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by vsair <gi...@git.apache.org> on 2017/04/14 06:55:06 UTC
[GitHub] incubator-rocketmq pull request #92: [ROCKETMQ-175] Consumer may miss messag...
GitHub user vsair opened a pull request:
https://github.com/apache/incubator-rocketmq/pull/92
[ROCKETMQ-175] Consumer may miss messages because of inconsistent sub\u2026
For push mode, update subscription version when rebalance result changed, then send heart beat to broker immediately.
Maybe subscriptions management mechanism could be re-designed to meet more complicated scenarios.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/vsair/incubator-rocketmq ROCKETMQ-175
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rocketmq/pull/92.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #92
----
commit 61901647fb1851bbb36b4734195206a364a0c0bd
Author: vsair <li...@gmail.com>
Date: 2017-04-14T06:47:00Z
[ROCKETMQ-175] Consumer may miss messages because of inconsistent subscription
1. For push mode, update subscription version when rebalance result changed.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by vsair <gi...@git.apache.org>.
Github user vsair commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
Yeah, you're right. There is no way to solve the problem in existing design.
But the result is :
when user stop the consumer has diff subscription, broker could hold the correct info.
I also look forward to the solution in https://issues.apache.org/jira/browse/ROCKETMQ-94.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by vsair <gi...@git.apache.org>.
Github user vsair commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
This PR does't break the restrict that all consumers should have the same subscription. It only focus on broker should hold the online consumer's subscription. Miss messages is just one result caused by inconsistency subscriptions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by vsair <gi...@git.apache.org>.
Github user vsair commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
OK
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
[](https://coveralls.io/builds/11084533)
Coverage decreased (-0.2%) to 34.443% when pulling **61901647fb1851bbb36b4734195206a364a0c0bd on vsair:ROCKETMQ-175** into **7bcb3b3eae1e3c441861f2a3cd79ff54a8e691b9 on apache:develop**.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by dongeforever <gi...@git.apache.org>.
Github user dongeforever commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
This PR is merged @vsair
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by Jaskey <gi...@git.apache.org>.
Github user Jaskey commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
@vsair
What I mean is that the scene this pr is going to fix is breaking the restrict that the same consumer group has the same subscriptions.
Though this may help improve the refresh the subscriptions, but the same problem is still there remains no solution in the existing design.
IMO, we should solve this problem directly to solve all the related problem. Which is issued in https://issues.apache.org/jira/browse/ROCKETMQ-94
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq pull request #92: [ROCKETMQ-175] Consumer may miss messag...
Posted by vsair <gi...@git.apache.org>.
Github user vsair closed the pull request at:
https://github.com/apache/incubator-rocketmq/pull/92
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by Jaskey <gi...@git.apache.org>.
Github user Jaskey commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
Actually missing message can not be avoided when the diffrerent tags are subscribed in the same consumer groups.
Say for two queues q1 q2, two consumers c1 c2.
c1->TAG1
c2->TAGB
when allocating, c1->q1, c2->q2 100 messages are sent to this topic with TAG1.
C2 will recive nothing and c1 will recieve 50 messages. but the consumer offset of q2 is till moving on. Then 50 messages are missing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by dongeforever <gi...@git.apache.org>.
Github user dongeforever commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
@Jaskey It may take a long time to fix the issue ROCKETMQ-94.
And we could improve it step by step.
Now one consumer startup will refresh the subscriptions, but the consumer shutdown does not.
This maybe confusing.
IMO, it is reasonable to refresh when rebalance occurs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rocketmq issue #92: [ROCKETMQ-175] Consumer may miss messages beca...
Posted by dongeforever <gi...@git.apache.org>.
Github user dongeforever commented on the issue:
https://github.com/apache/incubator-rocketmq/pull/92
@zhouxinyu @vongosling
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---