You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/07/19 04:49:57 UTC
[GitHub] [pulsar] Technoboy- opened a new pull request, #16672: [improve][broker] Use the unique consumer identifier to make the range more spread out.
Technoboy- opened a new pull request, #16672:
URL: https://github.com/apache/pulsar/pull/16672
Fixes #16654
Master Issue: #16654
### Motivation
Use the consumer identifier to make the range more spread out to avoid confusing user.
### Documentation
- [x] `doc-not-needed`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on pull request #16672: [improve][broker] Use the unique consumer identifier to make the range more spread out.
Posted by GitBox <gi...@apache.org>.
merlimat commented on PR #16672:
URL: https://github.com/apache/pulsar/pull/16672#issuecomment-1190607026
I think this will remove the possibility of having a stable assignment logic. Right now, if you specify the same consumer names, you will be guaranteed to receive the same keys after reconnections.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on a diff in pull request #16672: [improve][broker] Add `keyHashRangeIndex` to distinguish the same hashRanges with the same consumerName
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on code in PR #16672:
URL: https://github.com/apache/pulsar/pull/16672#discussion_r953397190
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java:
##########
@@ -1088,6 +1088,13 @@ public SubscriptionStatsImpl getStats(Boolean getPreciseBacklog, boolean subscri
consumerStats.keyHashRanges = consumerKeyHashRanges.get(consumer).stream()
.map(Range::toString)
.collect(Collectors.toList());
+ long sameNameConsumerCount = consumerKeyHashRanges.keySet()
+ .stream()
+ .filter(c -> c.consumerName().equals(consumer.consumerName()))
+ .count();
Review Comment:
Can we get the consumer count without going through all the items of consumerKeyHashRanges? If we have many consumers under a topic, this might introduce CPU spike
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] leizhiyuan commented on a diff in pull request #16672: [improve][broker] Use the unique consumer identifier to make the range more spread out.
Posted by GitBox <gi...@apache.org>.
leizhiyuan commented on code in PR #16672:
URL: https://github.com/apache/pulsar/pull/16672#discussion_r939633832
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/ConsistentHashingStickyKeyConsumerSelector.java:
##########
@@ -143,4 +145,8 @@ public Map<Consumer, List<Range>> getConsumerKeyHashRanges() {
}
return result;
}
+
+ private String getConsumerUniqueId(Consumer consumer) {
Review Comment:
should we make consumerUniqueId a method of Consumer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on pull request #16672: [improve][broker] Add `keyHashRangeIndex` to distinguish the same hashRanges with the same consumerName
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on PR #16672:
URL: https://github.com/apache/pulsar/pull/16672#issuecomment-1224234881
> > I think this will remove the possibility of having a stable assignment logic. Right now, if you specify the same consumer names, you will be guaranteed to receive the same keys after reconnections.
>
> Oh, good point. Maybe we should try to add more information to the topic stats instead of changing the behavior.
Add a new field to help user distinguish the same hashRanges.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #16672: [improve][broker] Use the unique consumer identifier to make the range more spread out.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #16672:
URL: https://github.com/apache/pulsar/pull/16672#issuecomment-1188596614
@Technoboy- Please provide a correct documentation label for your PR.
Instructions see [Pulsar Documentation Label Guide](https://docs.google.com/document/d/1Qw7LHQdXWBW9t2-r-A7QdFDBwmZh6ytB4guwMoXHqc0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #16672: [improve][broker] Use the unique consumer identifier to make the range more spread out.
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on PR #16672:
URL: https://github.com/apache/pulsar/pull/16672#issuecomment-1191438111
> I think this will remove the possibility of having a stable assignment logic. Right now, if you specify the same consumer names, you will be guaranteed to receive the same keys after reconnections.
Oh, good point. Maybe we should try to add more information to the topic stats instead of changing the behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- closed pull request #16672: [improve][broker] Add `keyHashRangeIndex` to distinguish the same hashRanges with the same consumerName
Posted by GitBox <gi...@apache.org>.
Technoboy- closed pull request #16672: [improve][broker] Add `keyHashRangeIndex` to distinguish the same hashRanges with the same consumerName
URL: https://github.com/apache/pulsar/pull/16672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on pull request #16672: [improve][broker] Add `keyHashRangeIndex` to distinguish the same hashRanges with the same consumerName
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on PR #16672:
URL: https://github.com/apache/pulsar/pull/16672#issuecomment-1229780922
Decide to add more descriptions for this part.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org