You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/07/10 00:37:06 UTC
[GitHub] [kafka] guozhangwang opened a new pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks
guozhangwang opened a new pull request #9005:
URL: https://github.com/apache/kafka/pull/9005
Also piggy-back a small fix to use TreeMap other than HashMap to preserve iteration ordering.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on pull request #9005:
URL: https://github.com/apache/kafka/pull/9005#issuecomment-656814245
> Awesome, thanks for the fix! Build has a checkstyleTest violation but otherwise this looks good
Fixed the checkstyle.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on a change in pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on a change in pull request #9005:
URL: https://github.com/apache/kafka/pull/9005#discussion_r452562468
##########
File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignorTest.java
##########
@@ -1402,9 +1404,13 @@ public void shouldTriggerImmediateRebalanceOnTasksRevoked() {
final Map<String, Assignment> assignment = partitionAssignor.assign(metadata, new GroupSubscription(subscriptions)).groupAssignment();
// Verify at least one partition was revoked
- assertThat(assignment.get(CONSUMER_1).partitions(), not(allTasks));
+ assertThat(assignment.get(CONSUMER_1).partitions(), not(allPartitions));
assertThat(assignment.get(CONSUMER_2).partitions(), equalTo(emptyList()));
+ // Verify that stateless revoked tasks would not be assigned as standbys
+ assertThat(AssignmentInfo.decode(assignment.get(CONSUMER_2).userData()).activeTasks(), equalTo(emptyList()));
+ assertThat(AssignmentInfo.decode(assignment.get(CONSUMER_2).userData()).standbyTasks(), equalTo(emptyMap()));
Review comment:
Verified that without the fix, this line will fail.
##########
File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignorTest.java
##########
@@ -1402,9 +1404,13 @@ public void shouldTriggerImmediateRebalanceOnTasksRevoked() {
final Map<String, Assignment> assignment = partitionAssignor.assign(metadata, new GroupSubscription(subscriptions)).groupAssignment();
// Verify at least one partition was revoked
- assertThat(assignment.get(CONSUMER_1).partitions(), not(allTasks));
+ assertThat(assignment.get(CONSUMER_1).partitions(), not(allPartitions));
Review comment:
I found it was always true because we are comparing a list of tasks with a list of partitions ;)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang merged pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks
Posted by GitBox <gi...@apache.org>.
guozhangwang merged pull request #9005:
URL: https://github.com/apache/kafka/pull/9005
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] vvcephei commented on a change in pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks
Posted by GitBox <gi...@apache.org>.
vvcephei commented on a change in pull request #9005:
URL: https://github.com/apache/kafka/pull/9005#discussion_r453027992
##########
File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignorTest.java
##########
@@ -1402,9 +1404,13 @@ public void shouldTriggerImmediateRebalanceOnTasksRevoked() {
final Map<String, Assignment> assignment = partitionAssignor.assign(metadata, new GroupSubscription(subscriptions)).groupAssignment();
// Verify at least one partition was revoked
- assertThat(assignment.get(CONSUMER_1).partitions(), not(allTasks));
+ assertThat(assignment.get(CONSUMER_1).partitions(), not(allPartitions));
Review comment:
Oh, boy...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ableegoldman commented on a change in pull request #9005: KAFKA-10263: Do not assign standby for revoking stateless tasks
Posted by GitBox <gi...@apache.org>.
ableegoldman commented on a change in pull request #9005:
URL: https://github.com/apache/kafka/pull/9005#discussion_r452570044
##########
File path: streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignorTest.java
##########
@@ -1402,9 +1404,13 @@ public void shouldTriggerImmediateRebalanceOnTasksRevoked() {
final Map<String, Assignment> assignment = partitionAssignor.assign(metadata, new GroupSubscription(subscriptions)).groupAssignment();
// Verify at least one partition was revoked
- assertThat(assignment.get(CONSUMER_1).partitions(), not(allTasks));
+ assertThat(assignment.get(CONSUMER_1).partitions(), not(allPartitions));
Review comment:
Whoops
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org