You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/10/04 18:39:57 UTC

[GitHub] [kafka] piotrrzysko opened a new pull request #9371: KAFKA-10510: Validate replication factor consistency on reassignment

piotrrzysko opened a new pull request #9371:
URL: https://github.com/apache/kafka/pull/9371


   This patch introduces server-side validation that checks if applying requested reassignment won't lead to a situation where different partitions of the same topic have different replication factors. 
   
   If a given partition has a pending reassignment, then the target replication factor is used by the validator.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] piotrrzysko commented on pull request #9371: KAFKA-10510: Validate replication factor consistency on reassignment

Posted by GitBox <gi...@apache.org>.

piotrrzysko commented on pull request #9371:
URL: https://github.com/apache/kafka/pull/9371#issuecomment-704430920


   Thanks for the review, @stanislavkozlovski. If I understand your point clearly, you are wondering whether this validation will block changing the replication factor for all partitions (replication factor of a topic). Right? It will not if all requested reassignments in the batch have the same number of target replicas for a given topic. Please take look at the test `ReassignPartitionsIntegrationTest::testReassignmentFailOnInconsistentReplicationFactorBetweenPartitions` - topic `bar`. It is a simple case because the topic has only one partition, but it shows that changing the replication factor from 3 to 4 is possible. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] piotrrzysko commented on pull request #9371: KAFKA-10510: Validate replication factor consistency on reassignment

Posted by GitBox <gi...@apache.org>.

piotrrzysko commented on pull request #9371:
URL: https://github.com/apache/kafka/pull/9371#issuecomment-703823226


   Hi @stanislavkozlovski, would you mind taking a look at this PR?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] stanislavkozlovski commented on pull request #9371: KAFKA-10510: Validate replication factor consistency on reassignment

Posted by GitBox <gi...@apache.org>.

stanislavkozlovski commented on pull request #9371:
URL: https://github.com/apache/kafka/pull/9371#issuecomment-704300915


   Thanks for the PR @piotrrzysko. The changes look good although I'm not sure if we're not regressing the ability to change the replication factor of a topic if we're to outright block it.
   
   Perhaps we could have a workaround in the form of allowing the reassignment batch to run if it's changing all the partitions for the topic. Another solution I like more is adding a flag to the AlterPartitionReassignments API that allows it to change the replication factor and keeping this validation.
   
   In the long term, I think we may want to consider an API that allows you to alter the replication factor of a topic.
   
   cc @ijuma @gwenshap @cmccabe @hachikuji @soondenana do you happen to have any thoughts on the matter?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org