You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "edoardocomar (via GitHub)" <gi...@apache.org> on 2025/09/05 10:09:04 UTC
[PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
edoardocomar opened a new pull request, #20485:
URL: https://github.com/apache/kafka/pull/20485
Apply the same 10000 new partitions limit when creating partitions as it is already done when creating new topics.
Cluster unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "edoardocomar (via GitHub)" <gi...@apache.org>.
edoardocomar commented on PR #20485:
URL: https://github.com/apache/kafka/pull/20485#issuecomment-3257883002
I think it makes sense to backport to 4.0.1 and 4.1 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20485:
URL: https://github.com/apache/kafka/pull/20485#issuecomment-3673372949
This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.
<p>
If you are having difficulty finding a reviewer, please reach out on the
[mailing list](https://kafka.apache.org/contact).
<p>
If this PR is no longer valid or desired, please feel free to close it. If no activity
occurs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20485:
URL: https://github.com/apache/kafka/pull/20485#issuecomment-4196425217
This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.
<p>
If you are having difficulty finding a reviewer, please reach out on the
[mailing list](https://kafka.apache.org/contact).
<p>
If this PR is no longer valid or desired, please feel free to close it. If no activity
occurs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "edoardocomar (via GitHub)" <gi...@apache.org>.
edoardocomar commented on PR #20485:
URL: https://github.com/apache/kafka/pull/20485#issuecomment-3312558705
Thanks @kirktrue makes sense
- I would like the message to be the same for create partitions and create topic
- I thought the limit was temporary because of this comment
https://github.com/apache/kafka/pull/13766#issuecomment-1564748396
but it look it may stay for longer than what we knew then.
changing the error message in create topic may potentially impact clients (so does this change I'm proposing)
but a KIP seems overkill.
would you change both? only the new one and make it longer so that it contains both the well known message as well as the numbers ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #20485: KAFKA-19673: Disallow create too many partitions also in createPartitions
URL: https://github.com/apache/kafka/pull/20485
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20485:
URL: https://github.com/apache/kafka/pull/20485#issuecomment-4394207596
This PR has been closed since it has not had any activity in 120 days. If you feel like this
was a mistake, or you would like to continue working on it, please feel free to re-open the
PR and ask for a review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #20485:
URL: https://github.com/apache/kafka/pull/20485#issuecomment-3287413433
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19673: Disallow create too many partitions also in createPartitions [kafka]
Posted by "showuon (via GitHub)" <gi...@apache.org>.
showuon commented on code in PR #20485:
URL: https://github.com/apache/kafka/pull/20485#discussion_r2663595954
##########
metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java:
##########
@@ -1215,7 +1215,23 @@ static void validateTotalNumberOfPartitions(CreateTopicsRequestData request, int
} else {
totalPartitions += topic.assignments().size();
}
+ }
+ if (totalPartitions > MAX_PARTITIONS_PER_BATCH) {
+ throw new PolicyViolationException("Excessively large number of partitions per request.");
+ }
+ }
+
+ static void validateTotalNumberOfPartitions(List<CreatePartitionsTopic> topics) {
+ int totalPartitions = 0;
+ for (CreatePartitionsTopic topic: topics) {
+ if (topic.assignments() == null || topic.assignments().isEmpty()) {
+ if (topic.count() > 0) {
+ totalPartitions += topic.count();
+ }
+ } else {
+ totalPartitions += topic.assignments().size();
+ }
Review Comment:
Is this right? If a topic already has 9999 partitions, and 1 partition to be added, the `topic.count()` will be set to 10000, which exceeds the limit. However, it only needs to create 1 partition record in this case. If we really want to limit the size for createPartition, I think we should count the number of `to be created` partitions, instead of relying on the topic count only. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org