You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/11/29 10:05:46 UTC

[GitHub] [kafka] Asura7969 opened a new pull request #11548: [KAFKA-13487]-Create a topic partition directory based on the size of the directory

Asura7969 opened a new pull request #11548:
URL: https://github.com/apache/kafka/pull/11548


   In the actual production environment, the file size generated by each subject partition is different. As a result, the disk usage of the base directory is unbalanced.It is not convenient to use the existing script to migrate the topic replica(such as kafka-reassign-partitions.sh)
   
   Add Broker-level configuration items:**log.directory.select.strategy**, there are two values: **partition** and **size**
   * **partition**: Sort by the number of directories under each base directory (log.dirs) (the current version of the new partition creation strategy)
   * **size**: Sort by the size of each directory (log.dirs), the smallest directory has the highest allocation right
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] Asura7969 edited a comment on pull request #11548: KAFKA-13487:Create a topic partition directory based on the size of the directory

Posted by GitBox <gi...@apache.org>.
Asura7969 edited a comment on pull request #11548:
URL: https://github.com/apache/kafka/pull/11548#issuecomment-986908189


   > Thanks @Asura7969 for the PR. Since this is adding a new configuration, this change needs a KIP: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.
   
   Thank you for following this PR, I found an existing [KIP-178](https://cwiki.apache.org/confluence/display/KAFKA/KIP-178%3A+Size-based+log+directory+selection+strategy), but I cannot add KIP to this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] Asura7969 commented on pull request #11548: KAFKA-13487:Create a topic partition directory based on the size of the directory

Posted by GitBox <gi...@apache.org>.
Asura7969 commented on pull request #11548:
URL: https://github.com/apache/kafka/pull/11548#issuecomment-986908189


   > Thanks @Asura7969 for the PR. Since this is adding a new configuration, this change needs a KIP: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.
   
   I found an existing [KIP-178](https://cwiki.apache.org/confluence/display/KAFKA/KIP-178%3A+Size-based+log+directory+selection+strategy), but I cannot add KIP to this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] mimaison commented on pull request #11548: KAFKA-13487:Create a topic partition directory based on the size of the directory

Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11548:
URL: https://github.com/apache/kafka/pull/11548#issuecomment-986634102


   Thanks @Asura7969 for the PR. Since this is adding a new configuration, this change needs a KIP: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] mimaison commented on pull request #11548: KAFKA-13487:Create a topic partition directory based on the size of the directory

Posted by GitBox <gi...@apache.org>.
mimaison commented on pull request #11548:
URL: https://github.com/apache/kafka/pull/11548#issuecomment-986948615


   KIP-178 was never voted. I recommend checking the discussion to see what happened at the time. Considering it's really old, you may want to create a brand new KIP instead of restarting the discussion on KIP-178.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org