You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/07/05 20:53:33 UTC
[GitHub] [kafka] mingaliu opened a new pull request #8982: Adding metrics to measure the idempotent producers
mingaliu opened a new pull request #8982:
URL: https://github.com/apache/kafka/pull/8982
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mingaliu commented on pull request #8982: Adding metrics to measure the idempotent producers
Posted by GitBox <gi...@apache.org>.
mingaliu commented on pull request #8982:
URL: https://github.com/apache/kafka/pull/8982#issuecomment-654612942
Thanks @abbccdda. The JIRA is https://issues.apache.org/jira/browse/KAFKA-10242.
I wonder why do we need a KIP here? Given there is no public interface or any protocol change, and the change is small and low risk.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mingaliu closed pull request #8982: Adding metrics to measure the idempotent producers
Posted by GitBox <gi...@apache.org>.
mingaliu closed pull request #8982:
URL: https://github.com/apache/kafka/pull/8982
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mingaliu commented on pull request #8982: Adding metrics to measure the idempotent producers
Posted by GitBox <gi...@apache.org>.
mingaliu commented on pull request #8982:
URL: https://github.com/apache/kafka/pull/8982#issuecomment-658356953
@abbccdda , can you review this PR? The failing checks seems due to unreliable tests (not related to this PR).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mingaliu commented on pull request #8982: Adding metrics to measure the idempotent producers
Posted by GitBox <gi...@apache.org>.
mingaliu commented on pull request #8982:
URL: https://github.com/apache/kafka/pull/8982#issuecomment-653938131
We found it is very useful to track the number of idempotent producers that broker is tracking.
This is especially true for client (< 2.4) where the client retry might generate too many different idempotent producers which might trigger broker GC.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] abbccdda commented on pull request #8982: Adding metrics to measure the idempotent producers
Posted by GitBox <gi...@apache.org>.
abbccdda commented on pull request #8982:
URL: https://github.com/apache/kafka/pull/8982#issuecomment-654040592
Thanks for the PR. We basically need a JIRA and a KIP to proceed on this change, see:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mingaliu commented on pull request #8982: Adding metrics to measure the idempotent producers
Posted by GitBox <gi...@apache.org>.
mingaliu commented on pull request #8982:
URL: https://github.com/apache/kafka/pull/8982#issuecomment-653939755
Can you review the change? @ijuma ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mingaliu edited a comment on pull request #8982: Adding metrics to measure the idempotent producers
Posted by GitBox <gi...@apache.org>.
mingaliu edited a comment on pull request #8982:
URL: https://github.com/apache/kafka/pull/8982#issuecomment-653938131
We found it is very useful to track the number of idempotent producers that broker is tracking.
In our production environment, we have many idempotent producers to a cluster and sometimes that number increased to very high number which requires some attention to mitigate.
This is especially true for client (< 2.4) where the client retry might generate too many different idempotent producers which can trigger broker GC.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org