You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/01/05 14:38:08 UTC
[GitHub] [kafka] huangyiminghappy opened a new pull request #9823: KAFKA-10903:Optimize producerBatch order performance
huangyiminghappy opened a new pull request #9823:
URL: https://github.com/apache/kafka/pull/9823
please. see https://issues.apache.org/jira/browse/KAFKA-10903,
sa wo konw,if we need sort the producerBatch by sequence,now we use the new batch compare with the first batch in deque, and if the first batch in deque is less than the new batch,we will loop the deque and let the new batch insert the right position.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] huangyiminghappy edited a comment on pull request #9823: KAFKA-10903:Optimize producerBatch order performance
Posted by GitBox <gi...@apache.org>.
huangyiminghappy edited a comment on pull request #9823:
URL: https://github.com/apache/kafka/pull/9823#issuecomment-762572624
> Thanks for the PR. Did you see a performance issue as a result of this? Do you have some numbers to share?
thank you reply, Sometimes we don't need data to support it. may be we can use common sense to judge,When we insert data into an ordered deque, we will determine whether it is inside or outside the deque,and may better than always loop the deque
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] huangyiminghappy commented on pull request #9823: KAFKA-10903:Optimize producerBatch order performance
Posted by GitBox <gi...@apache.org>.
huangyiminghappy commented on pull request #9823:
URL: https://github.com/apache/kafka/pull/9823#issuecomment-762572624
> Thanks for the PR. Did you see a performance issue as a result of this? Do you have some numbers to share?
thank you reply, sorry,i don't have the data support, but we can make very simple reasoning,we don't always need loop the deque,Just need to add a judgment,and May do less recursion，it is Obvious
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] huangyiminghappy edited a comment on pull request #9823: KAFKA-10903:Optimize producerBatch order performance
Posted by GitBox <gi...@apache.org>.
huangyiminghappy edited a comment on pull request #9823:
URL: https://github.com/apache/kafka/pull/9823#issuecomment-762572624
> Thanks for the PR. Did you see a performance issue as a result of this? Do you have some numbers to share?
thank you reply, Sometimes we don't need data to support it. we can use common sense to judge,When we insert data into an ordered deque, we will determine whether it is inside or outside the deque,and may better than always loop the deque
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #9823: KAFKA-10903:Optimize producerBatch order performance
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #9823:
URL: https://github.com/apache/kafka/pull/9823#issuecomment-761899771
Thanks for the PR. Did you see a performance issue as a result of this? Do you have some numbers to share?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on pull request #9823: KAFKA-10903:Optimize producerBatch order performance
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on pull request #9823:
URL: https://github.com/apache/kafka/pull/9823#issuecomment-763225099
@huangyiminghappy it seems that your assumption is, for most cases, there will just be one inflight request, and hence the batch to be enqueued would most likely need to be add to the last of the queue since very likely none of the batches in `deque` would have a sequence number, is that right?
Logically I think I agree with your reasoning here, and this optimization makes sense. But I'm wondering how much it is actually optimizing in practice -- looping over the `deque` should usually be very quick since the `deque` should be relatively short, like having single-digit number of batches.
Do you observe in your production environment, that the number of batches in `deque` could be very large and hence looping over it to the end could be very costly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org