You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/08/25 17:49:12 UTC

[GitHub] [kafka] chia7712 opened a new pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

chia7712 opened a new pull request #9220:
URL: https://github.com/apache/kafka/pull/9220


   issue: https://issues.apache.org/jira/browse/KAFKA-10433
   
   It is hot method so reusing the ByteBuffer can reduce a bunch of memory usage if the compression type supports BufferSupplier.
   
   **experiment**
   - duration: 1 minute
   - compression: LZ4
   - memory usage of byte array allocation: 4.52 GB -> 0.68 GB
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ijuma commented on pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #9220:
URL: https://github.com/apache/kafka/pull/9220#issuecomment-683342367


   @chia7712 here's what I had in mind:
   https://github.com/apache/kafka/pull/9229
   
   What do you think?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] chia7712 commented on pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

Posted by GitBox <gi...@apache.org>.
chia7712 commented on pull request #9220:
URL: https://github.com/apache/kafka/pull/9220#issuecomment-680194628


   > Would you be OK if I submit that as a PR and we can compare?
   
   Please feel free to submit another PR. We all love to see the better solution :)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] ijuma commented on pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #9220:
URL: https://github.com/apache/kafka/pull/9220#issuecomment-680182676


   @chia7712 Thanks for the PR. The intent is good, but I think the approach should be a bit different. As it happens, I have implemented this other approach. Would you be OK if I submit that as a PR and we can compare?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] chia7712 closed pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

Posted by GitBox <gi...@apache.org>.
chia7712 closed pull request #9220:
URL: https://github.com/apache/kafka/pull/9220


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [kafka] chia7712 commented on pull request #9220: KAFKA-10433 Reuse the ByteBuffer in validating compressed records

Posted by GitBox <gi...@apache.org>.
chia7712 commented on pull request #9220:
URL: https://github.com/apache/kafka/pull/9220#issuecomment-683400353


   close as there is a better approach (#9229)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org