You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "flore77 (via GitHub)" <gi...@apache.org> on 2026/03/06 09:51:50 UTC
[PR] KIP-646: Serializer API should support ByteBuffer [kafka]
flore77 opened a new pull request, #21656:
URL: https://github.com/apache/kafka/pull/21656
This implements: https://cwiki.apache.org/confluence/display/KAFKA/KIP-646+Serializer+API+should+support+ByteBuffer
## Summary
- Adds serializeToByteBuffer() default method to the Serializer interface, returning ByteBuffer instead of byte[]
- Overrides in built-in serializers (ByteArray, ByteBuffer, Bytes, Integer, Long, Short, Float, Double, Boolean, Void) with optimized implementations that avoid unnecessary copies
- Updates the producer pipeline (KafkaProducer → RecordAccumulator → ProducerBatch) to use ByteBuffer end-to-end, eliminating byte[] → ByteBuffer wrapping in MemoryRecordsBuilder
- Adds ByteBuffer overload to Partitioner interface with zero-copy optimization for exact backing arrays
- Extends Utils.murmur2() and BuiltInPartitioner.partitionForKey() to accept ByteBuffer natively, with byte[] versions delegating to them
## Test plan
- SerializationTest — verifies serializeToByteBuffer() matches serialize() output for all built-in serializers, null handling, and ByteBuffer zero-copy
- PartitionerTest — verifies default ByteBuffer partition method passes backing array directly (zero-copy) and copies when buffer is a slice
- KafkaProducerTest — end-to-end producer pipeline with ByteBuffer path
- RecordAccumulatorTest, ProducerBatchTest, SenderTest, MockProducerTest — existing tests pass via byte[] overloads that delegate to ByteBuffer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-4059390058
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-4942253663
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-4921662966
This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.
<p>
If you are having difficulty finding a reviewer, please reach out on the
[mailing list](https://kafka.apache.org/contact).
<p>
If this PR is no longer valid or desired, please feel free to close it. If no activity
occurs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "clolov (via GitHub)" <gi...@apache.org>.
clolov commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-4012005529
Hello @flore77! As far as I have seen KIP-646 was never voted on and we can't really make a change before it passes a vote. If you are interested in taking the KIP forward I would first suggest you bump the DISCUSS thread so that people can have the opportunity to raise suggestions. Afterwards, open a VOTE thread and if that passes successfully we can come back to reviewing this PR 😊
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-4220258981
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "flore77 (via GitHub)" <gi...@apache.org>.
flore77 commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-4015817855
@clolov ack. I was not aware of the voting process, I requested an ASF account to drive this KIP forward. Thanks for letting me know!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-6093441953
This PR is being marked as stale since it has not had any activity in 90 days. If you
would like to keep this PR alive, please leave a comment asking for a review. If the PR has
merge conflicts, update it with the latest from the base branch.
<p>
If you are having difficulty finding a reviewer, please reach out on the
[mailing list](https://kafka.apache.org/contact).
<p>
If this PR is no longer valid or desired, please feel free to close it. If no activity
occurs in the next 30 days, it will be automatically closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-5761: Serializer API should support ByteBuffer [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21656:
URL: https://github.com/apache/kafka/pull/21656#issuecomment-6117261430
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org