You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Jason Gustafson (Jira)" <ji...@apache.org> on 2020/08/11 22:18:00 UTC
[jira] [Updated] (KAFKA-10389) Fix zero copy tagged arrays
[ https://issues.apache.org/jira/browse/KAFKA-10389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Gustafson updated KAFKA-10389:
------------------------------------
Description:
We're missing a bunch of zero-copy checks in the message generator for tagged fields. As an example, try adding the following field to `SimpleExampleMessage.json`:
{code}
{ "name": "taggedZeroCopyByteBuffer", "versions": "1+", "type": "bytes", "zeroCopy": true,
"taggedVersions": "1+", "tag": 8, "ignorable": true },
{code}
The generated code has a few compilation errors because it assumes the byte array format. For example:
{code}
if (taggedZeroCopyByteBuffer.hasRemaining()) {
_writable.writeUnsignedVarint(8);
_writable.writeUnsignedVarint(this.taggedZeroCopyByteBuffer.length + ByteUtils.sizeOfUnsignedVarint(this.taggedZeroCopyByteBuffer.length + 1));
_writable.writeUnsignedVarint(this.taggedZeroCopyByteBuffer.length + 1);
_writable.writeByteArray(this.taggedZeroCopyByteBuffer);
}
{code}
The `toStruct` and `fromStruct` methods also seem to be missing checks.
was:
We're missing a bunch of zero-copy checks in the message generator. As an example, try adding the following field to `SimpleExampleMessage.json`:
{code}
{ "name": "taggedZeroCopyByteBuffer", "versions": "1+", "type": "bytes", "zeroCopy": true,
"taggedVersions": "1+", "tag": 8, "ignorable": true },
{code}
The generated code has a few compilation errors because it assumes the byte array format. For example:
{code}
if (taggedZeroCopyByteBuffer.hasRemaining()) {
_writable.writeUnsignedVarint(8);
_writable.writeUnsignedVarint(this.taggedZeroCopyByteBuffer.length + ByteUtils.sizeOfUnsignedVarint(this.taggedZeroCopyByteBuffer.length + 1));
_writable.writeUnsignedVarint(this.taggedZeroCopyByteBuffer.length + 1);
_writable.writeByteArray(this.taggedZeroCopyByteBuffer);
}
{code}
The `toStruct` and `fromStruct` methods also seem to be missing checks.
> Fix zero copy tagged arrays
> ---------------------------
>
> Key: KAFKA-10389
> URL: https://issues.apache.org/jira/browse/KAFKA-10389
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Priority: Major
>
> We're missing a bunch of zero-copy checks in the message generator for tagged fields. As an example, try adding the following field to `SimpleExampleMessage.json`:
> {code}
> { "name": "taggedZeroCopyByteBuffer", "versions": "1+", "type": "bytes", "zeroCopy": true,
> "taggedVersions": "1+", "tag": 8, "ignorable": true },
> {code}
> The generated code has a few compilation errors because it assumes the byte array format. For example:
> {code}
> if (taggedZeroCopyByteBuffer.hasRemaining()) {
> _writable.writeUnsignedVarint(8);
> _writable.writeUnsignedVarint(this.taggedZeroCopyByteBuffer.length + ByteUtils.sizeOfUnsignedVarint(this.taggedZeroCopyByteBuffer.length + 1));
> _writable.writeUnsignedVarint(this.taggedZeroCopyByteBuffer.length + 1);
> _writable.writeByteArray(this.taggedZeroCopyByteBuffer);
> }
> {code}
> The `toStruct` and `fromStruct` methods also seem to be missing checks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)