You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Divij Vaidya (Jira)" <ji...@apache.org> on 2023/02/07 09:49:00 UTC

[jira] [Commented] (KAFKA-14635) Compression optimization: Use direct memory buffers for (de)compression

    [ https://issues.apache.org/jira/browse/KAFKA-14635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685183#comment-17685183 ] 

Divij Vaidya commented on KAFKA-14635:
--------------------------------------

This might not have much benefit since Zstd-JNI uses primitiveArray API of JNI to access the heap buffer directly without any copying. Hence, usage of direct buffer may not produce much impact. This needs to be validated with a prototype and benchmark.

> Compression optimization: Use direct memory buffers for (de)compression
> -----------------------------------------------------------------------
>
>                 Key: KAFKA-14635
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14635
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: Divij Vaidya
>            Priority: Major
>
> Provide a pool of direct buffers to zstd-jni for it’s internal usage. Direct buffers is an ideal use case for scenarios where data is transferred across JNI such as the case in (de) compression. The latest version of zstd-jni works with direct buffers.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)