You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Sahil Takiar (JIRA)" <ji...@apache.org> on 2019/02/21 18:46:00 UTC
[jira] [Created] (HDFS-14306) Re-use Java buffers across libhdfs
read calls
Sahil Takiar created HDFS-14306:
-----------------------------------
Summary: Re-use Java buffers across libhdfs read calls
Key: HDFS-14306
URL: https://issues.apache.org/jira/browse/HDFS-14306
Project: Hadoop HDFS
Issue Type: Improvement
Components: hdfs-client, libhdfs, native
Reporter: Sahil Takiar
Assignee: Sahil Takiar
Filesystem connectors that don't natively support {{ByteBuffer}}s cannot fully take advantage of the direct read optimizations added in HDFS-2834 / HDFS-3110. In this case, they fall back to the regular read path which allocates a Java byte array for every read. This leads to a lot of perf overhead wasted in {{TypeArrayKlass::allocate_common}}.
A performance optimization would be to re-use Java byte arrays between calls to {{hdfsRead}} / {{hdfsPread}} so that new arrays don't need to be allocate for every single call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org