You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2018/07/05 22:11:00 UTC
[jira] [Created] (IMPALA-7253) JNI calls should pass parameters
using direct byte buffers
Todd Lipcon created IMPALA-7253:
-----------------------------------
Summary: JNI calls should pass parameters using direct byte buffers
Key: IMPALA-7253
URL: https://issues.apache.org/jira/browse/IMPALA-7253
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Todd Lipcon
Currently the convenience methods for calling Java functions that take Thrift objects as parameters all work by serializing the parameter into a byte array, and then passing that byte array. This has a few downsides:
- the serialized buffer has an extra memcpy from the native heap buffer into the Java heap byte[]
- the byte[] on the java heap increases the peak memory usage, since it can be relatively large in some cases
- the byte[] allocation requires a large span of contiguous memory which may be difficult to find if the Java heap has become fragmented. In that case, the allocation will trigger a stop-the-world full GC even when using a concurrent collector.
Instead, the native side could use NewDirectByteBuffer to directly wrap the native allocation, and then the Java side can deserialize using TByteBuffer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org