[jira] [Commented] (FLINK-15094) Warning about using private constructor of java.nio.DirectByteBuffer in Java 11

["Andrey Zagrebin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-15094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040801#comment-17040801 ] 

Andrey Zagrebin commented on FLINK-15094:
-----------------------------------------

Just to silence the warnings, we can use Unsafe to instantiate DirectByteBuffer and set its fields. It is still a hacky but quick solution ([checked|https://github.com/azagrebin/flink/commit/25fd29a532c8de912293c3d1f88ab64844d2aaf6], running on travis and PR can be opened).

Another more official solution is to write a native allocator and return a DirectByteBuffer object over JNI (NewDirectByteBuffer(addressOfNativelyAllocatedMemory)). This is probably slower and also tedious because requires cross-platform builds like in RocksDB.

These options do not contribute to MaxDirectMemory which allows to use interchangeably the native memory for managed batch and stream RocksDB.

Just for note: implementing our custom ByteBuffer (similar to DirectByteBuffer) will slow down DirectByteBuffer#put operation if it accepts our ByteBuffer because it will not be recognised as native DirectByteBuffer and the fast unsafe copying will not be applied. Adding our DirectByteBuffer into the same java.nio package in our code base to extend java DirectByteBuffer fails and potentially illegal.

> Warning about using private constructor of java.nio.DirectByteBuffer in Java 11
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-15094
>                 URL: https://issues.apache.org/jira/browse/FLINK-15094
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Task
>            Reporter: Andrey Zagrebin
>            Priority: Major
>
> The unsafe off-heap in [FLINK-13985|https://jira.apache.org/jira/browse/FLINK-13985] was implemented by hacking into a private constructor of java.nio.DirectByteBuffer. This causes undesirable warnings in Java 11:
> {code:java}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by org.apache.flink.core.memory.MemoryUtils (file:/C:/Development/repos/flink/flink-core/target/classes/) to constructor java.nio.DirectByteBuffer(long,int)
> WARNING: Please consider reporting this to the maintainers of org.apache.flink.core.memory.MemoryUtils
> WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
> WARNING: All illegal access operations will be denied in a future release{code}
> We should look into this, confirm that it is harmless and try to silence it to avoid confusion for users.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)