[jira] [Comment Edited] (FLINK-18581) Cannot find GC cleaner with java version previous 8u202

["Till Rohrmann (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162073#comment-17162073 ] 

Till Rohrmann edited comment on FLINK-18581 at 7/21/20, 2:36 PM:
-----------------------------------------------------------------

I found the first occurrence of {{Reference.tryHandlePending}} to be introduced in OpenJDK with the version jdk8u72-b01 http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/5ad1e9e8e841/src/share/classes/java/lang/ref/Reference.java. This version should be already a couple of years old.

[~azagrebin] I think calling {{Reference.ReferenceHandler.run}} would not work because it is an infinite loop.

I think this is price we are paying for using internal JVM methods which can change across minor versions and which is very unfortunate. Let's keep this as a warning for the future and let's try to avoid it.

I see two ways forward:

1) Only support Java8 version later or equal than jdk8u72-b01. This would entail that we throw an appropriate exception with a clear message saying that one needs to update the java version. The main problem I see with this approach is that other JVM implementation (other than OpenJDK) might not have these private methods. Have we ever tried running Flink with a different JVM implementation [~azagrebin]? Moreover, there is no guarantee that this method won't disappear in some future version.
2) Getting rid of the clean up mechanism which relies on internal JVM methods and implement our own mechanism. This solution which involve a much higher effort, though.

Maybe there is also some middle ground by trying to call {{tryHandlerPending}} and if this method does not exist to fall back to triggering the normal GC or some other mean to trigger the clean up.


was (Author: till.rohrmann):
I found the first occurrence of {{Reference.tryHandlePending}} to be introduced in OpenJDK with the version jdk8u72-b01 http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/5ad1e9e8e841/src/share/classes/java/lang/ref/Reference.java. This version should be already a couple of years old.

[~azagrebin] I think calling {{Reference.ReferenceHandler.run}} would not work because it is an infinite loop.

I think this is price we are paying for using internal JVM methods which can change across minor versions and which is very unfortunate. Let's keep this as a warning for the future and let's try to avoid it.

I see two ways forward:

1) Only support Java8 version later or equal than jdk8u72-b01. This would entail that we throw an appropriate exception with a clear message saying that one needs to update the java version. The main problem I see with this approach is whether other JVM implementation (other than OpenJDK) actually have the same methods or not. Have we ever tried running Flink with a different JVM implementation [~azagrebin]? Moreover, there is no guarantee that this method won't disappear in some future version.
2) Getting rid of the clean up mechanism which relies on internal JVM methods and implement our own mechanism. This solution which involve a much higher effort, though.

Maybe there is also some middle ground by trying to call {{tryHandlerPending}} and if this method does not exist to fall back to triggering the normal GC or some other mean to trigger the clean up.

> Cannot find GC cleaner with java version previous 8u202
> -------------------------------------------------------
>
>                 Key: FLINK-18581
>                 URL: https://issues.apache.org/jira/browse/FLINK-18581
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination
>    Affects Versions: 1.11.0
>            Reporter: Xintong Song
>            Priority: Critical
>             Fix For: 1.12.0, 1.11.2
>
>
> {{JavaGcCleanerWrapper}} is looking for the package-private method {{Reference.tryHandlePending}} using reflection. However, the method is first introduced in the version 8u202. Therefore, if an older version JDK is used, the method cannot be found and Flink will fail.
> See also this [ML thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Error-GC-Cleaner-Provider-Flink-1-11-0-td36565.html].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)