[jira] [Comment Edited] (FLINK-18581) Cannot find GC cleaner with java version previous jdk8u72(-b01)

["Andrey Zagrebin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-18581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162096#comment-17162096 ] 

Andrey Zagrebin edited comment on FLINK-18581 at 7/21/20, 4:31 PM:
-------------------------------------------------------------------

True, I overlooked the loop in ReferenceHandler.run.
{quote}1) Only support Java8 version later or equal than jdk8u72-b01. This would entail that we throw an appropriate exception with a clear message saying that one needs to update the java version. The main problem I see with this approach is that other JVM implementation (other than OpenJDK) might not have these private methods. Have we ever tried running Flink with a different JVM implementation [~azagrebin]? Moreover, there is no guarantee that this method won't disappear in some future version.
{quote}
I have Oracle JVM locally which I tried. Oracle jdk8u72 has the required method.
{quote}I think this is price we are paying for using internal JVM methods which can change across minor versions and which is very unfortunate. Let's keep this as a warning for the future and let's try to avoid it.
{quote}
I totally agree
{quote}2) Getting rid of the clean up mechanism which relies on internal JVM methods and implement our own mechanism. This solution which involve a much higher effort, though.
{quote}
This is hard to achieve our goal. We need to run release once the memory link is ready for GC which can be detected only by GC. Or we have to change the memory API/semantics somehow.
{quote}Maybe there is also some middle ground by trying to call {{tryHandlerPending}} and if this method does not exist to fall back to triggering the normal GC or some other mean to trigger the clean up.
{quote}
This is an option. We can wait a bit before calling GC to call it not so often and give some time to the previous GC call. This is less performant and we can output warning about the JVM version in this case. I can try it.


was (Author: azagrebin):
True, I overlooked the loop in ReferenceHandler.run.
{quote}1) Only support Java8 version later or equal than jdk8u72-b01. This would entail that we throw an appropriate exception with a clear message saying that one needs to update the java version. The main problem I see with this approach is that other JVM implementation (other than OpenJDK) might not have these private methods. Have we ever tried running Flink with a different JVM implementation [~azagrebin]? Moreover, there is no guarantee that this method won't disappear in some future version.
{quote}
I have Oracle JVM locally which I tried. I can check that Oracle jdk8u72 has the required method.
{quote}I think this is price we are paying for using internal JVM methods which can change across minor versions and which is very unfortunate. Let's keep this as a warning for the future and let's try to avoid it.
{quote}
I totally agree
{quote}2) Getting rid of the clean up mechanism which relies on internal JVM methods and implement our own mechanism. This solution which involve a much higher effort, though.
{quote}
This is hard to achieve our goal. We need to run release once the memory link is ready for GC which can be detected only by GC. Or we have to change the memory API/semantics somehow.
{quote}Maybe there is also some middle ground by trying to call {{tryHandlerPending}} and if this method does not exist to fall back to triggering the normal GC or some other mean to trigger the clean up.
{quote}
This is an option. We can wait a bit before calling GC to call it not so often and give some time to the previous GC call. This is less performant and we can output warning about the JVM version in this case. I can try it.

> Cannot find GC cleaner with java version previous jdk8u72(-b01)
> ---------------------------------------------------------------
>
>                 Key: FLINK-18581
>                 URL: https://issues.apache.org/jira/browse/FLINK-18581
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination
>    Affects Versions: 1.11.0
>            Reporter: Xintong Song
>            Priority: Critical
>             Fix For: 1.12.0, 1.11.2
>
>
> {{JavaGcCleanerWrapper}} is looking for the package-private method {{Reference.tryHandlePending}} using reflection. However, the method is first introduced in the version jdk8u72(-b01). Therefore, if an older version JDK is used, the method cannot be found and Flink will fail.
> See also this [ML thread|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Error-GC-Cleaner-Provider-Flink-1-11-0-td36565.html].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)