You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/12/01 12:05:01 UTC
[jira] [Work logged] (HADOOP-12760) sun.misc.Cleaner has moved to a
new location in OpenJDK 9
[ https://issues.apache.org/jira/browse/HADOOP-12760?focusedWorklogId=518398&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-518398 ]
ASF GitHub Bot logged work on HADOOP-12760:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 01/Dec/20 12:04
Start Date: 01/Dec/20 12:04
Worklog Time Spent: 10m
Work Description: wangyum opened a new pull request #2506:
URL: https://github.com/apache/hadoop/pull/2506
This PR backport [HADOOP-12760](https://github.com/apache/hadoop/commit/5d084d7eca32cfa647a78ff6ed3c378659f5b186) to support JDK 11. Otherwise it will throw exception:
```
Exception in thread "DataStreamer for file /tmp/hive-b_carmel/b_carmel/47ca70a2-7790-4acb-aea2-617efafaa54d/inuse.info" java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'
at org.apache.hadoop.crypto.CryptoStreamUtils.freeDB(CryptoStreamUtils.java:41)
at org.apache.hadoop.crypto.CryptoInputStream.freeBuffers(CryptoInputStream.java:683)
at org.apache.hadoop.crypto.CryptoInputStream.close(CryptoInputStream.java:317)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:179)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeStream(DFSOutputStream.java:746)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:694)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:689)
Exception in thread "DataStreamer for file /user/b_carmel/.sparkStaging/application_1605512824298_0222/__spark_libs__13869030055338205442.zip" java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'
at org.apache.hadoop.crypto.CryptoStreamUtils.freeDB(CryptoStreamUtils.java:41)
at org.apache.hadoop.crypto.CryptoInputStream.freeBuffers(CryptoInputStream.java:683)
at org.apache.hadoop.crypto.CryptoInputStream.close(CryptoInputStream.java:317)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:179)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeStream(DFSOutputStream.java:746)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:694)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:689)
Exception in thread "DataStreamer for file /user/b_carmel/.sparkStaging/application_1605512824298_0222/__spark_conf__.zip" java.lang.NoSuchMethodError: 'sun.misc.Cleaner sun.nio.ch.DirectBuffer.cleaner()'
at org.apache.hadoop.crypto.CryptoStreamUtils.freeDB(CryptoStreamUtils.java:41)
at org.apache.hadoop.crypto.CryptoInputStream.freeBuffers(CryptoInputStream.java:683)
at org.apache.hadoop.crypto.CryptoInputStream.close(CryptoInputStream.java:317)
at java.base/java.io.FilterInputStream.close(FilterInputStream.java:179)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeStream(DFSOutputStream.java:746)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.closeInternal(DFSOutputStream.java:694)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:689)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 518398)
Remaining Estimate: 0h
Time Spent: 10m
> sun.misc.Cleaner has moved to a new location in OpenJDK 9
> ---------------------------------------------------------
>
> Key: HADOOP-12760
> URL: https://issues.apache.org/jira/browse/HADOOP-12760
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Chris Hegarty
> Assignee: Akira Ajisaka
> Priority: Major
> Fix For: 3.2.0
>
> Attachments: HADOOP-12760.00.patch, HADOOP-12760.01.patch, HADOOP-12760.02.patch, HADOOP-12760.03.patch, HADOOP-12760.04.patch, HADOOP-12760.05.patch, HADOOP-12760.06.patch, HADOOP-12760.07.patch, HADOOP-12760.08.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a heads-up: there are upcoming changes in JDK 9 that will require, at least, a small update to org.apache.hadoop.crypto.CryptoStreamUtils & org.apache.hadoop.io.nativeio.NativeIO.
> OpenJDK issue no. 8148117: "Move sun.misc.Cleaner to jdk.internal.ref" [1], will move the Cleaner class from sun.misc to jdk.internal.ref. There is ongoing discussion about the possibility of providing a public supported API, maybe in the JDK 9 timeframe, for releasing NIO direct buffer native memory, see the core-libs-dev mail thread [2]. At the very least CryptoStreamUtils & NativeIO [3] should be updated to have knowledge of the new location of the JDK Cleaner.
> [1] https://bugs.openjdk.java.net/browse/JDK-8148117
> [2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038243.html
> [3] https://github.com/apache/hadoop/search?utf8=✓&q=sun.misc.Cleaner
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org