You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2015/11/10 12:32:11 UTC
[jira] [Commented] (FLINK-2992) New Windowing code is using
SerializationUtils with wrong classloader
[ https://issues.apache.org/jira/browse/FLINK-2992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998429#comment-14998429 ]
ASF GitHub Bot commented on FLINK-2992:
---------------------------------------
GitHub user rmetzger opened a pull request:
https://github.com/apache/flink/pull/1343
Remove and forbid use of SerializationUtils. Fix FLINK-2992
The SerializationUtils are usually not using the right classloader, and they have some security issues.
I'm using our checkstyle rules to forbid the use of them.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmetzger/flink cancel_hotfix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1343.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1343
----
commit b6f00f96d4b38ee69e081ced189f5817c3f57fd4
Author: Robert Metzger <rm...@apache.org>
Date: 2015-11-10T10:32:20Z
[hotfix] Check for null in StreamSource.cancel()
commit 7b5650bccd8959dbf1547742f610793aed7aeebe
Author: Robert Metzger <rm...@apache.org>
Date: 2015-11-10T11:29:17Z
[FLINK-2992] Remove use of SerializationUtils
----
> New Windowing code is using SerializationUtils with wrong classloader
> ---------------------------------------------------------------------
>
> Key: FLINK-2992
> URL: https://issues.apache.org/jira/browse/FLINK-2992
> Project: Flink
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 0.10
> Reporter: Robert Metzger
> Assignee: Robert Metzger
> Priority: Critical
>
> During release testing, I found the following issue
> {code}
> robert@hn0-flink0:~/flink010-26-211/flink-0.10.0$ ./bin/flink run ../../scratch/target/flink0.10-scala2.11-1.0-SNAPSHOT.jar --input hdfs:///user/robert/file.txt --out hdfs:///user/robert/result
> Found YARN properties file /tmp/.yarn-properties-robert
> Using JobManager address from YARN properties 10.0.0.5/10.0.0.5:59812
> org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.
> at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:512)
> at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:395)
> at org.apache.flink.client.program.Client.runBlocking(Client.java:252)
> at org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:675)
> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:326)
> at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:977)
> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1027)
> Caused by: org.apache.commons.lang.SerializationException: java.lang.ClassNotFoundException: com.dataartisans.Job$$anon$3$$anon$2
> at org.apache.commons.lang.SerializationUtils.deserialize(SerializationUtils.java:166)
> at org.apache.commons.lang.SerializationUtils.deserialize(SerializationUtils.java:193)
> at org.apache.commons.lang.SerializationUtils.clone(SerializationUtils.java:81)
> at org.apache.flink.streaming.api.datastream.WindowedStream.reduce(WindowedStream.java:172)
> at org.apache.flink.streaming.api.scala.WindowedStream.aggregate(WindowedStream.scala:352)
> at org.apache.flink.streaming.api.scala.WindowedStream.aggregate(WindowedStream.scala:332)
> at org.apache.flink.streaming.api.scala.WindowedStream.sum(WindowedStream.scala:300)
> at com.dataartisans.Job$.main(Job.scala:59)
> at com.dataartisans.Job.main(Job.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:497)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: com.dataartisans.Job$$anon$3$$anon$2
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:278)
> at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:625)
> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612)
> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1997)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1921)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1997)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1921)
> at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> at org.apache.commons.lang.SerializationUtils.deserialize(SerializationUtils.java:163)
> ... 19 more
> The exception above occurred while trying to run your command.
> {code}
> The problem is that we are using the org.apache.commons.lang.SerializationUtils with the wrong classloader.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)