You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Dominik Bruhn <do...@dbruhn.de> on 2016/12/15 16:10:54 UTC
1.2 SNAPSHOT - java.lang.VerifyError on YARN
Hey,
I'm trying out 1.2 because I need the new Kafka 0.10 connector.
I compiled my job using the SNAPSHOT releases of flink 1.2 and the job
works fine in local mode.
The problem is that deploying the job to our EMR/YARN cluster fails with
a
-----
java.lang.VerifyError: Cannot inherit from final class
at java.lang.ClassLoader.defineClass1(Native Method)
-----
The full log is available here:
https://gist.github.com/theomega/18a40c36b5b12e62dad629c5cc4bc017
The job is from my local machine to our EMR cluster. My local machine
runs a Flink 1.2 SNAPSHOT.
Is this some kind of version conflict?
The dependencies for the job are (sbt project):
----
resolvers += "Relayr Repo" at
"https://s3-eu-west-1.amazonaws.com/relayr-maven"
resolvers += "Apache Snapshots" at
"https://repository.apache.org/content/groups/snapshots/"
val flinkDependencies = Seq(
"org.apache.flink" %% "flink-scala" % flinkVersion % "provided"
exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
"org.apache.flink" %% "flink-streaming-scala" % flinkVersion %
"provided" exclude("log4j", "log4j") exclude("org.slf4j",
"slf4j-log4j12"),
"org.apache.flink" %% "flink-connector-kafka-0.10" % flinkVersion
exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
"org.apache.flink" %% "flink-connector-cassandra" % flinkVersion
exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
"org.json" % "json" % "20160212",
"org.clapper" %% "grizzled-slf4j" % "1.3.0",
"ch.qos.logback" % "logback-classic" % "1.1.7" % "provided",
"com.datadoghq" % "java-dogstatsd-client" % "2.3",
"io.dropwizard.metrics" % "metrics-core" % "3.1.2",
"org.scalatest" %% "scalatest" % "2.2.6" % "test"
)
----
How can I debug this further?
Thanks,
Dominik
Re: 1.2 SNAPSHOT - java.lang.VerifyError on YARN
Posted by Dominik Bruhn <do...@dbruhn.de>.
Tracked down the issue at least to some extend:
We are running into a JAR-Hell because the job-jar uses the typesafe
config package in version 1.3, but flink enforces 1.1.
We are trying to migrate away from typesafe for our configs.
Thanks,
Dominik
On 16.12.2016 10:47, Stephan Ewen wrote:
> This looks at a first glance like you are mixing up some versions of
> different modules. Either some Scala version mix, or Flink dependencies
> from different versions.
>
> Can you double check that?
>
> On Thu, Dec 15, 2016 at 5:10 PM, Dominik Bruhn <do...@dbruhn.de> wrote:
>
>> Hey,
>> I'm trying out 1.2 because I need the new Kafka 0.10 connector.
>>
>> I compiled my job using the SNAPSHOT releases of flink 1.2 and the job
>> works fine in local mode.
>>
>> The problem is that deploying the job to our EMR/YARN cluster fails with a
>>
>> -----
>> java.lang.VerifyError: Cannot inherit from final class
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> -----
>> The full log is available here:
>> https://gist.github.com/theomega/18a40c36b5b12e62dad629c5cc4bc017
>>
>>
>> The job is from my local machine to our EMR cluster. My local machine runs
>> a Flink 1.2 SNAPSHOT.
>>
>> Is this some kind of version conflict?
>>
>> The dependencies for the job are (sbt project):
>> ----
>> resolvers += "Relayr Repo" at "https://s3-eu-west-1.amazonaw
>> s.com/relayr-maven"
>> resolvers += "Apache Snapshots" at "https://repository.apache.org
>> /content/groups/snapshots/"
>>
>> val flinkDependencies = Seq(
>> "org.apache.flink" %% "flink-scala" % flinkVersion % "provided"
>> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
>> "org.apache.flink" %% "flink-streaming-scala" % flinkVersion %
>> "provided" exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
>>
>> "org.apache.flink" %% "flink-connector-kafka-0.10" % flinkVersion
>> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
>> "org.apache.flink" %% "flink-connector-cassandra" % flinkVersion
>> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
>>
>> "org.json" % "json" % "20160212",
>> "org.clapper" %% "grizzled-slf4j" % "1.3.0",
>> "ch.qos.logback" % "logback-classic" % "1.1.7" % "provided",
>> "com.datadoghq" % "java-dogstatsd-client" % "2.3",
>> "io.dropwizard.metrics" % "metrics-core" % "3.1.2",
>>
>> "org.scalatest" %% "scalatest" % "2.2.6" % "test"
>> )
>> ----
>>
>>
>> How can I debug this further?
>>
>> Thanks,
>> Dominik
>>
>
Re: 1.2 SNAPSHOT - java.lang.VerifyError on YARN
Posted by Stephan Ewen <se...@apache.org>.
This looks at a first glance like you are mixing up some versions of
different modules. Either some Scala version mix, or Flink dependencies
from different versions.
Can you double check that?
On Thu, Dec 15, 2016 at 5:10 PM, Dominik Bruhn <do...@dbruhn.de> wrote:
> Hey,
> I'm trying out 1.2 because I need the new Kafka 0.10 connector.
>
> I compiled my job using the SNAPSHOT releases of flink 1.2 and the job
> works fine in local mode.
>
> The problem is that deploying the job to our EMR/YARN cluster fails with a
>
> -----
> java.lang.VerifyError: Cannot inherit from final class
> at java.lang.ClassLoader.defineClass1(Native Method)
> -----
> The full log is available here:
> https://gist.github.com/theomega/18a40c36b5b12e62dad629c5cc4bc017
>
>
> The job is from my local machine to our EMR cluster. My local machine runs
> a Flink 1.2 SNAPSHOT.
>
> Is this some kind of version conflict?
>
> The dependencies for the job are (sbt project):
> ----
> resolvers += "Relayr Repo" at "https://s3-eu-west-1.amazonaw
> s.com/relayr-maven"
> resolvers += "Apache Snapshots" at "https://repository.apache.org
> /content/groups/snapshots/"
>
> val flinkDependencies = Seq(
> "org.apache.flink" %% "flink-scala" % flinkVersion % "provided"
> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
> "org.apache.flink" %% "flink-streaming-scala" % flinkVersion %
> "provided" exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
>
> "org.apache.flink" %% "flink-connector-kafka-0.10" % flinkVersion
> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
> "org.apache.flink" %% "flink-connector-cassandra" % flinkVersion
> exclude("log4j", "log4j") exclude("org.slf4j", "slf4j-log4j12"),
>
> "org.json" % "json" % "20160212",
> "org.clapper" %% "grizzled-slf4j" % "1.3.0",
> "ch.qos.logback" % "logback-classic" % "1.1.7" % "provided",
> "com.datadoghq" % "java-dogstatsd-client" % "2.3",
> "io.dropwizard.metrics" % "metrics-core" % "3.1.2",
>
> "org.scalatest" %% "scalatest" % "2.2.6" % "test"
> )
> ----
>
>
> How can I debug this further?
>
> Thanks,
> Dominik
>