You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Fabian Böhnlein <fa...@gmail.com> on 2017/11/15 10:00:40 UTC
Coping with conflicting Jackson version in Zeppelin and Spark 2.1+
Hi all,
how are you resolving the conflicts of jackson packages? zeppelin/lib/*
brings version 2.5.3 while Spark 2.1+ has version 2.6.2. For example this
issue
<https://stackoverflow.com/questions/44697706/zeppelin-jackson-version-is-too-old-error-when-running-spark-streaming>
.
This <https://issues.apache.org/jira/browse/ZEPPELIN-2402> might solve it
as well - but there must be a way people are currently doing it? Thanks.
Which causes the exception (see full attached).
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Jackson
version is too old 2.5.3
at
com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:56)
at
com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
at
com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:651)
at org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)
Thanks,
Fabian
ERROR [2017-11-15 09:35:52,882] ({pool-2-thread-4}
Utils.java[invokeMethod]:40) -
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
at
org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:368)
at
org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:233)
at
org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:841)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:491)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ExceptionInInitializerError
at
org.apache.spark.scheduler.EventLoggingListener$.initEventLog(EventLoggingListener.scala:254)
at
org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:121)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:531)
at
org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
... 20 more
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Jackson
version is too old 2.5.3
at
com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:56)
at
com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
at
com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:651)
at org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)
at org.apache.spark.util.JsonProtocol$.<clinit>(JsonProtocol.scala)
... 28 more
Re: Coping with conflicting Jackson version in Zeppelin and Spark 2.1+
Posted by Jeff Zhang <zj...@gmail.com>.
Do you have sample code reproduce it ? Zeppelin CI pass for spark 2.1, so I
guess it happens in some specific cases.
Fabian Böhnlein <fa...@gmail.com>于2017年11月15日周三 下午6:01写道：
> Hi all,
>
> how are you resolving the conflicts of jackson packages? zeppelin/lib/*
> brings version 2.5.3 while Spark 2.1+ has version 2.6.2. For example this
> issue
> <https://stackoverflow.com/questions/44697706/zeppelin-jackson-version-is-too-old-error-when-running-spark-streaming>
> .
>
> This <https://issues.apache.org/jira/browse/ZEPPELIN-2402> might solve it
> as well - but there must be a way people are currently doing it? Thanks.
>
> Which causes the exception (see full attached).
>
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: Jackson
> version is too old 2.5.3
> at
> com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:56)
> at
> com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
> at
> com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:651)
> at
> org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)
>
> Thanks,
> Fabian
>
> ERROR [2017-11-15 09:35:52,882] ({pool-2-thread-4}
> Utils.java[invokeMethod]:40) -
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
> at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
> at
> org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:368)
> at
> org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:233)
> at
> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:841)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:491)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
> at
> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ExceptionInInitializerError
> at
> org.apache.spark.scheduler.EventLoggingListener$.initEventLog(EventLoggingListener.scala:254)
> at
> org.apache.spark.scheduler.EventLoggingListener.start(EventLoggingListener.scala:121)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:531)
> at
> org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
> at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
> at
> org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
> at scala.Option.getOrElse(Option.scala:121)
> at
> org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
> ... 20 more
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: Jackson
> version is too old 2.5.3
> at
> com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:56)
> at
> com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
> at
> com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:651)
> at
> org.apache.spark.util.JsonProtocol$.<init>(JsonProtocol.scala:59)
> at org.apache.spark.util.JsonProtocol$.<clinit>(JsonProtocol.scala)
> ... 28 more
>
>
>