You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Paul Brenner <pb...@placeiq.com> on 2019/04/19 16:30:12 UTC
java.lang.NumberFormatException: Not a version: 9
I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82. However, we find that when including certain dependencies on the newer versions of zeppelin spark interpreters won’t start and instead throw the error I’ll paste below. This is if we have zeppelin.spark.useNew set to true. Setting it to false just gives a null pointer exception instead. An example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0, but there are also some internal packages we use that we haven’t been able to find anything that should be causing this conflict.
I found someone with a similar problem on ZEPPELIN-4074 but no one seems to have any idea what to do. Any ideas? It sounds like isJavaAtLeast got fixed to work with JDK 9 in scala 12 (see here), but who invited JDK 9 to this party? I certainly didn’t.
Zeppelin 0.8 is really struggling for us so we are desperate to upgrade but this is holding us back, would really appreciate any help at all!
java.lang.NumberFormatException: Not a version: 9 at scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276) at scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283) at scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293) at scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293) at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28) at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131) at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276) at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at scala.tools.nsc.Global.rootMirror(Global.scala:71) at scala.tools.nsc.Global.rootMirror(Global.scala:39) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390) at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139) at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161) at org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85) at org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
PAUL BRENNER
Sr. Data Scientist
pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
twitter @placeiq linkedin /placeiq
Re: java.lang.NumberFormatException: Not a version: 9
Posted by Jeff Zhang <zj...@gmail.com>.
Right, we should remove option 1 from doc, as it is not standard way to
adding additional jar for spark.
Created https://jira.apache.org/jira/browse/ZEPPELIN-4130
Paul Brenner <pb...@placeiq.com> 于2019年4月23日周二 上午8:55写道：
> Amazing. Using spark.jars.packages instead of the dependencies section of
> the interpreter completely fixes it. Thanks for the help!
>
> I noticed that the method I was originally using is listed as option #1 in
> the docs (here
> <http://zeppelin.apache.org/docs/0.8.0/interpreter/spark.html#dependency-management>).
> Does it make sense to include a note that options 2 and 3 are the
> “standard” way and may succeed when method #1 fails? Or is that going into
> unnecessary detail for the docs or not even true in most cases?
>
>
>
> PAUL BRENNER
>
> Sr. Data Scientist
> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
> <https://www.linkedin.com/company/placeiq/>
> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
> On Apr 21, 2019, 10:28 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
>
> Hi Paul,
>
> I could not reproduce this issue in branch-0.8 (0.8.2) even using your
> configuration( spark 2.4, yarn-client, add dependency via zeppelin
> interpreter dependencies interface)
>
> One thing you can try is to add dependency via setting spark.jars.packages
> which is the standard way to set additional jars/packages.
>
>
>
>
> Paul Brenner <pb...@placeiq.com> 于2019年4月21日周日 下午9:23写道：
>
>>
>> 1. I used Spark 2.4.0
>> 2. I used yarn-client mode
>> 3. Dependency is added via zeppelin’s interpreter dependencies
>> interface.
>>
>> Thanks for looking into this! We still don’t have a working fix in place
>> so are held back from migrating to the wonderful and glorious features of
>> 0.82.
>>
>>
>>
>> PAUL BRENNER
>>
>> Sr. Data Scientist
>> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
>> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
>> <https://www.linkedin.com/company/placeiq/>
>> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
>> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
>> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
>> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>> On Apr 21, 2019, 1:44 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
>>
>> Hi Paul,
>>
>> Could you let me know the following questions ?
>>
>> 1. Which spark version do you use ?
>> 2. Which mode do you use ? local / yarn /standalone
>> 3. How did you add dependency ? via spark.jars.package or via zeppelin's
>> interpreter dependencies ?
>>
>> Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 下午8:00写道：
>>
>>> Yes.
>>>
>>> Add com.databricks:spark-avro_2.11:3.2.0 as a dependency to a spark
>>> interpreter then try using that interpreter to run any code in a notebook
>>> to reproduce.
>>>
>>> For our internal library that was causing it I went through and tried
>>> excluding everything that that library imported. It turns out it was
>>> importing org.scala-lang:scala-library. Excluding that may have resolved
>>> the issue for our internal package.
>>>
>>>
>>>
>>> PAUL BRENNER
>>>
>>> Sr. Data Scientist
>>> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
>>> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
>>> <https://www.linkedin.com/company/placeiq/>
>>> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
>>> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
>>> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
>>> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>>> On Apr 20, 2019, 3:30 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
>>>
>>> Hi Paul,
>>>
>>> Could you describe how to reproduce it ?
>>>
>>> Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 上午12:30写道：
>>>
>>>> I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82.
>>>> However, we find that when including certain dependencies on the newer
>>>> versions of zeppelin spark interpreters won’t start and instead throw the
>>>> error I’ll paste below. This is if we have zeppelin.spark.useNew set
>>>> to true. Setting it to false just gives a null pointer exception instead.
>>>> An example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0,
>>>> but there are also some internal packages we use that we haven’t been able
>>>> to find anything that should be causing this conflict.
>>>>
>>>> I found someone with a similar problem on ZEPPELIN-4074
>>>> <https://jira.apache.org/jira/browse/ZEPPELIN-4074> but no one seems
>>>> to have any idea what to do. Any ideas? It sounds like isJavaAtLeast got
>>>> fixed to work with JDK 9 in scala 12 (see here
>>>> <https://github.com/scala/scala/pull/5276>), but who invited JDK 9 to
>>>> this party? I certainly didn’t.
>>>>
>>>> Zeppelin 0.8 is really struggling for us so we are desperate to upgrade
>>>> but this is holding us back, would really appreciate any help at all!
>>>>
>>>> java.lang.NumberFormatException: Not a version: 9 at
>>>> scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at
>>>> scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at
>>>> scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at
>>>> scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276)
>>>> at
>>>> scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283)
>>>> at
>>>> scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293)
>>>> at
>>>> scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293)
>>>> at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at
>>>> scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at
>>>> scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at
>>>> scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at
>>>> scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28)
>>>> at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at
>>>> scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131)
>>>> at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at
>>>> scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at
>>>> scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321)
>>>> at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at
>>>> scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330)
>>>> at
>>>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336)
>>>> at
>>>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336)
>>>> at
>>>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276)
>>>> at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at
>>>> scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at
>>>> scala.tools.nsc.Global.rootMirror(Global.scala:71) at
>>>> scala.tools.nsc.Global.rootMirror(Global.scala:39) at
>>>> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257)
>>>> at
>>>> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257)
>>>> at
>>>> scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390)
>>>> at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at
>>>> scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139)
>>>> at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161)
>>>> at
>>>> org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85)
>>>> at
>>>> org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102)
>>>> at
>>>> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62)
>>>> at
>>>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
>>>> at
>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616)
>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at
>>>> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>> at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>> at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>> at java.lang.Thread.run(Thread.java:748)
>>>>
>>>>
>>>>
>>>> PAUL BRENNER
>>>>
>>>> Sr. Data Scientist
>>>> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
>>>> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
>>>> <https://www.linkedin.com/company/placeiq/>
>>>> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
>>>> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
>>>> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
>>>> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
Re: java.lang.NumberFormatException: Not a version: 9
Posted by Paul Brenner <pb...@placeiq.com>.
Amazing. Using spark.jars.packages instead of the dependencies section of the interpreter completely fixes it. Thanks for the help!
I noticed that the method I was originally using is listed as option #1 in the docs (here). Does it make sense to include a note that options 2 and 3 are the “standard” way and may succeed when method #1 fails? Or is that going into unnecessary detail for the docs or not even true in most cases?
PAUL BRENNER
Sr. Data Scientist
pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
twitter @placeiq linkedin /placeiq
On Apr 21, 2019, 10:28 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
> Hi Paul,
>
> I could not reproduce this issue in branch-0.8 (0.8.2) even using your configuration( spark 2.4, yarn-client, add dependency via zeppelin interpreter dependencies interface)
>
> One thing you can try is to add dependency via setting spark.jars.packages which is the standard way to set additional jars/packages.
>
>
>
>
> > Paul Brenner <pb...@placeiq.com> 于2019年4月21日周日 下午9:23写道：
> > >
> > > 1. I used Spark 2.4.0
> > > 2. I used yarn-client mode
> > > 3. Dependency is added via zeppelin’s interpreter dependencies interface.
> > >
> > > Thanks for looking into this! We still don’t have a working fix in place so are held back from migrating to the wonderful and glorious features of 0.82.
> > >
> > >
> > >
> > > PAUL BRENNER
> > > Sr. Data Scientist
> > > pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> > > twitter @placeiq linkedin /placeiq
> > > On Apr 21, 2019, 1:44 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
> > > > Hi Paul,
> > > >
> > > > Could you let me know the following questions ?
> > > >
> > > > 1. Which spark version do you use ?
> > > > 2. Which mode do you use ? local / yarn /standalone
> > > > 3. How did you add dependency ? via spark.jars.package or via zeppelin's interpreter dependencies ?
> > > >
> > > > > Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 下午8:00写道：
> > > > > > Yes.
> > > > > >
> > > > > > Add com.databricks:spark-avro_2.11:3.2.0 as a dependency to a spark interpreter then try using that interpreter to run any code in a notebook to reproduce.
> > > > > >
> > > > > > For our internal library that was causing it I went through and tried excluding everything that that library imported. It turns out it was importing org.scala-lang:scala-library. Excluding that may have resolved the issue for our internal package.
> > > > > >
> > > > > >
> > > > > >
> > > > > > PAUL BRENNER
> > > > > > Sr. Data Scientist
> > > > > > pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> > > > > > twitter @placeiq linkedin /placeiq
> > > > > > On Apr 20, 2019, 3:30 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
> > > > > > > Hi Paul,
> > > > > > >
> > > > > > > Could you describe how to reproduce it ?
> > > > > > >
> > > > > > > > Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 上午12:30写道：
> > > > > > > > > I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82. However, we find that when including certain dependencies on the newer versions of zeppelin spark interpreters won’t start and instead throw the error I’ll paste below. This is if we have zeppelin.spark.useNew set to true. Setting it to false just gives a null pointer exception instead. An example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0, but there are also some internal packages we use that we haven’t been able to find anything that should be causing this conflict.
> > > > > > > > >
> > > > > > > > > I found someone with a similar problem on ZEPPELIN-4074 but no one seems to have any idea what to do. Any ideas? It sounds like isJavaAtLeast got fixed to work with JDK 9 in scala 12 (see here), but who invited JDK 9 to this party? I certainly didn’t.
> > > > > > > > >
> > > > > > > > > Zeppelin 0.8 is really struggling for us so we are desperate to upgrade but this is holding us back, would really appreciate any help at all!
> > > > > > > > >
> > > > > > > > > java.lang.NumberFormatException: Not a version: 9 at scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276) at scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283) at scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293) at scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293) at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28) at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131) at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276) at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at scala.tools.nsc.Global.rootMirror(Global.scala:71) at scala.tools.nsc.Global.rootMirror(Global.scala:39) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390) at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139) at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161) at org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85) at org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > PAUL BRENNER
> > > > > > > > > Sr. Data Scientist
> > > > > > > > > pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> > > > > > > > > twitter @placeiq linkedin /placeiq
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards
> > > > > > >
> > > > > > > Jeff Zhang
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > >
> > > > Jeff Zhang
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: java.lang.NumberFormatException: Not a version: 9
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Paul,
I could not reproduce this issue in branch-0.8 (0.8.2) even using your
configuration( spark 2.4, yarn-client, add dependency via zeppelin
interpreter dependencies interface)
One thing you can try is to add dependency via setting spark.jars.packages
which is the standard way to set additional jars/packages.
Paul Brenner <pb...@placeiq.com> 于2019年4月21日周日 下午9:23写道：
>
> 1. I used Spark 2.4.0
> 2. I used yarn-client mode
> 3. Dependency is added via zeppelin’s interpreter dependencies
> interface.
>
> Thanks for looking into this! We still don’t have a working fix in place
> so are held back from migrating to the wonderful and glorious features of
> 0.82.
>
>
>
> PAUL BRENNER
>
> Sr. Data Scientist
> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
> <https://www.linkedin.com/company/placeiq/>
> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
> On Apr 21, 2019, 1:44 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
>
> Hi Paul,
>
> Could you let me know the following questions ?
>
> 1. Which spark version do you use ?
> 2. Which mode do you use ? local / yarn /standalone
> 3. How did you add dependency ? via spark.jars.package or via zeppelin's
> interpreter dependencies ?
>
> Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 下午8:00写道：
>
>> Yes.
>>
>> Add com.databricks:spark-avro_2.11:3.2.0 as a dependency to a spark
>> interpreter then try using that interpreter to run any code in a notebook
>> to reproduce.
>>
>> For our internal library that was causing it I went through and tried
>> excluding everything that that library imported. It turns out it was
>> importing org.scala-lang:scala-library. Excluding that may have resolved
>> the issue for our internal package.
>>
>>
>>
>> PAUL BRENNER
>>
>> Sr. Data Scientist
>> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
>> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
>> <https://www.linkedin.com/company/placeiq/>
>> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
>> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
>> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
>> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>> On Apr 20, 2019, 3:30 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
>>
>> Hi Paul,
>>
>> Could you describe how to reproduce it ?
>>
>> Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 上午12:30写道：
>>
>>> I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82.
>>> However, we find that when including certain dependencies on the newer
>>> versions of zeppelin spark interpreters won’t start and instead throw the
>>> error I’ll paste below. This is if we have zeppelin.spark.useNew set to
>>> true. Setting it to false just gives a null pointer exception instead. An
>>> example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0,
>>> but there are also some internal packages we use that we haven’t been able
>>> to find anything that should be causing this conflict.
>>>
>>> I found someone with a similar problem on ZEPPELIN-4074
>>> <https://jira.apache.org/jira/browse/ZEPPELIN-4074> but no one seems to
>>> have any idea what to do. Any ideas? It sounds like isJavaAtLeast got fixed
>>> to work with JDK 9 in scala 12 (see here
>>> <https://github.com/scala/scala/pull/5276>), but who invited JDK 9 to
>>> this party? I certainly didn’t.
>>>
>>> Zeppelin 0.8 is really struggling for us so we are desperate to upgrade
>>> but this is holding us back, would really appreciate any help at all!
>>>
>>> java.lang.NumberFormatException: Not a version: 9 at
>>> scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at
>>> scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at
>>> scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at
>>> scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276)
>>> at
>>> scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283)
>>> at
>>> scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293)
>>> at
>>> scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293)
>>> at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at
>>> scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at
>>> scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at
>>> scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at
>>> scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28)
>>> at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at
>>> scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131)
>>> at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at
>>> scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at
>>> scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321)
>>> at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at
>>> scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330)
>>> at
>>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336)
>>> at
>>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336)
>>> at
>>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276)
>>> at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at
>>> scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at
>>> scala.tools.nsc.Global.rootMirror(Global.scala:71) at
>>> scala.tools.nsc.Global.rootMirror(Global.scala:39) at
>>> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257)
>>> at
>>> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257)
>>> at
>>> scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390)
>>> at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at
>>> scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139)
>>> at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161)
>>> at
>>> org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85)
>>> at
>>> org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102)
>>> at
>>> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62)
>>> at
>>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616)
>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at
>>> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>> at java.lang.Thread.run(Thread.java:748)
>>>
>>>
>>>
>>> PAUL BRENNER
>>>
>>> Sr. Data Scientist
>>> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
>>> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
>>> <https://www.linkedin.com/company/placeiq/>
>>> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
>>> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
>>> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
>>> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
Re: java.lang.NumberFormatException: Not a version: 9
Posted by Paul Brenner <pb...@placeiq.com>.
1. I used Spark 2.4.0
2. I used yarn-client mode
3. Dependency is added via zeppelin’s interpreter dependencies interface.
Thanks for looking into this! We still don’t have a working fix in place so are held back from migrating to the wonderful and glorious features of 0.82.
PAUL BRENNER
Sr. Data Scientist
pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
twitter @placeiq linkedin /placeiq
On Apr 21, 2019, 1:44 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
> Hi Paul,
>
> Could you let me know the following questions ?
>
> 1. Which spark version do you use ?
> 2. Which mode do you use ? local / yarn /standalone
> 3. How did you add dependency ? via spark.jars.package or via zeppelin's interpreter dependencies ?
>
> > Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 下午8:00写道：
> > > Yes.
> > >
> > > Add com.databricks:spark-avro_2.11:3.2.0 as a dependency to a spark interpreter then try using that interpreter to run any code in a notebook to reproduce.
> > >
> > > For our internal library that was causing it I went through and tried excluding everything that that library imported. It turns out it was importing org.scala-lang:scala-library. Excluding that may have resolved the issue for our internal package.
> > >
> > >
> > >
> > > PAUL BRENNER
> > > Sr. Data Scientist
> > > pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> > > twitter @placeiq linkedin /placeiq
> > > On Apr 20, 2019, 3:30 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
> > > > Hi Paul,
> > > >
> > > > Could you describe how to reproduce it ?
> > > >
> > > > > Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 上午12:30写道：
> > > > > > I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82. However, we find that when including certain dependencies on the newer versions of zeppelin spark interpreters won’t start and instead throw the error I’ll paste below. This is if we have zeppelin.spark.useNew set to true. Setting it to false just gives a null pointer exception instead. An example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0, but there are also some internal packages we use that we haven’t been able to find anything that should be causing this conflict.
> > > > > >
> > > > > > I found someone with a similar problem on ZEPPELIN-4074 but no one seems to have any idea what to do. Any ideas? It sounds like isJavaAtLeast got fixed to work with JDK 9 in scala 12 (see here), but who invited JDK 9 to this party? I certainly didn’t.
> > > > > >
> > > > > > Zeppelin 0.8 is really struggling for us so we are desperate to upgrade but this is holding us back, would really appreciate any help at all!
> > > > > >
> > > > > > java.lang.NumberFormatException: Not a version: 9 at scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276) at scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283) at scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293) at scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293) at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28) at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131) at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276) at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at scala.tools.nsc.Global.rootMirror(Global.scala:71) at scala.tools.nsc.Global.rootMirror(Global.scala:39) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390) at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139) at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161) at org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85) at org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> > > > > >
> > > > > >
> > > > > >
> > > > > > PAUL BRENNER
> > > > > > Sr. Data Scientist
> > > > > > pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> > > > > > twitter @placeiq linkedin /placeiq
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > >
> > > > Jeff Zhang
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: java.lang.NumberFormatException: Not a version: 9
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Paul,
Could you let me know the following questions ?
1. Which spark version do you use ?
2. Which mode do you use ? local / yarn /standalone
3. How did you add dependency ? via spark.jars.package or via zeppelin's
interpreter dependencies ?
Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 下午8:00写道：
> Yes.
>
> Add com.databricks:spark-avro_2.11:3.2.0 as a dependency to a spark
> interpreter then try using that interpreter to run any code in a notebook
> to reproduce.
>
> For our internal library that was causing it I went through and tried
> excluding everything that that library imported. It turns out it was
> importing org.scala-lang:scala-library. Excluding that may have resolved
> the issue for our internal package.
>
>
>
> PAUL BRENNER
>
> Sr. Data Scientist
> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
> <https://www.linkedin.com/company/placeiq/>
> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
> On Apr 20, 2019, 3:30 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
>
> Hi Paul,
>
> Could you describe how to reproduce it ?
>
> Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 上午12:30写道：
>
>> I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82.
>> However, we find that when including certain dependencies on the newer
>> versions of zeppelin spark interpreters won’t start and instead throw the
>> error I’ll paste below. This is if we have zeppelin.spark.useNew set to
>> true. Setting it to false just gives a null pointer exception instead. An
>> example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0,
>> but there are also some internal packages we use that we haven’t been able
>> to find anything that should be causing this conflict.
>>
>> I found someone with a similar problem on ZEPPELIN-4074
>> <https://jira.apache.org/jira/browse/ZEPPELIN-4074> but no one seems to
>> have any idea what to do. Any ideas? It sounds like isJavaAtLeast got fixed
>> to work with JDK 9 in scala 12 (see here
>> <https://github.com/scala/scala/pull/5276>), but who invited JDK 9 to
>> this party? I certainly didn’t.
>>
>> Zeppelin 0.8 is really struggling for us so we are desperate to upgrade
>> but this is holding us back, would really appreciate any help at all!
>>
>> java.lang.NumberFormatException: Not a version: 9 at
>> scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at
>> scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at
>> scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at
>> scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276)
>> at
>> scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283)
>> at
>> scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293)
>> at
>> scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293)
>> at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at
>> scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at
>> scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at
>> scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at
>> scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28)
>> at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at
>> scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131)
>> at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at
>> scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at
>> scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321)
>> at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at
>> scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330)
>> at
>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336)
>> at
>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336)
>> at
>> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276)
>> at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at
>> scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at
>> scala.tools.nsc.Global.rootMirror(Global.scala:71) at
>> scala.tools.nsc.Global.rootMirror(Global.scala:39) at
>> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257)
>> at
>> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257)
>> at
>> scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390)
>> at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at
>> scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139)
>> at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161)
>> at
>> org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85)
>> at
>> org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102)
>> at
>> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62)
>> at
>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
>> at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at
>> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>>
>> PAUL BRENNER
>>
>> Sr. Data Scientist
>> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
>> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
>> <https://www.linkedin.com/company/placeiq/>
>> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
>> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
>> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
>> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
--
Best Regards
Jeff Zhang
Re: java.lang.NumberFormatException: Not a version: 9
Posted by Paul Brenner <pb...@placeiq.com>.
Yes.
Add com.databricks:spark-avro_2.11:3.2.0 as a dependency to a spark interpreter then try using that interpreter to run any code in a notebook to reproduce.
For our internal library that was causing it I went through and tried excluding everything that that library imported. It turns out it was importing org.scala-lang:scala-library. Excluding that may have resolved the issue for our internal package.
PAUL BRENNER
Sr. Data Scientist
pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
twitter @placeiq linkedin /placeiq
On Apr 20, 2019, 3:30 AM -0400, Jeff Zhang <zj...@gmail.com>, wrote:
> Hi Paul,
>
> Could you describe how to reproduce it ?
>
> > Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 上午12:30写道：
> > > I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82. However, we find that when including certain dependencies on the newer versions of zeppelin spark interpreters won’t start and instead throw the error I’ll paste below. This is if we have zeppelin.spark.useNew set to true. Setting it to false just gives a null pointer exception instead. An example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0, but there are also some internal packages we use that we haven’t been able to find anything that should be causing this conflict.
> > >
> > > I found someone with a similar problem on ZEPPELIN-4074 but no one seems to have any idea what to do. Any ideas? It sounds like isJavaAtLeast got fixed to work with JDK 9 in scala 12 (see here), but who invited JDK 9 to this party? I certainly didn’t.
> > >
> > > Zeppelin 0.8 is really struggling for us so we are desperate to upgrade but this is holding us back, would really appreciate any help at all!
> > >
> > > java.lang.NumberFormatException: Not a version: 9 at scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276) at scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283) at scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293) at scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293) at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28) at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131) at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336) at scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276) at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at scala.tools.nsc.Global.rootMirror(Global.scala:71) at scala.tools.nsc.Global.rootMirror(Global.scala:39) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257) at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390) at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139) at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161) at org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85) at org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102) at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616) at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> > >
> > >
> > >
> > > PAUL BRENNER
> > > Sr. Data Scientist
> > > pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> > > twitter @placeiq linkedin /placeiq
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: java.lang.NumberFormatException: Not a version: 9
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Paul,
Could you describe how to reproduce it ?
Paul Brenner <pb...@placeiq.com> 于2019年4月20日周六 上午12:30写道：
> I’m trying to move my company up from zeppelin 0.8 to 0.81 or 0.82.
> However, we find that when including certain dependencies on the newer
> versions of zeppelin spark interpreters won’t start and instead throw the
> error I’ll paste below. This is if we have zeppelin.spark.useNew set to
> true. Setting it to false just gives a null pointer exception instead. An
> example of a dependency that causes this is com.databricks:spark-avro_2.11:3.2.0,
> but there are also some internal packages we use that we haven’t been able
> to find anything that should be causing this conflict.
>
> I found someone with a similar problem on ZEPPELIN-4074
> <https://jira.apache.org/jira/browse/ZEPPELIN-4074> but no one seems to
> have any idea what to do. Any ideas? It sounds like isJavaAtLeast got fixed
> to work with JDK 9 in scala 12 (see here
> <https://github.com/scala/scala/pull/5276>), but who invited JDK 9 to
> this party? I certainly didn’t.
>
> Zeppelin 0.8 is really struggling for us so we are desperate to upgrade
> but this is holding us back, would really appreciate any help at all!
>
> java.lang.NumberFormatException: Not a version: 9 at
> scala.util.PropertiesTrait$class.parts$1(Properties.scala:184) at
> scala.util.PropertiesTrait$class.isJavaAtLeast(Properties.scala:187) at
> scala.util.Properties$.isJavaAtLeast(Properties.scala:17) at
> scala.tools.util.PathResolverBase$Calculated$.javaBootClasspath(PathResolver.scala:276)
> at
> scala.tools.util.PathResolverBase$Calculated$.basis(PathResolver.scala:283)
> at
> scala.tools.util.PathResolverBase$Calculated$.containers$lzycompute(PathResolver.scala:293)
> at
> scala.tools.util.PathResolverBase$Calculated$.containers(PathResolver.scala:293)
> at scala.tools.util.PathResolverBase.containers(PathResolver.scala:309) at
> scala.tools.util.PathResolver.computeResult(PathResolver.scala:341) at
> scala.tools.util.PathResolver.computeResult(PathResolver.scala:332) at
> scala.tools.util.PathResolverBase.result(PathResolver.scala:314) at
> scala.tools.nsc.backend.JavaPlatform$class.classPath(JavaPlatform.scala:28)
> at scala.tools.nsc.Global$GlobalPlatform.classPath(Global.scala:115) at
> scala.tools.nsc.Global.scala$tools$nsc$Global$$recursiveClassPath(Global.scala:131)
> at scala.tools.nsc.Global$GlobalMirror.rootLoader(Global.scala:64) at
> scala.reflect.internal.Mirrors$Roots$RootClass.<init>(Mirrors.scala:307) at
> scala.reflect.internal.Mirrors$Roots.RootClass$lzycompute(Mirrors.scala:321)
> at scala.reflect.internal.Mirrors$Roots.RootClass(Mirrors.scala:321) at
> scala.reflect.internal.Mirrors$Roots$EmptyPackageClass.<init>(Mirrors.scala:330)
> at
> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass$lzycompute(Mirrors.scala:336)
> at
> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:336)
> at
> scala.reflect.internal.Mirrors$Roots.EmptyPackageClass(Mirrors.scala:276)
> at scala.reflect.internal.Mirrors$RootsBase.init(Mirrors.scala:250) at
> scala.tools.nsc.Global.rootMirror$lzycompute(Global.scala:73) at
> scala.tools.nsc.Global.rootMirror(Global.scala:71) at
> scala.tools.nsc.Global.rootMirror(Global.scala:39) at
> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:257)
> at
> scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:257)
> at
> scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1390)
> at scala.tools.nsc.Global$Run.<init>(Global.scala:1242) at
> scala.tools.nsc.interpreter.IMain.scala$tools$nsc$interpreter$IMain$$_initialize(IMain.scala:139)
> at scala.tools.nsc.interpreter.IMain.initializeSynchronous(IMain.scala:161)
> at
> org.apache.zeppelin.spark.SparkScala211Interpreter.open(SparkScala211Interpreter.scala:85)
> at
> org.apache.zeppelin.spark.NewSparkInterpreter.open(NewSparkInterpreter.java:102)
> at
> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:62)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:616)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:188) at
> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
>
>
> PAUL BRENNER
>
> Sr. Data Scientist
> pbrenner@placeiq.com | (217) 390-3033 | www.placeiq.com
> twitter @placeiq <https://twitter.com/PlaceIQ> linkedin /placeiq
> <https://www.linkedin.com/company/placeiq/>
> [image: PlaceIQ is named an Oracle Data Cloud Premier Data Provider]
> <https://www.placeiq.com/2019/04/placeiq-named-a-premier-data-provider-and-the-sole-location-data-company-in-official-launch-of-oracle-data-cloud-program/>
> [image: PlaceIQ: Oracle Data Cloud Premier Data Provider]
> <http://go.placeiq.com/WC01DPG0l0N3Gq000000e00>
>
--
Best Regards
Jeff Zhang