[jira] [Comment Edited] (SPARK-25783) Spark shell fails because of jline incompatibility

["koert kuipers (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657455#comment-16657455 ] 

koert kuipers edited comment on SPARK-25783 at 10/19/18 9:25 PM:
-----------------------------------------------------------------

i was just arriving at a similar conclusion. i couldn't find jline anywhere within my spark install. so its using jline from my hadoop install, and that version is jline 2.11

repl should explicitly rely on jline if it uses it, no?


was (Author: koert):
i was just to same conclusion. i couldn't find jline anywhere within my spark install. so its using jline from my hadoop install, and that version is jline 2.11

repl should explicitly rely on jline if it uses it, no?

> Spark shell fails because of jline incompatibility
> --------------------------------------------------
>
>                 Key: SPARK-25783
>                 URL: https://issues.apache.org/jira/browse/SPARK-25783
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Shell
>    Affects Versions: 2.4.0
>         Environment: spark 2.4.0-rc3 on hadoop 2.6.0 (cdh 5.15.1) with -Phadoop-provided
>            Reporter: koert kuipers
>            Priority: Minor
>
> error i get when launching spark-shell is:
> {code:bash}
> Spark context Web UI available at http://client:4040
> Spark context available as 'sc' (master = yarn, app id = application_xxx).
> Spark session available as 'spark'.
> Exception in thread "main" java.lang.NoSuchMethodError: jline.console.completer.CandidateListCompletionHandler.setPrintSpaceAfterFullCompletion(Z)V
> 	at scala.tools.nsc.interpreter.jline.JLineConsoleReader.initCompletion(JLineReader.scala:139)
> 	at scala.tools.nsc.interpreter.jline.InteractiveReader.postInit(JLineReader.scala:54)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$1.apply(SparkILoop.scala:190)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$1.apply(SparkILoop.scala:188)
> 	at scala.tools.nsc.interpreter.SplashReader.postInit(InteractiveReader.scala:130)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$org$apache$spark$repl$SparkILoop$$anonfun$$loopPostInit$1$1.apply$mcV$sp(SparkILoop.scala:214)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$org$apache$spark$repl$SparkILoop$$anonfun$$loopPostInit$1$1.apply(SparkILoop.scala:199)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$org$apache$spark$repl$SparkILoop$$anonfun$$loopPostInit$1$1.apply(SparkILoop.scala:199)
> 	at scala.tools.nsc.interpreter.ILoop$$anonfun$mumly$1.apply(ILoop.scala:189)
> 	at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:221)
> 	at scala.tools.nsc.interpreter.ILoop.mumly(ILoop.scala:186)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1.org$apache$spark$repl$SparkILoop$$anonfun$$loopPostInit$1(SparkILoop.scala:199)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$startup$1$1.apply(SparkILoop.scala:267)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1$$anonfun$startup$1$1.apply(SparkILoop.scala:247)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1.withSuppressedSettings$1(SparkILoop.scala:235)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1.startup$1(SparkILoop.scala:247)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:282)
> 	at org.apache.spark.repl.SparkILoop.runClosure(SparkILoop.scala:159)
> 	at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:182)
> 	at org.apache.spark.repl.Main$.doMain(Main.scala:78)
> 	at org.apache.spark.repl.Main$.main(Main.scala:58)
> 	at org.apache.spark.repl.Main.main(Main.scala)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> 	at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
> 	at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
> 	at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
> 	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
> 	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
> 	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:935)
> 	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}
> spark 2.4.0-rc3 which i build with:
> {code:bash}
> dev/make-distribution.sh --name provided --tgz -Phadoop-2.6 -Dhadoop.version=2.6.0 -Pyarn -Phadoop-provided
> {code}
> and deployed with in spark-env.sh:
> {code:bash}
> export SPARK_DIST_CLASSPATH=$(hadoop classpath)
> {code}
> hadoop version is 2.6.0 (CDH 5.15.1)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org