You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@toree.apache.org by "Jakob Odersky (JIRA)" <ji...@apache.org> on 2017/03/30 19:21:41 UTC
[jira] [Commented] (TOREE-399) Make Spark Kernel work on Windows
[ https://issues.apache.org/jira/browse/TOREE-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949642#comment-15949642 ]
Jakob Odersky commented on TOREE-399:
-------------------------------------
Hi Aldo,
the run.sh script is a launcher script that basically starts Toree as a Spark app by calling spark-submit. To add support on windows, the easiest is to probably create a custom run/bat script that does equivalent of run.sh on windows and create a custiom kernel.json to call it.
Furthermore, maybe Toree works without modification on the linux subsytem on windows 10.
> Make Spark Kernel work on Windows
> ---------------------------------
>
> Key: TOREE-399
> URL: https://issues.apache.org/jira/browse/TOREE-399
> Project: TOREE
> Issue Type: New Feature
> Environment: Windows 7/8/10
> Reporter: aldo
>
> After a successful install of the Spark Kernel the error: "Failed to run command:" occurs when from jupyter we select a Scala Notebook.
> The error happens because the kernel.json runs C:\\ProgramData\\jupyter\\kernels\\apache_toree_scala\\bin\\run.sh which is bash shell script and hence cannot work on windows.
> Can you give me some direction to fix this, and I will implement it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
RE: [jira] [Commented] (TOREE-399) Make Spark Kernel work on Windows
Posted by "Pahor Aldo (Consulente)" <Al...@allianz.it>.
Hi Jakob,
I created a quick run.bat with hardcoded values
%SPARK_HOME%/bin/spark-submit --class org.apache.toree.Main C:\ProgramData\jupyter\kernels\apache_toree_scala\lib\toree-assembly-0.2.0.dev1-incubating-SNAPSHOT.jar
This passes the previous error, but still getting errors. I guess related to some scala config see below. Any idea?
Besides the error, in view to create a windows version of the run.sh, is not clear to me how kernel.json var are passed to the run.bat and how can I refer to them in run.bat.
Any direction?
17/03/31 09:55:29 [WARN] o.a.h.u.NativeCodeLoader - Unable to load native-hadoop library for your platform... using buil
tin-java classes where applicable
17/03/31 09:55:30 [INFO] o.a.t.b.l.StandardComponentInitialization$$anon$1 - Connecting to spark.master local[*]
[init] error: error while loading Object, Missing dependency 'object scala in compiler mirror', required by C:\Program F
iles\Java\jdk1.8.0_121\jre\lib\rt.jar(java/lang/Object.class)
Failed to initialize compiler: object scala in compiler mirror not found.
** Note that as of 2.8 scala does not assume use of the java classpath.
** For the old behavior pass -usejavacp to scala, or if using a Settings
** object programmatically, settings.usejavacp.value = true.
Failed to initialize compiler: object scala in compiler mirror not found.
** Note that as of 2.8 scala does not assume use of the java classpath.
** For the old behavior pass -usejavacp to scala, or if using a Settings
** object programmatically, settings.usejavacp.value = true.
Exception in thread "main" java.lang.NullPointerException
at scala.reflect.internal.SymbolTable.exitingPhase(SymbolTable.scala:256)
at scala.tools.nsc.interpreter.IMain$Request.x$20$lzycompute(IMain.scala:896)
at scala.tools.nsc.interpreter.IMain$Request.x$20(IMain.scala:895)
at scala.tools.nsc.interpreter.IMain$Request.headerPreamble$lzycompute(IMain.scala:895)
at scala.tools.nsc.interpreter.IMain$Request.headerPreamble(IMain.scala:895)
at scala.tools.nsc.interpreter.IMain$Request$Wrapper.preamble(IMain.scala:918)
at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$apply$23.apply(IMain.scala:1337)
at scala.tools.nsc.interpreter.IMain$CodeAssembler$$anonfun$apply$23.apply(IMain.scala:1336)
at scala.tools.nsc.util.package$.stringFromWriter(package.scala:64)
at scala.tools.nsc.interpreter.IMain$CodeAssembler$class.apply(IMain.scala:1336)
at scala.tools.nsc.interpreter.IMain$Request$Wrapper.apply(IMain.scala:908)
at scala.tools.nsc.interpreter.IMain$Request.compile$lzycompute(IMain.scala:1002)
at scala.tools.nsc.interpreter.IMain$Request.compile(IMain.scala:997)
at scala.tools.nsc.interpreter.IMain.compile(IMain.scala:579)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:567)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
at org.apache.toree.kernel.interpreter.scala.ScalaInterpreterSpecific$$anonfun$start$1.apply(ScalaInterpreterSpe
cific.scala:295)
at org.apache.toree.kernel.interpreter.scala.ScalaInterpreterSpecific$$anonfun$start$1.apply(ScalaInterpreterSpe
cific.scala:289)
at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
at org.apache.toree.kernel.interpreter.scala.ScalaInterpreterSpecific$class.start(ScalaInterpreterSpecific.scala
:289)
at org.apache.toree.kernel.interpreter.scala.ScalaInterpreter.start(ScalaInterpreter.scala:44)
at org.apache.toree.kernel.interpreter.scala.ScalaInterpreter.init(ScalaInterpreter.scala:87)
at org.apache.toree.boot.layer.InterpreterManager$$anonfun$initializeInterpreters$1.apply(InterpreterManager.sca
la:35)
-----Original Message-----
From: Jakob Odersky (JIRA) [mailto:jira@apache.org]
Sent: Thursday 30 March 2017 9:22
To: dev@toree.incubator.apache.org
Subject: [jira] [Commented] (TOREE-399) Make Spark Kernel work on Windows
[ https://issues.apache.org/jira/browse/TOREE-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949642#comment-15949642 ]
Jakob Odersky commented on TOREE-399:
-------------------------------------
Hi Aldo,
the run.sh script is a launcher script that basically starts Toree as a Spark app by calling spark-submit. To add support on windows, the easiest is to probably create a custom run/bat script that does equivalent of run.sh on windows and create a custiom kernel.json to call it.
Furthermore, maybe Toree works without modification on the linux subsytem on windows 10.
> Make Spark Kernel work on Windows
> ---------------------------------
>
> Key: TOREE-399
> URL: https://issues.apache.org/jira/browse/TOREE-399
> Project: TOREE
> Issue Type: New Feature
> Environment: Windows 7/8/10
> Reporter: aldo
>
> After a successful install of the Spark Kernel the error: "Failed to run command:" occurs when from jupyter we select a Scala Notebook.
> The error happens because the kernel.json runs C:\\ProgramData\\jupyter\\kernels\\apache_toree_scala\\bin\\run.sh which is bash shell script and hence cannot work on windows.
> Can you give me some direction to fix this, and I will implement it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Le informazioni contenute in questo messaggio e gli eventuali allegati (il
"Messaggio") si intendono inviate a uno o piu' specifici destinatari. Il
contenuto del Messaggio puo' essere confidenziale, riservato e comunque
protetto dalla legge applicabile. Se non siete i destinatari del Messaggio,
siete pregati di informare immediatamente il mittente, cancellate questo
Messaggio, non rivelatelo, non distribuitelo ne' inoltratelo a terzi, non
copiatelo ne' fatene alcun uso.
Questo Messaggio ha natura aziendale e non personale. Pertanto le risposte
inviate a seguito di questa comunicazione potranno essere conosciute
nell'organizzazione di appartenenza del mittente nei limiti e secondo le
regole fissate dall'azienda, nel rispetto della vigente normativa e del
disciplinare interno. Qualsiasi pubblicazione, utilizzo o diffusione, anche
parziale di questo Messaggio deve essere preventivamente autorizzata.
I messaggi di posta elettronica non sono sicuri e sono soggetti ad
alterazioni, possono essere trasmettitori di Virus informatici o soggetti
a ritardi nella distribuzione. Il mittente del Messaggio non puo' essere
in alcun modo considerato responsabile per queste evenienze. Il mittente si
riserva il diritto di archiviare, ritenere e controllare i messaggi di posta
elettronica.
The information contained in this electronic message and any attachments (the
"Message") is intended for one or more specific individuals or entities, and
may be confidential, proprietary, privileged or otherwise protected by law.
If you are not the intended recipient, please notify the sender immediately,
delete the Message and do not disclose, distribute, or copy it to any third
party or otherwise use the Message.
The Message has business character and not private and/or individual
character. Your reply to the Message could be disclosed to the sender entity,
according to and in respect of its internal rules, under the applicable laws,
regulations or internal policies. Any publication, use or circulation of the
Message, both total and partial, have to be previously authorized.
Electronic messages are not secure or error free and can contain viruses or
may be delayed, and the sender is not liable for any of these occurrences.
The sender reserves the right to monitor, record and retain electronic
messages.