[jira] [Commented] (LENS-515) Support to run spark in yarn-cluster mode

["Yash Sharma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/LENS-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571401#comment-14571401 ] 

Yash Sharma commented on LENS-515:
----------------------------------

No luck yet.
Dead end at NPE as mentioned by [~sharadag]

Error log for reference -

java.lang.NullPointerException
	at org.apache.spark.deploy.yarn.ApplicationMaster$.sparkContextInitialized(ApplicationMaster.scala:581)
	at org.apache.spark.scheduler.cluster.YarnClusterScheduler.postStartHook(YarnClusterScheduler.scala:32)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:541)
	at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
	at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:75)
	at org.apache.lens.ml.algo.spark.SparkMLDriver.start(SparkMLDriver.java:198)
	at org.apache.lens.ml.impl.LensMLImpl.start(LensMLImpl.java:318)
	at org.apache.lens.ml.server.MLServiceImpl.start(MLServiceImpl.java:169)
	at org.apache.hive.service.CompositeService.start(CompositeService.java:70)
	at org.apache.lens.server.LensServices.start(LensServices.java:241)
	at org.apache.lens.server.LensJerseyTest.startAll(LensJerseyTest.java:123)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	

> Support to run spark in yarn-cluster mode
> -----------------------------------------
>
>                 Key: LENS-515
>                 URL: https://issues.apache.org/jira/browse/LENS-515
>             Project: Apache Lens
>          Issue Type: Improvement
>          Components: ml
>            Reporter: Sharad Agarwal
>            Assignee: Yash Sharma
>         Attachments: Lens-515-1.patch
>
>
> currently spark only runs in embedded mode. support to run in yarn-cluster mode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)