You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by "Arjun J (Jira)" <ji...@apache.org> on 2022/05/18 08:14:00 UTC

[jira] [Created] (ZEPPELIN-5741) unable to run spark job within Zeppelin 0.10.1

Arjun J created ZEPPELIN-5741:
---------------------------------

             Summary: unable to run spark job within Zeppelin 0.10.1
                 Key: ZEPPELIN-5741
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-5741
             Project: Zeppelin
          Issue Type: Bug
          Components: spark, spark-submit, zeppelin-client, zeppelin-interpreter
    Affects Versions: 0.10.1
            Reporter: Arjun J
             Fix For: 0.10.1
         Attachments: Docker merged.txt, Zepp-Spark.jpg, spark master log.txt, spark slave log.txt, spark.jpg, zepp log.txt

Hi,

I downloaded the Zeppelin docker file from the below site and then i took the spark standalone docker file and merged them into one (attached). I was able to create the image and run the cluster.  

1) Zeppelin 0.10.1 : [https://hub.docker.com/r/apache/zeppelin/dockerfile] 
2) Spark standalone : [https://github.com/apache/zeppelin/blob/master/scripts/docker/spark-cluster-managers/spark_standalone/Dockerfile|https://github.com/apache/zeppelin/blob/master/scripts/docker/spark-cluster-managers/spark_standalone/Dockerfile.]

Command used to run the container :

docker run -it \
-p 8080:8080 \
-p 7077:7077 \
-p 8888:8888 \
-p 8081:8081 \
-h sparkmaster \
--name imagename\
imagename bash;

3) I am able to connect to Zeppelin Ui : http://hostname:8080/and run basic r/ python notebook but when I run a simple spark code it keeps running and does not finish & its gets stuck. I have restart zep-daemeon everytime to test it again. The application id doesn't even show in the spark master ui: [http://hostname:8081/.|http://hostname:8081/] 

4) I am able to connect to spark-shell, pyspark and do spark-submit within the container and can see the application in the spark master url but none works from Zeppelin. I have been struggling to make it run from past 1 week :(. Please help.

Note : I had set the spark_home : /usr/local/spark/ and master : spark://sparkmaster:7077 in the interpreter setting for spark but it didn't help. There is also no interpreter log being generated within the zeppelin logs folder.

I have attached the logs (spark master , slave and zepp logs) but no errors in it. 



--
This message was sent by Atlassian Jira
(v8.20.7#820007)