[jira] [Created] (ZEPPELIN-3950) zeppelin can't retrive data from spark native k8s cluster correctly

["YuanzhangZhu (JIRA)" <ji...@apache.org>]

YuanzhangZhu created ZEPPELIN-3950:
--------------------------------------

             Summary: zeppelin can't retrive data from spark native  k8s cluster correctly
                 Key: ZEPPELIN-3950
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-3950
             Project: Zeppelin
          Issue Type: Bug
          Components: zeppelin-interpreter
    Affects Versions: 0.8.0
         Environment: centos 7.5 

java 1.8

zeppelin 0.8.0 0.9.0 SNAPSHOT

spark 2.4.0
            Reporter: YuanzhangZhu


I set up native spark on K8S acording to http://spark.apache.org/docs/latest/running-on-kubernetes.html . and set spark interpreter's master :k8s://https://10.64.17.119:6443 

I tried 2 case:

case 1:

import spark.implicits._
val df = Seq(("Judy", "50"),("Randy", "60")).toDF("int_column", "string_column", "date_column")
df.registerTempTable("peopleTB")
%sql
select * from peopleTB

case 2:

%spark
case class WordCountTable(word: String, count: Integer)
val textFile = sc.textFile("hdfs://10.64.17.119:9000/wordcounter.txt")
val wordCount = textFile.map(line => line.split(",")).map(line => WordCountTable(line(0),line(1).toInt)).toDF()
wordCount.registerTempTable("wordCountTB")

%sql
select * from wordCountTB

 

 

Case1 works well , Case 2 alway hung at "select * from wordCountTB"

when I change spark interpreter's master to local[*], the 2 case all works well.

 

It seems  that zeppelin can't retrive data from spark's native k8s cluster correctly

 

 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)