You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by "Tang, Jie" <jt...@ebay.com> on 2015/08/20 00:23:30 UTC
RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Hello,
I'm having the same issue, even just running a simple sc.parallelize(1 to 100), I got the same error. Any ideas how to solve it? It worked fine in spark 1.3.0. Only happening after I upgrade to spark 1.4.1.
thanks!
Jie
From: Bob Beauchemin [mailto:bobb@sqlskills.com]
Sent: Saturday, July 25, 2015 1:47 PM
To: users@zeppelin.incubator.apache.org
Subject: Error rendering Zeppelin sample notebook using Spark 1.4.1
Downloaded and built Zeppelin on Centos 6. Built with no issues for Spark 1.4.1 using "mvn clean install -Pspark-1.4 -DskipTests" or "mvn clean install -Pspark-1.4 -DskipTests -Dhadoop.version=2.6.0 -Phadoop-2.6". Running the default configuration.
It seems to be set up to use the right version, just typing, running "sc" returns an object reference or "sc.version" returns 1.4.1.
Running the bank example in the sample notebook returns (first line of a long stack dump):
com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
at [Source: {"id":"0","name":"parallelize"}; line: 1, column: 1]...
...first Spark-related frame:
at org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:82)
The bank code executes fine in the Spark Scala shell.
In fact, anything that uses Spark past the "sc" command, such as:
val data = Array(1, 2, 3, 4, 5)
val distData = sc.parallelize(data)
Produces the exact same mapping error. So I'm guessing it might be in the rendering. Tried using Firefox on Centos, Firefox on Windows, even IE on Windows. Same error.
Does Zeppelin support Spark 1.4? It seems to imply this in the readme. Or is there something else I'm missing?
Thanks, Bob
RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Posted by "Tang, Jie" <jt...@ebay.com>.
Thanks Bob! That was the problem. I got the latest master. After I tried the 0.5.0 snapshot, the error went away.
Thanks
Jie
From: Bob Beauchemin [mailto:bobb@sqlskills.com]
Sent: Wednesday, August 19, 2015 5:01 PM
To: Tang, Jie; users@zeppelin.incubator.apache.org
Subject: RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Hi Jie,
The problem went away when I downloaded the pre-built distribution (zeppelin-0.5.0-incubating-bin-spark-1.4.0_hadoop-2-3) from the snapshot, run with SPARK_HOME set to my 1.4.1 home. Never did figure out why the version I built didn't work. sc.version did report 1.4.1.
Were you compiling against the 0.5.1-SNAPSHOT source? I got the error against a pre-snapshot "current version at the time" from the master, downloaded about a 1-2 weeks before the snapshot dropped.
HTH, Cheers, Bob
From: Tang, Jie [mailto:jtang3@ebay.com]
Sent: Wednesday, August 19, 2015 3:28 PM
To: Bob Beauchemin <bo...@sqlskills.com>>
Subject: RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Hey Bob,
Have you been able to figure out the solution for this? thanks!
Jie
From: Tang, Jie [mailto:jtang3@ebay.com]
Sent: Wednesday, August 19, 2015 3:24 PM
To: users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>
Subject: RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Hello,
I'm having the same issue, even just running a simple sc.parallelize(1 to 100), I got the same error. Any ideas how to solve it? It worked fine in spark 1.3.0. Only happening after I upgrade to spark 1.4.1.
thanks!
Jie
From: Bob Beauchemin [mailto:bobb@sqlskills.com]
Sent: Saturday, July 25, 2015 1:47 PM
To: users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>
Subject: Error rendering Zeppelin sample notebook using Spark 1.4.1
Downloaded and built Zeppelin on Centos 6. Built with no issues for Spark 1.4.1 using "mvn clean install -Pspark-1.4 -DskipTests" or "mvn clean install -Pspark-1.4 -DskipTests -Dhadoop.version=2.6.0 -Phadoop-2.6". Running the default configuration.
It seems to be set up to use the right version, just typing, running "sc" returns an object reference or "sc.version" returns 1.4.1.
Running the bank example in the sample notebook returns (first line of a long stack dump):
com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
at [Source: {"id":"0","name":"parallelize"}; line: 1, column: 1]...
...first Spark-related frame:
at org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:82)
The bank code executes fine in the Spark Scala shell.
In fact, anything that uses Spark past the "sc" command, such as:
val data = Array(1, 2, 3, 4, 5)
val distData = sc.parallelize(data)
Produces the exact same mapping error. So I'm guessing it might be in the rendering. Tried using Firefox on Centos, Firefox on Windows, even IE on Windows. Same error.
Does Zeppelin support Spark 1.4? It seems to imply this in the readme. Or is there something else I'm missing?
Thanks, Bob
RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Posted by Bob Beauchemin <bo...@sqlskills.com>.
Hi Jie,
The problem went away when I downloaded the pre-built distribution (zeppelin-0.5.0-incubating-bin-spark-1.4.0_hadoop-2-3) from the snapshot, run with SPARK_HOME set to my 1.4.1 home. Never did figure out why the version I built didn't work. sc.version did report 1.4.1.
Were you compiling against the 0.5.1-SNAPSHOT source? I got the error against a pre-snapshot "current version at the time" from the master, downloaded about a 1-2 weeks before the snapshot dropped.
HTH, Cheers, Bob
From: Tang, Jie [mailto:jtang3@ebay.com]
Sent: Wednesday, August 19, 2015 3:28 PM
To: Bob Beauchemin <bo...@sqlskills.com>
Subject: RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Hey Bob,
Have you been able to figure out the solution for this? thanks!
Jie
From: Tang, Jie [mailto:jtang3@ebay.com]
Sent: Wednesday, August 19, 2015 3:24 PM
To: users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>
Subject: RE: Error rendering Zeppelin sample notebook using Spark 1.4.1
Hello,
I'm having the same issue, even just running a simple sc.parallelize(1 to 100), I got the same error. Any ideas how to solve it? It worked fine in spark 1.3.0. Only happening after I upgrade to spark 1.4.1.
thanks!
Jie
From: Bob Beauchemin [mailto:bobb@sqlskills.com]
Sent: Saturday, July 25, 2015 1:47 PM
To: users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>
Subject: Error rendering Zeppelin sample notebook using Spark 1.4.1
Downloaded and built Zeppelin on Centos 6. Built with no issues for Spark 1.4.1 using "mvn clean install -Pspark-1.4 -DskipTests" or "mvn clean install -Pspark-1.4 -DskipTests -Dhadoop.version=2.6.0 -Phadoop-2.6". Running the default configuration.
It seems to be set up to use the right version, just typing, running "sc" returns an object reference or "sc.version" returns 1.4.1.
Running the bank example in the sample notebook returns (first line of a long stack dump):
com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
at [Source: {"id":"0","name":"parallelize"}; line: 1, column: 1]...
...first Spark-related frame:
at org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:82)
The bank code executes fine in the Spark Scala shell.
In fact, anything that uses Spark past the "sc" command, such as:
val data = Array(1, 2, 3, 4, 5)
val distData = sc.parallelize(data)
Produces the exact same mapping error. So I'm guessing it might be in the rendering. Tried using Firefox on Centos, Firefox on Windows, even IE on Windows. Same error.
Does Zeppelin support Spark 1.4? It seems to imply this in the readme. Or is there something else I'm missing?
Thanks, Bob