You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by "Luca Menichetti (JIRA)" <ji...@apache.org> on 2015/08/21 00:17:46 UTC
[jira] [Created] (ZEPPELIN-246) org.apache.hadoop.mapred.JobConf
cannot be cast to org.apache.spark.rdd.RDD
Luca Menichetti created ZEPPELIN-246:
----------------------------------------
Summary: org.apache.hadoop.mapred.JobConf cannot be cast to org.apache.spark.rdd.RDD
Key: ZEPPELIN-246
URL: https://issues.apache.org/jira/browse/ZEPPELIN-246
Project: Zeppelin
Issue Type: Bug
Components: Core
Environment: Hadoop: 2.5.0-cdh5.3.3
Spark: 1.2.0
Reporter: Luca Menichetti
All operations with RDDs (except few like "saveAsTextFile") are producing the same output (see below). This is happening with Zeppeling build with:
mvn clean package -Pspark-1.2 -Dhadoop.version=2.5.0-cdh5.3.3 -Phadoop-2.4 -Pyarn -DskipTests
SPARK_JAR and Hadoop conf dir are properly set.
SPARK_JAR= hdfs://p23409239480:9000/project/hadoop/spark-assembly-1.2.0-cdh5.3.3-hadoop2.5.0-cdh5.3.3.jar
Problem also reported here: https://github.com/NFLabs/zeppelin/issues/368
---
WARN [2015-08-20 23:14:09,526] ({task-result-getter-0} Logging.scala[logWarning]:71) - Lost task 89.0 in stage 0.0 (TID 0, p01001532975913.cern.ch): java.lang.ClassCastException: org.apache.hadoop.mapred.JobConf cannot be cast to org.apache.spark.rdd.RDD
at org.apache.spark.SparkContext$$anonfun$27.apply(SparkContext.scala:1050)
at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:170)
at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:170)
at scala.Option.map(Option.scala:145)
at org.apache.spark.rdd.HadoopRDD.getJobConf(HadoopRDD.scala:170)
at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:214)
at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:210)
at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:99)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.rdd.MappedRDD.compute(MappedRDD.scala:31)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:263)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:230)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
at org.apache.spark.scheduler.Task.run(Task.scala:56)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:196)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)