You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Dhimant <dh...@gmail.com> on 2014/09/04 08:56:21 UTC
error: type mismatch while assigning RDD to RDD val object
I am receiving following error in Spark-Shell while executing following code.
/class LogRecrod(logLine: String) extends Serializable {
val splitvals = logLine.split(",");
val strIp: String = splitvals(0)
val hostname: String = splitvals(1)
val server_name: String = splitvals(2)
}/
/var logRecordRdd: org.apache.spark.rdd.RDD[LogRecrod] = _/
/ val sourceFile =
sc.textFile("hdfs://192.168.1.30:9000/Data/Log_1406794333258.log", 2)/
14/09/04 12:08:28 INFO storage.MemoryStore: ensureFreeSpace(179585) called
with curMem=0, maxMem=309225062
14/09/04 12:08:28 INFO storage.MemoryStore: Block broadcast_0 stored as
values to memory (estimated size 175.4 KB, free 294.7 MB)
sourceFile: org.apache.spark.rdd.RDD[String] = MappedRDD[1] at textFile at
<console>:12
/scala> logRecordRdd = sourceFile.map(line => new LogRecrod(line))/
/<console>:18: error: type mismatch;
found : LogRecrod
required: LogRecrod
logRecordRdd = sourceFile.map(line => new LogRecrod(line))/
Any suggestions to resolve this problem?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/error-type-mismatch-while-assigning-RDD-to-RDD-val-object-tp13429.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: error: type mismatch while assigning RDD to RDD val object
Posted by Sean Owen <so...@cloudera.com>.
I think this is a known problem with the shell and case classes. Have
a look at JIRA.
https://issues.apache.org/jira/browse/SPARK-1199
On Thu, Sep 4, 2014 at 7:56 AM, Dhimant <dh...@gmail.com> wrote:
> I am receiving following error in Spark-Shell while executing following code.
>
> /class LogRecrod(logLine: String) extends Serializable {
> val splitvals = logLine.split(",");
> val strIp: String = splitvals(0)
> val hostname: String = splitvals(1)
> val server_name: String = splitvals(2)
> }/
>
> /var logRecordRdd: org.apache.spark.rdd.RDD[LogRecrod] = _/
>
> / val sourceFile =
> sc.textFile("hdfs://192.168.1.30:9000/Data/Log_1406794333258.log", 2)/
> 14/09/04 12:08:28 INFO storage.MemoryStore: ensureFreeSpace(179585) called
> with curMem=0, maxMem=309225062
> 14/09/04 12:08:28 INFO storage.MemoryStore: Block broadcast_0 stored as
> values to memory (estimated size 175.4 KB, free 294.7 MB)
> sourceFile: org.apache.spark.rdd.RDD[String] = MappedRDD[1] at textFile at
> <console>:12
>
>
> /scala> logRecordRdd = sourceFile.map(line => new LogRecrod(line))/
> /<console>:18: error: type mismatch;
> found : LogRecrod
> required: LogRecrod
> logRecordRdd = sourceFile.map(line => new LogRecrod(line))/
>
> Any suggestions to resolve this problem?
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/error-type-mismatch-while-assigning-RDD-to-RDD-val-object-tp13429.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org