You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2014/09/02 15:02:22 UTC

[jira] [Resolved] (SPARK-524) spark integration issue with Cloudera hadoop

     [ https://issues.apache.org/jira/browse/SPARK-524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-524.
-----------------------------
    Resolution: Won't Fix

I'm testing whether contributors can in fact close others JIRAs. It seems clear this one is stale and possibly already resolved.

I'm also interested in how comfortable people are with people like me modifying JIRAs where it seems like there is a clear update or status change that needs to happen, like, it's clearly already resolved and needs to be marked as such?

> spark integration issue with Cloudera hadoop
> --------------------------------------------
>
>                 Key: SPARK-524
>                 URL: https://issues.apache.org/jira/browse/SPARK-524
>             Project: Spark
>          Issue Type: Bug
>            Reporter: openreserach
>
> Hi, 
> 1. I am using single EC2 instance with pre-built mesos (ami-0fcb7966) (Same issue if I build mesos from source code in locall VM)
> 2. Follow instruction on https://github.com/mesos/spark/wiki/Running-spark-on-mesos with some tweaks.
> 3. I install Cloudera cdhu5 by yum (not using pre-built hadoop due to lack of document)
> 4. ./spartk-shell.sh
> import spark._
> val sc = new SparkContext("localhost:5050","passwd")
> val ec2 = sc.textFile("hdfs://localhost:8020/tmp/passwd")
> IF I keep val HADOOP_VERSION = "0.20.205.0" in project/SparkBuild.scala
> at val file = sc.textFile("hdfs://localhost:8020/tmp/passwd")
> I am getting error
> Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 61, server = 63)
> IF I set val HADOOP_VERSION = "0.20.2-cdh3u5" or val HADOOP_VERSION = "0.20.2-cdh3u3" 
> I am getting error at  ec2.count()
> ERROR spark.SimpleJob: Task 0:0 failed more than 4 times; aborting job
> like the one reported at http://mail-archives.apache.org/mod_mbox/incubator-mesos-dev/201108.mbox/%3CBD25AE7A-C9DC-4020-AD40-41C66DCAA56E@eecs.berkeley.edu%3E
> Please let me know if you cannot replicate this error, and give more instruction on how Spark integrate with Cloudera Hadoop 
> Thanks
> -QH



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org