You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by ch huang <ju...@gmail.com> on 2014/06/04 11:13:55 UTC
compile spark 1.0.0 error
hi,maillist:
i try to compile spark ,but failed, here is my compile command and
compile output
# SPARK_HADOOP_VERSION=2.0.0-cdh4.4.0 SPARK_YARN=true sbt/sbt assembly
[warn] 18 warnings found
[info] Compiling 53 Scala sources and 1 Java source to
/home/admserver/spark-1.0.0/sql/catalyst/target/scala-2.10/classes...
[info] Compiling 68 Scala sources and 2 Java sources to
/home/admserver/spark-1.0.0/streaming/target/scala-2.10/classes...
[info] Compiling 62 Scala sources and 1 Java source to
/home/admserver/spark-1.0.0/mllib/target/scala-2.10/classes...
[info] Compiling 14 Scala sources to
/home/admserver/spark-1.0.0/yarn/alpha/target/scala-2.10/classes...
[error]
/home/admserver/spark-1.0.0/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala:36:
object AMResponse is not a member of package
org.apache.hadoop.yarn.api.records
[error] import org.apache.hadoop.yarn.api.records.{AMResponse,
ApplicationAttemptId}
[error] ^
[error]
/home/admserver/spark-1.0.0/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala:110:
value getAMResponse is not a member of
org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse
[error] val amResp =
allocateExecutorResources(executorsToRequest).getAMResponse
[error] ^
[error] two errors found
[error] (yarn-alpha/compile:compile) Compilation failed
[error] Total time: 1815 s, completed Jun 4, 2014 5:07:56 PM
Re: compile spark 1.0.0 error
Posted by ch huang <ju...@gmail.com>.
if i compile spark with CDH4.6 and enable yarn support , it can run on
CDH4.4?
On Wed, Jun 4, 2014 at 5:59 PM, Sean Owen <so...@cloudera.com> wrote:
> I am not sure if it is exposed in the SBT build, but you may need the
> equivalent of the 'yarn-alpha' profile from the Maven build. This
> older build of CDH predates the newer YARN APIs.
>
> See also
> https://groups.google.com/forum/#!msg/spark-users/T1soH67C5M4/CmGYV8kfRkcJ
>
> Or, use a later CDH. In fact 4.6+ has a Spark parcel for you already.
>
> On Wed, Jun 4, 2014 at 10:13 AM, ch huang <ju...@gmail.com> wrote:
> > hi,maillist:
> > i try to compile spark ,but failed, here is my compile command
> and
> > compile output
> >
> >
> >
> > # SPARK_HADOOP_VERSION=2.0.0-cdh4.4.0 SPARK_YARN=true sbt/sbt assembly
> >
> > [warn] 18 warnings found
> > [info] Compiling 53 Scala sources and 1 Java source to
> > /home/admserver/spark-1.0.0/sql/catalyst/target/scala-2.10/classes...
> > [info] Compiling 68 Scala sources and 2 Java sources to
> > /home/admserver/spark-1.0.0/streaming/target/scala-2.10/classes...
> > [info] Compiling 62 Scala sources and 1 Java source to
> > /home/admserver/spark-1.0.0/mllib/target/scala-2.10/classes...
> > [info] Compiling 14 Scala sources to
> > /home/admserver/spark-1.0.0/yarn/alpha/target/scala-2.10/classes...
> > [error]
> >
> /home/admserver/spark-1.0.0/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala:36:
> > object AMResponse is not a member of package
> > org.apache.hadoop.yarn.api.records
> > [error] import org.apache.hadoop.yarn.api.records.{AMResponse,
> > ApplicationAttemptId}
> > [error] ^
> > [error]
> >
> /home/admserver/spark-1.0.0/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala:110:
> > value getAMResponse is not a member of
> > org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse
> > [error] val amResp =
> > allocateExecutorResources(executorsToRequest).getAMResponse
> > [error] ^
> > [error] two errors found
> >
> >
> > [error] (yarn-alpha/compile:compile) Compilation failed
> > [error] Total time: 1815 s, completed Jun 4, 2014 5:07:56 PM
>
Re: compile spark 1.0.0 error
Posted by Sean Owen <so...@cloudera.com>.
I am not sure if it is exposed in the SBT build, but you may need the
equivalent of the 'yarn-alpha' profile from the Maven build. This
older build of CDH predates the newer YARN APIs.
See also https://groups.google.com/forum/#!msg/spark-users/T1soH67C5M4/CmGYV8kfRkcJ
Or, use a later CDH. In fact 4.6+ has a Spark parcel for you already.
On Wed, Jun 4, 2014 at 10:13 AM, ch huang <ju...@gmail.com> wrote:
> hi,maillist:
> i try to compile spark ,but failed, here is my compile command and
> compile output
>
>
>
> # SPARK_HADOOP_VERSION=2.0.0-cdh4.4.0 SPARK_YARN=true sbt/sbt assembly
>
> [warn] 18 warnings found
> [info] Compiling 53 Scala sources and 1 Java source to
> /home/admserver/spark-1.0.0/sql/catalyst/target/scala-2.10/classes...
> [info] Compiling 68 Scala sources and 2 Java sources to
> /home/admserver/spark-1.0.0/streaming/target/scala-2.10/classes...
> [info] Compiling 62 Scala sources and 1 Java source to
> /home/admserver/spark-1.0.0/mllib/target/scala-2.10/classes...
> [info] Compiling 14 Scala sources to
> /home/admserver/spark-1.0.0/yarn/alpha/target/scala-2.10/classes...
> [error]
> /home/admserver/spark-1.0.0/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala:36:
> object AMResponse is not a member of package
> org.apache.hadoop.yarn.api.records
> [error] import org.apache.hadoop.yarn.api.records.{AMResponse,
> ApplicationAttemptId}
> [error] ^
> [error]
> /home/admserver/spark-1.0.0/yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala:110:
> value getAMResponse is not a member of
> org.apache.hadoop.yarn.api.protocolrecords.AllocateResponse
> [error] val amResp =
> allocateExecutorResources(executorsToRequest).getAMResponse
> [error] ^
> [error] two errors found
>
>
> [error] (yarn-alpha/compile:compile) Compilation failed
> [error] Total time: 1815 s, completed Jun 4, 2014 5:07:56 PM