You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by ABHISHEK <ab...@gmail.com> on 2014/05/26 04:51:48 UTC
Fails: Spark sbt/sbt publish local
Hi,
I'm trying to install Spark along with Shark.
Here's configuration details:
Spark 0.9.1
Shark 0.9.1
Scala 2.10.3
Spark assembly was successful but running "sbt/sbt publish-local" failed.
Please refer attached log for more details and advise.
Thanks,
Abhishek
Sparkhome>SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.4.0
Successfully build.
Sparkhome>sbt/sbt publish local ( It failed)
/home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.pom
[info] published spark-repl_2.10 to
/home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.jar
[info] published spark-repl_2.10 to
/home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-sources.jar
[info] published spark-repl_2.10 to
/home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-javadoc.jar
[info] published spark-repl_2.10 to
/home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/poms/spark-repl_2.10.pom
[info] published spark-repl_2.10 to
/home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/jars/spark-repl_2.10.jar
[info] published spark-repl_2.10 to
/home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/srcs/spark-repl_2.10-sources.jar
[info] published spark-repl_2.10 to
/home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/docs/spark-repl_2.10-javadoc.jar
[info] published ivy to
/home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/ivys/ivy.xml
[error] (streaming/compile:compile) Compilation failed
[error] Total time: 419 s, completed May 25, 2014 7:28:27 PM
Re: Fails: Spark sbt/sbt publish local
Posted by Aaron Davidson <il...@gmail.com>.
Googling that error, I came across something that appears relevant:
https://groups.google.com/forum/#!msg/spark-users/T1soH67C5M4/vihzNt92anYJ
I'd try just doing sbt/sbt clean first, and if that fails, digging deeper
into that thread.
(By the way, "sbt/sbt publish-local" IS what you want, otherwise it's
trying to publish to a remote repo.)
On Sun, May 25, 2014 at 8:24 PM, ABHISHEK <ab...@gmail.com> wrote:
> Thanks for reply Aaron.
> I tried with "sbt/sbt publish local" but got below error.
>
>
> [error]
> /home/cloudera/at_Installation/spark-0.9.1-bin-cdh4/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:669:
> type mismatch;
> [error] found : org.apache.spark.streaming.dstream.DStream[(K,
> (com.google.common.base.Optional[V], W))]
> [error] required:
> org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V],
> W)]
> [error] Note: implicit method fromPairDStream is not applicable here
> because it comes after the application point and it lacks an explicit
> result type
> [error] joinResult.mapValues{case (v, w) =>
> (JavaUtils.optionToOptional(v), w)}
> [error] ^
> [error] 44 errors found
> java.lang.RuntimeException: Repository for publishing is not specified.
> at scala.sys.package$.error(package.scala:27)
> at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
> at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
> at scala.Option.getOrElse(Option.scala:108)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ..........
> java.lang.RuntimeException: Repository for publishing is not specified.
> at scala.sys.package$.error(package.scala:27)
> at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
> at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
> at scala.Option.getOrElse(Option.scala:108)
> at sbt.Classpaths$.getPublishTo(Defaults.scala:997)
> ........
> [error] (repl/*:publish-configuration) Repository for publishing is not
> specified.
> [error] (graphx/*:publish-configuration) Repository for publishing is not
> specified.
> [error] (mllib/*:publish-configuration) Repository for publishing is not
> specified.
>
> [error] (streaming/compile:compile) Compilation failed
> [error] (core/*:publish-configuration) Repository for publishing is not
> specified.
> [error] (bagel/*:publish-configuration) Repository for publishing is not
> specified.
> [error] Total time: 58 s, completed May 25, 2014 8:20:46 PM
>
>
> On Mon, May 26, 2014 at 8:46 AM, Aaron Davidson <il...@gmail.com>wrote:
>
>> I suppose you actually ran "publish-local" and not "publish local" like
>> your example showed. That being the case, could you show the compile error
>> that occurs? It could be related to the hadoop version.
>>
>>
>> On Sun, May 25, 2014 at 7:51 PM, ABHISHEK <ab...@gmail.com> wrote:
>>
>>> Hi,
>>> I'm trying to install Spark along with Shark.
>>> Here's configuration details:
>>> Spark 0.9.1
>>> Shark 0.9.1
>>> Scala 2.10.3
>>>
>>> Spark assembly was successful but running "sbt/sbt publish-local" failed.
>>> Please refer attached log for more details and advise.
>>> Thanks,
>>> Abhishek
>>>
>>> Sparkhome>SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.4.0
>>> Successfully build.
>>>
>>> Sparkhome>sbt/sbt publish local ( It failed)
>>>
>>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.pom
>>> [info] published spark-repl_2.10 to
>>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.jar
>>> [info] published spark-repl_2.10 to
>>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-sources.jar
>>> [info] published spark-repl_2.10 to
>>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-javadoc.jar
>>> [info] published spark-repl_2.10 to
>>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/poms/spark-repl_2.10.pom
>>> [info] published spark-repl_2.10 to
>>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/jars/spark-repl_2.10.jar
>>> [info] published spark-repl_2.10 to
>>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/srcs/spark-repl_2.10-sources.jar
>>> [info] published spark-repl_2.10 to
>>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/docs/spark-repl_2.10-javadoc.jar
>>> [info] published ivy to
>>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/ivys/ivy.xml
>>> [error] (streaming/compile:compile) Compilation failed
>>> [error] Total time: 419 s, completed May 25, 2014 7:28:27 PM
>>>
>>>
>>>
>>>
>>>
>>
>
Re: Fails: Spark sbt/sbt publish local
Posted by ABHISHEK <ab...@gmail.com>.
Thanks for reply Aaron.
I tried with "sbt/sbt publish local" but got below error.
[error]
/home/cloudera/at_Installation/spark-0.9.1-bin-cdh4/streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaPairDStream.scala:669:
type mismatch;
[error] found : org.apache.spark.streaming.dstream.DStream[(K,
(com.google.common.base.Optional[V], W))]
[error] required:
org.apache.spark.streaming.api.java.JavaPairDStream[K,(com.google.common.base.Optional[V],
W)]
[error] Note: implicit method fromPairDStream is not applicable here
because it comes after the application point and it lacks an explicit
result type
[error] joinResult.mapValues{case (v, w) =>
(JavaUtils.optionToOptional(v), w)}
[error] ^
[error] 44 errors found
java.lang.RuntimeException: Repository for publishing is not specified.
at scala.sys.package$.error(package.scala:27)
at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
at scala.Option.getOrElse(Option.scala:108)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
..........
java.lang.RuntimeException: Repository for publishing is not specified.
at scala.sys.package$.error(package.scala:27)
at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:997)
at scala.Option.getOrElse(Option.scala:108)
at sbt.Classpaths$.getPublishTo(Defaults.scala:997)
........
[error] (repl/*:publish-configuration) Repository for publishing is not
specified.
[error] (graphx/*:publish-configuration) Repository for publishing is not
specified.
[error] (mllib/*:publish-configuration) Repository for publishing is not
specified.
[error] (streaming/compile:compile) Compilation failed
[error] (core/*:publish-configuration) Repository for publishing is not
specified.
[error] (bagel/*:publish-configuration) Repository for publishing is not
specified.
[error] Total time: 58 s, completed May 25, 2014 8:20:46 PM
On Mon, May 26, 2014 at 8:46 AM, Aaron Davidson <il...@gmail.com> wrote:
> I suppose you actually ran "publish-local" and not "publish local" like
> your example showed. That being the case, could you show the compile error
> that occurs? It could be related to the hadoop version.
>
>
> On Sun, May 25, 2014 at 7:51 PM, ABHISHEK <ab...@gmail.com> wrote:
>
>> Hi,
>> I'm trying to install Spark along with Shark.
>> Here's configuration details:
>> Spark 0.9.1
>> Shark 0.9.1
>> Scala 2.10.3
>>
>> Spark assembly was successful but running "sbt/sbt publish-local" failed.
>> Please refer attached log for more details and advise.
>> Thanks,
>> Abhishek
>>
>> Sparkhome>SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.4.0
>> Successfully build.
>>
>> Sparkhome>sbt/sbt publish local ( It failed)
>>
>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.pom
>> [info] published spark-repl_2.10 to
>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.jar
>> [info] published spark-repl_2.10 to
>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-sources.jar
>> [info] published spark-repl_2.10 to
>> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-javadoc.jar
>> [info] published spark-repl_2.10 to
>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/poms/spark-repl_2.10.pom
>> [info] published spark-repl_2.10 to
>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/jars/spark-repl_2.10.jar
>> [info] published spark-repl_2.10 to
>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/srcs/spark-repl_2.10-sources.jar
>> [info] published spark-repl_2.10 to
>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/docs/spark-repl_2.10-javadoc.jar
>> [info] published ivy to
>> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/ivys/ivy.xml
>> [error] (streaming/compile:compile) Compilation failed
>> [error] Total time: 419 s, completed May 25, 2014 7:28:27 PM
>>
>>
>>
>>
>>
>
Re: Fails: Spark sbt/sbt publish local
Posted by Aaron Davidson <il...@gmail.com>.
I suppose you actually ran "publish-local" and not "publish local" like
your example showed. That being the case, could you show the compile error
that occurs? It could be related to the hadoop version.
On Sun, May 25, 2014 at 7:51 PM, ABHISHEK <ab...@gmail.com> wrote:
> Hi,
> I'm trying to install Spark along with Shark.
> Here's configuration details:
> Spark 0.9.1
> Shark 0.9.1
> Scala 2.10.3
>
> Spark assembly was successful but running "sbt/sbt publish-local" failed.
> Please refer attached log for more details and advise.
> Thanks,
> Abhishek
>
> Sparkhome>SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.4.0
> Successfully build.
>
> Sparkhome>sbt/sbt publish local ( It failed)
>
> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.pom
> [info] published spark-repl_2.10 to
> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1.jar
> [info] published spark-repl_2.10 to
> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-sources.jar
> [info] published spark-repl_2.10 to
> /home/cloudera/.m2/repository/org/apache/spark/spark-repl_2.10/0.9.1/spark-repl_2.10-0.9.1-javadoc.jar
> [info] published spark-repl_2.10 to
> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/poms/spark-repl_2.10.pom
> [info] published spark-repl_2.10 to
> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/jars/spark-repl_2.10.jar
> [info] published spark-repl_2.10 to
> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/srcs/spark-repl_2.10-sources.jar
> [info] published spark-repl_2.10 to
> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/docs/spark-repl_2.10-javadoc.jar
> [info] published ivy to
> /home/cloudera/.ivy2/local/org.apache.spark/spark-repl_2.10/0.9.1/ivys/ivy.xml
> [error] (streaming/compile:compile) Compilation failed
> [error] Total time: 419 s, completed May 25, 2014 7:28:27 PM
>
>
>
>
>