You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jakob Odersky <jo...@gmail.com> on 2015/10/14 02:53:33 UTC
Building with SBT and Scala 2.11
I'm having trouble compiling Spark with SBT for Scala 2.11. The command I
use is:
dev/change-version-to-2.11.sh
build/sbt -Pyarn -Phadoop-2.11 -Dscala-2.11
followed by
compile
in the sbt shell.
The error I get specifically is:
spark/core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala:308:
no valid targets for annotation on value conf - it is discarded unused. You
may specify targets with meta-annotations, e.g. @(transient @param)
[error] private[netty] class NettyRpcEndpointRef(@transient conf: SparkConf)
[error]
However I am also getting a large amount of deprecation warnings, making me
wonder if I am supplying some incompatible/unsupported options to sbt. I am
using Java 1.8 and the latest Spark master sources.
Does someone know if I am doing anything wrong or is the sbt build broken?
thanks for you help,
--Jakob
Re: Building with SBT and Scala 2.11
Posted by Jakob Odersky <jo...@gmail.com>.
[Repost to mailing list]
Hey,
Sorry about the typo, I of course meant hadoop-2.6, not 2.11.
I suspect something bad happened with my Ivy cache, since when reverting
back to scala 2.10, I got a very strange IllegalStateException (something
something IvyNode, I can't remember the details).
Kilking the cache made 2.10 work at least, I'll retry with 2.11
Thx for your help
On Oct 14, 2015 6:52 AM, "Ted Yu" <yu...@gmail.com> wrote:
> Adrian:
> Likely you were using maven.
>
> Jakob's report was with sbt.
>
> Cheers
>
> On Tue, Oct 13, 2015 at 10:05 PM, Adrian Tanase <at...@adobe.com> wrote:
>
>> Do you mean hadoop-2.4 or 2.6? not sure if this is the issue but I'm also
>> compiling the 1.5.1 version with scala 2.11 and hadoop 2.6 and it works.
>>
>> -adrian
>>
>> Sent from my iPhone
>>
>> On 14 Oct 2015, at 03:53, Jakob Odersky <jo...@gmail.com> wrote:
>>
>> I'm having trouble compiling Spark with SBT for Scala 2.11. The command I
>> use is:
>>
>> dev/change-version-to-2.11.sh
>> build/sbt -Pyarn -Phadoop-2.11 -Dscala-2.11
>>
>> followed by
>>
>> compile
>>
>> in the sbt shell.
>>
>> The error I get specifically is:
>>
>> spark/core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala:308:
>> no valid targets for annotation on value conf - it is discarded unused. You
>> may specify targets with meta-annotations, e.g. @(transient @param)
>> [error] private[netty] class NettyRpcEndpointRef(@transient conf:
>> SparkConf)
>> [error]
>>
>> However I am also getting a large amount of deprecation warnings, making
>> me wonder if I am supplying some incompatible/unsupported options to sbt. I
>> am using Java 1.8 and the latest Spark master sources.
>> Does someone know if I am doing anything wrong or is the sbt build broken?
>>
>> thanks for you help,
>> --Jakob
>>
>>
>
Re: Building with SBT and Scala 2.11
Posted by Adrian Tanase <at...@adobe.com>.
You are correct, of course. Gave up on sbt for spark long ago, I never managed to get it working while mvn works great.
Sent from my iPhone
On 14 Oct 2015, at 16:52, Ted Yu <yu...@gmail.com>> wrote:
Adrian:
Likely you were using maven.
Jakob's report was with sbt.
Cheers
On Tue, Oct 13, 2015 at 10:05 PM, Adrian Tanase <at...@adobe.com>> wrote:
Do you mean hadoop-2.4 or 2.6? not sure if this is the issue but I'm also compiling the 1.5.1 version with scala 2.11 and hadoop 2.6 and it works.
-adrian
Sent from my iPhone
On 14 Oct 2015, at 03:53, Jakob Odersky <jo...@gmail.com>> wrote:
I'm having trouble compiling Spark with SBT for Scala 2.11. The command I use is:
dev/change-version-to-2.11.sh<http://change-version-to-2.11.sh>
build/sbt -Pyarn -Phadoop-2.11 -Dscala-2.11
followed by
compile
in the sbt shell.
The error I get specifically is:
spark/core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala:308: no valid targets for annotation on value conf - it is discarded unused. You may specify targets with meta-annotations, e.g. @(transient @param)
[error] private[netty] class NettyRpcEndpointRef(@transient conf: SparkConf)
[error]
However I am also getting a large amount of deprecation warnings, making me wonder if I am supplying some incompatible/unsupported options to sbt. I am using Java 1.8 and the latest Spark master sources.
Does someone know if I am doing anything wrong or is the sbt build broken?
thanks for you help,
--Jakob
Re: Building with SBT and Scala 2.11
Posted by Ted Yu <yu...@gmail.com>.
Adrian:
Likely you were using maven.
Jakob's report was with sbt.
Cheers
On Tue, Oct 13, 2015 at 10:05 PM, Adrian Tanase <at...@adobe.com> wrote:
> Do you mean hadoop-2.4 or 2.6? not sure if this is the issue but I'm also
> compiling the 1.5.1 version with scala 2.11 and hadoop 2.6 and it works.
>
> -adrian
>
> Sent from my iPhone
>
> On 14 Oct 2015, at 03:53, Jakob Odersky <jo...@gmail.com> wrote:
>
> I'm having trouble compiling Spark with SBT for Scala 2.11. The command I
> use is:
>
> dev/change-version-to-2.11.sh
> build/sbt -Pyarn -Phadoop-2.11 -Dscala-2.11
>
> followed by
>
> compile
>
> in the sbt shell.
>
> The error I get specifically is:
>
> spark/core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala:308:
> no valid targets for annotation on value conf - it is discarded unused. You
> may specify targets with meta-annotations, e.g. @(transient @param)
> [error] private[netty] class NettyRpcEndpointRef(@transient conf:
> SparkConf)
> [error]
>
> However I am also getting a large amount of deprecation warnings, making
> me wonder if I am supplying some incompatible/unsupported options to sbt. I
> am using Java 1.8 and the latest Spark master sources.
> Does someone know if I am doing anything wrong or is the sbt build broken?
>
> thanks for you help,
> --Jakob
>
>
Re: Building with SBT and Scala 2.11
Posted by Adrian Tanase <at...@adobe.com>.
Do you mean hadoop-2.4 or 2.6? not sure if this is the issue but I'm also compiling the 1.5.1 version with scala 2.11 and hadoop 2.6 and it works.
-adrian
Sent from my iPhone
On 14 Oct 2015, at 03:53, Jakob Odersky <jo...@gmail.com>> wrote:
I'm having trouble compiling Spark with SBT for Scala 2.11. The command I use is:
dev/change-version-to-2.11.sh<http://change-version-to-2.11.sh>
build/sbt -Pyarn -Phadoop-2.11 -Dscala-2.11
followed by
compile
in the sbt shell.
The error I get specifically is:
spark/core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala:308: no valid targets for annotation on value conf - it is discarded unused. You may specify targets with meta-annotations, e.g. @(transient @param)
[error] private[netty] class NettyRpcEndpointRef(@transient conf: SparkConf)
[error]
However I am also getting a large amount of deprecation warnings, making me wonder if I am supplying some incompatible/unsupported options to sbt. I am using Java 1.8 and the latest Spark master sources.
Does someone know if I am doing anything wrong or is the sbt build broken?
thanks for you help,
--Jakob
Re: Building with SBT and Scala 2.11
Posted by Ted Yu <yu...@gmail.com>.
See this thread: http://search-hadoop.com/m/q3RTtY7aX22B44dB
On Tue, Oct 13, 2015 at 5:53 PM, Jakob Odersky <jo...@gmail.com> wrote:
> I'm having trouble compiling Spark with SBT for Scala 2.11. The command I
> use is:
>
> dev/change-version-to-2.11.sh
> build/sbt -Pyarn -Phadoop-2.11 -Dscala-2.11
>
> followed by
>
> compile
>
> in the sbt shell.
>
> The error I get specifically is:
>
> spark/core/src/main/scala/org/apache/spark/rpc/netty/NettyRpcEnv.scala:308:
> no valid targets for annotation on value conf - it is discarded unused. You
> may specify targets with meta-annotations, e.g. @(transient @param)
> [error] private[netty] class NettyRpcEndpointRef(@transient conf:
> SparkConf)
> [error]
>
> However I am also getting a large amount of deprecation warnings, making
> me wonder if I am supplying some incompatible/unsupported options to sbt. I
> am using Java 1.8 and the latest Spark master sources.
> Does someone know if I am doing anything wrong or is the sbt build broken?
>
> thanks for you help,
> --Jakob
>
>