You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Stephen Boesch <ja...@gmail.com> on 2015/08/16 20:12:23 UTC
Spark on scala 2.11 build fails due to incorrect jline dependency in REPL
I am building spark with the following options - most notably the
**scala-2.11**:
. dev/switch-to-scala-2.11.sh
mvn -Phive -Pyarn -Phadoop-2.6 -Dhadoop2.6.2 -Pscala-2.11 -DskipTests
-Dmaven.javadoc.skip=true clean package
The build goes pretty far but fails in one of the minor modules *repl*:
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal on project spark-repl_2.11: Could not
resolve dependencies
for project org.apache.spark:spark-repl_2.11:jar:1.5.0-SNAPSHOT:
Could not find artifact org.scala-lang:jline:jar:2.11.7 in central
(https://repo1.maven.org/maven2) -> [Help 1]
Upon investigation - from 2.11.5 and later the scala version of jline is no
longer required: they use the default jline distribution.
And in fact the repl only shows dependency on jline for the 2.10.4 scala
version:
<profile>
<id>scala-2.10</id>
<activation>
<property><name>!scala-2.11</name></property>
</activation>
<properties>
<scala.version>2.10.4</scala.version>
<scala.binary.version>2.10</scala.binary.version>
<jline.version>${scala.version}</jline.version>
<jline.groupid>org.scala-lang</jline.groupid>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>${jline.groupid}</groupId>
<artifactId>jline</artifactId>
<version>${jline.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
</profile>
So then it is not clear why this error is occurring. Pointers appreciated.
Re: Spark on scala 2.11 build fails due to incorrect jline dependency
in REPL
Posted by Stephen Boesch <ja...@gmail.com>.
In 1.4 it is change-scala-version.sh 2.11
But the problem was it is a -Dscala-211 not a -P. I misread the doc's.
2015-08-17 14:17 GMT-07:00 Ted Yu <yu...@gmail.com>:
> You were building against 1.4.x, right ?
>
> In master branch, switch-to-scala-2.11.sh is gone. There is scala-2.11
> profile.
>
> FYI
>
> On Sun, Aug 16, 2015 at 11:12 AM, Stephen Boesch <ja...@gmail.com>
> wrote:
>
>>
>> I am building spark with the following options - most notably the
>> **scala-2.11**:
>>
>> . dev/switch-to-scala-2.11.sh
>> mvn -Phive -Pyarn -Phadoop-2.6 -Dhadoop2.6.2 -Pscala-2.11 -DskipTests
>> -Dmaven.javadoc.skip=true clean package
>>
>>
>> The build goes pretty far but fails in one of the minor modules *repl*:
>>
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal on project spark-repl_2.11: Could not
>> resolve dependencies
>> for project org.apache.spark:spark-repl_2.11:jar:1.5.0-SNAPSHOT:
>> Could not find artifact org.scala-lang:jline:jar:2.11.7 in central
>> (https://repo1.maven.org/maven2) -> [Help 1]
>>
>> Upon investigation - from 2.11.5 and later the scala version of jline is
>> no longer required: they use the default jline distribution.
>>
>> And in fact the repl only shows dependency on jline for the 2.10.4 scala
>> version:
>>
>> <profile>
>> <id>scala-2.10</id>
>> <activation>
>> <property><name>!scala-2.11</name></property>
>> </activation>
>> <properties>
>> <scala.version>2.10.4</scala.version>
>> <scala.binary.version>2.10</scala.binary.version>
>> <jline.version>${scala.version}</jline.version>
>> <jline.groupid>org.scala-lang</jline.groupid>
>> </properties>
>> <dependencyManagement>
>> <dependencies>
>> <dependency>
>> <groupId>${jline.groupid}</groupId>
>> <artifactId>jline</artifactId>
>> <version>${jline.version}</version>
>> </dependency>
>> </dependencies>
>> </dependencyManagement>
>> </profile>
>>
>> So then it is not clear why this error is occurring. Pointers appreciated.
>>
>>
>>
>
Re: Spark on scala 2.11 build fails due to incorrect jline dependency
in REPL
Posted by Ted Yu <yu...@gmail.com>.
You were building against 1.4.x, right ?
In master branch, switch-to-scala-2.11.sh is gone. There is scala-2.11
profile.
FYI
On Sun, Aug 16, 2015 at 11:12 AM, Stephen Boesch <ja...@gmail.com> wrote:
>
> I am building spark with the following options - most notably the
> **scala-2.11**:
>
> . dev/switch-to-scala-2.11.sh
> mvn -Phive -Pyarn -Phadoop-2.6 -Dhadoop2.6.2 -Pscala-2.11 -DskipTests
> -Dmaven.javadoc.skip=true clean package
>
>
> The build goes pretty far but fails in one of the minor modules *repl*:
>
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal on project spark-repl_2.11: Could not
> resolve dependencies
> for project org.apache.spark:spark-repl_2.11:jar:1.5.0-SNAPSHOT:
> Could not find artifact org.scala-lang:jline:jar:2.11.7 in central
> (https://repo1.maven.org/maven2) -> [Help 1]
>
> Upon investigation - from 2.11.5 and later the scala version of jline is
> no longer required: they use the default jline distribution.
>
> And in fact the repl only shows dependency on jline for the 2.10.4 scala
> version:
>
> <profile>
> <id>scala-2.10</id>
> <activation>
> <property><name>!scala-2.11</name></property>
> </activation>
> <properties>
> <scala.version>2.10.4</scala.version>
> <scala.binary.version>2.10</scala.binary.version>
> <jline.version>${scala.version}</jline.version>
> <jline.groupid>org.scala-lang</jline.groupid>
> </properties>
> <dependencyManagement>
> <dependencies>
> <dependency>
> <groupId>${jline.groupid}</groupId>
> <artifactId>jline</artifactId>
> <version>${jline.version}</version>
> </dependency>
> </dependencies>
> </dependencyManagement>
> </profile>
>
> So then it is not clear why this error is occurring. Pointers appreciated.
>
>
>