You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Bryan Jeffrey <br...@gmail.com> on 2015/10/26 14:01:29 UTC
Error Compiling Spark 1.4.1 w/ Scala 2.11 & Hive Support
All,
I'm seeing the following error compiling Spark 1.4.1 w/ Scala 2.11 & Hive
support. Any ideas?
mvn -Dhadoop.version=2.6.1 -Dscala-2.11 -DskipTests -Pyarn -Phive
-Phive-thriftserver package
[INFO] Spark Project Parent POM .......................... SUCCESS [4.124s]
[INFO] Spark Launcher Project ............................ SUCCESS [9.001s]
[INFO] Spark Project Networking .......................... SUCCESS [7.871s]
[INFO] Spark Project Shuffle Streaming Service ........... SUCCESS [3.904s]
[INFO] Spark Project Unsafe .............................. SUCCESS [3.095s]
[INFO] Spark Project Core ................................ SUCCESS [24.768s]
[INFO] Spark Project Bagel ............................... SUCCESS [2.029s]
[INFO] Spark Project GraphX .............................. SUCCESS [4.057s]
[INFO] Spark Project Streaming ........................... SUCCESS [9.774s]
[INFO] Spark Project Catalyst ............................ SUCCESS [6.804s]
[INFO] Spark Project SQL ................................. SUCCESS [9.606s]
[INFO] Spark Project ML Library .......................... SUCCESS [10.872s]
[INFO] Spark Project Tools ............................... SUCCESS [0.627s]
[INFO] Spark Project Hive ................................ SUCCESS [13.463s]
[INFO] Spark Project REPL ................................ SUCCESS [1.414s]
[INFO] Spark Project YARN ................................ SUCCESS [2.433s]
[INFO] Spark Project Hive Thrift Server .................. FAILURE [8.097s]
[ERROR]
/spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:25:
object ConsoleReader is not a member of package jline
[ERROR] import jline.{ConsoleReader, History}
[ERROR] ^
[WARNING] Class jline.Completor not found - continuing with a stub.
[WARNING] Class jline.ConsoleReader not found - continuing with a stub.
[ERROR]
/spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:171:
not found: type ConsoleReader
[ERROR] val reader = new ConsoleReader()
[ERROR] ^
[ERROR] Class jline.Completor not found - continuing with a stub.
Re: Error Compiling Spark 1.4.1 w/ Scala 2.11 & Hive Support
Posted by Sean Owen <so...@cloudera.com>.
Did you switch the build to Scala 2.11 by running the script in dev/? It
won't work otherwise, but does work if you do. @Ted 2.11 was supported in
1.4, not just 1.5.
On Mon, Oct 26, 2015 at 2:13 PM, Bryan Jeffrey <br...@gmail.com>
wrote:
> All,
>
> The error resolved to a bad version of jline pulling from Maven. The
> jline version is defined as 'scala.version' -- the 2.11 version does not
> exist in maven. Instead the following should be used:
>
> <dependency>
> <groupId>org.scala-lang</groupId>
> <artifactId>jline</artifactId>
> <version>2.11.0-M3</version>
> </dependency>
>
> Regards,
>
> Bryan Jeffrey
>
> On Mon, Oct 26, 2015 at 9:01 AM, Bryan Jeffrey <br...@gmail.com>
> wrote:
>
>> All,
>>
>> I'm seeing the following error compiling Spark 1.4.1 w/ Scala 2.11 & Hive
>> support. Any ideas?
>>
>> mvn -Dhadoop.version=2.6.1 -Dscala-2.11 -DskipTests -Pyarn -Phive
>> -Phive-thriftserver package
>>
>> [INFO] Spark Project Parent POM .......................... SUCCESS
>> [4.124s]
>> [INFO] Spark Launcher Project ............................ SUCCESS
>> [9.001s]
>> [INFO] Spark Project Networking .......................... SUCCESS
>> [7.871s]
>> [INFO] Spark Project Shuffle Streaming Service ........... SUCCESS
>> [3.904s]
>> [INFO] Spark Project Unsafe .............................. SUCCESS
>> [3.095s]
>> [INFO] Spark Project Core ................................ SUCCESS
>> [24.768s]
>> [INFO] Spark Project Bagel ............................... SUCCESS
>> [2.029s]
>> [INFO] Spark Project GraphX .............................. SUCCESS
>> [4.057s]
>> [INFO] Spark Project Streaming ........................... SUCCESS
>> [9.774s]
>> [INFO] Spark Project Catalyst ............................ SUCCESS
>> [6.804s]
>> [INFO] Spark Project SQL ................................. SUCCESS
>> [9.606s]
>> [INFO] Spark Project ML Library .......................... SUCCESS
>> [10.872s]
>> [INFO] Spark Project Tools ............................... SUCCESS
>> [0.627s]
>> [INFO] Spark Project Hive ................................ SUCCESS
>> [13.463s]
>> [INFO] Spark Project REPL ................................ SUCCESS
>> [1.414s]
>> [INFO] Spark Project YARN ................................ SUCCESS
>> [2.433s]
>> [INFO] Spark Project Hive Thrift Server .................. FAILURE
>> [8.097s]
>>
>>
>> [ERROR]
>> /spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:25:
>> object ConsoleReader is not a member of package jline
>> [ERROR] import jline.{ConsoleReader, History}
>> [ERROR] ^
>> [WARNING] Class jline.Completor not found - continuing with a stub.
>> [WARNING] Class jline.ConsoleReader not found - continuing with a stub.
>> [ERROR]
>> /spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:171:
>> not found: type ConsoleReader
>> [ERROR] val reader = new ConsoleReader()
>> [ERROR] ^
>> [ERROR] Class jline.Completor not found - continuing with a stub.
>>
>
>
Re: Error Compiling Spark 1.4.1 w/ Scala 2.11 & Hive Support
Posted by Bryan Jeffrey <br...@gmail.com>.
All,
The error resolved to a bad version of jline pulling from Maven. The jline
version is defined as 'scala.version' -- the 2.11 version does not exist in
maven. Instead the following should be used:
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>jline</artifactId>
<version>2.11.0-M3</version>
</dependency>
Regards,
Bryan Jeffrey
On Mon, Oct 26, 2015 at 9:01 AM, Bryan Jeffrey <br...@gmail.com>
wrote:
> All,
>
> I'm seeing the following error compiling Spark 1.4.1 w/ Scala 2.11 & Hive
> support. Any ideas?
>
> mvn -Dhadoop.version=2.6.1 -Dscala-2.11 -DskipTests -Pyarn -Phive
> -Phive-thriftserver package
>
> [INFO] Spark Project Parent POM .......................... SUCCESS [4.124s]
> [INFO] Spark Launcher Project ............................ SUCCESS [9.001s]
> [INFO] Spark Project Networking .......................... SUCCESS [7.871s]
> [INFO] Spark Project Shuffle Streaming Service ........... SUCCESS [3.904s]
> [INFO] Spark Project Unsafe .............................. SUCCESS [3.095s]
> [INFO] Spark Project Core ................................ SUCCESS
> [24.768s]
> [INFO] Spark Project Bagel ............................... SUCCESS [2.029s]
> [INFO] Spark Project GraphX .............................. SUCCESS [4.057s]
> [INFO] Spark Project Streaming ........................... SUCCESS [9.774s]
> [INFO] Spark Project Catalyst ............................ SUCCESS [6.804s]
> [INFO] Spark Project SQL ................................. SUCCESS [9.606s]
> [INFO] Spark Project ML Library .......................... SUCCESS
> [10.872s]
> [INFO] Spark Project Tools ............................... SUCCESS [0.627s]
> [INFO] Spark Project Hive ................................ SUCCESS
> [13.463s]
> [INFO] Spark Project REPL ................................ SUCCESS [1.414s]
> [INFO] Spark Project YARN ................................ SUCCESS [2.433s]
> [INFO] Spark Project Hive Thrift Server .................. FAILURE [8.097s]
>
>
> [ERROR]
> /spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:25:
> object ConsoleReader is not a member of package jline
> [ERROR] import jline.{ConsoleReader, History}
> [ERROR] ^
> [WARNING] Class jline.Completor not found - continuing with a stub.
> [WARNING] Class jline.ConsoleReader not found - continuing with a stub.
> [ERROR]
> /spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:171:
> not found: type ConsoleReader
> [ERROR] val reader = new ConsoleReader()
> [ERROR] ^
> [ERROR] Class jline.Completor not found - continuing with a stub.
>
Re: Error Compiling Spark 1.4.1 w/ Scala 2.11 & Hive Support
Posted by Ted Yu <yu...@gmail.com>.
Scala 2.11 is supported in 1.5.1 release:
http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22spark-parent_2.11%22
Can you upgrade ?
Cheers
On Mon, Oct 26, 2015 at 6:01 AM, Bryan Jeffrey <br...@gmail.com>
wrote:
> All,
>
> I'm seeing the following error compiling Spark 1.4.1 w/ Scala 2.11 & Hive
> support. Any ideas?
>
> mvn -Dhadoop.version=2.6.1 -Dscala-2.11 -DskipTests -Pyarn -Phive
> -Phive-thriftserver package
>
> [INFO] Spark Project Parent POM .......................... SUCCESS [4.124s]
> [INFO] Spark Launcher Project ............................ SUCCESS [9.001s]
> [INFO] Spark Project Networking .......................... SUCCESS [7.871s]
> [INFO] Spark Project Shuffle Streaming Service ........... SUCCESS [3.904s]
> [INFO] Spark Project Unsafe .............................. SUCCESS [3.095s]
> [INFO] Spark Project Core ................................ SUCCESS
> [24.768s]
> [INFO] Spark Project Bagel ............................... SUCCESS [2.029s]
> [INFO] Spark Project GraphX .............................. SUCCESS [4.057s]
> [INFO] Spark Project Streaming ........................... SUCCESS [9.774s]
> [INFO] Spark Project Catalyst ............................ SUCCESS [6.804s]
> [INFO] Spark Project SQL ................................. SUCCESS [9.606s]
> [INFO] Spark Project ML Library .......................... SUCCESS
> [10.872s]
> [INFO] Spark Project Tools ............................... SUCCESS [0.627s]
> [INFO] Spark Project Hive ................................ SUCCESS
> [13.463s]
> [INFO] Spark Project REPL ................................ SUCCESS [1.414s]
> [INFO] Spark Project YARN ................................ SUCCESS [2.433s]
> [INFO] Spark Project Hive Thrift Server .................. FAILURE [8.097s]
>
>
> [ERROR]
> /spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:25:
> object ConsoleReader is not a member of package jline
> [ERROR] import jline.{ConsoleReader, History}
> [ERROR] ^
> [WARNING] Class jline.Completor not found - continuing with a stub.
> [WARNING] Class jline.ConsoleReader not found - continuing with a stub.
> [ERROR]
> /spark/spark-1.4.1.hive.bak/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:171:
> not found: type ConsoleReader
> [ERROR] val reader = new ConsoleReader()
> [ERROR] ^
> [ERROR] Class jline.Completor not found - continuing with a stub.
>