You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Night Wolf <ni...@gmail.com> on 2015/03/30 10:02:26 UTC
Spark 1.3 build with hive support fails on JLine
Hey,
Trying to build Spark 1.3 with Scala 2.11 supporting yarn & hive (with
thrift server).
Running;
*mvn -e -DskipTests -Pscala-2.11 -Dscala-2.11 -Pyarn -Pmapr4 -Phive
-Phive-thriftserver clean install*
The build fails with;
INFO] Compiling 9 Scala sources to
/var/lib/jenkins/workspace/cse-Apache-Spark-YARN-2.11-1.3-build/sql/hive-thriftserver/target/scala-2.11/classes...[ERROR]
/var/lib/jenkins/workspace/cse-Apache-Spark-YARN-2.11-1.3-build/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:25:
object ConsoleReader is not a member of package jline[ERROR] import
jline.{ConsoleReader, History}[ERROR] ^[WARNING] Class
jline.Completor not found - continuing with a stub.[WARNING] Class
jline.ConsoleReader not found - continuing with a stub.[ERROR]
/var/lib/jenkins/workspace/cse-Apache-Spark-YARN-2.11-1.3-build/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:165:
not found: type ConsoleReader[ERROR] val reader = new
ConsoleReader()[ERROR] ^[ERROR] Class
jline.Completor not found - continuing with a stub.[WARNING] Class
com.google.protobuf.Parser not found - continuing with a
stub.[WARNING] Class com.google.protobuf.Parser not found - continuing
with a stub.[WARNING] Class com.google.protobuf.Parser not found -
continuing with a stub.[WARNING] Class com.google.protobuf.Parser not
found - continuing with a stub.[WARNING] 6 warnings found[ERROR] three
errors found[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 15.731 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 46.667 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 28.508 s]
[INFO] Spark Project Core ................................. SUCCESS [07:45 min]
[INFO] Spark Project Bagel ................................ SUCCESS [01:10 min]
[INFO] Spark Project GraphX ............................... SUCCESS [02:42 min]
[INFO] Spark Project Streaming ............................ SUCCESS [03:22 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [04:42 min]
[INFO] Spark Project SQL .................................. SUCCESS [05:17 min]
[INFO] Spark Project ML Library ........................... SUCCESS [05:36 min]
[INFO] Spark Project Tools ................................ SUCCESS [ 46.976 s]
[INFO] Spark Project Hive ................................. SUCCESS [04:08 min]
[INFO] Spark Project REPL ................................. SUCCESS [01:58 min]
[INFO] Spark Project YARN ................................. SUCCESS [01:47 min]
[INFO] Spark Project Hive Thrift Server ................... FAILURE [ 20.731 s]
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Spark Project External Twitter ..................... SKIPPED
[INFO] Spark Project External Flume Sink .................. SKIPPED
[INFO] Spark Project External Flume ....................... SKIPPED
[INFO] Spark Project External MQTT ........................ SKIPPED
[INFO] Spark Project External ZeroMQ ...................... SKIPPED
[INFO] Spark Project Examples ............................. SKIPPED
[INFO] Spark Project YARN Shuffle Service ................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 41:59 min
[INFO] Finished at: 2015-03-30T07:06:49+00:00
[INFO] Final Memory: 94M/868M
[INFO] ------------------------------------------------------------------------[ERROR]
Failed to execute goal
net.alchim31.maven:scala-maven-plugin:3.2.0:compile
(scala-compile-first) on project spark-hive-thriftserver_2.11:
Execution scala-compile-first of goal
net.alchim31.maven:scala-maven-plugin:3.2.0:compile failed.
CompileFailed -> [Help
1]org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile
(scala-compile-first) on project spark-hive-thriftserver_2.11:
Execution scala-compile-first of goal
net.alchim31.maven:scala-maven-plugin:3.2.0:compile failed.
Any ideas?
Cheers,
N
Re: Spark 1.3 build with hive support fails on JLine
Posted by Ted Yu <yu...@gmail.com>.
Please invoke dev/change-version-to-2.11.sh before running mvn.
Cheers
On Mon, Mar 30, 2015 at 1:02 AM, Night Wolf <ni...@gmail.com> wrote:
> Hey,
>
> Trying to build Spark 1.3 with Scala 2.11 supporting yarn & hive (with
> thrift server).
>
> Running;
>
> *mvn -e -DskipTests -Pscala-2.11 -Dscala-2.11 -Pyarn -Pmapr4 -Phive
> -Phive-thriftserver clean install*
>
> The build fails with;
>
> INFO] Compiling 9 Scala sources to /var/lib/jenkins/workspace/cse-Apache-Spark-YARN-2.11-1.3-build/sql/hive-thriftserver/target/scala-2.11/classes...[ERROR] /var/lib/jenkins/workspace/cse-Apache-Spark-YARN-2.11-1.3-build/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:25: object ConsoleReader is not a member of package jline[ERROR] import jline.{ConsoleReader, History}[ERROR] ^[WARNING] Class jline.Completor not found - continuing with a stub.[WARNING] Class jline.ConsoleReader not found - continuing with a stub.[ERROR] /var/lib/jenkins/workspace/cse-Apache-Spark-YARN-2.11-1.3-build/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala:165: not found: type ConsoleReader[ERROR] val reader = new ConsoleReader()[ERROR] ^[ERROR] Class jline.Completor not found - continuing with a stub.[WARNING] Class com.google.protobuf.Parser not found - continuing with a stub.[WARNING] Class com.google.protobuf.Parser not found - continuing with a stub.[WARNING] Class com.google.protobuf.Parser not found - continuing with a stub.[WARNING] Class com.google.protobuf.Parser not found - continuing with a stub.[WARNING] 6 warnings found[ERROR] three errors found[INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project Parent POM ........................... SUCCESS [ 15.731 s]
> [INFO] Spark Project Networking ........................... SUCCESS [ 46.667 s]
> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 28.508 s]
> [INFO] Spark Project Core ................................. SUCCESS [07:45 min]
> [INFO] Spark Project Bagel ................................ SUCCESS [01:10 min]
> [INFO] Spark Project GraphX ............................... SUCCESS [02:42 min]
> [INFO] Spark Project Streaming ............................ SUCCESS [03:22 min]
> [INFO] Spark Project Catalyst ............................. SUCCESS [04:42 min]
> [INFO] Spark Project SQL .................................. SUCCESS [05:17 min]
> [INFO] Spark Project ML Library ........................... SUCCESS [05:36 min]
> [INFO] Spark Project Tools ................................ SUCCESS [ 46.976 s]
> [INFO] Spark Project Hive ................................. SUCCESS [04:08 min]
> [INFO] Spark Project REPL ................................. SUCCESS [01:58 min]
> [INFO] Spark Project YARN ................................. SUCCESS [01:47 min]
> [INFO] Spark Project Hive Thrift Server ................... FAILURE [ 20.731 s]
> [INFO] Spark Project Assembly ............................. SKIPPED
> [INFO] Spark Project External Twitter ..................... SKIPPED
> [INFO] Spark Project External Flume Sink .................. SKIPPED
> [INFO] Spark Project External Flume ....................... SKIPPED
> [INFO] Spark Project External MQTT ........................ SKIPPED
> [INFO] Spark Project External ZeroMQ ...................... SKIPPED
> [INFO] Spark Project Examples ............................. SKIPPED
> [INFO] Spark Project YARN Shuffle Service ................. SKIPPED
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 41:59 min
> [INFO] Finished at: 2015-03-30T07:06:49+00:00
> [INFO] Final Memory: 94M/868M
> [INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile (scala-compile-first) on project spark-hive-thriftserver_2.11: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile failed. CompileFailed -> [Help 1]org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile (scala-compile-first) on project spark-hive-thriftserver_2.11: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile failed.
>
>
> Any ideas?
>
> Cheers,
> N
>