You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Kengo Seki (Jira)" <ji...@apache.org> on 2020/04/03 02:04:00 UTC
[jira] [Created] (BIGTOP-3333) Fix Hive build after upgrading Spark
to 2.4.5
Kengo Seki created BIGTOP-3333:
----------------------------------
Summary: Fix Hive build after upgrading Spark to 2.4.5
Key: BIGTOP-3333
URL: https://issues.apache.org/jira/browse/BIGTOP-3333
Project: Bigtop
Issue Type: Bug
Components: hive
Reporter: Kengo Seki
Assignee: Kengo Seki
After BIGTOP-3165, building Hive came to fail as follows, due to the API change of Spark:
{code}
$ ./gradlew hive-pkg
(snip)
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/sekikn/repos/bigtop/output/hive/hive-2.3.6/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java:[87,26] cannot find symbol
symbol: method addListener(org.apache.hadoop.hive.ql.exec.spark.status.impl.JobMetricsListener)
location: class org.apache.spark.scheduler.LiveListenerBus
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Hive 2.3.6:
[INFO]
[INFO] Hive ............................................... SUCCESS [ 2.115 s]
[INFO] Hive Shims Common .................................. SUCCESS [ 3.889 s]
[INFO] Hive Shims 0.23 .................................... SUCCESS [ 2.686 s]
[INFO] Hive Shims Scheduler ............................... SUCCESS [ 0.976 s]
[INFO] Hive Shims ......................................... SUCCESS [ 0.793 s]
[INFO] Hive Common ........................................ SUCCESS [ 5.762 s]
[INFO] Hive Service RPC ................................... SUCCESS [ 2.480 s]
[INFO] Hive Serde ......................................... SUCCESS [ 4.136 s]
[INFO] Hive Metastore ..................................... SUCCESS [ 16.040 s]
[INFO] Hive Vector-Code-Gen Utilities ..................... SUCCESS [ 0.306 s]
[INFO] Hive Llap Common ................................... SUCCESS [ 3.953 s]
[INFO] Hive Llap Client ................................... SUCCESS [ 2.334 s]
[INFO] Hive Llap Tez ...................................... SUCCESS [ 2.591 s]
[INFO] Spark Remote Client ................................ SUCCESS [ 3.028 s]
[INFO] Hive Query Language ................................ FAILURE [ 17.783 s]
[INFO] Hive Llap Server ................................... SKIPPED
[INFO] Hive Service ....................................... SKIPPED
[INFO] Hive Accumulo Handler .............................. SKIPPED
[INFO] Hive JDBC .......................................... SKIPPED
[INFO] Hive Beeline ....................................... SKIPPED
[INFO] Hive CLI ........................................... SKIPPED
[INFO] Hive Contrib ....................................... SKIPPED
[INFO] Hive Druid Handler ................................. SKIPPED
[INFO] Hive HBase Handler ................................. SKIPPED
[INFO] Hive JDBC Handler .................................. SKIPPED
[INFO] Hive HCatalog ...................................... SKIPPED
[INFO] Hive HCatalog Core ................................. SKIPPED
[INFO] Hive HCatalog Pig Adapter .......................... SKIPPED
[INFO] Hive HCatalog Server Extensions .................... SKIPPED
[INFO] Hive HCatalog Webhcat Java Client .................. SKIPPED
[INFO] Hive HCatalog Webhcat .............................. SKIPPED
[INFO] Hive HCatalog Streaming ............................ SKIPPED
[INFO] Hive HPL/SQL ....................................... SKIPPED
[INFO] Hive Llap External Client .......................... SKIPPED
[INFO] Hive Shims Aggregator .............................. SKIPPED
[INFO] Hive TestUtils ..................................... SKIPPED
[INFO] Hive Packaging ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:09 min
[INFO] Finished at: 2020-04-03T10:55:51+09:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hive-exec: Compilation failure
[ERROR] /home/sekikn/repos/bigtop/output/hive/hive-2.3.6/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java:[87,26] cannot find symbol
[ERROR] symbol: method addListener(org.apache.hadoop.hive.ql.exec.spark.status.impl.JobMetricsListener)
[ERROR] location: class org.apache.spark.scheduler.LiveListenerBus
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :hive-exec
debian/rules:32: recipe for target 'override_dh_auto_build' failed
make[1]: *** [override_dh_auto_build] Error 1
make[1]: Leaving directory '/home/sekikn/repos/bigtop/output/hive/hive-2.3.6'
debian/rules:29: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
debuild: fatal error at line 1152:
dpkg-buildpackage -rfakeroot -us -uc -ui -b
failed
> Task :hive-deb FAILED
FAILURE: Build failed with an exception.
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)