You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Yang Jie (Jira)" <ji...@apache.org> on 2021/02/01 08:19:00 UTC
[jira] [Commented] (SPARK-34098) HadoopVersionInfoSuite faild when
maven test in Scala 2.13
[ https://issues.apache.org/jira/browse/SPARK-34098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276133#comment-17276133 ]
Yang Jie commented on SPARK-34098:
----------------------------------
cc [~hyukjin.kwon] Can you help close this jira? seems not a problem... thx ~
> HadoopVersionInfoSuite faild when maven test in Scala 2.13
> ----------------------------------------------------------
>
> Key: SPARK-34098
> URL: https://issues.apache.org/jira/browse/SPARK-34098
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: Yang Jie
> Priority: Minor
>
>
> {code:java}
> mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl sql/hive -Pscala-2.13 -DwildcardSuites=org.apache.spark.sql.hive.client.HadoopVersionInfoSuite -Dtest=none
> {code}
> Independent test HadoopVersionInfoSuite all case passed, but execute
>
> {code:java}
> mvn clean test -Phadoop-3.2 -Phive-2.3 -Phadoop-cloud -Pmesos -Pyarn -Pkinesis-asl -Phive-thriftserver -Pspark-ganglia-lgpl -Pkubernetes -Phive -pl sql/hive -Pscala-2.13
> {code}
>
> HadoopVersionInfoSuite failed as follow:
> {code:java}
> HadoopVersionInfoSuite: 22:32:30.310 WARN org.apache.spark.sql.hive.client.IsolatedClientLoader: Failed to resolve Hadoop artifacts for the version 2.7.4. We will change the hadoop version from 2.7.4 to 2.7.4 and try again. It is recommended to set jars used by Hive metastore client through spark.sql.hive.metastore.jars in the production environment. - SPARK-32256: Hadoop VersionInfo should be preloaded *** FAILED *** java.lang.RuntimeException: [unresolved dependency: org.apache.hive#hive-metastore;2.0.1: not found, unresolved dependency: org.apache.hive#hive-exec;2.0.1: not found, unresolved dependency: org.apache.hive#hive-common;2.0.1: not found, unresolved dependency: org.apache.hive#hive-serde;2.0.1: not found, unresolved dependency: com.google.guava#guava;14.0.1: not found, unresolved dependency: org.apache.hadoop#hadoop-client;2.7.4: not found] at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1423) at org.apache.spark.sql.hive.client.IsolatedClientLoader$.$anonfun$downloadVersion$2(IsolatedClientLoader.scala:122) at org.apache.spark.sql.catalyst.util.package$.quietly(package.scala:42) at org.apache.spark.sql.hive.client.IsolatedClientLoader$.downloadVersion(IsolatedClientLoader.scala:122) at org.apache.spark.sql.hive.client.IsolatedClientLoader$.liftedTree1$1(IsolatedClientLoader.scala:75) at org.apache.spark.sql.hive.client.IsolatedClientLoader$.forVersion(IsolatedClientLoader.scala:63) at org.apache.spark.sql.hive.client.HadoopVersionInfoSuite.$anonfun$new$1(HadoopVersionInfoSuite.scala:46) at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> {code}
> need some investigate.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org