You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/17 10:02:09 UTC
[GitHub] [spark] wangyum commented on a change in pull request #27492:
[SPARK-30755][SQL] Update migration guide for HIVE-15167
wangyum commented on a change in pull request #27492: [SPARK-30755][SQL] Update migration guide for HIVE-15167
URL: https://github.com/apache/spark/pull/27492#discussion_r380086236
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -318,7 +318,14 @@ private[hive] class HiveClientImpl(
// with the HiveConf in `state` to override the context class loader of the current
// thread.
shim.setCurrentSessionState(state)
- val ret = try f finally {
Review comment:
Yes. It cover all the cases:
```java
20/02/17 02:55:59 ERROR SparkSQLDriver: Failed in [insert into t2 values('1', '2')]
java.lang.ClassNotFoundException: The SerDe interface removed since Hive 2.3(HIVE-15167). Please migrate your custom SerDes to Hive 2.3 or build your own Spark with hive-1.2 profile. See HIVE-15167 for more details.
at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$withHiveState$1(HiveClientImpl.scala:328)
at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:255)
at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:254)
at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:304)
at org.apache.spark.sql.hive.client.HiveClientImpl.getTableOption(HiveClientImpl.scala:450)
at org.apache.spark.sql.hive.client.HiveClient.getTable(HiveClient.scala:81)
```
```java
20/02/17 02:56:45 ERROR SparkSQLDriver: Failed in [select * from t2]
java.lang.ClassNotFoundException: The SerDe interface removed since Hive 2.3(HIVE-15167). Please migrate your custom SerDes to Hive 2.3 or build your own Spark with hive-1.2 profile. See HIVE-15167 for more details.
at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$withHiveState$1(HiveClientImpl.scala:328)
at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:255)
at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:254)
at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:304)
at org.apache.spark.sql.hive.client.HiveClientImpl.getTableOption(HiveClientImpl.scala:450)
at org.apache.spark.sql.hive.client.HiveClient.getTable(HiveClient.scala:81)
at org.apache.spark.sql.hive.client.HiveClient.getTable$(HiveClient.scala:80)
at org.apache.spark.sql.hive.client.HiveClientImpl.getTable(HiveClientImpl.scala:91)
at org.apache.spark.sql.hive.HiveExternalCatalog.getRawTable(HiveExternalCatalog.scala:120)
at org.apache.spark.sql.hive.HiveExternalCatalog.$anonfun$getTable$1(HiveExternalCatalog.scala:711)
at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:99)
at org.apache.spark.sql.hive.HiveExternalCatalog.getTable(HiveExternalCatalog.scala:711)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.getTable(ExternalCatalogWithListener.scala:138)
at org.apache.spark.sql.catalyst.catalog.SessionCatalog.getTableMetadata(SessionCatalog.scala:445)
at org.apache.spark.sql.execution.datasources.v2.V2SessionCatalog.loadTable(V2SessionCatalog.scala:66)
at org.apache.spark.sql.connector.catalog.CatalogV2Util$.loadTable(CatalogV2Util.scala:281)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.loaded$1(Analyzer.scala:923)
```
```java
spark-sql> truncate table t2;
20/02/17 02:59:19 INFO HiveMetaStore: 0: get_database: default
20/02/17 02:59:19 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_database: default
20/02/17 02:59:19 INFO HiveMetaStore: 0: get_table : db=default tbl=t2
20/02/17 02:59:19 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=t2
20/02/17 02:59:19 INFO HiveMetaStore: 0: get_table : db=default tbl=t2
20/02/17 02:59:19 INFO audit: ugi=root ip=unknown-ip-addr cmd=get_table : db=default tbl=t2
20/02/17 02:59:19 ERROR SparkSQLDriver: Failed in [truncate table t2]
java.lang.ClassNotFoundException: The SerDe interface removed since Hive 2.3(HIVE-15167). Please migrate your custom SerDes to Hive 2.3 or build your own Spark with hive-1.2 profile. See HIVE-15167 for more details.
at org.apache.spark.sql.hive.client.HiveClientImpl.$anonfun$withHiveState$1(HiveClientImpl.scala:328)
at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:255)
at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:254)
at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:304)
at org.apache.spark.sql.hive.client.HiveClientImpl.getTableOption(HiveClientImpl.scala:450)
at org.apache.spark.sql.hive.client.HiveClient.getTable(HiveClient.scala:81)
at org.apache.spark.sql.hive.client.HiveClient.getTable$(HiveClient.scala:80)
at org.apache.spark.sql.hive.client.HiveClientImpl.getTable(HiveClientImpl.scala:91)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org