You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Lian (JIRA)" <ji...@apache.org> on 2014/11/03 10:17:33 UTC
[jira] [Commented] (SPARK-4199) Drop table if exists raises "table
not found" exception in HiveContext
[ https://issues.apache.org/jira/browse/SPARK-4199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194376#comment-14194376 ]
Cheng Lian commented on SPARK-4199:
-----------------------------------
Hi [~huangjs], which version/commit are you using? Could you please provide, for example, a {{spark-shell}} session snippet that helps reproduce this issue? Just tried both 1.1.0 and the most recent master (https://github.com/apache/spark/tree/76386e1a23c55a58c0aeea67820aab2bac71b24b) with this under {{spark-shell}}:
{code}
import org.apache.spark.sql.hive.HiveContext
import org.apache.spark.sql.catalyst.types._
import java.sql.Date
val sparkContext = sc
import sparkContext._
val hiveContext = new HiveContext(sparkContext)
import hiveContext._
sql("DROP TABLE IF EXISTS xxx")
{code}
The only ERROR log (which is expected) I found is:
{code}
14/11/03 17:12:56 ERROR metadata.Hive: NoSuchObjectException(message:default.xxx table not found)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1560)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
at com.sun.proxy.$Proxy16.get_table(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
...
{code}
And the DROP statement itself completed successfully.
> Drop table if exists raises "table not found" exception in HiveContext
> ----------------------------------------------------------------------
>
> Key: SPARK-4199
> URL: https://issues.apache.org/jira/browse/SPARK-4199
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.2.0
> Reporter: Jianshi Huang
>
> Try this:
> sql("DROP TABLE IF EXISTS some_table")
> The exception looks like this:
> 14/11/02 19:55:29 INFO ParseDriver: Parsing command: DROP TABLE IF EXISTS some_table
> 14/11/02 19:55:29 INFO ParseDriver: Parse Completed
> 14/11/02 19:55:29 INFO Driver: </PERFLOG method=parse start=1414986929678 end=1414986929678 duration=0>
> 14/11/02 19:55:29 INFO Driver: <PERFLOG method=semanticAnalyze>
> 14/11/02 19:55:29 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
> 14/11/02 19:55:29 INFO ObjectStore: ObjectStore, initialize called
> 14/11/02 19:55:29 ERROR Driver: FAILED: SemanticException [Error 10001]: Table not found some_table
> org.apache.hadoop.hive.ql.parse.SemanticException: Table not found some_table
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.getTable(DDLSemanticAnalyzer.java:3294)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.getTable(DDLSemanticAnalyzer.java:3281)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:824)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:249)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:284)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:441)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:342)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:977)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
> at org.apache.spark.sql.hive.HiveContext.runHive(HiveContext.scala:294)
> at org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:273)
> at org.apache.spark.sql.hive.execution.DropTable.sideEffectResult$lzycompute(commands.scala:58)
> at org.apache.spark.sql.hive.execution.DropTable.sideEffectResult(commands.scala:56)
> at org.apache.spark.sql.execution.Command$class.execute(commands.scala:44)
> at org.apache.spark.sql.hive.execution.DropTable.execute(commands.scala:51)
> at org.apache.spark.sql.hive.HiveContext$QueryExecution.toRdd$lzycompute(HiveContext.scala:353)
> at org.apache.spark.sql.hive.HiveContext$QueryExecution.toRdd(HiveContext.scala:353)
> at org.apache.spark.sql.SchemaRDDLike$class.$init$(SchemaRDDLike.scala:58)
> at org.apache.spark.sql.SchemaRDD.<init>(SchemaRDD.scala:104)
> at org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:98)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org