You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Cheng Lian (JIRA)" <ji...@apache.org> on 2014/06/12 11:16:04 UTC
[jira] [Commented] (SPARK-2129) NPE thrown while lookup a view
[ https://issues.apache.org/jira/browse/SPARK-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028956#comment-14028956 ]
Cheng Lian commented on SPARK-2129:
-----------------------------------
Exception also got eaten up in the {{HiveCompatibilitySuite.mergejoins_mixed}} test case, but the underlying cause of the exception is different.
> NPE thrown while lookup a view
> ------------------------------
>
> Key: SPARK-2129
> URL: https://issues.apache.org/jira/browse/SPARK-2129
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.0.0
> Reporter: Cheng Lian
>
> {code}
> scala> hql("create table t1(key int, value string)")
> ...
> scala> hql("create view v as select * from t1")
> ...
> scala> table("v")
> ...
> java.lang.NullPointerException
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:190)
> at org.apache.spark.sql.hive.MetastoreRelation.<init>(HiveMetastoreCatalog.scala:261)
> at org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation(HiveMetastoreCatalog.scala:66)
> at org.apache.spark.sql.hive.HiveContext$$anon$2.org$apache$spark$sql$catalyst$analysis$OverrideCatalog$$super$lookupRelation(HiveContext.scala:149)
> at org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:96)
> at org.apache.spark.sql.catalyst.analysis.OverrideCatalog$$anonfun$lookupRelation$3.apply(Catalog.scala:96)
> at scala.Option.getOrElse(Option.scala:120)
> at org.apache.spark.sql.catalyst.analysis.OverrideCatalog$class.lookupRelation(Catalog.scala:96)
> at org.apache.spark.sql.hive.HiveContext$$anon$2.lookupRelation(HiveContext.scala:155)
> at org.apache.spark.sql.SQLContext.table(SQLContext.scala:161)
> at .<init>(<console>:40)
> at .<clinit>(<console>)
> at .<init>(<console>:7)
> at .<clinit>(<console>)
> at $print(<console>)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:734)
> at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:983)
> at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:573)
> at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:604)
> at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:568)
> at scala.tools.nsc.interpreter.ILoop.reallyInterpret$1(ILoop.scala:760)
> at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:805)
> at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:717)
> at scala.tools.nsc.interpreter.ILoop.processLine$1(ILoop.scala:581)
> at scala.tools.nsc.interpreter.ILoop.innerLoop$1(ILoop.scala:588)
> at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:591)
> at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:882)
> at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:837)
> at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:837)
> at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
> at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:837)
> at scala.tools.nsc.interpreter.ILoop.main(ILoop.scala:904)
> at xsbt.ConsoleInterface.run(ConsoleInterface.scala:69)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at sbt.compiler.AnalyzingCompiler.call(AnalyzingCompiler.scala:102)
> at sbt.compiler.AnalyzingCompiler.console(AnalyzingCompiler.scala:77)
> at sbt.Console.sbt$Console$$console0$1(Console.scala:23)
> at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply$mcV$sp(Console.scala:24)
> at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply(Console.scala:24)
> at sbt.Console$$anonfun$apply$2$$anonfun$apply$1.apply(Console.scala:24)
> at sbt.Logger$$anon$4.apply(Logger.scala:90)
> at sbt.TrapExit$App.run(TrapExit.scala:244)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> This issue can also be triggered by console version of the {{join_view}} test case of the {{HiveCompatibilitySuite}}:
> {code}
> == Console version of this test ==
> val q0 = hql("""drop table invites"""); q0.collect()
> val q1 = hql("""drop table invites2"""); q1.collect()
> val q2 = hql("""create table invites (foo int, bar string) partitioned by (ds string)"""); q2.collect()
> val q3 = hql("""create table invites2 (foo int, bar string) partitioned by (ds string)"""); q3.collect()
> val q4 = hql("""set hive.mapred.mode=strict"""); q4.collect()
> val q5 = hql("""-- test join views: see HIVE-1989
> create view v as select invites.bar, invites2.foo, invites2.ds from invites join invites2 on invites.ds=invites2.ds"""); q5.collect()
> val q6 = hql("""explain select * from v where ds='2011-09-01'"""); q6.collect()
> val q7 = hql("""drop view v"""); q7.collect()
> val q8 = hql("""drop table invites"""); q8.collect()
> val q9 = hql("""drop table invites2"""); q9.collect()
> {code}
> When running inside {{HiveCompatibilitySuite}}, the exception is eaten up by {{SQLContext.stringOrError}}.
> When attacking [SPARK-2094|https://issues.apache.org/jira/browse/SPARK-2094], we added physical plans for DDL and commands, no longer taking the {{stringOnError}} route, and the exception is thrown.
--
This message was sent by Atlassian JIRA
(v6.2#6252)