You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Jianshi Huang <ji...@gmail.com> on 2014/12/05 08:31:56 UTC
drop table if exists throws exception
Hi,
I got exception saying Hive: NoSuchObjectException(message:<table> table
not found)
when running "DROP TABLE IF EXISTS <table>"
Looks like a new regression in Hive module.
Anyone can confirm this?
Thanks,
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
Re: drop table if exists throws exception
Posted by Jianshi Huang <ji...@gmail.com>.
I see. The resulting SchemaRDD is returned so like Michael said, the
exception does not propogate to user code.
However printing out the following log is confusing :)
scala> sql("drop table if exists abc")
14/12/05 16:27:02 INFO ParseDriver: Parsing command: drop table if exists
abc
14/12/05 16:27:02 INFO ParseDriver: Parse Completed
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=Driver.run
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=TimeToSubmit
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO Driver: Concurrency mode is disabled, not creating a
lock manager
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=compile
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=parse
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO ParseDriver: Parsing command: DROP TABLE IF EXISTS
abc
14/12/05 16:27:02 INFO ParseDriver: Parse Completed
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=parse
start=1417825622650 end=1417825622650 duration=0
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=semanticAnalyze
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO HiveMetaStore: 0: get_table : db=default tbl=abc
14/12/05 16:27:02 INFO audit: ugi=jianshuang ip=unknown-ip-addr
cmd=get_table : db=default tbl=abc
14/12/05 16:27:02 INFO Driver: Semantic Analysis Completed
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=semanticAnalyze
start=1417825622650 end=1417825622653 duration=3
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO Driver: Returning Hive schema:
Schema(fieldSchemas:null, properties:null)
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=compile
start=1417825622650 end=1417825622654 duration=4
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=Driver.execute
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO Driver: Starting command: DROP TABLE IF EXISTS abc
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=TimeToSubmit
start=1417825622650 end=1417825622654 duration=4
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=runTasks
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=task.DDL.Stage-0
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO HiveMetaStore: 0: get_table : db=default tbl=abc
14/12/05 16:27:02 INFO audit: ugi=jianshuang ip=unknown-ip-addr
cmd=get_table : db=default tbl=abc
14/12/05 16:27:02 ERROR Hive: NoSuchObjectException(message:default.abc
table not found)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1560)
at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
On Sat, Dec 6, 2014 at 3:45 AM, Mark Hamstra <ma...@clearstorydata.com>
wrote:
> And that is no different from how Hive has worked for a long time.
>
> On Fri, Dec 5, 2014 at 11:42 AM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
>> The command run fine for me on master. Note that Hive does print an
>> exception in the logs, but that exception does not propogate to user code.
>>
>> On Thu, Dec 4, 2014 at 11:31 PM, Jianshi Huang <ji...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I got exception saying Hive: NoSuchObjectException(message:<table> table
>> > not found)
>> >
>> > when running "DROP TABLE IF EXISTS <table>"
>> >
>> > Looks like a new regression in Hive module.
>> >
>> > Anyone can confirm this?
>> >
>> > Thanks,
>> > --
>> > Jianshi Huang
>> >
>> > LinkedIn: jianshi
>> > Twitter: @jshuang
>> > Github & Blog: http://huangjs.github.com/
>> >
>>
>
>
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
Re: drop table if exists throws exception
Posted by Jianshi Huang <ji...@gmail.com>.
I see. The resulting SchemaRDD is returned so like Michael said, the
exception does not propogate to user code.
However printing out the following log is confusing :)
scala> sql("drop table if exists abc")
14/12/05 16:27:02 INFO ParseDriver: Parsing command: drop table if exists
abc
14/12/05 16:27:02 INFO ParseDriver: Parse Completed
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=Driver.run
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=TimeToSubmit
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO Driver: Concurrency mode is disabled, not creating a
lock manager
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=compile
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=parse
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO ParseDriver: Parsing command: DROP TABLE IF EXISTS
abc
14/12/05 16:27:02 INFO ParseDriver: Parse Completed
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=parse
start=1417825622650 end=1417825622650 duration=0
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=semanticAnalyze
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO HiveMetaStore: 0: get_table : db=default tbl=abc
14/12/05 16:27:02 INFO audit: ugi=jianshuang ip=unknown-ip-addr
cmd=get_table : db=default tbl=abc
14/12/05 16:27:02 INFO Driver: Semantic Analysis Completed
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=semanticAnalyze
start=1417825622650 end=1417825622653 duration=3
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO Driver: Returning Hive schema:
Schema(fieldSchemas:null, properties:null)
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=compile
start=1417825622650 end=1417825622654 duration=4
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=Driver.execute
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO Driver: Starting command: DROP TABLE IF EXISTS abc
14/12/05 16:27:02 INFO PerfLogger: </PERFLOG method=TimeToSubmit
start=1417825622650 end=1417825622654 duration=4
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=runTasks
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO PerfLogger: <PERFLOG method=task.DDL.Stage-0
from=org.apache.hadoop.hive.ql.Driver>
14/12/05 16:27:02 INFO HiveMetaStore: 0: get_table : db=default tbl=abc
14/12/05 16:27:02 INFO audit: ugi=jianshuang ip=unknown-ip-addr
cmd=get_table : db=default tbl=abc
14/12/05 16:27:02 ERROR Hive: NoSuchObjectException(message:default.abc
table not found)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1560)
at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
On Sat, Dec 6, 2014 at 3:45 AM, Mark Hamstra <ma...@clearstorydata.com>
wrote:
> And that is no different from how Hive has worked for a long time.
>
> On Fri, Dec 5, 2014 at 11:42 AM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
>> The command run fine for me on master. Note that Hive does print an
>> exception in the logs, but that exception does not propogate to user code.
>>
>> On Thu, Dec 4, 2014 at 11:31 PM, Jianshi Huang <ji...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > I got exception saying Hive: NoSuchObjectException(message:<table> table
>> > not found)
>> >
>> > when running "DROP TABLE IF EXISTS <table>"
>> >
>> > Looks like a new regression in Hive module.
>> >
>> > Anyone can confirm this?
>> >
>> > Thanks,
>> > --
>> > Jianshi Huang
>> >
>> > LinkedIn: jianshi
>> > Twitter: @jshuang
>> > Github & Blog: http://huangjs.github.com/
>> >
>>
>
>
--
Jianshi Huang
LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/
Re: drop table if exists throws exception
Posted by Mark Hamstra <ma...@clearstorydata.com>.
And that is no different from how Hive has worked for a long time.
On Fri, Dec 5, 2014 at 11:42 AM, Michael Armbrust <mi...@databricks.com>
wrote:
> The command run fine for me on master. Note that Hive does print an
> exception in the logs, but that exception does not propogate to user code.
>
> On Thu, Dec 4, 2014 at 11:31 PM, Jianshi Huang <ji...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I got exception saying Hive: NoSuchObjectException(message:<table> table
> > not found)
> >
> > when running "DROP TABLE IF EXISTS <table>"
> >
> > Looks like a new regression in Hive module.
> >
> > Anyone can confirm this?
> >
> > Thanks,
> > --
> > Jianshi Huang
> >
> > LinkedIn: jianshi
> > Twitter: @jshuang
> > Github & Blog: http://huangjs.github.com/
> >
>
Re: drop table if exists throws exception
Posted by Mark Hamstra <ma...@clearstorydata.com>.
And that is no different from how Hive has worked for a long time.
On Fri, Dec 5, 2014 at 11:42 AM, Michael Armbrust <mi...@databricks.com>
wrote:
> The command run fine for me on master. Note that Hive does print an
> exception in the logs, but that exception does not propogate to user code.
>
> On Thu, Dec 4, 2014 at 11:31 PM, Jianshi Huang <ji...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I got exception saying Hive: NoSuchObjectException(message:<table> table
> > not found)
> >
> > when running "DROP TABLE IF EXISTS <table>"
> >
> > Looks like a new regression in Hive module.
> >
> > Anyone can confirm this?
> >
> > Thanks,
> > --
> > Jianshi Huang
> >
> > LinkedIn: jianshi
> > Twitter: @jshuang
> > Github & Blog: http://huangjs.github.com/
> >
>
Re: drop table if exists throws exception
Posted by Michael Armbrust <mi...@databricks.com>.
The command run fine for me on master. Note that Hive does print an
exception in the logs, but that exception does not propogate to user code.
On Thu, Dec 4, 2014 at 11:31 PM, Jianshi Huang <ji...@gmail.com>
wrote:
> Hi,
>
> I got exception saying Hive: NoSuchObjectException(message:<table> table
> not found)
>
> when running "DROP TABLE IF EXISTS <table>"
>
> Looks like a new regression in Hive module.
>
> Anyone can confirm this?
>
> Thanks,
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
>
Re: drop table if exists throws exception
Posted by Michael Armbrust <mi...@databricks.com>.
The command run fine for me on master. Note that Hive does print an
exception in the logs, but that exception does not propogate to user code.
On Thu, Dec 4, 2014 at 11:31 PM, Jianshi Huang <ji...@gmail.com>
wrote:
> Hi,
>
> I got exception saying Hive: NoSuchObjectException(message:<table> table
> not found)
>
> when running "DROP TABLE IF EXISTS <table>"
>
> Looks like a new regression in Hive module.
>
> Anyone can confirm this?
>
> Thanks,
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
>