You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Stephen Boesch <ja...@gmail.com> on 2015/03/22 03:15:10 UTC
How to clean up a table for which the underlying hdfs file no longer exists
There is a hive table for which the metadata points to a non-existing hdfs
file. Simply calling
drop table <mytable>
results in:
Failed to load metadata for table: db.mytable
Caused by TAbleLoadingException: Failed to load metadata for table: db.mytable
File does not exist: hdfs://....
Caused by FileNotFoundException: File does not exist: hdfs:// ..
So: the file does not exist in hdfs , and it is not possible to remove the
metadata for it directly. Is the next step going to be: "run some sql
commands against the metastore to manually delete the associated rows"? If
so, what are those delete commands?
thanks
Re: How to clean up a table for which the underlying hdfs file no longer exists
Posted by Daniel Haviv <da...@veracity-group.com>.
You can (as a workaround) just create it's directory and then drop it
Daniel
> On 22 במרץ 2015, at 04:15, Stephen Boesch <ja...@gmail.com> wrote:
>
>
> There is a hive table for which the metadata points to a non-existing hdfs file. Simply calling
>
> drop table <mytable>
>
> results in:
>
> Failed to load metadata for table: db.mytable
> Caused by TAbleLoadingException: Failed to load metadata for table: db.mytable
> File does not exist: hdfs://....
> Caused by FileNotFoundException: File does not exist: hdfs:// ..
>
> So: the file does not exist in hdfs , and it is not possible to remove the metadata for it directly. Is the next step going to be: "run some sql commands against the metastore to manually delete the associated rows"? If so, what are those delete commands?
>
> thanks
Re: How to clean up a table for which the underlying hdfs file no
longer exists
Posted by Stephen Boesch <ja...@gmail.com>.
The folder exists: just not the file. I tried both of Daniel's suggestions
and they ended up with same error afterwards.
2015-03-22 2:07 GMT-07:00 Wollert, Fabian <fa...@zalando.de>:
> can you create the specified folder (just leave it empty) and then delete
> again?
>
> Cheers
> Fabian
>
> 2015-03-22 3:15 GMT+01:00 Stephen Boesch <ja...@gmail.com>:
>
>>
>> There is a hive table for which the metadata points to a non-existing
>> hdfs file. Simply calling
>>
>> drop table <mytable>
>>
>> results in:
>>
>> Failed to load metadata for table: db.mytable
>> Caused by TAbleLoadingException: Failed to load metadata for table: db.mytable
>> File does not exist: hdfs://....
>> Caused by FileNotFoundException: File does not exist: hdfs:// ..
>>
>>
>> So: the file does not exist in hdfs , and it is not possible to remove
>> the metadata for it directly. Is the next step going to be: "run some sql
>> commands against the metastore to manually delete the associated rows"? If
>> so, what are those delete commands?
>>
>> thanks
>>
>
>
>
> --
> *Fabian Wollert*
> Business Intelligence
>
>
>
> *POSTANSCHRIFT*
> Zalando SE
> 11501 Berlin
>
> *STANDORT*
> Zalando SE
> Mollstraße 1
> 10178 Berlin
> Germany
>
> Phone: +49 30 20968 1819
> Fax: +49 30 27594 693
> E-Mail: fabian.wollert@zalando.de
> Web: www.zalando.de
> Jobs: jobs.zalando.de
>
> Zalando SE, Tamara-Danz-Straße 1, 10243 Berlin
> Handelsregister: Amtsgericht Charlottenburg, HRB 158855 B
> Steuer-Nr. 29/560/00596 * USt-ID-Nr. DE 260543043
> Vorstand: Robert Gentz, David Schneider, Rubin Ritter
> Vorsitzende des Aufsichtsrates: Cristina Stenbeck
> Sitz der Gesellschaft: Berlin
>
Re: How to clean up a table for which the underlying hdfs file no
longer exists
Posted by "Wollert, Fabian" <fa...@zalando.de>.
can you create the specified folder (just leave it empty) and then delete
again?
Cheers
Fabian
2015-03-22 3:15 GMT+01:00 Stephen Boesch <ja...@gmail.com>:
>
> There is a hive table for which the metadata points to a non-existing hdfs
> file. Simply calling
>
> drop table <mytable>
>
> results in:
>
> Failed to load metadata for table: db.mytable
> Caused by TAbleLoadingException: Failed to load metadata for table: db.mytable
> File does not exist: hdfs://....
> Caused by FileNotFoundException: File does not exist: hdfs:// ..
>
>
> So: the file does not exist in hdfs , and it is not possible to remove
> the metadata for it directly. Is the next step going to be: "run some sql
> commands against the metastore to manually delete the associated rows"? If
> so, what are those delete commands?
>
> thanks
>
--
*Fabian Wollert*
Business Intelligence
*POSTANSCHRIFT*
Zalando SE
11501 Berlin
*STANDORT*
Zalando SE
Mollstraße 1
10178 Berlin
Germany
Phone: +49 30 20968 1819
Fax: +49 30 27594 693
E-Mail: fabian.wollert@zalando.de
Web: www.zalando.de
Jobs: jobs.zalando.de
Zalando SE, Tamara-Danz-Straße 1, 10243 Berlin
Handelsregister: Amtsgericht Charlottenburg, HRB 158855 B
Steuer-Nr. 29/560/00596 * USt-ID-Nr. DE 260543043
Vorstand: Robert Gentz, David Schneider, Rubin Ritter
Vorsitzende des Aufsichtsrates: Cristina Stenbeck
Sitz der Gesellschaft: Berlin
Re: How to clean up a table for which the underlying hdfs file no longer exists
Posted by Daniel Haviv <da...@veracity-group.com>.
You can also use
ALTER TABLE <table> SET TBLPROPERTIES('EXTERNAL'='TRUE')
And then drop it
Daniel
> On 22 במרץ 2015, at 04:15, Stephen Boesch <ja...@gmail.com> wrote:
>
>
> There is a hive table for which the metadata points to a non-existing hdfs file. Simply calling
>
> drop table <mytable>
>
> results in:
>
> Failed to load metadata for table: db.mytable
> Caused by TAbleLoadingException: Failed to load metadata for table: db.mytable
> File does not exist: hdfs://....
> Caused by FileNotFoundException: File does not exist: hdfs:// ..
>
> So: the file does not exist in hdfs , and it is not possible to remove the metadata for it directly. Is the next step going to be: "run some sql commands against the metastore to manually delete the associated rows"? If so, what are those delete commands?
>
> thanks