[jira] [Commented] (HIVE-19994) Impala "drop table" fails with Hive Metastore exception

["Peter Vary (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-19994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523542#comment-16523542 ] 

Peter Vary commented on HIVE-19994:
-----------------------------------

My guess would be, that somehow a new partition is added to the table concurrently, or something like that happened.

This part of the code is responsible to set the CD to null before dropping a table:

[https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L1426]

The method code:
{code:java}
/**
* Called right before an action that would drop a storage descriptor.
* This function makes the SD's reference to a CD null, and then deletes the CD
* if it no longer is referenced in the table.
* @param msd the storage descriptor to drop
*/
private void preDropStorageDescriptor(MStorageDescriptor msd) {
if (msd == null || msd.getCD() == null) {
return;
}
MColumnDescriptor mcd = msd.getCD();
// Because there is a 1-N relationship between CDs and SDs,
// we must set the SD's CD to null first before dropping the storage descriptor
// to satisfy foreign key constraints.
msd.setCD(null);
removeUnusedColumnDescriptor(mcd);
}{code}

> Impala "drop table" fails with Hive Metastore exception
> -------------------------------------------------------
>
>                 Key: HIVE-19994
>                 URL: https://issues.apache.org/jira/browse/HIVE-19994
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 1.1.0
>         Environment: Hadoop distribution: CHD 5.14.2
> Hive version:  1.1.0-cdh5.14.2
> Impala version: 2.11.0
> Kudu version: 1.6.0
>  
>            Reporter: Rodion Myronov
>            Priority: Major
>         Attachments: metastore_exception.txt
>
>
> "drop table" statement in Impala shell fails with the following exception:
> {{ImpalaRuntimeException: Error making 'dropTable' RPC to Hive Metastore: CAUSED BY: MetaException: One or more instances could not be deleted}}
>  
> Metastore log file shows that "DELETE FROM `PARTITION_KEYS` WHERE `TBL_ID`=?" statement fails because of foreign key violation (full stacktrace will be added):
> {{Caused by: java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails ("hivemetastore_emtig3vtq7qp1tiooo07sb70ud"."COLUMNS_V2", CONSTRAINT "COLUMNS_V2_FK1" FOREIGN KEY ("CD_ID") REFERENCES "CDS" ("CD_ID"))}}
>  
> The table is created and then dropped as a part of ETL process executed every hour. Most of the time it works fine, the issue is not reproducible at will.
> Table creation script is:
> {{CREATE TABLE IF NOT EXISTS price_advisor_ouput.t_switching_coef_source}}
> {{( }}
> {{...fields here...}}
> {{PRIMARY KEY (...PK field here...)}}
> {{)}}
> {{PARTITION BY HASH(matrix_pcd) PARTITIONS 3}}
> {{STORED AS KUDU;}}
>  
> Not sure how to approach diagnostics and fix, so any input will be really appreciated. 
> Thanks in advance, 
> Rodion Myronov



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)