You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/03/03 15:25:00 UTC
[jira] [Work logged] (HIVE-25989) CTLT HBaseStorageHandler is dropping underlying HBase table when failed
[ https://issues.apache.org/jira/browse/HIVE-25989?focusedWorklogId=736072&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-736072 ]
ASF GitHub Bot logged work on HIVE-25989:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 03/Mar/22 15:24
Start Date: 03/Mar/22 15:24
Worklog Time Spent: 10m
Work Description: marton-bod opened a new pull request #3076:
URL: https://github.com/apache/hive/pull/3076
… the underlying HBase table
### What changes were proposed in this pull request?
Introduce a CTLT flag to check if a table was created using CTLT, the same way it's done for CTAS.
### Why are the changes needed?
In the case of HBase tables, if you use a CTLT command, your new table will point to the same underlying HBase table as your original table. In case of a failure, currently the HBaseMetaHook rollback logic deletes the HBase table, leading to data loss. However, using the CTLT flag, we can stop that from doing so.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 736072)
Remaining Estimate: 0h
Time Spent: 10m
> CTLT HBaseStorageHandler is dropping underlying HBase table when failed
> -----------------------------------------------------------------------
>
> Key: HIVE-25989
> URL: https://issues.apache.org/jira/browse/HIVE-25989
> Project: Hive
> Issue Type: Bug
> Reporter: Naresh P R
> Assignee: Marton Bod
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> With hive.strict.managed.tables & hive.create.as.acid,
> Hive-Hbase rollback code is assuming it is a createTable failure instead of CTLT & removing underlying hbase table while rolling back at here.
> [https://github.com/apache/hive/blob/master/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseMetaHook.java#L187-L195]
>
> Repro
>
> {code:java}
> hbase
> =====
> hbase shell
> create 'hbase_hive_table', 'cf'
> beeline
> =======
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> set hive.strict.managed.tables=true;
> set hive.create.as.acid=true;
> set hive.create.as.insert.only=true;
> set hive.default.fileformat.managed=ORC;
> > CREATE EXTERNAL TABLE `hbase_hive_table`(
> `key` int COMMENT '',
> `value` string COMMENT '')
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
> STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> 'hbase.columns.mapping'=':key,cf:cf')
> TBLPROPERTIES ('hbase.table.name'='hbase_hive_table');
> > select * from hbase_hive_table;
> +-----------------------+-------------------------+
> | hbase_hive_table.key | hbase_hive_table.value |
> +-----------------------+-------------------------+
> +-----------------------+-------------------------+
> > create table new_hbase_hive_table like hbase_hive_table;
> Caused by: org.apache.hadoop.hive.metastore.api.MetaException: The table must be stored using an ACID compliant format (such as ORC): default.new_hbase_hive_table
> > select * from hbase_hive_table;
> Error: java.io.IOException: org.apache.hadoop.hbase.TableNotFoundException: hbase_hive_table
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)