[jira] [Updated] (HIVE-13406) HBase metastore needs to support ACID

["Alan Gates (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-13406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Gates updated HIVE-13406:
------------------------------
    Attachment: HIVE-13406.patch

Rather than directly translate the RDBMS implementation I propose to take a somewhat different approach for HBase.  In the RDBMS implementation the lock and transaction trees are kept in the db and reconstructed for each begin, lock, commit, or rollback.  This leads to a lot of DB activity as well as slower than required response times.  I propose instead to keep the lock and txn trees in memory.  This implementation does this by using an HBase co-processor.

A co-processor was chosen so that HBase can manage the location and HA aspects of the service rather than needing to re-invent that.  In other words, this patch effectively uses HBase's co-processor facility as an app server.

This patch makes one other fundamental change, it requires read operations to open transactions.  Associating all locks with a transaction greatly simplifies the code.

This patch is not ready for merging.  I haven't finished testing it or carried forward into the RDBMS implementation changes such as requiring transactions on reads.  It is just posted as a prototype to show the direction I'm thinking this should go.

> HBase metastore needs to support ACID
> -------------------------------------
>
>                 Key: HIVE-13406
>                 URL: https://issues.apache.org/jira/browse/HIVE-13406
>             Project: Hive
>          Issue Type: Improvement
>          Components: HBase Metastore, Transactions
>    Affects Versions: 2.0.0
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: HIVE-13406.patch
>
>
> The current HBase metastore code does not support any of the acid operations.  It needs to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)