You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Vladimir Rodionov (JIRA)" <ji...@apache.org> on 2017/09/12 22:03:01 UTC
[jira] [Comment Edited] (HBASE-17852) Add Fault tolerance to
HBASE-14417 (Support bulk loaded files in incremental backup)
[ https://issues.apache.org/jira/browse/HBASE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163729#comment-16163729 ]
Vladimir Rodionov edited comment on HBASE-17852 at 9/12/17 10:02 PM:
---------------------------------------------------------------------
Some notes after initial analysis of implementation of bulk-loaded files support
# RegionObserver coprocessor has pre-commit and post-commit code (kind of Tx), but only on a region level. No Tx support on a table level. This means that if (for some regions), loading files fail, we will be left with a partial data in both: table store physical location and in backup system table. What we need here is Tx support for the whole bulk load operation cluster-wide (prepare, commit). And hooks exposed for custom coprocessor (Master?).
# What will happen if bulk load operation runs at the same time as incremental backup? Backup will see partial updates, but it should not.
# We add files to backup system tables, but I have not found where and when we delete those files.
was (Author: vrodionov):
Some notes after initial analysis of implementation of bulk-loaded files support
# RegionObserver coprocessor has pre-commit and post-commit code (kind of Tx), but only on a region level. No Tx support on a table level. This means that if (for some regions), loading files fail, we will be left with a partial data in both: table store physical location and in backup system table. What we need here is Tx support for the whole bulk load operation cluster-wide (prepare, commit). And hooks exposed for custom coprocessor (Master?).
# What will happen if bulk load operation runs at the same time as incremental backup? Backup will see partial updates, but it should not.
# We add files to backup system tables, but I have not found where and when we deleted those files.
> Add Fault tolerance to HBASE-14417 (Support bulk loaded files in incremental backup)
> ------------------------------------------------------------------------------------
>
> Key: HBASE-17852
> URL: https://issues.apache.org/jira/browse/HBASE-17852
> Project: HBase
> Issue Type: Sub-task
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Fix For: 2.0.0
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)