You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hbase.apache.org by "Ujjawal Kumar (Jira)" <ji...@apache.org> on 2023/12/18 15:38:00 UTC

[jira] [Created] (HBASE-28268) Add option to skip wal while using TableOutputFormat

Ujjawal Kumar created HBASE-28268:
-------------------------------------

             Summary: Add option to skip wal while using TableOutputFormat 
                 Key: HBASE-28268
                 URL: https://issues.apache.org/jira/browse/HBASE-28268
             Project: HBase
          Issue Type: Improvement
          Components: mapreduce
            Reporter: Ujjawal Kumar


If we have replication setup between clusters A <-> B, during production incidents like where HFiles get corrupted (e.g. via hdfs missing blocks) on cluster A, we can use CopyTable job to copy the data from B

In those cases considering the data is being copied from B to A via CopyTable, it doesn't make sense to write to WALs on A using TableOutputFormat (since the mutations being copied already exist on B). 
We can add a config to customise durability ([https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Durability.java)]  within TableOutputFormat so that an operator can use it to skip WAL writes



--
This message was sent by Atlassian Jira
(v8.20.10#820010)