[jira] [Created] (HBASE-8566) Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsisten

["Demai Ni (JIRA)" <ji...@apache.org>]

Demai Ni created HBASE-8566:
-------------------------------

             Summary: Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsisten
                 Key: HBASE-8566
                 URL: https://issues.apache.org/jira/browse/HBASE-8566
             Project: HBase
          Issue Type: Bug
          Components: Replication
    Affects Versions: 0.94.3
         Environment: two 2-nodes clusters setup as Master and Slave for replication of table 'usertable'
            Reporter: Demai Ni
            Priority: Minor


after successfully setup the replcation. put some rows into 'usertable' , 

At Master cluster
$truncate 'usertable'

The truncate(or mass delete from user perspective) request isn't sent over to slave cluster. 

>From internal, the truncate is 'disable', 'drop' and 'create'. Such operations are not designed for replication. However, from external/user perspective, this is a 'delete everything' operation, which should be part of the replication. 

This JIRA is to add this support
---------------------------
additional information. I did a few loads using YCSB into 'usertable', with different # of rows(from 1000 to 100000). And did truncate a couple times in between. Then the slave cluster began to throw errors:
{code:title=count failed on slave cluster|borderStyle=solid}

ERROR: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=7, exceptions:
Thu May 16 15:00:13 PDT 2013, org.apache.hadoop.hbase.client.ScannerCallable@4c474c47, java.net.ConnectException: Connection refused
Thu May 16 15:00:32 PDT 2013, org.apache.hadoop.hbase.client.ScannerCallable@4c474c47, org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find region for usertable,,99999999999999 after 7 tries.
Thu May 16 15:00:51 PDT 2013, org.apache.hadoop.hbase.client.ScannerCallable@4c474c47, org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find region for usertable,,99999999999999 after 7 tries.
Thu May 16 15:01:11 PDT 2013, org.apache.hadoop.hbase.client.ScannerCallable@4c474c47, org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find region for usertable,,99999999999999 after 7 tries.
{code}
The regionserver log of slave cluster throws :
{code:title=regionserver log of slave cluster|borderStyle=solid}
2013-05-16 14:59:59,655 ERROR org.apache.hadoop.hbase.replication.regionserver.ReplicationSink: Unable to accept edit because:
java.io.IOException: java.lang.InterruptedException
        at org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:220)
        at org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.replicateEntries(ReplicationSink.java:154)
        at org.apache.hadoop.hbase.replication.regionserver.Replication.replicateLogEntries(Replication.java:140)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.replicateLogEntries(HRegionServer.java:3797)
        at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
Caused by: java.lang.InterruptedException
        at java.lang.Thread.sleep(Native Method)
        at java.lang.Thread.sleep(Thread.java:853)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1507)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1400)
        at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:699)
        at org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.batch(ReplicationSink.java:217)
        ... 8 more
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira