[jira] [Created] (TRAFODION-1820) HBase error after a failed bulk load in Trafodion

["Hans Zeller (JIRA)" <ji...@apache.org>]

Hans Zeller created TRAFODION-1820:
--------------------------------------

             Summary: HBase error after a failed bulk load in Trafodion
                 Key: TRAFODION-1820
                 URL: https://issues.apache.org/jira/browse/TRAFODION-1820
             Project: Apache Trafodion
          Issue Type: Bug
          Components: sql-general
    Affects Versions: 1.1 (pre-incubation)
         Environment: Any
            Reporter: Hans Zeller


When using the Trafodion bulk loader (LOAD statement), if the load fails in the middle, it may leave HBase in a state where the HMaster produces security-related errors. In the case I saw, the HMaster failed to start, due to these errors. Here is an example:

Unhandled exception. Starting shutdown.
java.io.IOException: Received error when attempting to archive files ([class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://ip-10-0-0-132.ec2.internal:8020/hbase/.tmp/data/default/TRAFODION.SEABASE.LSTATS1/03a60516389fcc3e02e5de07f936cbb1/#1, class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://ip-10-0-0-132.ec2.internal:8020/hbase/.tmp/data/default/TRAFODION.SEABASE.LSTATS1/03a60516389fcc3e02e5de07f936cbb1/mt_, class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://ip-10-0-0-132.ec2.internal:8020/hbase/.tmp/data/default/TRAFODION.SEABASE.LSTATS1/03a60516389fcc3e02e5de07f936cbb1/recovered.edits]), cannot delete region directory. 
	at org.apache.hadoop.hbase.backup.HFileArchiver.archiveRegion(HFileArchiver.java:148)
	at org.apache.hadoop.hbase.master.MasterFileSystem.checkTempDir(MasterFileSystem.java:508)
	at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:149)
	at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:126)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:611)
	at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:169)
	at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1484)
	at java.lang.Thread.run(Thread.java:745)


The solution is to do a "chown hbase:hbase" command in HDFS, to give the files to the hbase user. A fix for this problem might do this chown operation earlier in the Trafodion code path, or use some other method to avoid the security error.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)