[jira] [Created] (HBASE-27061) two phase bulkload is broken when SFT is in use.

["Sergey Soldatov (Jira)" <ji...@apache.org>]

Sergey Soldatov created HBASE-27061:
---------------------------------------

             Summary: two phase bulkload is broken when SFT is in use.
                 Key: HBASE-27061
                 URL: https://issues.apache.org/jira/browse/HBASE-27061
             Project: HBase
          Issue Type: Bug
    Affects Versions: 2.4.12
            Reporter: Sergey Soldatov


In HBASE-26707 for the SFT case, we are writing files directly to the region location. For that we are using HRegion.regionDir as the staging directory. The problem is that in reality, this dir is pointing to the WAL dir, so for S3 deployments that would be pointing to the hdfs. As the result during the execution of LoadIncrementalHFiles the process failed with the exception:
{noformat}
2022-05-24 03:31:23,656 ERROR org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager: Failed to complete bulk load
java.lang.IllegalArgumentException: Wrong FS hdfs://ns1//hbase-wals/data/default/employees/4f367b303da4fed7667fff07fd4c6066/department/acd971097924463da6d6e3a15f9527da -expected s3a://hbase
        at org.apache.hadoop.fs.s3native.S3xLoginHelper.checkPath(S3xLoginHelper.java:224)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.checkPath(S3AFileSystem.java:1375)
        at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:647)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.makeQualified(S3AFileSystem.java:1337)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.qualify(S3AFileSystem.java:1363)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:3521)
        at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:511)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:397)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:387)
        at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
        at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$SecureBulkLoadListener.prepareBulkLoad(SecureBulkLoadManager.java:397)
        at org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:6994)
        at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$1.run(SecureBulkLoadManager.java:291)
        at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$1.run(SecureBulkLoadManager.java:266)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1879)
        at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager.secureBulkLoadHFiles(SecureBulkLoadManager.java:266)
        at org.apache.hadoop.hbase.regionserver.RSRpcServices.bulkLoadHFile(RSRpcServices.java:2453)
        at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45821)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:359)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:339)

 {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)