[jira] [Resolved] (HBASE-27061) two phase bulkload is broken when SFT is in use.

["Wellington Chevreuil (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-27061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wellington Chevreuil resolved HBASE-27061.
------------------------------------------
    Resolution: Fixed

Thanks for the fix, [~sergey.soldatov]. Had merged into master, branch-2 and branch-2.5.

> two phase bulkload is broken when SFT is in use.
> ------------------------------------------------
>
>                 Key: HBASE-27061
>                 URL: https://issues.apache.org/jira/browse/HBASE-27061
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.5.0, 3.0.0-alpha-2, 2.6.0
>            Reporter: Sergey Soldatov
>            Assignee: Sergey Soldatov
>            Priority: Major
>             Fix For: 2.5.0, 2.6.0, 3.0.0-alpha-3
>
>
> In HBASE-26707 for the SFT case, we are writing files directly to the region location. For that we are using HRegion.regionDir as the staging directory. The problem is that in reality, this dir is pointing to the WAL dir, so for S3 deployments that would be pointing to the hdfs. As the result during the execution of LoadIncrementalHFiles the process failed with the exception:
> {noformat}
> 2022-05-24 03:31:23,656 ERROR org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager: Failed to complete bulk load
> java.lang.IllegalArgumentException: Wrong FS hdfs://ns1//hbase-wals/data/default/employees/4f367b303da4fed7667fff07fd4c6066/department/acd971097924463da6d6e3a15f9527da -expected s3a://hbase
>         at org.apache.hadoop.fs.s3native.S3xLoginHelper.checkPath(S3xLoginHelper.java:224)
>         at org.apache.hadoop.fs.s3a.S3AFileSystem.checkPath(S3AFileSystem.java:1375)
>         at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:647)
>         at org.apache.hadoop.fs.s3a.S3AFileSystem.makeQualified(S3AFileSystem.java:1337)
>         at org.apache.hadoop.fs.s3a.S3AFileSystem.qualify(S3AFileSystem.java:1363)
>         at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:3521)
>         at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:511)
>         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:397)
>         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:387)
>         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
>         at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$SecureBulkLoadListener.prepareBulkLoad(SecureBulkLoadManager.java:397)
>         at org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:6994)
>         at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$1.run(SecureBulkLoadManager.java:291)
>         at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager$1.run(SecureBulkLoadManager.java:266)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:360)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1879)
>         at org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager.secureBulkLoadHFiles(SecureBulkLoadManager.java:266)
>         at org.apache.hadoop.hbase.regionserver.RSRpcServices.bulkLoadHFile(RSRpcServices.java:2453)
>         at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:45821)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:392)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:140)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:359)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:339)
>  {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)