[jira] [Created] (HBASE-22200) WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir

["Wellington Chevreuil (JIRA)" <ji...@apache.org>]

Wellington Chevreuil created HBASE-22200:
--------------------------------------------

             Summary: WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir
                 Key: HBASE-22200
                 URL: https://issues.apache.org/jira/browse/HBASE-22200
             Project: HBase
          Issue Type: Bug
            Reporter: Wellington Chevreuil
            Assignee: Wellington Chevreuil


*WALSplitter.hasRecoveredEdits* should use same FS instance from WAL region dir when checking for recovered.edits files, instead of taking FS instance as additional method parameter. When specifying different file systems for *wal dir* and *root dir*,  *WALSplitter.hasRecoveredEdits* current implementation will crash or give wrong results. As of now, it's being used indirectly by *SplitTableRegionProcedure*. When running tests with *WAL dir* on HDFS and *root dir* on S3, for example, noticed region split failing with below error:

{noformat}
2019-04-08 13:53:58,064 ERROR org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught runtime exception: pid=98, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=test-tbl, parent=4c5db01611e97e3abbe02e781e867212, daughterA=28a0a5e4ef7618899f6bd6dfb5335fe7, daughterB=05fa26feaf03ebf9e87e099cbd1eabac
java.lang.IllegalArgumentException: Path hdfs://host-1.example.com:8020/wal_dir/default/test-tbl/4c5db01611e97e3abbe02e781e867212/recovered.edits scheme must be s3a
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:115)
        at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.checkPath(DynamoDBMetadataStore.java:1127)
        at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:437)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:2110)
        at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:2088)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1668)
        at org.apache.hadoop.hbase.wal.WALSplitter.getSplitEditFilesSorted(WALSplitter.java:576)
        at org.apache.hadoop.hbase.wal.WALSplitter.hasRecoveredEdits(WALSplitter.java:558)
        at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.hasRecoveredEdits(SplitTableRegionProcedure.java:148)
        at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:255)
{noformat}

Since *WALSplitter.hasRecoveredEdits* already resolves the proper WAL dir for the region, we can simply re-use FS instance from the path instance for the WAL dir region, when searching for recovered.edits.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)