[jira] [Commented] (HBASE-16527) IOExceptions from DFS client still can cause CatalogJanitor to delete referenced files

["Vladimir Rodionov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-16527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15447743#comment-15447743 ] 

Vladimir Rodionov commented on HBASE-16527:
-------------------------------------------

CatalogJanitor:
{code}
  Pair<Boolean, Boolean> checkDaughterInFs(final HRegionInfo parent, final HRegionInfo daughter)

SKIP-SKIP
    boolean references = false;
    HTableDescriptor parentDescriptor = getTableDescriptor(parent.getTable());
    for (HColumnDescriptor family: parentDescriptor.getFamilies()) {
      if ((references = regionFs.hasReferences(family.getNameAsString()))) {
        break;
      }
    }
    return new Pair<Boolean, Boolean>(Boolean.TRUE, Boolean.valueOf(references));
{code}

HRegionFileSystem:
{code}
  /**
   * Returns true if the specified family has reference files
   * @param familyName Column Family Name
   * @return true if family contains reference files
   * @throws IOException
   */
  public boolean hasReferences(final String familyName) throws IOException {
    FileStatus[] files = FSUtils.listStatus(fs, getStoreDir(familyName),
        new FSUtils.ReferenceFileFilter(fs));
    return files != null && files.length > 0;
  }
{code}

 FSUtils.ReferenceFileFilter:

{code}
    @Override
    protected boolean accept(Path p, @CheckForNull Boolean isDir) {
      if (!StoreFileInfo.isReference(p)) {
        return false;
      }

      try {
        // only files can be references.
        return isFile(fs, isDir, p);
      } catch (IOException ioe) {
        // Maybe the file was moved or the fs was disconnected.
        LOG.warn("Skipping file " + p +" due to IOException", ioe);
        return false;
      }
    }
{code}

isFile can throw IOException and we return false (not reference) - which is bad.


> IOExceptions from DFS client still can cause CatalogJanitor to delete referenced files
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-16527
>                 URL: https://issues.apache.org/jira/browse/HBASE-16527
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>             Fix For: 2.0.0
>
>
> that was fixed partially in HBASE-13331, but issue still exists , now a little bit deeper in the code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)