[jira] [Comment Edited] (HBASE-13128) Make HBCK's lock file retry creation and deletion

["Sean Busbey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-13128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345525#comment-14345525 ] 

Sean Busbey edited comment on HBASE-13128 at 3/3/15 7:05 PM:
-------------------------------------------------------------

This looks good Victoria. One last thing:

{code}
+      do {
+        try {
+          IOUtils.closeStream(hbckOutFd);
+          FSUtils.delete(FSUtils.getCurrentFileSystem(getConf()),
+              HBCK_LOCK_PATH, true);
+          return;
+        } catch (IOException ioe) {
+          LOG.info("Failed to delete " + HBCK_LOCK_PATH + ", try="
+              + (retryCounter.getAttemptTimes() + 1) + " of "
+              + retryCounter.getMaxAttempts());
+          LOG.debug("Failed to delete " + HBCK_LOCK_PATH, ioe);
+          try {
+            retryCounter.sleepUntilNextRetry();
+          } catch (InterruptedException ie) {
+            Thread.currentThread().interrupt();
+            LOG.warn("Interrupted while deleting lock file" +
+                HBCK_LOCK_PATH);
+            return;
+          }
+        }
+      } while (retryCounter.shouldRetry());
{code}

Could you break this into a retry block for each of closing and deleting per Elliot's request above?

Otherwise, +1 lgtm


was (Author: busbey):
This looks good Victoria. One last thing:

{quote}
+      do {
+        try {
+          IOUtils.closeStream(hbckOutFd);
+          FSUtils.delete(FSUtils.getCurrentFileSystem(getConf()),
+              HBCK_LOCK_PATH, true);
+          return;
+        } catch (IOException ioe) {
+          LOG.info("Failed to delete " + HBCK_LOCK_PATH + ", try="
+              + (retryCounter.getAttemptTimes() + 1) + " of "
+              + retryCounter.getMaxAttempts());
+          LOG.debug("Failed to delete " + HBCK_LOCK_PATH, ioe);
+          try {
+            retryCounter.sleepUntilNextRetry();
+          } catch (InterruptedException ie) {
+            Thread.currentThread().interrupt();
+            LOG.warn("Interrupted while deleting lock file" +
+                HBCK_LOCK_PATH);
+            return;
+          }
+        }
+      } while (retryCounter.shouldRetry());
{quote}

Could you break this into a retry block for each of closing and deleting per Elliot's request above?

Otherwise, +1 lgtm

> Make HBCK's lock file retry creation and deletion
> -------------------------------------------------
>
>                 Key: HBASE-13128
>                 URL: https://issues.apache.org/jira/browse/HBASE-13128
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbck
>            Reporter: Victoria
>            Assignee: Victoria
>            Priority: Minor
>         Attachments: hbck_files_fix-v1.patch, hbck_files_fix-v2.patch, hbck_files_fix-v3.patch, hbck_files_fix.patch
>
>
> When hbck runs it creates a lock file to ensure that no two hbck instances are running. We've been seeing creating and removing that file fail sometimes.
> This improvement should make the creation, closing of the file, and the deletion retry multiple times. This should allow our alerting which uses this command to be more reliable and have fewer false positives.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)