[jira] [Commented] (HBASE-12642) LoadIncrementalHFiles does not throw exception after hitting hbase.bulkload.retries.number setting

["Erik Ingle (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-12642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236111#comment-14236111 ] 

Erik Ingle commented on HBASE-12642:
------------------------------------

Seems reasonable to me if you put it there or in the finally block.  The error is more specific where you have placed it.

This is somewhat related to an issue I am seeing with [HBASE-10549|https://issues.apache.org/jira/browse/HBASE-10549] where retries split infinitely.
This usually occurs near the time of automated compactions.

What I am trying to do is give up after too many retries and try the entire bulk load again later.  This patch would suit that purpose.
Thanks.


> LoadIncrementalHFiles does not throw exception after hitting hbase.bulkload.retries.number setting
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-12642
>                 URL: https://issues.apache.org/jira/browse/HBASE-12642
>             Project: HBase
>          Issue Type: Bug
>          Components: mapreduce
>    Affects Versions: 0.98.1
>         Environment: RedHat 6.6
> Java 1.7.0_65
> HBase 0.98.1-cdh5.1.0
> Hadoop 2.3.0-mr1-cdh5.1.0
>            Reporter: Erik Ingle
>            Priority: Minor
>         Attachments: 12642-v1.txt
>
>
> Related to [HBASE-8367|https://issues.apache.org/jira/browse/HBASE-8367] there is no programmatic way to determine if the hbase.bulkload.retries.number value has been hit.  Callers of doBulkLoad have no way to determine other than log examination whether the bulk load was actually successful.
> Here is the relevant code sections:
> {code:title=LoadIncrementalHFiles.java|borderStyle=solid}
> public void doBulkLoad ...
> {
>     try {
>     ...
>         if (maxRetries != 0 && count >= maxRetries) {
>             LOG.error("Retry attempted " + count + " times without completing, bailing out");
>             return;
>         }
>     ...
>     } finally {
>     ...
>       pool.shutdown();
>       if (queue != null && !queue.isEmpty()) {
>         StringBuilder err = new StringBuilder();
>         err.append("-------------------------------------------------\n");
>         err.append("Bulk load aborted with some files not yet loaded:\n");
>         err.append("-------------------------------------------------\n");
>         for (LoadQueueItem q : queue) {
>           err.append("  ").append(q.hfilePath).append('\n');
>         }
>         LOG.error(err);
>       }
>     }
>     if (queue != null && !queue.isEmpty()) {
>         throw new RuntimeException("Bulk load aborted with some files not yet loaded."
>           + "Please check log for more details.");
> }
> {code}
> One possible solution is to move the throw new RuntimeException clause at the end into the finally block.  The exception is currently never thrown for max retries due to the return in the initial if statement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)