[jira] [Created] (HIVE-25746) Compaction Failure Counter counted incorrectly

["Viktor Csomor (Jira)" <ji...@apache.org>]

Viktor Csomor created HIVE-25746:
------------------------------------

             Summary: Compaction Failure Counter counted incorrectly
                 Key: HIVE-25746
                 URL: https://issues.apache.org/jira/browse/HIVE-25746
             Project: Hive
          Issue Type: Bug
          Components: Metastore
    Affects Versions: 4.0.0
            Reporter: Viktor Csomor
            Assignee: Viktor Csomor


The count of the below metrics counted incorrectly upon an exception.
- {{compaction_initator_failure_counter}}
- {{compaction_cleaner_failure_counter}}

Reasoning:
In the {{Initator}}/{{Cleaner}} class creates a list of {{CompletableFuture}} which {{Runnable}} core exception is being wrapped to {{RuntimeExceptions}}. 
The below code-snippet waits all cleaners to complete (Initiators does it similarly).
{code:java}
        try {
           ....
            for (CompactionInfo compactionInfo : readyToClean) {
              cleanerList.add(CompletableFuture.runAsync(CompactorUtil.ThrowingRunnable.unchecked(() ->
                      clean(compactionInfo, cleanerWaterMark, metricsEnabled)), cleanerExecutor));
            }
            CompletableFuture.allOf(cleanerList.toArray(new CompletableFuture[0])).join();
          }
        } catch (Throwable t) {
          // the lock timeout on AUX lock, should be ignored.
          if (metricsEnabled && handle != null) {
            failuresCounter.inc();
          }
{code}

If the {{CompleteableFututre#join}} throws an Exception then the failure counter is incremented.

Let's suppose we have 10 cleaners and the 2nd throws an exception. The {{catch}} block will be initiated and the {{failuresCounter}} will be incremented. If there is any consecutive error amongst the remaining cleaners the counter won't be incremented. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)