[jira] [Updated] (HIVE-25914) Cleaner updates Initiator cycle metric

["Viktor Csomor (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Viktor Csomor updated HIVE-25914:
---------------------------------
    Description: 
The Cleaner updates the Initiator metric.

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java#L175

Create some test for Initiator and Cleaner to avoid this happen.

  was:
The count of the below metrics counted incorrectly upon an exception.
- {{compaction_initator_failure_counter}}
- {{compaction_cleaner_failure_counter}}

Reasoning:
In the {{Initator}}/{{Cleaner}} class creates a list of {{CompletableFuture}} which {{Runnable}} core exception is being wrapped to {{RuntimeExceptions}}. 
The below code-snippet waits all cleaners to complete (Initiators does it similarly).
{code:java}
        try {
           ....
            for (CompactionInfo compactionInfo : readyToClean) {
              cleanerList.add(CompletableFuture.runAsync(CompactorUtil.ThrowingRunnable.unchecked(() ->
                      clean(compactionInfo, cleanerWaterMark, metricsEnabled)), cleanerExecutor));
            }
            CompletableFuture.allOf(cleanerList.toArray(new CompletableFuture[0])).join();
          }
        } catch (Throwable t) {
          // the lock timeout on AUX lock, should be ignored.
          if (metricsEnabled && handle != null) {
            failuresCounter.inc();
          }
{code}

If the {{CompleteableFututre#join}} throws an Exception then the failure counter is incremented.

Docs:
{code}

    /**
     * Returns the result value when complete, or throws an
     * (unchecked) exception if completed exceptionally. To better
     * conform with the use of common functional forms, if a
     * computation involved in the completion of this
     * CompletableFuture threw an exception, this method throws an
     * (unchecked) {@link CompletionException} with the underlying
     * exception as its cause.
     *
     * @return the result value
     * @throws CancellationException if the computation was cancelled
     * @throws CompletionException if this future completed
     * exceptionally or a completion computation threw an exception
     */
    public T join() {
        Object r;
        return reportJoin((r = result) == null ? waitingGet(false) : r);
    }
{code}

(!) Let's suppose we have 10 cleaners and the 2nd throws an exception. The {{catch}} block will be initiated and the {{failuresCounter}} will be incremented. If there is any consecutive error amongst the remaining cleaners the counter won't be incremented. 


> Cleaner updates Initiator cycle metric
> --------------------------------------
>
>                 Key: HIVE-25914
>                 URL: https://issues.apache.org/jira/browse/HIVE-25914
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>    Affects Versions: 4.0.0
>            Reporter: Viktor Csomor
>            Assignee: Viktor Csomor
>            Priority: Minor
>              Labels: pull-request-available
>
> The Cleaner updates the Initiator metric.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java#L175
> Create some test for Initiator and Cleaner to avoid this happen.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)