[jira] [Created] (HIVE-16385) StatsNoJobTask could exit early before all partitions have been processed

["Chao Sun (JIRA)" <ji...@apache.org>]

Chao Sun created HIVE-16385:
-------------------------------

             Summary: StatsNoJobTask could exit early before all partitions have been processed
                 Key: HIVE-16385
                 URL: https://issues.apache.org/jira/browse/HIVE-16385
             Project: Hive
          Issue Type: Bug
          Components: Statistics
            Reporter: Chao Sun
            Assignee: Chao Sun


For a partitioned table, the class {{StatsNoJobTask}} is supposed to launch threads for all partitions and compute their stats. However, it could exit early after at most 100 seconds:

{code}
  private void shutdownAndAwaitTermination(ExecutorService threadPool) {

    // Disable new tasks from being submitted
    threadPool.shutdown();
    try {

      // Wait a while for existing tasks to terminate
      if (!threadPool.awaitTermination(100, TimeUnit.SECONDS)) {
        // Cancel currently executing tasks
        threadPool.shutdownNow();

        // Wait a while for tasks to respond to being cancelled
        if (!threadPool.awaitTermination(100, TimeUnit.SECONDS)) {
          LOG.debug("Stats collection thread pool did not terminate");
        }
      }
    } catch (InterruptedException ie) {

      // Cancel again if current thread also interrupted
      threadPool.shutdownNow();

      // Preserve interrupt status
      Thread.currentThread().interrupt();
    }
  }
{code}

The {{shutdown}} call does not wait for all submitted tasks to complete, and the {{awaitTermination}} call waits at most 100 seconds. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)