[jira] [Created] (HIVE-13132) Hive should lazily load and cache metastore (permanent) functions

["Anthony Hsu (JIRA)" <ji...@apache.org>]

Anthony Hsu created HIVE-13132:
----------------------------------

             Summary: Hive should lazily load and cache metastore (permanent) functions
                 Key: HIVE-13132
                 URL: https://issues.apache.org/jira/browse/HIVE-13132
             Project: Hive
          Issue Type: Improvement
    Affects Versions: 0.13.1
            Reporter: Anthony Hsu
            Assignee: Anthony Hsu


In Hive 0.13.1, we have noticed that as the number of databases increases, the start-up time of the Hive interactive shell increases. This is because during start-up, all databases are iterated over to fetch the permanent functions to display in the {{SHOW FUNCTIONS}} output.

{noformat:title=FunctionRegistry.java}
  private static Set<String> getFunctionNames(boolean searchMetastore) {
    Set<String> functionNames = mFunctions.keySet();
    if (searchMetastore) {
      functionNames = new HashSet<String>(functionNames);
      try {
        Hive db = getHive();
        List<String> dbNames = db.getAllDatabases();

        for (String dbName : dbNames) {
          List<String> funcNames = db.getFunctions(dbName, "*");
          for (String funcName : funcNames) {
            functionNames.add(FunctionUtils.qualifyFunctionName(funcName, dbName));
          }
        }
      } catch (Exception e) {
        LOG.error(e);
        // Continue on, we can still return the functions we've gotten to this point.
      }
    }
    return functionNames;
  }
{noformat}

Instead of eagerly loading all metastore functions, we should only load them the first time {{SHOW FUNCTIONS}} is invoked. We should also cache the results.

Note that this issue may have been fixed by HIVE-2573, though I haven't verified this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)