[jira] [Created] (IMPALA-5835) Severe slowdown in catalogd startup after 2.1 → 2.5 upgrade with > 200,000 databases

["Ben Breakstone (JIRA)" <ji...@apache.org>]

Ben Breakstone created IMPALA-5835:
--------------------------------------

             Summary: Severe slowdown in catalogd startup after 2.1 → 2.5 upgrade with > 200,000 databases
                 Key: IMPALA-5835
                 URL: https://issues.apache.org/jira/browse/IMPALA-5835
             Project: IMPALA
          Issue Type: Bug
    Affects Versions: Impala 2.5.5
            Reporter: Ben Breakstone


After an upgrade from Impala 2.1 (CDH 5.3.9) to Impala 2.5 (CDH 5.7.5), starting up Catalog Server takes around eight to ten hours. It took around twenty minutes before the upgrade. 

There are over 200,000 databases in use. Looking in the catalogd log as it starts up for hours, it says 
"Loading native functions for database..." and then 
"Loading Java functions for database..." for each database. Based on this, it appears the introduction of persistent UDFs/UDAs is causing the slowdown. 

Only one of the databases actually has any UDFs defined. num_metadata_loading_threads is set to 64. Background loading of metadata is disabled. 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)