[jira] [Resolved] (IMPALA-5835) Severe slowdown in catalogd startup after 2.1 → 2.5 upgrade with > 200,000 databases

["bharath v (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

bharath v resolved IMPALA-5835.
-------------------------------
       Resolution: Cannot Reproduce
    Fix Version/s: Not Applicable

Please try out the latest bits and reopen if needed.

> Severe slowdown in catalogd startup after 2.1 → 2.5 upgrade with > 200,000 databases
> ------------------------------------------------------------------------------------
>
>                 Key: IMPALA-5835
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5835
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog
>    Affects Versions: Impala 2.6.0, Impala 2.7.0, Impala 2.5.5, Impala 2.8.0, Impala 2.9.0, Impala 2.10.0
>            Reporter: Ben Breakstone
>            Assignee: bharath v
>            Priority: Major
>              Labels: performance
>             Fix For: Not Applicable
>
>
> After an upgrade from Impala 2.1 (CDH 5.3.9) to Impala 2.5 (CDH 5.7.5), starting up Catalog Server takes around eight to ten hours. It took around twenty minutes before the upgrade. 
> There are over 200,000 databases in use. Looking in the catalogd log as it starts up for hours, it says 
> "Loading native functions for database..." and then 
> "Loading Java functions for database..." for each database. Based on this, it appears the introduction of persistent UDFs/UDAs is causing the slowdown. 
> Only one of the databases actually has any UDFs defined. num_metadata_loading_threads is set to 64. Background loading of metadata is disabled. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org