You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Kurt Deschler (Jira)" <ji...@apache.org> on 2021/08/04 13:23:00 UTC
[jira] [Updated] (IMPALA-10355) Use lower
statestore_update_frequency_ms default with local catalog
[ https://issues.apache.org/jira/browse/IMPALA-10355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kurt Deschler updated IMPALA-10355:
-----------------------------------
Issue Type: Improvement (was: Bug)
Summary: Use lower statestore_update_frequency_ms default with local catalog (was: DROP FUNCTION IF EXISTS taking 4-5 sec to execute when function does not exist)
> Use lower statestore_update_frequency_ms default with local catalog
> -------------------------------------------------------------------
>
> Key: IMPALA-10355
> URL: https://issues.apache.org/jira/browse/IMPALA-10355
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Kurt Deschler
> Priority: Major
>
> DROP FUNCTION IF EXISTS is excessively slow when the function does not exist. Preliminary analysis shows this line is taking 4+ sec for the drop function if exists call:
> impala-server.cc:1778 WaitForCatalogUpdate(catalog_update_result.version, catalog_service_id);
> The 4+ sec wait here is due to statestore updates being done every 2 sec by default.
> Setting statestore_update_frequency_ms to 100 significantly improves performance of drop function if exists.
> Query: create FUNCTION test_udf(DECIMAL(38,36)) RETURNS STRING LOCATION '/tmp/test_udf.so' SYMBOL='test_udf'
> real 0m0.104s
> user 0m0.062s
> sys 0m0.016s
> Query: drop FUNCTION test_udf(DECIMAL(38,36))
> real 0m0.097s
> user 0m0.049s
> sys 0m0.032s
> Query: drop FUNCTION IF EXISTS test_udf(DECIMAL(38,36)) --function exists
> real 0m0.098s
> user 0m0.059s
> sys 0m0.020s
> Query: drop FUNCTION IF EXISTS test_udf(DECIMAL(38,36)) --function does not exists
> real 0m4.542s
> user 0m0.068s
> sys 0m0.022s
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org