You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Attila Magyar (Jira)" <ji...@apache.org> on 2020/04/15 19:26:00 UTC
[jira] [Assigned] (HIVE-23196) Reduce number of delete calls to NN
during Context::clear
[ https://issues.apache.org/jira/browse/HIVE-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Attila Magyar reassigned HIVE-23196:
------------------------------------
Assignee: Attila Magyar
> Reduce number of delete calls to NN during Context::clear
> ---------------------------------------------------------
>
> Key: HIVE-23196
> URL: https://issues.apache.org/jira/browse/HIVE-23196
> Project: Hive
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Attila Magyar
> Priority: Major
>
> {{Context::clear()}} ends up deleting same directories (or its subdirs) multiple times. It would be good to reduce the number of delete calls to NN for latency sensitive queries. This also has an impact on concurrent queries.
> {noformat}
> 2020-04-14T04:22:28,703 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899 HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting result dir: hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-10000
> 2020-04-14T04:22:28,721 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899 HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir: hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13
> 2020-04-14T04:22:28,737 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899 HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir: hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-10000/.hive-staging_hive_2020-04-14_04-22-24_335_8573832618972595103-13{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)