You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Vaibhav Gumashta (JIRA)" <ji...@apache.org> on 2018/03/06 23:21:00 UTC
[jira] [Updated] (HIVE-17495) CachedStore: prewarm improvement
(avoid multiple sql calls to read partition column stats), refactoring and
caching some aggregate stats
[ https://issues.apache.org/jira/browse/HIVE-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vaibhav Gumashta updated HIVE-17495:
------------------------------------
Issue Type: Sub-task (was: Bug)
Parent: HIVE-16520
> CachedStore: prewarm improvement (avoid multiple sql calls to read partition column stats), refactoring and caching some aggregate stats
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-17495
> URL: https://issues.apache.org/jira/browse/HIVE-17495
> Project: Hive
> Issue Type: Sub-task
> Components: Metastore
> Reporter: Vaibhav Gumashta
> Assignee: Vaibhav Gumashta
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-17495.1.patch, HIVE-17495.10.patch, HIVE-17495.10.patch, HIVE-17495.2.patch, HIVE-17495.3.patch, HIVE-17495.4.patch, HIVE-17495.5.patch, HIVE-17495.6.patch, HIVE-17495.7.patch, HIVE-17495.8.patch, HIVE-17495.9.patch
>
>
> Only when CachedStore is enabled, we would like to make the following optimizations:
> 1. During CachedStore prewarm, use one sql call to retrieve column stats objects for a db and store it in the cache.
> 2. Cache some aggregate stats (e.g. aggregate stats for all partitions, which seems to be commonly used) for query compilation speedup.
> 3. There was a bug in {{MetaStoreUtils.aggrPartitionStats}}, which would use an iterator.next w/o checking with iterator.hasNext. This patch refactors some code to fix that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)