You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2017/09/12 00:16:00 UTC
[jira] [Comment Edited] (HIVE-17495) CachedStore: prewarm
improvements, refactoring and caching some aggregate stats
[ https://issues.apache.org/jira/browse/HIVE-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162268#comment-16162268 ]
Sergey Shelukhin edited comment on HIVE-17495 at 9/12/17 12:15 AM:
-------------------------------------------------------------------
Some minor comments; my main concern - I don't see prewarm improvements and it looks like this will make prewarm even slower.
There's an issue that prewarm triggers on any access to metastore (e.g. LLAP setup getting UDF lists, or MetaTool) and it can take forever on large metastores. So at the very least it should be off by default and explicitly triggered in HS2 only.
Also even for regular prewarm for slow metastore, given how it runs, either startup, or many threads in HS2 may be blocked doing it as far as I see (firstTime is only set after the call and there's no synchronization). HS2 should not be unusable because of cache prewarm; it should be done on a background thread. Otherwise HS2 restarts become slow from users perspective.
Please do add "prewarm improvements" ;)
I'm strongly -0 (-0.1 :)) on piling new things onto this path before these improvements.
was (Author: sershe):
Some minor comments; my main concern - I don't see prewarm improvements and it looks like this will make prewarm even slower.
There's an issue that prewarm triggers on any access to metastore (e.g. LLAP setup getting UDF lists, or MetaTool) and it can take forever on large metastores. So at the very least it should be off by default and explicitly triggered in HS2 only.
Also even for regular prewarm for slow metastore, given how it runs, either startup, or many threads in HS2 may be blocked doing it as far as I see (firstTime is only set after the call and there's no synchronization). HS2 should not be unusable because of cache prewarm; it should be done on a background thread.
> CachedStore: prewarm improvements, refactoring and caching some aggregate stats
> -------------------------------------------------------------------------------
>
> Key: HIVE-17495
> URL: https://issues.apache.org/jira/browse/HIVE-17495
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Reporter: Vaibhav Gumashta
> Assignee: Vaibhav Gumashta
> Attachments: HIVE-17495.1.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)