You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Soumyakanti Das (Jira)" <ji...@apache.org> on 2020/07/29 20:05:00 UTC

[jira] [Created] (HIVE-23949) Introduce caching layer in HS2 to accelerate query compilation

Soumyakanti Das created HIVE-23949:
--------------------------------------

             Summary: Introduce caching layer in HS2 to accelerate query compilation
                 Key: HIVE-23949
                 URL: https://issues.apache.org/jira/browse/HIVE-23949
             Project: Hive
          Issue Type: New Feature
          Components: HiveServer2
            Reporter: Soumyakanti Das
            Assignee: Soumyakanti Das


One of the major contributors to compilation latency is the retrieval of metadata from HMS. This JIRA introduces a caching layer in HS2 for this metadata. Its design is simple, relying on snapshot information being queried to cache and invalidate the metadata. This will help us to reduce the time spent in compilation by using HS2 memory more effectively, and it will allow us to improve HMS throughput for multi-tenant workloads by reducing the number of calls it needs to serve.
This patch only caches partition retrieval information, which is often one of the most costly metadata operations. It also sets the foundation to cache additional calls in follow-up work.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)