You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Yu-Wen Lai (Jira)" <ji...@apache.org> on 2021/07/15 22:21:00 UTC
[jira] [Created] (IMPALA-10801) Check the latest compaction Id
before serving request
Yu-Wen Lai created IMPALA-10801:
-----------------------------------
Summary: Check the latest compaction Id before serving request
Key: IMPALA-10801
URL: https://issues.apache.org/jira/browse/IMPALA-10801
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Reporter: Yu-Wen Lai
Assignee: Yu-Wen Lai
Cache compaction Id for a given table/file-metadata in CatalogD.
Whenever there is a read request to CatalogD, get the latest compaction event Id from HMS, compare it with what is cached in CatalogD, and based on that decide whether to serve the data from cache or to refresh it from the filesystem. This can avoid notification based cache invalidation.
Also, since there will be an open txn for the current long running query which is being served from CatalogD, we can be sure that current file-metadata being served is not already deleted by the cleaner.
This proposal will use a new HMS API (https://issues.apache.org/jira/browse/HIVE-24828) to get the latest compaction id for a table.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org