[jira] [Commented] (IMPALA-7538) Support HDFS caching with LocalCatalog

["Quanlong Huang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-7538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17129231#comment-17129231 ] 

Quanlong Huang commented on IMPALA-7538:
----------------------------------------

I did a quick investigation on this. The most missing block is LocalCatalog.getHdfsCachePool(String). Coordinators use this to analyze DDL statements to make sure the given cache pool exists. The DDLs are executed in catalogd so we already have the functionality.

Only the cache pool names are propagated to coordinators. For cache replica locations, they are in the FileDescriptors which we already have in partition metadata.

So I think we can implement this by:
 * Sync cache pool names to CatalogdMetaProvider as what we do in catalog-v1.
 * DirectMetaProvider fetches the cache pool names from HDFS directly.
 * Don't need to introduce FeFsCachePool. Because if we do so and derive HdfsCachePool and LocalFsCachePool from it, there will be no difference between HdfsCachePool and LocalFsCachePool.

cc [~vihangk1], [~tarmstrong@cloudera.com]

> Support HDFS caching with LocalCatalog
> --------------------------------------
>
>                 Key: IMPALA-7538
>                 URL: https://issues.apache.org/jira/browse/IMPALA-7538
>             Project: IMPALA
>          Issue Type: Sub-task
>            Reporter: Todd Lipcon
>            Assignee: Quanlong Huang
>            Priority: Major
>              Labels: catalog-v2
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org