You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2019/09/16 23:06:00 UTC

[jira] [Resolved] (IMPALA-1463) Improve HDFS Caching DDL performance for partitioned table

     [ https://issues.apache.org/jira/browse/IMPALA-1463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-1463.
-----------------------------------
    Resolution: Later

Don't think there's much interest in this one.

> Improve HDFS Caching DDL performance for partitioned table
> ----------------------------------------------------------
>
>                 Key: IMPALA-1463
>                 URL: https://issues.apache.org/jira/browse/IMPALA-1463
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Catalog
>    Affects Versions: Impala 1.4.1, Impala 2.0
>            Reporter: Alan Choi
>            Priority: Minor
>              Labels: performance
>
> Enabling HDFS caching for partitioned table requires two steps. For each partition, it'll have to set a directive to HDFS. Then, it'll issue an "alter table" to HiveMetaStore. This is done in a single-threaded loop.
> Issuing the "atler table" to HiveMetaStore is very slow. In my experiment, each call is ~1sec. For a 4k partition table, it took 4k seconds.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org