[jira] [Closed] (DRILL-7115) Improve Hive schema show tables performance

["Denys Ordynskiy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Denys Ordynskiy closed DRILL-7115.
----------------------------------

Successfully tested on the release Drill v1.16 rc2
Created 8136 tables in hive.
In sqlline:
{noformat}
use hive;
show tables;
{noformat}
On the Drill before fix (commit id - 4627973bde9847a4eb2672c44941136c167326a1)
8,136 rows selected (106.361 seconds)
8,136 rows selected (113.649 seconds)
8,136 rows selected (110.421 seconds)
8,136 rows selected (106.415 seconds)
8,136 rows selected (106.863 seconds)

On the release Drill v1.16 rc2:
8,136 rows selected (6.578 seconds)
8,136 rows selected (6.387 seconds)
8,136 rows selected (6.461 seconds)
8,136 rows selected (7.123 seconds)
8,136 rows selected (6.303 seconds)

On the hive shell:
{noformat}
show tables;
{noformat}
Time taken: 0.867 seconds, Fetched: 8136 row(s)
Time taken: 0.049 seconds, Fetched: 8136 row(s)
Time taken: 0.05 seconds, Fetched: 8136 row(s)
Time taken: 0.047 seconds, Fetched: 8136 row(s)
Time taken: 0.045 seconds, Fetched: 8136 row(s)

> Improve Hive schema show tables performance
> -------------------------------------------
>
>                 Key: DRILL-7115
>                 URL: https://issues.apache.org/jira/browse/DRILL-7115
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - Hive, Storage - Information Schema
>    Affects Versions: 1.15.0
>            Reporter: Igor Guzenko
>            Assignee: Igor Guzenko
>            Priority: Major
>              Labels: ready-to-commit
>             Fix For: 1.16.0
>
>
> In Sqlline(Drill), "show tables" on a Hive schema is taking nearly 15mins to 20mins. The schema has nearly ~8000 tables.
> Whereas the same in beeline(Hive) is throwing the result in a split second(~ 0.2 secs).
> I tested the same in my test cluster by creating 6000 tables(empty!) in Hive and then doing "show tables" in Drill. It took more than 2 mins(~140 secs).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)