You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/03/22 01:40:00 UTC
[jira] [Work logged] (HIVE-21485) Hive desc operation takes more
than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4
[ https://issues.apache.org/jira/browse/HIVE-21485?focusedWorklogId=217089&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-217089 ]
ASF GitHub Bot logged work on HIVE-21485:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Mar/19 01:39
Start Date: 22/Mar/19 01:39
Worklog Time Spent: 10m
Work Description: TopGunViper commented on pull request #577: HIVE-21485: Add flag to turn off fetching partition stats in DESCRIBE…
URL: https://github.com/apache/hive/pull/577
## What changes were proposed in this pull request?
Hive DESCRIBE [formatted|extended] operation cost more than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4. This is mainly caused by showing stats for partitioned tables which was introduced by [HIVE-16098](https://issues.apache.org/jira/browse/HIVE-16098) when the partitioned tables have a large amount of partitions.
So，could we add a flag that determines whether 'DESCRIBE [EXTENDED|FORMATTED]' operation display partitioned table stats or not?
## How was this patch tested?
Query Unit Test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 217089)
Time Spent: 10m
Remaining Estimate: 0h
> Hive desc operation takes more than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4
> ----------------------------------------------------------------------------------------
>
> Key: HIVE-21485
> URL: https://issues.apache.org/jira/browse/HIVE-21485
> Project: Hive
> Issue Type: Bug
> Components: CLI, Hive
> Affects Versions: 2.3.4
> Reporter: Qingxin Wu
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive desc [formatted|extended] operation cost more than 100 seconds after upgrading from Hive 1.2.1 to 2.3.4. This is mainly caused by showing stats for partitioned tables which was introduced by HIVE-16098 when the partitioned tables have a large amount of partitions. In our case, the number of partition is 187221.
> {code:java}
> hive> desc bus.kafka_data;
> OK
> id string
> ...
> d map<string,string>
> stat_date string
> log_id string
> # Partition Information
> # col_name data_type comment
> stat_date string
> log_id string
> Time taken: 115.342 seconds, Fetched: 42 row(s)
> {code}
> same operation executed in hive-1.2.1 and only cost 2 seconds.
> {code:java}
> hive> desc bus.kafka_data;
> OK
> id string
> ...
> d map<string,string>
> stat_date string
> log_id string
> # Partition Information
> # col_name data_type comment
> stat_date string
> log_id string
> Time taken: 2.037 seconds, Fetched: 42 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)