[jira] [Work logged] (HIVE-24965) Describe table partition fetch should be configurable

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-24965?focusedWorklogId=576101&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-576101 ]

ASF GitHub Bot logged work on HIVE-24965:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 02/Apr/21 13:50
            Start Date: 02/Apr/21 13:50
    Worklog Time Spent: 10m 
      Work Description: sankarh commented on a change in pull request #2141:
URL: https://github.com/apache/hive/pull/2141#discussion_r606244819



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/info/desc/DescTableOperation.java
##########
@@ -148,7 +148,8 @@ private void getColumnsNoColumnPath(Table table, Partition partition, List<Field
 
     // Fetch partition statistics only for describe extended or formatted.
     if (desc.isExtended() || desc.isFormatted()) {
-      if (table.isPartitioned() && partition == null) {
+      boolean shouldGetPartStats = MetastoreConf.getBoolVar(context.getConf(), MetastoreConf.ConfVars.DESCTABLE_ENABLE_PARTITION_STATS);

Review comment:
       We need to ensure the stats field itself is removed in the output if this config is enabled. Also, keep it uniform for both table and partition stats. Returning incorrect stats misleads user.

##########
File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
##########
@@ -559,6 +559,8 @@ public static ConfVars getMetaConf(String name) {
     DELEGATION_TOKEN_STORE_CLS("metastore.cluster.delegation.token.store.class",
         "hive.cluster.delegation.token.store.class", METASTORE_DELEGATION_MANAGER_CLASS,
         "Class to store delegation tokens"),
+    DESCTABLE_ENABLE_PARTITION_STATS("desctable.enable.partitionstats", "desctable.enable.partitionstats", true, 

Review comment:
       This config is used by HS2 and so move it to HiveConf.java. Also, use the name "hive.describe.table.ignore.stats" and keep the default as false. 

##########
File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/table/info/desc/DescTableOperation.java
##########
@@ -148,7 +148,8 @@ private void getColumnsNoColumnPath(Table table, Partition partition, List<Field
 
     // Fetch partition statistics only for describe extended or formatted.
     if (desc.isExtended() || desc.isFormatted()) {
-      if (table.isPartitioned() && partition == null) {
+      boolean shouldGetPartStats = MetastoreConf.getBoolVar(context.getConf(), MetastoreConf.ConfVars.DESCTABLE_ENABLE_PARTITION_STATS);
+      if (table.isPartitioned() && partition == null && shouldGetPartStats) {

Review comment:
       Need to add a unit test to verify the output with and without this config is set.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 576101)
    Time Spent: 20m  (was: 10m)

> Describe table partition fetch should be configurable
> -----------------------------------------------------
>
>                 Key: HIVE-24965
>                 URL: https://issues.apache.org/jira/browse/HIVE-24965
>             Project: Hive
>          Issue Type: Improvement
>    Affects Versions: 3.0.0, 3.1.1, 3.1.2
>            Reporter: Kevin Cheung
>            Assignee: Kevin Cheung
>            Priority: Minor
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> In certain 3rd party hive UIs, they issue a describe table command, and usually it's either "describe formatted table" or "describe extended table", and it will take a very long time for highly partitioned tables (10k+). In the UI, there is no way to change the settings (to use "describe table"), thus impairing the user experience. Prior to the feature where partition stats was gathered, the experience with using the UI had no problems.
> This will allow users to selectively change the describe formatted/extended table behavior at any time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)