You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Satheesh Akuthota (Jira)" <ji...@apache.org> on 2023/01/24 10:09:00 UTC

[jira] [Updated] (HIVE-26979) bucketing_version properly not coming for like table

     [ https://issues.apache.org/jira/browse/HIVE-26979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Satheesh Akuthota updated HIVE-26979:
-------------------------------------
    Description: 
Steps to reproduce : 
1.create table T6(a int , b int) clustered by (a) into 2 buckets TBLPROPERTIES ('transient_lastDdlTime'='1521230300);
2.create table T6like like T6;
3.  desc formatted T6;
4.desc formatted T6like;

Output of desc formatted T6;

{code:java}
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
|           col_name            |                     data_type                      |                      comment                       |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| # col_name                    | data_type                                          | comment                                            |
| a                             | int                                                |                                                    |
| b                             | int                                                |                                                    |
|                               | NULL                                               | NULL                                               |
| # Detailed Table Information  | NULL                                               | NULL                                               |
| Database:                     | default                                            | NULL                                               |
| OwnerType:                    | USER                                               | NULL                                               |
| Owner:                        | hive                                               | NULL                                               |
| CreateTime:                   | Mon Jan 23 06:16:27 GMT 2023                       | NULL                                               |
| LastAccessTime:               | UNKNOWN                                            | NULL                                               |
| Retention:                    | 0                                                  | NULL                                               |
| Location:                     | hdfs:///warehouse/tablespace/external/hive/t6 | NULL                                               |
| Table Type:                   | EXTERNAL_TABLE                                     | NULL                                               |
| Table Parameters:             | NULL                                               | NULL                                               |
|                               | COLUMN_STATS_ACCURATE                              | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\"}} |
|                               | EXTERNAL                                           | TRUE                                               |
|                               | *bucketing_version*                                  | 2                                                  |
|                               | external.table.purge                               | true                                               |
|                               | numFiles                                           | 2                                                  |
|                               | numRows                                            | 6                                                  |
|                               | rawDataSize                                        | 22                                                 |
|                               | totalSize                                          | 39                                                 |
|                               | transient_lastDdlTime                              | 1674454919                                         |
|                               | NULL                                               | NULL                                               |
| # Storage Information         | NULL                                               | NULL                                               |
| SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL                                               |
| InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat           | NULL                                               |
| OutputFormat:                 | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL                                               |
| Compressed:                   | No                                                 | NULL                                               |
| Num Buckets:                  | 2                                                  | NULL                                               |
| Bucket Columns:               | [a]                                                | NULL                                               |
| Sort Columns:                 | []                                                 | NULL                                               |
| Storage Desc Params:          | NULL                                               | NULL                                               |
|                               | serialization.format                               | 1                                                  |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+

{code}

Output of desc formatted T6like;

{code:java}
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
|           col_name            |                     data_type                      |                      comment                       |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| # col_name                    | data_type                                          | comment                                            |
| a                             | int                                                |                                                    |
| b                             | int                                                |                                                    |
|                               | NULL                                               | NULL                                               |
| # Detailed Table Information  | NULL                                               | NULL                                               |
| Database:                     | default                                            | NULL                                               |
| OwnerType:                    | USER                                               | NULL                                               |
| Owner:                        | hive                                               | NULL                                               |
| CreateTime:                   | Mon Jan 23 06:16:59 GMT 2023                       | NULL                                               |
| LastAccessTime:               | UNKNOWN                                            | NULL                                               |
| Retention:                    | 0                                                  | NULL                                               |
| Location:                     | hdfs:///warehouse/tablespace/external/hive/t6like | NULL                                               |
| Table Type:                   | EXTERNAL_TABLE                                     | NULL                                               |
| Table Parameters:             | NULL                                               | NULL                                               |
|                               | COLUMN_STATS_ACCURATE                              | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\"}} |
|                               | EXTERNAL                                           | TRUE                                               |
|                               | external.table.purge                               | true                                               |
|                               | numFiles                                           | 0                                                  |
|                               | numRows                                            | 0                                                  |
|                               | rawDataSize                                        | 0                                                  |
|                               | totalSize                                          | 0                                                  |
|                               | transient_lastDdlTime                              | 1674454619                                         |
|                               | NULL                                               | NULL                                               |
| # Storage Information         | NULL                                               | NULL                                               |
| SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL                                               |
| InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat           | NULL                                               |
| OutputFormat:                 | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL                                               |
| Compressed:                   | No                                                 | NULL                                               |
| Num Buckets:                  | 2                                                  | NULL                                               |
| Bucket Columns:               | [a]                                                | NULL                                               |
| Sort Columns:                 | []                                                 | NULL                                               |
| Storage Desc Params:          | NULL                                               | NULL                                               |
|                               | serialization.format                               | 1                                                  |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
33 rows selected (0.172 seconds)

{code}





  was:
Steps to reproduce : 
1.create table T6(a int , b int) clustered by (a) into 2 buckets TBLPROPERTIES ('transient_lastDdlTime'='1521230300);
2.create table T6like like T6;
3.  desc formatted T6;
4.desc formatted T6like;

Output of desc formatted T6;

{code:java}
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
|           col_name            |                     data_type                      |                      comment                       |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| # col_name                    | data_type                                          | comment                                            |
| a                             | int                                                |                                                    |
| b                             | int                                                |                                                    |
|                               | NULL                                               | NULL                                               |
| # Detailed Table Information  | NULL                                               | NULL                                               |
| Database:                     | default                                            | NULL                                               |
| OwnerType:                    | USER                                               | NULL                                               |
| Owner:                        | hive                                               | NULL                                               |
| CreateTime:                   | Mon Jan 23 06:16:27 GMT 2023                       | NULL                                               |
| LastAccessTime:               | UNKNOWN                                            | NULL                                               |
| Retention:                    | 0                                                  | NULL                                               |
| Location:                     | hdfs:///warehouse/tablespace/external/hive/t6 | NULL                                               |
| Table Type:                   | EXTERNAL_TABLE                                     | NULL                                               |
| Table Parameters:             | NULL                                               | NULL                                               |
|                               | COLUMN_STATS_ACCURATE                              | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\"}} |
|                               | EXTERNAL                                           | TRUE                                               |
|                               | *bucketing_version*                                  | 2                                                  |
|                               | external.table.purge                               | true                                               |
|                               | numFiles                                           | 2                                                  |
|                               | numRows                                            | 6                                                  |
|                               | rawDataSize                                        | 22                                                 |
|                               | totalSize                                          | 39                                                 |
|                               | transient_lastDdlTime                              | 1674454919                                         |
|                               | NULL                                               | NULL                                               |
| # Storage Information         | NULL                                               | NULL                                               |
| SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL                                               |
| InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat           | NULL                                               |
| OutputFormat:                 | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL                                               |
| Compressed:                   | No                                                 | NULL                                               |
| Num Buckets:                  | 2                                                  | NULL                                               |
| Bucket Columns:               | [a]                                                | NULL                                               |
| Sort Columns:                 | []                                                 | NULL                                               |
| Storage Desc Params:          | NULL                                               | NULL                                               |
|                               | serialization.format                               | 1                                                  |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+

{code}


{code:java}
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
|           col_name            |                     data_type                      |                      comment                       |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| # col_name                    | data_type                                          | comment                                            |
| a                             | int                                                |                                                    |
| b                             | int                                                |                                                    |
|                               | NULL                                               | NULL                                               |
| # Detailed Table Information  | NULL                                               | NULL                                               |
| Database:                     | default                                            | NULL                                               |
| OwnerType:                    | USER                                               | NULL                                               |
| Owner:                        | hive                                               | NULL                                               |
| CreateTime:                   | Mon Jan 23 06:16:59 GMT 2023                       | NULL                                               |
| LastAccessTime:               | UNKNOWN                                            | NULL                                               |
| Retention:                    | 0                                                  | NULL                                               |
| Location:                     | hdfs:///warehouse/tablespace/external/hive/t6like | NULL                                               |
| Table Type:                   | EXTERNAL_TABLE                                     | NULL                                               |
| Table Parameters:             | NULL                                               | NULL                                               |
|                               | COLUMN_STATS_ACCURATE                              | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\"}} |
|                               | EXTERNAL                                           | TRUE                                               |
|                               | external.table.purge                               | true                                               |
|                               | numFiles                                           | 0                                                  |
|                               | numRows                                            | 0                                                  |
|                               | rawDataSize                                        | 0                                                  |
|                               | totalSize                                          | 0                                                  |
|                               | transient_lastDdlTime                              | 1674454619                                         |
|                               | NULL                                               | NULL                                               |
| # Storage Information         | NULL                                               | NULL                                               |
| SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL                                               |
| InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat           | NULL                                               |
| OutputFormat:                 | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL                                               |
| Compressed:                   | No                                                 | NULL                                               |
| Num Buckets:                  | 2                                                  | NULL                                               |
| Bucket Columns:               | [a]                                                | NULL                                               |
| Sort Columns:                 | []                                                 | NULL                                               |
| Storage Desc Params:          | NULL                                               | NULL                                               |
|                               | serialization.format                               | 1                                                  |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
33 rows selected (0.172 seconds)

{code}






> bucketing_version properly not coming for like table
> ----------------------------------------------------
>
>                 Key: HIVE-26979
>                 URL: https://issues.apache.org/jira/browse/HIVE-26979
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Satheesh Akuthota
>            Priority: Major
>
> Steps to reproduce : 
> 1.create table T6(a int , b int) clustered by (a) into 2 buckets TBLPROPERTIES ('transient_lastDdlTime'='1521230300);
> 2.create table T6like like T6;
> 3.  desc formatted T6;
> 4.desc formatted T6like;
> Output of desc formatted T6;
> {code:java}
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> |           col_name            |                     data_type                      |                      comment                       |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> | # col_name                    | data_type                                          | comment                                            |
> | a                             | int                                                |                                                    |
> | b                             | int                                                |                                                    |
> |                               | NULL                                               | NULL                                               |
> | # Detailed Table Information  | NULL                                               | NULL                                               |
> | Database:                     | default                                            | NULL                                               |
> | OwnerType:                    | USER                                               | NULL                                               |
> | Owner:                        | hive                                               | NULL                                               |
> | CreateTime:                   | Mon Jan 23 06:16:27 GMT 2023                       | NULL                                               |
> | LastAccessTime:               | UNKNOWN                                            | NULL                                               |
> | Retention:                    | 0                                                  | NULL                                               |
> | Location:                     | hdfs:///warehouse/tablespace/external/hive/t6 | NULL                                               |
> | Table Type:                   | EXTERNAL_TABLE                                     | NULL                                               |
> | Table Parameters:             | NULL                                               | NULL                                               |
> |                               | COLUMN_STATS_ACCURATE                              | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\"}} |
> |                               | EXTERNAL                                           | TRUE                                               |
> |                               | *bucketing_version*                                  | 2                                                  |
> |                               | external.table.purge                               | true                                               |
> |                               | numFiles                                           | 2                                                  |
> |                               | numRows                                            | 6                                                  |
> |                               | rawDataSize                                        | 22                                                 |
> |                               | totalSize                                          | 39                                                 |
> |                               | transient_lastDdlTime                              | 1674454919                                         |
> |                               | NULL                                               | NULL                                               |
> | # Storage Information         | NULL                                               | NULL                                               |
> | SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL                                               |
> | InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat           | NULL                                               |
> | OutputFormat:                 | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL                                               |
> | Compressed:                   | No                                                 | NULL                                               |
> | Num Buckets:                  | 2                                                  | NULL                                               |
> | Bucket Columns:               | [a]                                                | NULL                                               |
> | Sort Columns:                 | []                                                 | NULL                                               |
> | Storage Desc Params:          | NULL                                               | NULL                                               |
> |                               | serialization.format                               | 1                                                  |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> {code}
> Output of desc formatted T6like;
> {code:java}
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> |           col_name            |                     data_type                      |                      comment                       |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> | # col_name                    | data_type                                          | comment                                            |
> | a                             | int                                                |                                                    |
> | b                             | int                                                |                                                    |
> |                               | NULL                                               | NULL                                               |
> | # Detailed Table Information  | NULL                                               | NULL                                               |
> | Database:                     | default                                            | NULL                                               |
> | OwnerType:                    | USER                                               | NULL                                               |
> | Owner:                        | hive                                               | NULL                                               |
> | CreateTime:                   | Mon Jan 23 06:16:59 GMT 2023                       | NULL                                               |
> | LastAccessTime:               | UNKNOWN                                            | NULL                                               |
> | Retention:                    | 0                                                  | NULL                                               |
> | Location:                     | hdfs:///warehouse/tablespace/external/hive/t6like | NULL                                               |
> | Table Type:                   | EXTERNAL_TABLE                                     | NULL                                               |
> | Table Parameters:             | NULL                                               | NULL                                               |
> |                               | COLUMN_STATS_ACCURATE                              | {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"a\":\"true\",\"b\":\"true\"}} |
> |                               | EXTERNAL                                           | TRUE                                               |
> |                               | external.table.purge                               | true                                               |
> |                               | numFiles                                           | 0                                                  |
> |                               | numRows                                            | 0                                                  |
> |                               | rawDataSize                                        | 0                                                  |
> |                               | totalSize                                          | 0                                                  |
> |                               | transient_lastDdlTime                              | 1674454619                                         |
> |                               | NULL                                               | NULL                                               |
> | # Storage Information         | NULL                                               | NULL                                               |
> | SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL                                               |
> | InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat           | NULL                                               |
> | OutputFormat:                 | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL                                               |
> | Compressed:                   | No                                                 | NULL                                               |
> | Num Buckets:                  | 2                                                  | NULL                                               |
> | Bucket Columns:               | [a]                                                | NULL                                               |
> | Sort Columns:                 | []                                                 | NULL                                               |
> | Storage Desc Params:          | NULL                                               | NULL                                               |
> |                               | serialization.format                               | 1                                                  |
> +-------------------------------+----------------------------------------------------+----------------------------------------------------+
> 33 rows selected (0.172 seconds)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)