You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/11/23 21:49:00 UTC
[jira] [Commented] (IMPALA-10288) Functionality to display table
history for Iceberg tables
[ https://issues.apache.org/jira/browse/IMPALA-10288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237711#comment-17237711 ]
ASF subversion and git services commented on IMPALA-10288:
----------------------------------------------------------
Commit b66045c8a5d48c268a4dfad967021ff9bcbdd937 in impala's branch refs/heads/master from Gabor Kaszab
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b66045c ]
IMPALA-10288: Implement DESCRIBE HISTORY for Iceberg tables
The DESCRIBE HISTORY works for Iceberg tables and displays the
snapshot history of the table.
An example output:
DESCRIBE HISTORY iceberg_multi_snapshots;
+----------------------------+---------------------+---------------------+---------------------+
| creation_time | snapshot_id | parent_id | is_current_ancestor |
+----------------------------+---------------------+---------------------+---------------------+
| 2020-10-13 14:01:07.234000 | 4400379706200951771 | NULL | TRUE |
| 2020-10-13 14:01:19.307000 | 4221472712544505868 | 4400379706200951771 | TRUE |
+----------------------------+---------------------+---------------------+---------------------+
The purpose here was to have similar output with this new feature as
what SparkSql returns for "SELECT * from tablename.history".
See "History" section of
https://iceberg.apache.org/spark/#inspecting-tables
Testing:
- iceberg-negative.test was extended to check that DESCRIBE HISTORY
is not applicable for non-Iceberg tables.
- iceberg-table-history.test: Covers basic usage of DESCRIBE
HISTORY. Tests on tables created with Impala and also with Spark.
Change-Id: I56a4b92c27e8e4a79109696cbae62735a00750e5
Reviewed-on: http://gerrit.cloudera.org:8080/16599
Reviewed-by: Zoltan Borok-Nagy <bo...@cloudera.com>
Reviewed-by: wangsheng <sk...@163.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Functionality to display table history for Iceberg tables
> ---------------------------------------------------------
>
> Key: IMPALA-10288
> URL: https://issues.apache.org/jira/browse/IMPALA-10288
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Reporter: Gabor Kaszab
> Assignee: Gabor Kaszab
> Priority: Major
>
> For time travel it would be useful to display the table history (previous snapshot, creation times etc.) so that the user can pick which snapshot to use.
> Spark already has this functionality:
> https://iceberg.apache.org/spark/#time-travel
> {code:java}
> SELECT * FROM prod.db.table.history
> {code}
> {code:java}
> +-------------------------+---------------------+---------------------+---------------------+
> | made_current_at | snapshot_id | parent_id | is_current_ancestor |
> +-------------------------+---------------------+---------------------+---------------------+
> | 2019-02-08 03:29:51.215 | 5781947118336215154 | NULL | true |
> | 2019-02-08 03:47:55.948 | 5179299526185056830 | 5781947118336215154 | true |
> | 2019-02-09 16:24:30.13 | 296410040247533544 | 5179299526185056830 | false |
> | 2019-02-09 16:32:47.336 | 2999875608062437330 | 5179299526185056830 | true |
> | 2019-02-09 19:42:03.919 | 8924558786060583479 | 2999875608062437330 | true |
> | 2019-02-09 19:49:16.343 | 6536733823181975045 | 8924558786060583479 | true |
> +-------------------------+---------------------+---------------------+---------------------+
> {code}
> We can have the same output columns for the Impala implementation, however, the actual syntax to run the query is not decided yet.
> There are various solutions on the field:
> 1) SELECT * from tablename.history; (Spark)
> 2) DESCRIBE HISTORY tablename; (Delta)
> 3) select * from 'table.history'; (Presto)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org