You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "hililiwei (via GitHub)" <gi...@apache.org> on 2023/07/04 12:48:34 UTC

[GitHub] [incubator-paimon] hililiwei opened a new pull request, #1497: [test][doc] Spark time travel with data frame test and doc

hililiwei opened a new pull request, #1497:
URL: https://github.com/apache/incubator-paimon/pull/1497

   <!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
   
   ### Purpose
   
   <!-- Linking this pull request to the issue -->
   Linked issue: close #xxx
   
   <!-- What is the purpose of the change -->
   add test and doc for spark time travel with data frame 
   
   ### Tests
   
   <!-- List UT and IT cases to verify this change -->
   org.apache.paimon.spark.SparkTimeTravelWithDataFrameITCase
   
   
   ### API and Format
   
   <!-- Does this change affect API or storage format -->
   
   ### Documentation
   
   <!-- Does this change introduce a new feature -->
   
   Add quering with data frame in spark3.md


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-paimon] JingsongLi merged pull request #1497: [test][doc] Spark time travel with data frame test and doc

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #1497:
URL: https://github.com/apache/incubator-paimon/pull/1497


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-paimon] JingsongLi commented on a diff in pull request #1497: [test][doc] Spark time travel with data frame test and doc

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #1497:
URL: https://github.com/apache/incubator-paimon/pull/1497#discussion_r1253828274


##########
docs/content/engines/spark3.md:
##########
@@ -205,6 +205,50 @@ dataset.createOrReplaceTempView("my_table")
 spark.sql("SELECT * FROM my_table").show()
 ```
 
+## Time travel
+
+### SQL
+
+Spark 3.3 and later supports time travel in SQL queries using `TIMESTAMP AS OF` or `VERSION AS OF` clauses.
+
+
+```sql 
+-- read the snapshot with id 1L (use snapshot id as version)
+SELECT * FROM t VERSION AS OF 1;
+
+-- read the snapshot from specified timestamp 
+SELECT * FROM t TIMESTAMP AS OF '2023-06-01 00:00:00.123';
+
+-- read the snapshot from specified timestamp in unix seconds
+SELECT * FROM t TIMESTAMP AS OF 1678883047;
+
+-- read tag 'my-tag'
+SELECT * FROM t VERSION AS OF 'my-tag';
+```
+
+### DataFrame
+
+To select a specific table snapshot or the snapshot at some time in the DataFrame API, Paimon supports:

Review Comment:
   Can we just put this into query-table page? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-paimon] hililiwei commented on a diff in pull request #1497: [test][doc] Spark time travel with data frame test and doc

Posted by "hililiwei (via GitHub)" <gi...@apache.org>.
hililiwei commented on code in PR #1497:
URL: https://github.com/apache/incubator-paimon/pull/1497#discussion_r1253985046


##########
docs/content/engines/spark3.md:
##########
@@ -205,6 +205,50 @@ dataset.createOrReplaceTempView("my_table")
 spark.sql("SELECT * FROM my_table").show()
 ```
 
+## Time travel
+
+### SQL
+
+Spark 3.3 and later supports time travel in SQL queries using `TIMESTAMP AS OF` or `VERSION AS OF` clauses.
+
+
+```sql 
+-- read the snapshot with id 1L (use snapshot id as version)
+SELECT * FROM t VERSION AS OF 1;
+
+-- read the snapshot from specified timestamp 
+SELECT * FROM t TIMESTAMP AS OF '2023-06-01 00:00:00.123';
+
+-- read the snapshot from specified timestamp in unix seconds
+SELECT * FROM t TIMESTAMP AS OF 1678883047;
+
+-- read tag 'my-tag'
+SELECT * FROM t VERSION AS OF 'my-tag';
+```
+
+### DataFrame
+
+To select a specific table snapshot or the snapshot at some time in the DataFrame API, Paimon supports:

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org