[GitHub] [iceberg] singhpk234 commented on a diff in pull request #5160: Spark 3.3: [FOLLOWUP] TimeTravel using SPARK SQL and DataFrameReaders

[GitBox <gi...@apache.org>]

singhpk234 commented on code in PR #5160:
URL: https://github.com/apache/iceberg/pull/5160#discussion_r911616713


##########
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/IcebergSource.java:
##########
@@ -173,7 +174,19 @@ public Optional<String> extractTimeTravelVersion(CaseInsensitiveStringMap option
 
   @Override
   public Optional<String> extractTimeTravelTimestamp(CaseInsensitiveStringMap options) {
-    return Optional.ofNullable(PropertyUtil.propertyAsString(options, "timestampAsOf", null));
+    String timestampAsOf = PropertyUtil.propertyAsString(options, "timestampAsOf", null);
+    if (timestampAsOf == null) {
+      return Optional.empty();
+    }
+
+    try {
+      // timestamp provided should be at a seconds precision.
+      // TODO: remove once https://issues.apache.org/jira/browse/SPARK-39633 is resolved
+      long timestampAsOfAsLong = Long.parseLong(timestampAsOf);
+      return Optional.of(DateTimeUtil.formatTimestampMillisWithLocalTime(timestampAsOfAsLong * 1000));

Review Comment:
   Makes sense, calling spark api's via iceberg will also not be ok i think then as this now indirectly puts the translation to iceberg (as which spark api to use). Should I  add a TODO in the code with the issue linking it and add the UT added in this PR in ignore as soon as we upgrade to 3.3.1 we will have it (The PR for [SPARK-39633](https://issues.apache.org/jira/browse/SPARK-39633) is merged in upstream). 
   
   Meanwhile we have existing ways to TT via dataframe options we can specify [as-of-timestamp](https://iceberg.apache.org/docs/latest/spark-queries/#time-travel) in milliseconds.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org