You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "Yohahaha (via GitHub)" <gi...@apache.org> on 2026/10/21 08:57:47 UTC

[PR] [spark] Add commit-time snapshot TVF [fluss]

Yohahaha opened a new pull request, #4778:
URL: https://github.com/apache/fluss/pull/4778

   <!-- Generated-by: OpenAI Codex (gpt-6.1-sol implementation; Codex review) following https://github.com/apache/fluss/blob/main/AGENTS.md -->
   
   ### Purpose
   
   Linked issue: close #4776
   
   Support complete table-state queries at a Fluss commit-time boundary from Spark, such as a historical midnight. Fluss may retain only the recent log, while Paimon snapshots do not align with the requested time. The new TVF restores a retained Paimon base plus bounded Fluss replay instead of treating a retained changelog tail as a complete primary-key table.
   
   ```sql
   SELECT * FROM fluss_snapshot_at_timestamp('fluss_catalog.db.orders', DATE '2026-10-20');
   SELECT * FROM fluss_snapshot_at_timestamp('fluss_catalog.db.orders', now());
   ```
   
   The boundary is exclusive (`commit timestamp < T`). Future timestamps are rejected, and `now()` / `current_timestamp()` are fixed during analysis.
   
   ### Brief change log
   
   - Register `fluss_snapshot_at_timestamp(table, timestamp)` through the existing Spark TVF and catalog resolution path.
   - Calibrate per-bucket timestamp offsets against a fixed committed prefix and select an offset-compatible Paimon snapshot through an optional internal lake capability.
   - Reuse existing append union and primary-key lake merge readers; apply filters and limits after restoration and preserve log-table duplicates.
   - Fix bounded-reader handling of exclusive stops, progress-only batches, empty replay intervals and failure cleanup; validate table identity/schema during execution.
   - Fail immediately with table, snapshot ID and path when a candidate references a missing `fluss-offsets` file. Do not fall back to an older candidate.
   
   Initial scope is Spark batch on non-partitioned tables with stable schema and bucket mappings, Paimon maintained exclusively by Fluss tiering, and deduplicate primary-key tables without KV TTL. FULL and WAL after-images are supported; DV candidates require complete offsets and no L0 files. Streaming, historical DDL, custom merge engines and `read.optimized` are outside this scope.
   
   **Retention limitation:** a retained Paimon snapshot alone is insufficient. Its referenced offsets file and the required Fluss replay interval must also exist. Ordinary non-DV tiering currently removes older offsets files when a newer base is registered. This PR does not change retention or add archival, and therefore does not guarantee arbitrary historical dates under the default lifecycle.
   
   ### Tests
   
   - 22-module dependency build passed with Java 11, Spark 3.5 and Paimon 2.1.0; RAT, Checkstyle and Spotless enabled.
   - 42 focused tests passed:
     - `LakeSnapshotAndLogSplitScannerTest` and `KvSnapshotAndLogBatchScannerTest`: 6.
     - `PaimonHistoricalSnapshotSelectorTest`, `PaimonLakeSourceTest`, `PaimonSplitPlannerTest`, `PaimonSortedRecordReaderTest`: 15.
     - `TimestampBoundaryResolverTest`: 7.
     - `SparkSnapshotAtTimestampTvfTest`: 4.
     - `SparkTimeRangeTvfTest` and `SparkLakePaimonTimeRangeReadTest`: 10.
   - After tightening the missing-offsets behavior, reran the selector and snapshot SQL suites: all 12 tests passed.
   - Snapshot SQL tests exercise updates, deletes, unchanged keys, filters/projection/aggregation/LIMIT, duplicate log rows, WAL partial updates, `now()`, future-time rejection and missing-history errors.
   - The primary-key historical test first verifies the real default failure after old offsets are deleted, then restores the original offsets file **only in the fixture** to test recovery when the required metadata is retained.
   - Rebased onto `b25ba46d8`; rebuilt dependencies and reran all four snapshot SQL cases on the rebased commit.
   - `git diff --check` passed. Full-repository tests, performance benchmarks and real DV SQL end-to-end tests were not run.
   
   ### API and Format
   
   Adds one Spark SQL TVF and an internal query-local `scan.as-of.timestamp` transport option. Adds an optional internal lake-source capability; no server RPC, storage-format or tiering-retention changes.
   
   ### Documentation
   
   Adds the “Snapshot at a Commit Timestamp” section to `website/docs/engine-spark/reads.md`, covering examples, timestamp semantics, supported scope and retention limitations.
   
   Generative AI disclosure: **Yes — OpenAI Codex**, including gpt-6.1-sol for implementation and Codex-assisted review.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "KnightChess (via GitHub) via issues" <is...@fluss.apache.org>.
KnightChess commented on PR #4778:
URL: https://github.com/apache/fluss/pull/4778#issuecomment-6578790076

   @Yohahaha A snapshot at T, followed by replaying a changelog window [S, E), where S can be later than T.
   A snapshot at T, followed by replaying changelog continuously from T to any later E—either in one window [T, E) or in consecutive windows without gaps.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "KnightChess (via GitHub) via issues" <is...@fluss.apache.org>.
KnightChess commented on PR #4778:
URL: https://github.com/apache/fluss/pull/4778#issuecomment-6594622842

   @Yohahaha
   > if fluss_snapshot_at_timestamp becomes available, would querying snapshots at the required timestamps be sufficient for your use case?
   
   yes
   
   > It sounds like your goal is to obtain the table state at each target time. Do you really need the changelog to incrementally update an existing downstream state, or would reading the complete snapshot at each timestamp also work?
   
   There are two phases: a one-time initialization phase and multiple incremental pull-and-update phases. Snapshot reads are performed at a certain timestamp during initialization, followed by periodically scheduled incremental reads. One important note: the incremental read range should be **(T, E]**, not **[T, E)**, since data at timestamp T has already been ingested in initialization and must not be re-pulled in incremental loads.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "KnightChess (via GitHub) via issues" <is...@fluss.apache.org>.
KnightChess commented on PR #4778:
URL: https://github.com/apache/fluss/pull/4778#issuecomment-6578782642

   @Yohahaha 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "Yohahaha (via GitHub) via issues" <is...@fluss.apache.org>.
Yohahaha commented on PR #4778:
URL: https://github.com/apache/fluss/pull/4778#issuecomment-6576648865

   > Hi @Yohahaha , really appreciate the effort on this PR.
   > 
   > We have a recommended/online feature pipeline that strongly depends on this capability:
   > 
   > 1. Downstream initialization step: query the table snapshot at a specific timestamp to build the base state;
   > 2. Periodic offline batch job: reuse exactly that timestamp as the start position, read changelog within [startTs, endTs] time range.
   > 
   > The key requirement is **time-boundary consistency**: the snapshot view at T and the changelog starting from T must match, so we can safely combine snapshot + time-range changelog without hole or duplication. Currently we don't have a native way to bind snapshot timestamp and changelog start timestamp together.
   > 
   > Just let us know if there are any design questions or things we can assist with. Thanks!
   
   hi @KnightChess  thanks for sharing the use case.
   
   could you clarify which replay pattern you need?
   
   1. A snapshot at T, followed by replaying a changelog window [S, E), where S can be later than T.
   2. A snapshot at T, followed by replaying changelog continuously from T to any later E—either in one window [T, E) or in consecutive windows without gaps.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "KnightChess (via GitHub) via issues" <is...@fluss.apache.org>.
KnightChess commented on PR #4778:
URL: https://github.com/apache/fluss/pull/4778#issuecomment-6574436181

   Hi @Yohahaha , really appreciate the effort on this PR.
   
   We have a recommended/online feature pipeline that strongly depends on this capability:
   1. Downstream initialization step: query the table snapshot at a specific timestamp to build the base state;
   2. Periodic offline batch job: reuse exactly that timestamp as the start position, read changelog within [startTs, endTs] time range.
   
   The key requirement is **time-boundary consistency**: the snapshot view at T and the changelog starting from T must match, so we can safely combine snapshot + time-range changelog without hole or duplication.
   Currently we don't have a native way to bind snapshot timestamp and changelog start timestamp together.
   
   Just let us know if there are any design questions or things we can assist with.
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "Yohahaha (via GitHub) via issues" <is...@fluss.apache.org>.
Yohahaha commented on PR #4778:
URL: https://github.com/apache/fluss/pull/4778#issuecomment-6593760554

   > @Yohahaha
   > 
   > > A snapshot at T, followed by replaying a changelog window [S, E), where S can be later than T.
   > > A snapshot at T, followed by replaying changelog continuously from T to any later E—either in one window [T, E) or in consecutive windows without gaps.
   > 
   > second, we perform strict state computation and need to guarantee exactly-once semantics for the data
   
   @KnightChess
   if `fluss_snapshot_at_timestamp` becomes available, would querying snapshots at the required timestamps be sufficient for your use case?
   
   It sounds like your goal is to obtain the table state at each target time. Do you really need the changelog to incrementally update an existing downstream state, or would reading the complete snapshot at each timestamp also work?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "KnightChess (via GitHub) via issues" <is...@fluss.apache.org>.
KnightChess commented on PR #4778:
URL: https://github.com/apache/fluss/pull/4778#issuecomment-6578825935

   @Yohahaha 
   > A snapshot at T, followed by replaying a changelog window [S, E), where S can be later than T.
   > A snapshot at T, followed by replaying changelog continuously from T to any later E—either in one window [T, E) or in consecutive windows without gaps.
   second, we perform strict state computation and need to guarantee exactly-once semantics for the data


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Add commit-time snapshot TVF [fluss]

Posted by "beryllw (via GitHub) via issues" <is...@fluss.apache.org>.
beryllw commented on code in PR #4778:
URL: https://github.com/apache/fluss/pull/4778#discussion_r4522869301


##########
fluss-spark/fluss-spark-common/src/main/scala/org/apache/fluss/spark/catalyst/plans/logical/FlussTableValuedFunctions.scala:
##########
@@ -256,3 +267,24 @@ case class IncrementalBetweenTimestamp(override val args: Seq[Expression])
     FlussOffsetInitializers.requireValidWindow(start, end, startMs, endMs)
   }
 }
+
+/** Plan for a complete snapshot with commit timestamp strictly before the requested time. */
+case class SnapshotAtTimestamp(override val args: Seq[Expression])
+  extends FlussTableValueFunction(SNAPSHOT_AT_TIMESTAMP) {
+
+  override def parseArgs(argsWithoutTable: Seq[Expression]): Map[String, String] = {
+    require(
+      argsWithoutTable.size == 1,
+      s"$SNAPSHOT_AT_TIMESTAMP requires exactly two arguments: a table identifier and a timestamp.")
+    val queryNowMs = System.currentTimeMillis()

Review Comment:
   Each TVF evaluates System.currentTimeMillis() independently, so two now() calls in the same SQL can produce different snapshot boundaries. Please use one query-level timestamp for both argument evaluation and future-time validation.



##########
fluss-spark/fluss-spark-common/src/main/scala/org/apache/fluss/spark/catalyst/plans/logical/FlussTableValuedFunctions.scala:
##########
@@ -256,3 +267,24 @@ case class IncrementalBetweenTimestamp(override val args: Seq[Expression])
     FlussOffsetInitializers.requireValidWindow(start, end, startMs, endMs)
   }
 }
+
+/** Plan for a complete snapshot with commit timestamp strictly before the requested time. */
+case class SnapshotAtTimestamp(override val args: Seq[Expression])
+  extends FlussTableValueFunction(SNAPSHOT_AT_TIMESTAMP) {
+
+  override def parseArgs(argsWithoutTable: Seq[Expression]): Map[String, String] = {
+    require(
+      argsWithoutTable.size == 1,
+      s"$SNAPSHOT_AT_TIMESTAMP requires exactly two arguments: a table identifier and a timestamp.")
+    val queryNowMs = System.currentTimeMillis()
+    val key = SparkFlussConf.SCAN_AS_OF_TIMESTAMP.key()
+    val timestamp = FlussOffsetInitializers.parseTimestamp(
+      toTimestampOptionValue(SNAPSHOT_AT_TIMESTAMP, argsWithoutTable.head, Some(queryNowMs)),

Review Comment:
   Converting TIMESTAMP values to milliseconds truncates the boundary and can silently omit rows. A record at 1000ms should be included when T is 1000.5ms, but currently it is excluded. Please preserve the strict < T semantics or reject unsupported precision explicitly.



##########
fluss-lake/fluss-lake-paimon/src/main/java/org/apache/fluss/lake/paimon/source/PaimonHistoricalSnapshotSelector.java:
##########
@@ -0,0 +1,205 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.fluss.lake.paimon.source;
+
+import org.apache.fluss.fs.FSDataInputStream;
+import org.apache.fluss.fs.FsPath;
+import org.apache.fluss.lake.source.LakeSource.HistoricalSnapshot;
+import org.apache.fluss.metadata.TableBucket;
+import org.apache.fluss.metadata.TableInfo;
+import org.apache.fluss.utils.IOUtils;
+import org.apache.fluss.utils.json.TableBucketOffsets;
+
+import org.apache.paimon.Snapshot;
+import org.apache.paimon.manifest.FileKind;
+import org.apache.paimon.table.FileStoreTable;
+import org.apache.paimon.utils.SnapshotManager;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.ByteArrayOutputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+import static org.apache.fluss.lake.committer.LakeCommitter.FLUSS_LAKE_SNAP_BUCKET_OFFSET_PROPERTY;
+import static org.apache.fluss.lake.paimon.tiering.PaimonLakeTieringFactory.FLUSS_LAKE_TIERING_COMMIT_USER;
+import static org.apache.fluss.lake.paimon.utils.PaimonConversions.toFlussRowType;
+
+/** Selects an offset-compatible complete historical base without scanning table data. */
+final class PaimonHistoricalSnapshotSelector {
+    private static final Logger LOG =
+            LoggerFactory.getLogger(PaimonHistoricalSnapshotSelector.class);
+
+    private final FileStoreTable table;
+
+    PaimonHistoricalSnapshotSelector(FileStoreTable table) {
+        this.table = table;
+    }
+
+    Optional<HistoricalSnapshot> select(
+            TableInfo tableInfo, Map<TableBucket, Long> earliest, Map<TableBucket, Long> target)
+            throws IOException {
+        if (!toFlussRowType(table.rowType()).equals(tableInfo.getRowType())
+                || !table.primaryKeys().equals(tableInfo.getPrimaryKeys())
+                || !table.partitionKeys().isEmpty()) {
+            throw new IOException(
+                    "Historical snapshot schema or keys differ from Fluss table "
+                            + tableInfo.getTableId());
+        }
+        if (!table.primaryKeys().isEmpty()
+                && (!"deduplicate"
+                                .equals(table.options().getOrDefault("merge-engine", "deduplicate"))
+                        || Integer.parseInt(table.options().getOrDefault("bucket", "-1"))
+                                != tableInfo.getNumBuckets())) {
+            throw new IOException(
+                    "Historical reads require deduplicate merge engine and stable buckets");
+        }
+        SnapshotManager manager = table.snapshotManager();
+        Long latest = manager.latestSnapshotId();
+        Long first = manager.earliestSnapshotId();
+        if (latest == null || first == null) {
+            return Optional.empty();
+        }
+        List<Snapshot> snapshots = new ArrayList<>();
+        // Validate the whole retained lineage before accepting a candidate. A Fluss commit can
+        // otherwise inherit data from a preceding foreign writer or an overwrite.
+        for (long id = latest; id >= first; id--) {
+            Snapshot snapshot;
+            try {
+                snapshot = manager.tryGetSnapshot(id);
+            } catch (FileNotFoundException e) {
+                continue;
+            }
+            if (snapshot == null) {
+                continue;
+            }
+            if (snapshot.commitKind() == Snapshot.CommitKind.OVERWRITE
+                    || (snapshot.commitKind() == Snapshot.CommitKind.APPEND
+                            && !snapshot.commitUser().startsWith(FLUSS_LAKE_TIERING_COMMIT_USER))) {
+                throw new IOException(
+                        "Unsupported historical lineage: snapshot "
+                                + id
+                                + " was written by "
+                                + snapshot.commitUser()
+                                + " with "
+                                + snapshot.commitKind());
+            }
+            snapshots.add(snapshot);
+        }
+        Map<String, TableBucketOffsets> cachedOffsets = new HashMap<>();
+        for (Snapshot snapshot : snapshots) {
+            if (snapshot.schemaId() != table.schema().id()) {
+                continue;
+            }
+            String offsetsPath =
+                    snapshot.properties() == null
+                            ? null
+                            : snapshot.properties().get(FLUSS_LAKE_SNAP_BUCKET_OFFSET_PROPERTY);
+            if (offsetsPath == null) {
+                continue;
+            }
+            TableBucketOffsets offsets = cachedOffsets.get(offsetsPath);
+            if (offsets == null) {
+                try {
+                    offsets = readOffsets(offsetsPath);
+                } catch (FileNotFoundException e) {
+                    throw new IOException(
+                            "Missing fluss-offsets file: table="
+                                    + tableInfo.getTablePath()
+                                    + ", tableId="
+                                    + tableInfo.getTableId()
+                                    + ", snapshotId="
+                                    + snapshot.id()
+                                    + ", path="
+                                    + offsetsPath,
+                            e);
+                }
+                cachedOffsets.put(offsetsPath, offsets);
+            }
+            if (offsets.getTableId() != tableInfo.getTableId()) {
+                throw new IOException(
+                        "Historical offsets table identity mismatch at snapshot " + snapshot.id());
+            }
+            if (!compatible(offsets.getOffsets(), earliest, target)) {
+                continue;
+            }
+            if (Boolean.parseBoolean(
+                            table.options().getOrDefault("deletion-vectors.enabled", "false"))
+                    && table
+                            .store()
+                            .newScan()
+                            .withSnapshot(snapshot.id())
+                            .plan()
+                            .files(FileKind.ADD)
+                            .stream()
+                            .anyMatch(entry -> entry.level() == 0)) {
+                // The normal DV reader skips L0. Tiered offsets are readable only if no L0 exists.
+                continue;
+            }
+            LOG.info(
+                    "Selected historical snapshot {} for table {} after checking {} retained versions",
+                    snapshot.id(),
+                    tableInfo.getTableId(),
+                    snapshots.size());
+            return Optional.of(new HistoricalSnapshot(snapshot.id(), offsets.getOffsets()));
+        }
+        return Optional.empty();
+    }
+
+    private TableBucketOffsets readOffsets(String value) throws IOException {
+        if (value.trim().startsWith("{")) {
+            throw new IOException("Unsupported legacy embedded historical offsets: " + value);
+        }
+        FsPath path = new FsPath(value);
+        try (FSDataInputStream in = path.getFileSystem().open(path);
+                ByteArrayOutputStream out = new ByteArrayOutputStream()) {
+            IOUtils.copyBytes(in, out, false);
+            try {
+                return TableBucketOffsets.fromJsonBytes(out.toByteArray());
+            } catch (RuntimeException e) {
+                throw new IOException("Corrupt historical offsets file " + path, e);
+            }
+        }
+    }
+
+    static boolean compatible(
+            Map<TableBucket, Long> offsets,
+            Map<TableBucket, Long> earliest,
+            Map<TableBucket, Long> target) {
+        if (!offsets.keySet().equals(target.keySet())

Review Comment:
   The exact key-set comparison rejects valid snapshots when never-written buckets are absent from the offsets file. This reproduces with a two-bucket table containing one row. Please handle provably empty buckets without treating arbitrary missing offsets as zero.



##########
fluss-spark/fluss-spark-ut/src/test/scala/org/apache/fluss/spark/lake/SparkSnapshotAtTimestampTvfTest.scala:
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.fluss.spark.lake
+
+import org.apache.fluss.client.admin.OffsetSpec
+import org.apache.fluss.config.{ConfigOptions, Configuration}
+import org.apache.fluss.metadata.{DataLakeFormat, TableBucket}
+import org.apache.fluss.record.ChangeType
+import org.apache.fluss.row.GenericRow
+import org.apache.fluss.spark.SparkConnectorOptions.{BUCKET_NUMBER, PRIMARY_KEY}
+import org.apache.fluss.spark.SparkFlussConf
+import org.apache.fluss.spark.read.lake.FlussLakeUpsertInputPartition
+
+import org.apache.spark.sql.Row
+import org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation
+import org.assertj.core.api.Assertions.assertThat
+
+import java.io.IOException
+import java.nio.file.{Files, Paths}
+import java.time.Duration
+
+import scala.collection.JavaConverters._
+import scala.collection.mutable
+
+/** Exercises the SQL entry point against real Paimon bases and Fluss commit timestamps. */
+@SparkLakeTest
+class SparkSnapshotAtTimestampTvfTest extends SparkLakeTableReadTestBase {
+  private val TVF = "fluss_snapshot_at_timestamp"
+  private val key = SparkFlussConf.SCAN_AS_OF_TIMESTAMP.key()
+
+  override protected def dataLakeFormat: DataLakeFormat = DataLakeFormat.PAIMON
+
+  override protected def flussConf: Configuration = {
+    val config = super.flussConf
+    config.setString("datalake.format", "paimon")
+    config.setString("datalake.paimon.metastore", "filesystem")
+    config.setString("datalake.paimon.cache-enabled", "false")
+    warehousePath = Files.createTempDirectory("snapshot-tvf-paimon").resolve("warehouse").toString
+    config.setString("datalake.paimon.warehouse", warehousePath)
+    config
+  }
+
+  override protected def lakeCatalogConf: Configuration = {
+    val config = new Configuration()
+    config.setString("metastore", "filesystem")
+    config.setString("warehouse", warehousePath)
+    config
+  }
+
+  private def create(name: String, primaryKey: Boolean): Unit = {
+    val pk = if (primaryKey) s", '${PRIMARY_KEY.key()}'='id'" else ""
+    sql(s"""CREATE TABLE $DEFAULT_DATABASE.$name (id INT, v INT)
+           | TBLPROPERTIES ('${ConfigOptions.TABLE_DATALAKE_ENABLED.key()}'='true',
+           | '${ConfigOptions.TABLE_DATALAKE_FRESHNESS.key()}'='1s',
+           | '${BUCKET_NUMBER.key()}'='1' $pk)""".stripMargin)
+  }
+
+  private def latest(name: String): Map[TableBucket, Long] = {
+    val path = createTablePath(name)
+    val id = admin.getTableInfo(path).get().getTableId
+    admin
+      .listOffsets(path, Seq(Int.box(0)).asJava, new OffsetSpec.LatestSpec())
+      .all()
+      .get()
+      .asScala
+      .map { case (bucket, offset) => new TableBucket(id, bucket) -> offset.longValue() }
+      .toMap
+  }
+
+  private def logs(name: String): Seq[(Long, Long, ChangeType, Int, Int)] = {
+    val table = loadFlussTable(createTablePath(name))
+    val scanner = table.newScan().createLogScanner()
+    try {
+      val stop = latest(name).values.head
+      scanner.subscribe(0, 0L)
+      val result = mutable.ArrayBuffer.empty[(Long, Long, ChangeType, Int, Int)]
+      var consumed = 0L
+      val deadline = System.nanoTime() + Duration.ofSeconds(30).toNanos
+      while (consumed < stop && System.nanoTime() < deadline) {
+        val batch = scanner.poll(Duration.ofSeconds(1))
+        batch.iterator().asScala.foreach {
+          r =>
+            result += ((
+              r.logOffset(),
+              r.timestamp(),
+              r.getChangeType,
+              r.getRow.getInt(0),
+              r.getRow.getInt(1)))
+            consumed = r.logOffset() + 1
+        }
+        Option(batch.consumedUpToOffset(new TableBucket(table.getTableInfo.getTableId, 0)))
+          .foreach(offset => consumed = math.max(consumed, offset.longValue()))
+      }
+      assertThat(Long.box(consumed)).isGreaterThanOrEqualTo(Long.box(stop))
+      result.toSeq
+    } finally {
+      scanner.close()
+      table.close()
+    }
+  }
+
+  test("SQL snapshot restores an older base plus exact bounded upserts and deletes") {
+    withTable("snapshot_pk") {
+      val name = "snapshot_pk"
+      create(name, primaryKey = true)
+      val table = loadFlussTable(createTablePath(name))
+      val writer = table.newUpsert().createWriter()
+      try {
+        Seq((1, 10), (2, 10), (3, 10), (9, 99)).foreach {
+          case (id, value) => writer.upsert(GenericRow.of(Int.box(id), Int.box(value))).get()
+        }
+        writer.flush()
+        tierToLake(name)
+        val base = admin.getReadableLakeSnapshot(createTablePath(name)).get()
+        val baseOffsetsPath = Paths.get(
+          flussServer.getZooKeeperClient
+            .getLakeTable(table.getTableInfo.getTableId)
+            .get()
+            .getLatestLakeSnapshotMetadata
+            .getTieredOffsetsFilePath
+            .toUri)
+        val retainedOffsetBytes = Files.readAllBytes(baseOffsetsPath)
+        writer.upsert(GenericRow.of(Int.box(1), Int.box(20))).get()
+        writer.delete(GenericRow.of(Int.box(2), null)).get()
+        writer.upsert(GenericRow.of(Int.box(4), Int.box(20))).get()
+        writer.flush()
+        val beforeMarker = logs(name).map(_._2).max
+        while (System.currentTimeMillis() <= beforeMarker) {
+          Thread.`yield`()
+        }
+        writer.upsert(GenericRow.of(Int.box(1), Int.box(30))).get()
+        writer.flush()
+        val timestamp = logs(name)
+          .filter(
+            r =>
+              r._4 == 1 && r._5 == 30 &&
+                r._3 != ChangeType.UPDATE_BEFORE)
+          .map(_._2)
+          .max
+        writer.upsert(GenericRow.of(Int.box(4), Int.box(40))).get()
+        writer.upsert(GenericRow.of(Int.box(2), Int.box(50))).get()
+        writer.flush()
+        tierToLake(name)
+        assertThat(
+          Long.box(admin.getReadableLakeSnapshot(createTablePath(name)).get().getSnapshotId))
+          .isGreaterThan(Long.box(base.getSnapshotId))
+
+        // Ordinary non-DV tiering currently discards older offsets even while Paimon retains
+        // the snapshots. Exercise that real failure before explicitly modeling retained metadata.
+        assertThat(Files.exists(baseOffsetsPath)).isFalse()
+        val missing = intercept[IOException] {
+          sql(s"SELECT * FROM $TVF('$name', $timestamp)").collect()
+        }
+        assertThat(missing.getMessage)
+          .contains(
+            "Missing fluss-offsets file",
+            s"snapshotId=${base.getSnapshotId}",
+            baseOffsetsPath.toString)
+        // Fixture only: restore the exact original offset file, representing an independently
+        // retained complete base. Production code never recreates or guesses deleted offsets.
+        Files.write(baseOffsetsPath, retainedOffsetBytes)

Review Comment:
   Normal non-DV tiering deletes older offset files, so retaining Paimon snapshots alone does not make historical queries work. The integration test manually restores these files. Although documented, this limits the main use case. What is the supported retention strategy?



##########
fluss-spark/fluss-spark-ut/src/test/scala/org/apache/fluss/spark/read/TimestampBoundaryResolverTest.scala:
##########
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.fluss.spark.read
+
+import org.apache.fluss.client.table.scanner.ScanRecord
+import org.apache.fluss.client.table.scanner.log.{LogScanner, ScanRecords}
+import org.apache.fluss.metadata.TableBucket
+import org.apache.fluss.record.ChangeType
+import org.apache.fluss.row.GenericRow
+
+import org.assertj.core.api.Assertions.assertThat
+import org.mockito.ArgumentMatchers.any
+import org.mockito.Mockito.{mock, verify, when}

Review Comment:
   Please add coverage for empty buckets, multiple TVFs in one query, and sub-millisecond timestamps. The plan-pinning test should actually create a newer snapshot between planning attempts. Please also replace new Mockito usage with lightweight test implementations and bound the clock-wait loops.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org