You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "plusplusjiajia (via GitHub)" <gi...@apache.org> on 2026/10/13 08:59:33 UTC

[PR] feat(datafusion): support timestamp time travel for external engines [paimon-rust]

plusplusjiajia opened a new pull request, #1121:
URL: https://github.com/apache/paimon-rust/pull/1121

   ### Purpose
   
   Support `TIMESTAMP AS OF '<yyyy-MM-dd HH:mm:ss>'` for registered external table engines through `SQLContext` and DataFusion `SessionContext`:
   
   ```sql
   SELECT * FROM lake.db.events TIMESTAMP AS OF '2026-10-01 00:00:00';
   ```
   
   The timestamp is parsed as for Paimon tables (local time zone, epoch milliseconds) and passed to the engine, which picks the snapshot.
   
   Follow-up to #1113.
   
   ### Brief change log
   
   - Add `TableEngineResolver::resolve_table_at_timestamp`. The default implementation rejects timestamp time travel, so existing resolvers keep failing closed.
   - Route `TIMESTAMP AS OF` on external data tables through the same read-only wrapper and session option checks as `VERSION AS OF`.
   - Keep the existing boundaries: `SQLContext` resolves without loading the current provider; raw `SessionContext` preloads it and propagates its errors; branches, metadata tables and `FOR SYSTEM_TIME AS OF` remain rejected.
   - Update the DataFusion README.
   
   ### Tests
   
   - Timestamps at and between engine snapshot commit times reach the resolver as exact epoch milliseconds and return that snapshot's data and schema.
   - `SQLContext` reads a timestamp snapshot when the current provider cannot be resolved; raw `SessionContext` propagates the current resolution error.
   - Missing snapshots, backend failures and missing tables are reported without falling back to current data; resolvers without timestamp support and unregistered engines are rejected.
   - Timestamp providers reject writes; session scan options and query-auth are checked before the engine is called.
   - Paimon tables keep Paimon's own timestamp resolution, including when joined with an external timestamp snapshot.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(datafusion): support timestamp time travel for external engines [paimon-rust]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #1121:
URL: https://github.com/apache/paimon-rust/pull/1121


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org