You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "Xiao-zhen-Liu (via GitHub)" <gi...@apache.org> on 2026/06/22 05:29:26 UTC
[I] [Cache PR 1] Storage foundation: cache table, read/write code, and cache key [texera]
Xiao-zhen-Liu opened a new issue, #5882:
URL: https://github.com/apache/texera/issues/5882
Parent: #5881 ยท Design: #5880
## Goal
Add the storage layer for the cache: a table to hold cache entries, the code that reads and writes it, and the cache-key computation that identifies the computation behind an output port. Adds code only; nothing uses it during execution yet, so it changes no behavior.
## What is included
- New table `operator_port_cache`, primary key `(workflow_id, global_port_id, subdag_hash)`. Columns: the cache key and its hash, the result location, an optional tuple count, the source execution, and a database-managed updated-at timestamp. Added to the schema file and as a migration.
- `OperatorPortCacheDao`: get by key, list by workflow, upsert, delete.
- `OperatorPortCacheService`: look up matched ports for a plan, write an entry when a port finishes, and invalidate entries. Computes cache keys and serializes the port id.
- The cache-key computation: from an output port's upstream operators, their parameters, their output schemas, and the wiring, produce a value whose hash is the cache key. Same computation gives the same key; any upstream edit changes it.
- The `CachedOutput` type and a defaulted, empty `cachedOutputs` field on `WorkflowSettings`.
## Why this is safe
The table starts empty and nothing reads or writes it during execution in this PR. The new `WorkflowSettings` field defaults to empty. The cache-key computation is a pure function.
## Generated code
The DAO uses generated jOOQ classes for the new table; generated classes are not committed, so this PR ships the `.sql` schema and migration and the build regenerates the rest. Adding to the schema file sends an automatic dev-list notification, so I will post a short heads-up about the table first.
## Depends on
Nothing. PRs 3 and 4 depend on this.
## Out of scope
No cost or eviction columns, and no execution-time use of the cache (PRs 3 and 4).
## Size
About 900 lines of code, plus tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Cache PR 1] Storage foundation: cache table, read/write code, and cache key [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #5882:
URL: https://github.com/apache/texera/issues/5882#issuecomment-4765203146
<!-- texera:template-compliance -->
๐ Thanks for opening this issue, @Xiao-zhen-Liu!
It looks like the issue description doesn't quite follow our template yet:
- This issue doesn't appear to use one of our templates (Bug, Feature, or Task). Please open it using a template so the required details are captured.
Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed.
You can find the template prompts by editing the description, or see [CONTRIBUTING.md](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) for the full contribution flow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add operator output port cache storage (table and cache key) [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on issue #5882:
URL: https://github.com/apache/texera/issues/5882#issuecomment-4860271417
is this done? @Xiao-zhen-Liu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add operator output port cache storage (table and cache key) [texera]
Posted by "Xiao-zhen-Liu (via GitHub)" <gi...@apache.org>.
Xiao-zhen-Liu commented on issue #5882:
URL: https://github.com/apache/texera/issues/5882#issuecomment-4910915508
> is this done? [@Xiao-zhen-Liu](https://github.com/Xiao-zhen-Liu)
Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add operator output port cache storage (table and cache key) [texera]
Posted by "Xiao-zhen-Liu (via GitHub)" <gi...@apache.org>.
Xiao-zhen-Liu closed issue #5882: Add operator output port cache storage (table and cache key)
URL: https://github.com/apache/texera/issues/5882
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org