[I] [FIP-28] Manage historical lookup local cache capacity [fluss]

["luoyuxia (via GitHub)" <gi...@apache.org>]

luoyuxia opened a new issue, #3862:
URL: https://github.com/apache/fluss/issues/3862

   ### Search before asking\n\n- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.\n\n### Description\n\nParent issue: #3631\n\nHistorical partition lookups materialize local Paimon lookup files on TabletServers. Without server-level cache management, cached lookupers from many tables can consume unbounded local disk and configuration changes can leave stale lookup state behind.\n\nAdd bounded local cache management for historical lookup with the following behavior:\n\n- Provide a server-wide historical lookup cache capacity and a per-table cache size.\n- Reserve configured capacity when admitting a table lookuper and evict inactive cached lookupers in least-recently-used order when capacity is needed.\n- Reject invalid per-table cache sizes and throttle admission when a new lookuper cannot fit.\n- Use stable per-table lookup cache directories and clean stale cache directories during startup.\n- Recreate cached lookupers when relevant t
 able or lake configuration changes.\n- Allow the server-wide capacity to be updated dynamically through the server configuration procedure.\n- Expose focused metrics for cached table count, capacity evictions, historical lookup traffic, in-flight requests, and Paimon lookup file materialization.\n\n### Willingness to contribute\n\n- [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [FIP-28] Manage historical lookup local cache capacity [fluss]

["wuchong (via GitHub)" <gi...@apache.org>]

wuchong closed issue #3862: [FIP-28] Manage historical lookup local cache capacity
URL: https://github.com/apache/fluss/issues/3862


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org