[I] perf: measure filesystem costs and address evidenced scaling limits [opendal-yinyang]

["Xuanwo (via GitHub)" <gi...@apache.org>]

Xuanwo opened a new issue, #78:
URL: https://github.com/apache/opendal-yinyang/issues/78

   Implementation roadmap: #67. Dependencies: none; this work can start independently.
   
   ## Problem
   
   The core has indexed cost regression checks, but there is no representative product-level throughput/capacity claim. Mount still stages whole files and Sync materializes a baseline before replacement; native enumeration also has explicit scalability limits.
   
   ## Scope
   
   Measure representative workflows at the current implementation, then optimize only demonstrated bottlenecks. Separate local scan/read, network payload, metadata/proof requests, persistent space and contention costs.
   
   ## Acceptance criteria
   
   - [ ] Measure small-file trees, large-file range reads and sparse updates, concurrent writers, retained history, cold/warm opens and restart.
   - [ ] Include whole-file staging, directory publish/restore and recursive File Provider enumeration so core range-read behavior is not mistaken for end-to-end scalability.
   - [ ] Record commit, backend, dataset, cache state, completion semantics, bytes, requests, latency and speculative work with reproducible commands.
   - [ ] Compare equivalent workflows; never compare Mount remote fsync with File Provider local save as the same completion guarantee.
   - [ ] Use results to close encoding/verification-unit decisions and maintain targeted cost regressions for changes worth enforcing.
   
   ## Planning
   
   Milestone: M3. Primary direction: Shared core / Managed, then supported Direct combinations. Architecture index: #19.
   
   ## Baseline
   
   [Current contract](https://github.com/apache/opendal-yinyang/blob/1146781a127a7fa3ad2afee3e3ca8c823d659013/specs/transactional-filesystem.md) at main `1146781`, verified 2026-10-26. Existing native observations describe the documented prototype; this issue does not claim new acceptance results.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org