You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "baibaichen (via GitHub)" <gi...@apache.org> on 2026/08/01 07:24:40 UTC
[PR] [GLUTEN-12671][VL] Bound the Velox memCache by memory Spark can spare [gluten]
baibaichen opened a new pull request, #12672:
URL: https://github.com/apache/gluten/pull/12672
## What changes are proposed in this pull request?
Fixes #12671
`SparkMmapAllocator` subclasses `MmapAllocator` and governs the cache capacity in **two** places,
because reporting it and enforcing it do different jobs:
- **Reporting** is the virtual `capacity()`, which `AsyncDataCache::canTryAllocate()` measures
headroom against, so lowering it is what makes the cache evict older entries instead of growing.
It returns `max(governed, allocated)` rather than the governed value alone: the headroom is an
*unsigned* subtraction, so a capacity below the allocated bytes would wrap to a huge number and
turn the cap into an unlimited allowance. Clamped, the headroom is exactly zero.
- **Enforcing** is the three admission points, which take the bound as a parameter. Reporting alone
would not do: `canTryAllocate()` runs without a lock by design, so readers passing it together
took more than the cap — one entry over a 32MB grant with 16 threads, measured.
A periodic checker on the JVM side moves the capacity — halving the room above the floor when
storage tightens, reclaiming a fixed step when it lifts — and does **all** of the Spark borrow and
repay. Growing borrows first and then raises the cap; shrinking lowers the cap first and then repays
what was actually given up. Native code never calls back into Java.
### What this bound does and does not guarantee
It is a **soft bound that settles, rather than one that holds**:
- The checker runs **periodically** (1s by default), so the capacity follows Spark's pressure with a
lag of up to one interval. It is not synchronous with allocation.
- Lowering the capacity is a *request*. `AsyncDataCache` evicts what it can, but **pinned entries
survive it**. When a lot is pinned the cache stays above the new target and there is nothing to do
but wait — only what was **actually** freed is repaid to Spark, so the reservation is never
under-stated while that lasts.
- Cache usage therefore **converges to below** what is reported to Spark. It does not step over the
reservation and stay there.
- The one window where usage can exceed the reservation is an allocation **already in flight** when
the capacity changes: the bound is read once on the way in, so lowering it cannot reach a call
already past that point. It is bounded by concurrent readers times `loadQuantum` (forced <= 8MB
when the cache is on), and closes as those allocations finish.
### Why not a hard bound
The original plan was to hook the cache into the memory manager's LRU list, so eviction could be
driven synchronously and the bound made hard. **Pins are what make that impossible**: there is no
answer to "how much can you release right now". A synchronous reclaim would have to either block on
readers or return an unpredictable amount, and the memory manager can act on neither. Backing off
periodically, and repaying only what was really freed, is what pins actually permit.
### Dependency — please read before merging
This depends on facebookincubator/velox#18358, which parameterises the three `MmapAllocator`
admission paths as protected `...WithCapacity` overloads (a pure refactor; the existing entry points
forward the configured capacity). It is unavoidable: `Allocation::append` is private to three named
`friend` classes and **friendship is not inherited**, so copying `MmapAllocator` into Gluten does not
compile.
The last commit is marked `[TMP]` and sets `UPSTREAM_VELOX_PR_ID` in `get-velox.sh` so that CI can
build against that PR. **It must be reverted before this merges**, once the PR has landed and the
pinned branch has picked it up.
Opened as a draft for that reason — the design and the Gluten side are ready for review.
The design, including the invariants, the rejected alternatives and the known limitations, is in
`docs/superpowers/specs/2026-07-31-velox-cache-backpressure-design.md`.
## How was this patch tested?
New tests:
- `cpp/velox/tests/SparkMmapAllocatorTest.cc` — 17 cases, covering the invariants, page alignment,
concurrent readers filling the cache together, and the shutdown contract.
- `PeriodicMemoryCheckerSuite` — 28 cases, covering the adjustment algorithm, a refused borrow, a
shrink blocked by pins, and the startup configuration checks.
- `CacheStorageReservationDynamicSizingSuite` — 1 case, that the reservation counts towards the
process memory gate under dynamic off-heap sizing.
All passing locally, along with `dev/format-scala-code.sh` and clang-format-15.
Not yet done, and worth a reviewer's attention: the cross-profile compile (spark-3.3 / 3.5 /
scala-2.12) has not been run locally, and the defaults for `minCacheSize`, `cacheRatio` and
`stepSize` have not been calibrated against a real cluster.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12671][VL] Bound the Velox memCache by memory Spark can spare [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12672:
URL: https://github.com/apache/gluten/pull/12672#issuecomment-5707123232
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12671][VL] Bound the Velox memCache by memory Spark can spare [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #12672: [GLUTEN-12671][VL] Bound the Velox memCache by memory Spark can spare
URL: https://github.com/apache/gluten/pull/12672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12671][VL] Bound the Velox memCache by memory Spark can spare [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12672:
URL: https://github.com/apache/gluten/pull/12672#issuecomment-5851729668
This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org