You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by GitBox <gi...@apache.org> on 2026/05/17 10:24:11 UTC
[GH] (pekko/optimize-flatten-merge-avoid-materialization): Workflow run "Generate doc check" is working again!
The GitHub Actions job "Generate doc check" on pekko.git/optimize-flatten-merge-avoid-materialization has succeeded.
Run started by GitHub user He-Pin (triggered by He-Pin).
Head commit for run:
281303b1b0907bb400a0597780db6318cd1489f8 / He-Pin <he...@gmail.com>
optimize: FlattenMerge avoids substream materialization for value-presented sources
Motivation:
FlattenMerge previously only fast-pathed `SingleSource`. For all other
inner sources -- `Source.empty`, `Source(List)`, `Source.fromJavaStream`,
`Source.future(Future.successful(...))`, range/iterator/repeat sources --
each one paid the cost of materializing a `SubSinkInlet` and
`subFusingMaterializer.materialize(...)`. FlattenConcat already does this
optimization via `TraversalBuilder.getValuePresentedSource`; FlattenMerge
should benefit too, especially because the single-arg `flatMapConcat(f)`
internally uses `FlattenMerge(1)` and so depends on FlattenMerge for its
hot path.
Modification:
- Generalize FlattenMerge to dispatch on `getValuePresentedSource`
(instead of `getSingleSource`) and consume `SingleSource`,
`IterableSource`, `IteratorSource`, `RangeSource`, `RepeatSource`,
`JavaStreamSource`, `FutureSource`, `FailedSource`, and empty sources
in-place without materialization, mirroring FlattenConcat.
- Add an `InflightSource[T]` family inside the new `FlattenMerge`
companion to occupy a breadth slot for multi-element value-presented
sources. Track them via a new `pendingInflightSources` counter so
`activeSources` correctly bounds the breadth budget.
- Preserve merge semantics: when an inflight source still has more
elements after a push, re-enqueue it so other concurrent sources keep
interleaving (instead of draining one source first, which is the
concat behaviour).
- Fold completed `Future`s and `FailedSource` directly: success pushes
or queues a single element, failure calls `failStage`.
- Pending `Future`s register a callback via `getAsyncCallback` and
occupy a breadth slot until completion.
- Empty inner sources are discarded in place (no slot consumed).
Result:
- `flatMapMerge(breadth, ...)` and the default `flatMapConcat(...)`
(which routes through `FlattenMerge(1)`) skip substream materialization
for value-presented inner sources, reducing per-source GC and stage
overhead.
- All existing FlattenMerge / flatMapConcat tests pass; new tests cover
empty / single / iterable / range / java stream / completed and
delayed future / failed inner sources across breadth = 1..128.
- Internal API only (`@InternalApi private[pekko]`); MiMa is clean.
References:
- The optimization mirrors FlattenConcat's value-presented-source
handling introduced in 1.2.0.
Report URL: https://github.com/apache/pekko/actions/runs/25987981634
With regards,
GitHub Actions via GitBox
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org