You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@pekko.apache.org by GitBox <gi...@apache.org> on 2026/05/17 10:24:11 UTC

[GH] (pekko/optimize-flatten-merge-avoid-materialization): Workflow run "Generate doc check" is working again!

The GitHub Actions job "Generate doc check" on pekko.git/optimize-flatten-merge-avoid-materialization has succeeded.
Run started by GitHub user He-Pin (triggered by He-Pin).

Head commit for run:
281303b1b0907bb400a0597780db6318cd1489f8 / He-Pin <he...@gmail.com>
optimize: FlattenMerge avoids substream materialization for value-presented sources

Motivation:
FlattenMerge previously only fast-pathed `SingleSource`. For all other
inner sources -- `Source.empty`, `Source(List)`, `Source.fromJavaStream`,
`Source.future(Future.successful(...))`, range/iterator/repeat sources --
each one paid the cost of materializing a `SubSinkInlet` and
`subFusingMaterializer.materialize(...)`. FlattenConcat already does this
optimization via `TraversalBuilder.getValuePresentedSource`; FlattenMerge
should benefit too, especially because the single-arg `flatMapConcat(f)`
internally uses `FlattenMerge(1)` and so depends on FlattenMerge for its
hot path.

Modification:
- Generalize FlattenMerge to dispatch on `getValuePresentedSource`
  (instead of `getSingleSource`) and consume `SingleSource`,
  `IterableSource`, `IteratorSource`, `RangeSource`, `RepeatSource`,
  `JavaStreamSource`, `FutureSource`, `FailedSource`, and empty sources
  in-place without materialization, mirroring FlattenConcat.
- Add an `InflightSource[T]` family inside the new `FlattenMerge`
  companion to occupy a breadth slot for multi-element value-presented
  sources. Track them via a new `pendingInflightSources` counter so
  `activeSources` correctly bounds the breadth budget.
- Preserve merge semantics: when an inflight source still has more
  elements after a push, re-enqueue it so other concurrent sources keep
  interleaving (instead of draining one source first, which is the
  concat behaviour).
- Fold completed `Future`s and `FailedSource` directly: success pushes
  or queues a single element, failure calls `failStage`.
- Pending `Future`s register a callback via `getAsyncCallback` and
  occupy a breadth slot until completion.
- Empty inner sources are discarded in place (no slot consumed).

Result:
- `flatMapMerge(breadth, ...)` and the default `flatMapConcat(...)`
  (which routes through `FlattenMerge(1)`) skip substream materialization
  for value-presented inner sources, reducing per-source GC and stage
  overhead.
- All existing FlattenMerge / flatMapConcat tests pass; new tests cover
  empty / single / iterable / range / java stream / completed and
  delayed future / failed inner sources across breadth = 1..128.
- Internal API only (`@InternalApi private[pekko]`); MiMa is clean.

References:
- The optimization mirrors FlattenConcat's value-presented-source
  handling introduced in 1.2.0.

Report URL: https://github.com/apache/pekko/actions/runs/25987981634

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@pekko.apache.org
For additional commands, e-mail: notifications-help@pekko.apache.org