You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@asyncband.apache.org by "tisonkun (via GitHub)" <gi...@apache.org> on 2026/09/20 04:46:30 UTC
[PR] perf(mpmc): coalesce receiver notifications and simplify futures [asyncband]
tisonkun opened a new pull request, #321:
URL: https://github.com/apache/asyncband/pull/321
## Summary
Avoid waking another MPMC receiver when existing notifications already cover the buffered messages, and return the internal send/receive futures directly instead of wrapping them in another async state machine. This targets the blocking 1P/8C scheduling overhead and the small ready-path regression investigated after #314.
The two changes are separate commits: `d149619` handles notifications; `6de7e55` removes async forwarding. Existing `.await` usage, lazy execution, cancellation, and disconnection behavior remain supported. The notification tests now exercise stolen messages followed by another send, cancellation after that reuse, and parallel notification when a real backlog exists.
## Design Notes
Previously, if receiver A was notified, an active receiver could take its message before A ran. The next send then woke receiver B even though A was still scheduled to retry. Count notified receivers until they poll or cancel, and only issue another notification when buffered messages outnumber those outstanding notifications. Cancellation passes on a notification only if an uncovered message remains. A successful receive acknowledges one notification and consumes one message, so it needs no additional wake chain. Wake callbacks and retired waker destruction remain outside the queue lock.
A prototype limited the queue to one outstanding receiver notification and passed it on after receiving. It brought unbounded blocking 1P/8C down to about 1 ms, but slowed bounded 8P/8C by roughly 10–13%. This PR uses the backlog count instead, preserving parallel notification for real queued work rather than serializing receiver activation.
For the ready path, disassembly showed out-of-line polling/forwarding around the nested async wrappers. Returning the existing internal future removes that avoidable layer. The earlier 2.5% regression was code-generation-sensitive: rebuilding the unchanged merged tree did not reproduce its previous absolute timings. The results below are fresh comparisons against the same main revision, not a claim that a unique historical compiler decision has been isolated.
## Benchmark baseline
- Baseline: main `7a4e00025c8f5069cd9b89271654250755dfc2a4`; candidate: `6de7e55`.
- Apple M4 Max, 14 CPU cores, 36 GB RAM; macOS 26.6.2; `rustc 1.99.0-nightly (3d6c19bb9 2026-08-11)`, aarch64, default optimized bench profile. AC power was checked before and after every run. This was a normal desktop, without CPU pinning.
- Both revisions used unchanged benchmark sources and separate build directories, built with `cargo x bench --no-run`; benchmark binary hashes were checked to differ. No builds or tests ran during measurements.
- One discarded warmup pair per suite; eight measured pairs for microbenchmarks, six for bulk benchmarks. Main/PR order alternated. Each table entry is the median of the per-run medians; negative changes mean less time. Small differences should not be read as statistically established wins or losses.
- Micro: 1,000 samples × 4,096 iterations. Bulk: 30 samples × one 16,384-message batch; bounded capacity 64. Blocking consumers have equal quotas; Tokio consumers compete until disconnection. Thread/task creation is outside the timed batch.
- Coverage: all 7 Asyncband MPMC microbenchmarks and all 24 Asyncband bulk configurations. The ecosystem runs also covered 24 async-channel configurations and 16 Flume Tokio configurations (71 distinct configurations in total).
- Flume blocking comparisons were excluded after the previously observed bounded 1P/8C stall recurred. Its stack again showed the producer and one receiver parked while seven receivers waited at the completion barrier. The incomplete run was discarded, and the bulk suite restarted with the exclusion on both revisions. This PR does not diagnose or fix that separate adapter/benchmark stall.
### Target cases
| Case | Main | PR | Time change |
| --- | ---: | ---: | ---: |
| Unbounded blocking 1P/8C | 39.150 ms/batch | 20.680 ms/batch | -47.2% |
| Bounded ready send → recv | 14.685 ns/pair | 12.190 ns/pair | -17.0% |
| Bounded blocking 1P/8C | 45.075 ms/batch | 22.265 ms/batch | -50.6% |
For the unbounded blocking target, the six run medians ranged from 33.990–40.150 ms on main and 19.650–21.210 ms on the PR. These ranges describe repeated runs, not a latency percentile or confidence interval.
The remaining gap is substantial: the unchanged async-channel adapter measured 3.138 ms in the baseline binary and 2.932 ms in the candidate binary for unbounded blocking 1P/8C. This is a partial improvement to the existing design, not parity with that implementation.
One watchpoint in the full matrix is unbounded Tokio current-thread 8P/8C: 0.215 → 0.227 ms (+6.0%). An additional eight alternating pairs with 200 samples per configuration measured 0.224 → 0.218 ms (−2.9%; run medians 0.219–0.233 ms on main and 0.214–0.234 ms on the PR). The direction did not reproduce, so the original matrix entry is retained below rather than presented as a confirmed regression or replaced by the more favorable follow-up.
<details>
<summary>All 7 microbenchmarks (ns per iteration)</summary>
| Case | Main | PR | Time change |
| --- | ---: | ---: | ---: |
| bounded / send_then_recv | 14.685 | 12.190 | -17.0% |
| bounded / try_send_then_try_recv | 10.430 | 9.919 | -4.9% |
| bounded / wake_blocked_sender | 29.330 | 28.270 | -3.6% |
| unbounded / repoll_pending_receiver | 7.303 | 6.769 | -7.3% |
| unbounded / send_then_recv | 12.930 | 11.160 | -13.7% |
| unbounded / send_then_try_recv | 10.545 | 10.029 | -4.9% |
| unbounded / wake_pending_receiver | 31.600 | 29.285 | -7.3% |
</details>
<details>
<summary>All 24 Asyncband bulk configurations (ms per 16,384 messages)</summary>
#### Blocking threads
| Queue / producers:consumers | Main | PR | Time change |
| --- | ---: | ---: | ---: |
| bounded / 1:1 | 2.178 | 2.126 | -2.4% |
| bounded / 1:8 | 45.075 | 22.265 | -50.6% |
| bounded / 8:1 | 44.995 | 44.720 | -0.6% |
| bounded / 8:8 | 6.953 | 6.855 | -1.4% |
| unbounded / 1:1 | 0.565 | 0.482 | -14.6% |
| unbounded / 1:8 | 39.150 | 20.680 | -47.2% |
| unbounded / 8:1 | 0.680 | 0.657 | -3.5% |
| unbounded / 8:8 | 1.032 | 1.030 | -0.1% |
#### Tokio current-thread
| Queue / producers:consumers | Main | PR | Time change |
| --- | ---: | ---: | ---: |
| bounded / 1:1 | 0.247 | 0.239 | -3.3% |
| bounded / 1:8 | 0.317 | 0.293 | -7.4% |
| bounded / 8:1 | 0.287 | 0.285 | -0.8% |
| bounded / 8:8 | 0.298 | 0.276 | -7.4% |
| unbounded / 1:1 | 0.219 | 0.213 | -2.9% |
| unbounded / 1:8 | 0.228 | 0.216 | -5.3% |
| unbounded / 8:1 | 0.222 | 0.217 | -2.4% |
| unbounded / 8:8 | 0.215 | 0.227 | +6.0% |
#### Tokio, four workers
| Queue / producers:consumers | Main | PR | Time change |
| --- | ---: | ---: | ---: |
| bounded / 1:1 | 0.418 | 0.396 | -5.4% |
| bounded / 1:8 | 1.109 | 1.065 | -4.1% |
| bounded / 8:1 | 1.092 | 1.046 | -4.2% |
| bounded / 8:8 | 1.298 | 1.282 | -1.2% |
| unbounded / 1:1 | 0.264 | 0.266 | +0.5% |
| unbounded / 1:8 | 0.891 | 0.894 | +0.4% |
| unbounded / 8:1 | 0.633 | 0.606 | -4.2% |
| unbounded / 8:8 | 0.900 | 0.904 | +0.5% |
</details>
These batches use `usize` messages and no application work between operations. The blocking 1P/8C case is useful for investigating small-message dispatch and park/wake overhead; 8P/8C checks that improving fanout does not sacrifice a busy shared queue. The ready-path tests isolate immediate completion without executor scheduling. This suite does not establish p99 latency, fairness, slow-consumer behavior, or performance across other capacities and payload sizes.
<details>
<summary>Reproduction commands</summary>
Build each revision in its own checkout/build directory with `cargo x bench --no-run`, then run the executables printed by Cargo. Repeat in alternating order as described above.
```sh
"$BENCH_BIN" --bench --color never --timer os --sample-count 1000 --sample-size 4096 '^benchmarks::mpmc::' --skip 'blocking_threads|tokio_tasks'
"$ECOSYSTEM_BIN" --bench --color never --timer os --sample-count 30 --sample-size 1 '^ecosystem::mpmc::' --skip 'blocking_threads::Flume'
```
</details>
## Validation
- `cargo x test`: 607 passed.
- `cargo x check`: feature matrix passed.
- `cargo x lint`: Clippy, formatting, spelling, license headers, and rustdoc passed.
- `cargo x miri`: 149 passed; 8 OS-runtime tests ignored by their existing annotations.
- Benchmark sources are unchanged; baseline data and this report are kept in the PR description, not added to the repository.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@asyncband.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@asyncband.apache.org
For additional commands, e-mail: commits-help@asyncband.apache.org
Re: [PR] perf(mpmc): coalesce outstanding receiver notifications [asyncband]
Posted by "tisonkun (via GitHub)" <gi...@apache.org>.
tisonkun closed pull request #321: perf(mpmc): coalesce outstanding receiver notifications
URL: https://github.com/apache/asyncband/pull/321
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@asyncband.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@asyncband.apache.org
For additional commands, e-mail: commits-help@asyncband.apache.org