You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by "krishvishal (via GitHub)" <gi...@apache.org> on 2026/07/14 09:45:10 UTC
[PR] feat(simulator): deterministic future executor and server-ng dispatch shell [iggy]
krishvishal opened a new pull request, #3671:
URL: https://github.com/apache/iggy/pull/3671
## What
A deterministic executor for the simulator and a toggle that runs the **real** server-ng request-handling path (dispatch, login, produce, poll) under it, so seeded runs replay byte-for-byte and the async-concurrency bug class (a borrow held across `.await` while a concurrent task mutates the same state, the #3557 shape) becomes reachable under adversarial schedules.
## Why
The sim ran real consensus/metadata/shard over mocked I/O but never exercised the async dispatch path, and wall-clock/OS-random/HashMap-order/random-id inputs broke seeded replay.
## Key pieces
* **Deterministic executor** (`simulator/src/executor/`): seeded cooperative scheduling of `!Send` shard tasks, virtual time, rolling `schedule_hash` for replay assertions, generational task ids, crash = abort-in-place.
* **Dispatch shell (toggle)**: `dispatch`/`auth`/`responses`/`consumer_group` genericized over `<B, MJ, S>`; production instantiates the concrete shard unchanged. Bus gains `spawn`/`sleep`/`realtime_micros` seams (compio in prod, virtual in sim); `ConnectionInstaller` widened for the sim bus.
* **Determinism**: injected consensus clock + prepare-timestamp clamp against the observed log head; snapshot `created_at` from the injected clock; fixed-salt hash for the seeded root user; deterministic message ids; BTreeMap namespace fan-out.
* **Multi-shard sim**: S fully-meshed shards per replica wired like bootstrap (metadata on shard 0, partitions on hash owner), seeded entry-shard hop, real pump per shard.
* **Detector**: borrow-across-await tripwire fires under a forced suspension.
## Production impact
None at runtime. Toggle off = current fast fuzzer path; the generics bottom out at the concrete `ServerNgShard`. Sim-only helpers are behind a `simulator` cargo feature and excluded from a `-p iggy-server-ng` build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(simulator): deterministic future executor and server-ng dispatch shell [iggy]
Posted by "numinnex (via GitHub)" <gi...@apache.org>.
numinnex merged PR #3671:
URL: https://github.com/apache/iggy/pull/3671
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(simulator): deterministic future executor and server-ng dispatch shell [iggy]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #3671:
URL: https://github.com/apache/iggy/pull/3671#issuecomment-4967956384
## [Codecov](https://app.codecov.io/gh/apache/iggy/pull/3671?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 73.03%. Comparing base ([`c42eca7`](https://app.codecov.io/gh/apache/iggy/commit/c42eca779aea58bc6b1c64df6d4509c2aaade622?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`0fb1840`](https://app.codecov.io/gh/apache/iggy/commit/0fb18406c9ce8a5b1e4e92b12979b2d800cae582?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #3671 +/- ##
============================================
- Coverage 73.60% 73.03% -0.58%
Complexity 937 937
============================================
Files 1297 1234 -63
Lines 143769 138683 -5086
Branches 119329 118842 -487
============================================
- Hits 105822 101284 -4538
+ Misses 34582 34033 -549
- Partials 3365 3366 +1
```
| [Components](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [Rust Core](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.00% <ø> (+0.16%)` | :arrow_up: |
| [Java SDK](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.44% <ø> (ø)` | |
| [C# SDK](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `72.19% <ø> (ø)` | |
| [Python SDK](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `92.17% <ø> (ø)` | |
| [PHP SDK](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.29% <ø> (ø)` | |
| [Node SDK](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `87.27% <ø> (-4.08%)` | :arrow_down: |
| [Go SDK](https://app.codecov.io/gh/apache/iggy/pull/3671/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.28% <ø> (ø)` | |
| [Files with missing lines](https://app.codecov.io/gh/apache/iggy/pull/3671?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [core/consensus/src/impls.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fconsensus%2Fsrc%2Fimpls.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25zZW5zdXMvc3JjL2ltcGxzLnJz) | `77.56% <ø> (-0.02%)` | :arrow_down: |
| [core/message\_bus/src/installer/mod.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fmessage_bus%2Fsrc%2Finstaller%2Fmod.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9tZXNzYWdlX2J1cy9zcmMvaW5zdGFsbGVyL21vZC5ycw==) | `43.39% <ø> (ø)` | |
| [core/message\_bus/src/lib.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fmessage_bus%2Fsrc%2Flib.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9tZXNzYWdlX2J1cy9zcmMvbGliLnJz) | `91.09% <ø> (ø)` | |
| [core/metadata/src/impls/metadata.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fmetadata%2Fsrc%2Fimpls%2Fmetadata.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9tZXRhZGF0YS9zcmMvaW1wbHMvbWV0YWRhdGEucnM=) | `43.93% <ø> (+0.04%)` | :arrow_up: |
| [core/metadata/src/stm/mux.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fmetadata%2Fsrc%2Fstm%2Fmux.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9tZXRhZGF0YS9zcmMvc3RtL211eC5ycw==) | `94.11% <ø> (-0.12%)` | :arrow_down: |
| [core/metadata/src/stm/snapshot.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fmetadata%2Fsrc%2Fstm%2Fsnapshot.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9tZXRhZGF0YS9zcmMvc3RtL3NuYXBzaG90LnJz) | `85.89% <ø> (-0.12%)` | :arrow_down: |
| [core/metadata/src/stm/stream.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fmetadata%2Fsrc%2Fstm%2Fstream.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9tZXRhZGF0YS9zcmMvc3RtL3N0cmVhbS5ycw==) | `57.95% <ø> (ø)` | |
| [core/partitions/src/iggy\_partition.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fpartitions%2Fsrc%2Figgy_partition.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9wYXJ0aXRpb25zL3NyYy9pZ2d5X3BhcnRpdGlvbi5ycw==) | `54.61% <ø> (ø)` | |
| [core/partitions/src/iggy\_partitions.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fpartitions%2Fsrc%2Figgy_partitions.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9wYXJ0aXRpb25zL3NyYy9pZ2d5X3BhcnRpdGlvbnMucnM=) | `67.28% <ø> (-0.09%)` | :arrow_down: |
| [core/server-ng/src/auth.rs](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree&filepath=core%2Fserver-ng%2Fsrc%2Fauth.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXItbmcvc3JjL2F1dGgucnM=) | `0.00% <ø> (ø)` | |
| ... and [18 more](https://app.codecov.io/gh/apache/iggy/pull/3671?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | |
... and [81 files with indirect coverage changes](https://app.codecov.io/gh/apache/iggy/pull/3671/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org