You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by pi...@apache.org on 2026/08/20 12:09:34 UTC
(iggy-website) branch main updated: docs(site): rework protocol docs into Binary Protocol section (#67)
This is an automated email from the ASF dual-hosted git repository.
spetz pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iggy-website.git
The following commit(s) were added to refs/heads/main by this push:
new f369abf5 docs(site): rework protocol docs into Binary Protocol section (#67)
f369abf5 is described below
commit f369abf52056f0a96060d4dfb4ef5fc595a7cfe3
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Aug 20 14:09:28 2026 +0200
docs(site): rework protocol docs into Binary Protocol section (#67)
The 857-line schema page sat buried under Server and mixed every
protocol concern, the replica-to-replica wire was undocumented, and
the storage-engine write-pipeline diagram rendered unreadably while
contradicting the code.
Schema content now lives in a top-level Binary Protocol section
split by concern (framing, connection lifecycle, commands, shared
encodings, message batches), plus a new server-to-server page
written against the consensus code: command registry, frame seals,
handshake, replication, view changes, repair, state transfer, and
session forwarding. A meta-refresh stub keeps the old
/docs/server/schema/ URL alive since the static export cannot
redirect.
The write-pipeline diagram is redrawn without the back-edge cycle
and long edge label that scrambled the site's mermaid renderer, and
its claims are corrected against the server: stamping happens at
journal-append time before replication acks, buffering at prepare
time, the flush gate is a three-way OR evaluated on commit events,
and enforce_fsync does fdatasync on both the .log and .index.
Inherited errors fixed along the way: the header-kinds table was
missing code 8 (Int128); the server-stamped message time is the
flat batch base_timestamp, not base_timestamp + timestamp_delta;
and compression_algorithm is now documented as a placeholder (the
value is stored and reported, but nothing compresses payloads yet -
the message-headers examples show the client-side pattern).
Two rendering fixes ride along. The mermaid component bumped
font-size attributes after layout, overflowing node boxes sized for
the original metrics and colliding adjacent nodes; it now zooms the
whole SVG uniformly instead. And the fumadocs mobile TOC bar
([data-toc-popover]) is lifted to z-30 with an opaque background:
it was sticky at z-10 and transparent, so docs content using z-10
painted over it while scrolling under.
---
content/docs/binary-protocol/cluster.mdx | 169 ++++
content/docs/binary-protocol/commands.mdx | 425 ++++++++++
.../docs/binary-protocol/connection-lifecycle.mdx | 65 ++
content/docs/binary-protocol/encodings.mdx | 127 +++
content/docs/binary-protocol/framing.mdx | 194 +++++
content/docs/binary-protocol/index.mdx | 20 +
content/docs/binary-protocol/messages.mdx | 41 +
content/docs/binary-protocol/meta.json | 4 +
content/docs/introduction/concepts.mdx | 4 +-
content/docs/meta.json | 1 +
content/docs/server/meta.json | 2 +-
content/docs/server/networking.mdx | 2 +-
content/docs/server/schema.mdx | 857 ---------------------
content/docs/server/storage-engine.mdx | 43 +-
content/docs/server/topic-options.mdx | 2 +-
public/docs/server/schema/index.html | 31 +
src/app/global.css | 8 +
src/components/architecture-diagrams.tsx | 2 +-
src/components/mermaid.tsx | 13 +-
19 files changed, 1121 insertions(+), 889 deletions(-)
diff --git a/content/docs/binary-protocol/cluster.mdx b/content/docs/binary-protocol/cluster.mdx
new file mode 100644
index 00000000..0d0e2aff
--- /dev/null
+++ b/content/docs/binary-protocol/cluster.mdx
@@ -0,0 +1,169 @@
+---
+title: Server-to-server
+---
+
+Replicas talk to each other with the same 256-byte [framing](/docs/binary-protocol/framing) the client protocol uses, on a **dedicated TCP port**, with their own set of `command` discriminants. None of these frames ever appears on a client connection, and a client cannot reach this plane: it is a separate listener, gated by a handshake.
+
+This page documents the replica plane as of binary protocol 0.11.0. It is internal protocol: only Iggy servers speak it, and it can change between server releases without a client-facing version bump.
+
+## Transport
+
+- Every node in `[[cluster.nodes]]` exposes a `tcp_replica` port next to the client ports: `ports = { tcp = 8090, quic = 8080, http = 3000, websocket = 8092, tcp_replica = 9090 }`. The node's `ip` is the roster address for the replica plane (clients fall back to it when `advertised_address` is unset).
+- The replica plane is **TCP only**, by design: the prepare hash chain, cross-shard fd delegation, and view-change timing all assume an ordered byte stream.
+- **Directional dialing:** a replica dials only peers with a strictly greater `replica_id` and accepts inbound connections only from strictly lower ids. Exactly one connection exists per pair, with no tiebreaker races.
+- Frames are the standard `[256-byte header][optional body]` with `size` at offset 48. The maximum frame is **64 MiB** by default (`message_bus.max_message_size`). Headers are `#[repr(C)]`, decoded zero-copy, and require 16-byte alignment.
+- Nothing inside a frame marks it as replica traffic. The separation is structural: client listeners parse every inbound frame as a `RequestHeader` (command `5` only), the replica listener requires the first frame to be `ReplicaHello`, and the client-bound commands `Reply` (8) and `Eviction` (13) are rejected if they arrive on the replica plane.
+
+## Command discriminants
+
+The full `command` byte registry, client values included. Values above 29 are rejected.
+
+| Value | Command | Direction | Purpose |
+|-------|---------|-----------|---------|
+| 0 | `Reserved` | - | Invalid sentinel |
+| 1-4 | `Ping`, `Pong`, `PingClient`, `PongClient` | - | Reserved; no production traffic today |
+| 5 | `Request` | client to server | Client command ([framing](/docs/binary-protocol/framing)) |
+| 6 | `Prepare` | primary to backup, backup to next backup | Replicate one operation |
+| 7 | `PrepareOk` | backup to primary | Acknowledge a prepare |
+| 8 | `Reply` | server to client | Client reply; rejected on the replica plane |
+| 9 | `Commit` | primary to all | Commit-point broadcast, doubles as heartbeat |
+| 10 | `StartViewChange` | any to all | View-change proposal |
+| 11 | `DoViewChange` | any to all | View-change vote with log suffix |
+| 12 | `StartView` | new primary to all | Install the new view |
+| 13 | `Eviction` | server to client | Client eviction; rejected on the replica plane |
+| 14 | `ReplicaHello` | dialer to acceptor | Handshake step 1 |
+| 15 | `ReplicaChallenge` | acceptor to dialer | Handshake step 2 |
+| 16 | `ReplicaFinish` | dialer to acceptor | Handshake step 3 |
+| 17 | `RequestStartView` | recovering replica to all | Ask the current primary for a targeted `StartView` |
+| 18 | `RequestPrepares` | lagging replica to peer | Journal repair: request an op range |
+| 19 | `RepairPrepare` | serving peer to requester | One journaled prepare, verbatim |
+| 20 | `RepairDone` | serving peer to requester | Repair stream terminator |
+| 21 | `RangeEvicted` | serving peer to requester | Range no longer retained |
+| 22 | `RequestStateTransfer` | requester to primary | Bulk catch-up: ask for the target |
+| 23 | `StateTransferTarget` | primary to requester | Offer + state manifest |
+| 24 | `RequestStateChunk` | requester to primary | Fetch one artifact chunk |
+| 25 | `StateChunk` | primary to requester | Artifact bytes |
+| 26 | `ForwardRegister` | backup to primary | Forward a verified login |
+| 27 | `ForwardRegisterResult` | primary to backup | Login outcome |
+| 28 | `ForwardLogout` | backup to primary | Forward a logout |
+| 29 | `ForwardLogoutResult` | primary to backup | Logout outcome |
+
+There is no dedicated replica heartbeat: liveness is the `Commit` broadcast, sent on `cluster.commit_broadcast_interval`.
+
+## Frame integrity
+
+The `checksum` and `checksum_body` fields that client frames leave zero are live on the replica plane:
+
+- **Frame seal** (`checksum`, bytes 0..16): XxHash3-64 over header bytes 16..256, widened to u128. Sealed on every replica message **except** `Prepare`, `RepairPrepare`, and the three handshake frames (whose integrity comes from the keyed MAC). Verified once, at typed decode, before any field validation; a mismatch drops the frame.
+- **Prepare identity** (`Prepare` / `RepairPrepare` only): those two spend `checksum` on an identity hash instead - XxHash3-64 over the whole 256-byte header computed with `checksum = 0` and `view = 0`, so a retransmit that re-stamps `view` keeps the same identity. `PrepareOk` echoes it in `prepare_checksum`, and each prepare's `parent` field carries the previous prepare's identity, forming a hash chain.
+- **Body seal** (`checksum_body`, bytes 16..32): three regimes. Metadata-plane prepares seal their body with XxHash3-64. Partition-plane prepares leave it zero: the message batch inside already carries `batch_checksum`, verified at network ingress. `DoViewChange` and `StartView` seal their suffix bodies. Every other message leaves it zero (state-transfer bodies are verified per artifact, not per frame).
+- The seals are **unkeyed integrity checks, not authentication**. Peer authentication comes from the handshake below; without `cluster.auth` and TLS enabled, the replica port trusts any peer that can reach it. Do not expose `tcp_replica` beyond the cluster network.
+
+`Prepare`, `RepairPrepare`, and the session-forwarding headers validate their reserved regions as zero; the other replica headers leave reserved bytes covered by the frame seal only.
+
+## Handshake
+
+Commands 14-16, exchanged before any consensus traffic. These are raw `GenericHeader` frames (`size = 256`) using the per-command area at bytes 128..256:
+
+| Offset | Size | Content |
+|--------|------|---------|
+| 128 | 32 | Nonce (dialer's in `Hello`, acceptor's in `Challenge`) |
+| 160 | 32 | BLAKE3 keyed MAC (acceptor's in `Challenge`, dialer's in `Finish`) |
+| 192 | 1 | `Challenge` only: handshake status |
+
+Statuses: `0` Ok, `1` UnknownCommand, `2` ClusterMismatch, `3` DirectionalRule; values `4` and `5` are reserved (never sent), and a dialer treats any unknown status byte as a rejection. The MAC key derives from `cluster.auth.shared_secret` (32+ bytes; `previous_shared_secret` gives a rotation window), and with cluster TLS enabled (TLS 1.3 only, ALPN `iggy-replica`) the TLS exporter is folded into every MAC as channel binding. The handshake authenticates **cluster membership**, not per-re [...]
+
+The `cluster` header field (bytes 32..48) is the first 16 bytes of `blake3(cluster_name)` as a little-endian u128 on every replica frame, checked during the handshake, so nodes from a differently named cluster cannot connect even with auth disabled.
+
+## Replication
+
+### Prepare (6)
+
+Per-command fields (bytes 128..256):
+
+| Offset | Size | Field | Description |
+|--------|------|-------|-------------|
+| 128 | 16 | `client` | Originating client id |
+| 144 | 16 | `parent` | Previous prepare's identity checksum (hash chain) |
+| 160 | 16 | `request_checksum` | Verbatim from the admitted request |
+| 176 | 8 | `op` | Log position |
+| 184 | 8 | `commit` | Primary's commit point when the prepare was built |
+| 192 | 8 | `timestamp` | Primary-assigned monotonic timestamp |
+| 200 | 8 | `request` | Client request number |
+| 208 | 1 | `operation` | Operation discriminant (+7 padding) |
+| 216 | 8 | `group` | Consensus group (see [planes](#consensus-planes)) |
+| 224 | 4 | `user_id` | Authenticated user |
+| 228 | 28 | reserved | Zero |
+
+For metadata operations the body is the admitted command's payload **verbatim**. For `SendMessages` it is the [message batch](/docs/binary-protocol/messages) as stamped by the primary at journal append (`base_offset`, `base_timestamp`, and the recomputed `batch_checksum` are final); each backup re-derives the expected stamp from its own position and refuses a mismatch. Replication is **chain-form**: the primary sends to `(replica + 1) % replica_count`, each backup forwards to its own suc [...]
+
+### PrepareOk (7)
+
+Header-only. `parent` (echo) at 128, `prepare_checksum` (echo of the prepare's identity) at 144, then `op` 160, `commit` 168, `timestamp` 176, `request` 184, `operation` 192, `group` 200. Two fields are the acker's own, not echoes: header `view` is the acking replica's view, and `commit` is its own commit point. Sent to the view's primary.
+
+### Commit (9)
+
+Header-only broadcast; validation rejects `size != 256`. `timestamp_monotonic` at 144 is the liveness token (a receiver resets its heartbeat timer only on a strictly newer value), `commit` at 152 is the commit point, `group` at 168. Bytes 128..144 and 160..168 are declared (`commit_checksum`, `checkpoint_op`) but currently unused.
+
+## View changes
+
+- **StartViewChange (10):** header-only, just `group` at 128. Broadcast when a replica suspects the primary.
+- **DoViewChange (11):** `op` (highest op) 128, `commit` 136, `group` 144, `log_view` 152 (view when the sender was last in Normal status), `nack_bitset` 224 (bit i = "I never prepared suffix entry i"; truncation authority), `present_bitset` 240 (bit i = "I can serve entry i's body"). Body = the log suffix as up to **128 packed 256-byte `PrepareHeader`s**, headers only. Broadcast.
+- **StartView (12):** `op` 128, `commit` 136 (max over the DVC quorum), `group` 144, `incarnation` 240 (echo of a `RequestStartView` incarnation, zero when unsolicited). Body = the view's suffix as packed `PrepareHeader`s, highest op first; empty is legal. Unicast when answering a probe, broadcast otherwise.
+- **RequestStartView (17):** header-only recovery probe with `group` 128 and `incarnation` 240. A restarted replica broadcasts it instead of trusting stale local state; only the current view's primary answers, with a targeted `StartView`.
+
+## Journal repair
+
+A replica missing committed prepares (rejoin window or interior hole) pulls them from a Normal-status peer (the metadata plane also serves repair during a view change, so a forming primary cannot deadlock):
+
+- **RequestPrepares (18):** `nonce` 128, `from_op` 144, `to_op` 152, `group` 160. `from_op = 0` or `from_op > to_op` is rejected.
+- **RepairPrepare (19):** the journaled prepares, streamed in op order.
+- **RepairDone (20)** / **RangeEvicted (21):** one shared layout (`nonce` 128, `op` 144, `group` 152) under two command bytes. `RepairDone.op` = last op served; `RangeEvicted.op` = oldest op the peer still retains, the honest answer when the front of the range is gone.
+
+Pacing is bounded by `cluster.repair_chunk_max`, which must stay below `message_bus.peer_queue_capacity`: per-peer send queues drop overruns silently.
+
+## State transfer
+
+Bulk catch-up for a replica too far behind for journal repair. **Pull-based and lockstep** (at most one chunk in flight per artifact), because per-peer queues drop overruns:
+
+1. **RequestStateTransfer (22):** `nonce` 128, `group` 144.
+2. **StateTransferTarget (23):** `nonce` 128, `commit_op` 144, `group` 152, `available` 160, `unavailable_transient` 161, `commit_max` 168. `available = 0` is a header-only refusal. Body = the **state manifest**.
+3. **RequestStateChunk (24):** `nonce` 128, `offset` 144, `group` 152, `len` 160, `artifact` 164. `len` describes the reply and is clamped by the server.
+4. **StateChunk (25):** `nonce` 128, `offset` 144, `group` 152, `artifact` 160. Body = raw artifact bytes. Chunks carry no per-chunk integrity; each artifact is verified whole, by length and XxHash3-64, against its manifest entry.
+
+The manifest (little-endian):
+
+```text
+[magic: "ISM1"][count: u32][entry_len: u8]
+count x { kind: u8, frontier: u64, len: u64, checksum: u64 } # entry_len = 25
+[trailer: XxHash3-64 over everything above]
+```
+
+At most 65536 entries. Entries longer than 25 bytes decode with the tail skipped (forward compatibility); shorter entries are rejected.
+
+Artifact kinds:
+
+| Kind | Artifact | Plane | Content |
+|------|----------|-------|---------|
+| 0 | `METADATA_SNAPSHOT` | metadata | `snapshot.bin` verbatim (MessagePack, snapshot format version 3); `frontier` = sequence number |
+| 1 | `CLIENT_TABLE` | metadata | Client table encoding (magic `ICT2`); `frontier` = mutation frontier |
+| 2 | `SEGMENT_LOG` | partition | One retained segment's `.log` verbatim; `frontier` = segment base offset |
+| 3 | `CONSUMER_OFFSETS` | partition | Consumer + group offset tables and applied purge generation; `frontier` = offer's `commit_op` |
+
+## Session forwarding
+
+A client may dial a backup; authentication happens there, and only the verified identity travels to the primary (credentials never cross the replica plane):
+
+- **ForwardRegister (26):** `client` 128, `nonce` 144, `user_id` 160.
+- **ForwardRegisterResult (27):** `nonce` 128, `client` 144, `epoch` 160, `watermark` 168, outcome byte at **255**: `0` Ok, `1` NotPrimary, `2` NotCaughtUp, `3` PipelineFull, `4` InProgress, `5` Canceled, `6` ClientIdOwnedByAnotherUser. `epoch` and `watermark` must be zero on any non-Ok outcome.
+- **ForwardLogout (28):** `client` 128, `nonce` 144, `session` 160, `request` 168.
+- **ForwardLogoutResult (29):** `nonce` 128, `client` 144, `commit` 160, outcome at 255: `0` Ok, `1` NotPrimary, `2` PipelineFull, `3` InProgress, `4` Canceled.
+
+## Consensus planes
+
+Two consensus planes share this one message set; there are no plane-specific commands. Routing is by the `group: u64` field carried on every consensus header except the handshake and session-forwarding frames (those are implicitly metadata-plane):
+
+- `group = 1 << 63` is the **metadata plane** (streams, topics, users, consumer groups; durable on-disk journal).
+- Any other value is a packed stream/topic/partition key: a **partition plane** group (message batches, consumer offsets; in-memory journal).
+
+Same wire, a few divergent semantics: partition-plane prepares leave `checksum_body` zero and rely on the batch's own `batch_checksum`, their identity checksum covers the header alone, and `StateTransferTarget.unavailable_transient` / `commit_max` are read only by the partition arm.
diff --git a/content/docs/binary-protocol/commands.mdx b/content/docs/binary-protocol/commands.mdx
new file mode 100644
index 00000000..5a9687c6
--- /dev/null
+++ b/content/docs/binary-protocol/commands.mdx
@@ -0,0 +1,425 @@
+---
+title: Commands
+---
+
+## Catalog
+
+The u32 command codes. Replicated commands are identified on the wire by their `operation` byte (see [Operation discriminants](/docs/binary-protocol/framing#operation-discriminants)). The code column below is the protocol-level registry and, for non-replicated commands, the value carried in header bytes 196..200.
+
+```bash
+# System
+PING = 1 # non-replicated; works without login
+GET_STATS = 10 # non-replicated
+GET_SNAPSHOT = 11 # non-replicated
+GET_CLUSTER_METADATA = 12 # non-replicated
+DESCRIBE_OPTIONS = 13 # non-replicated
+GET_ME = 20 # non-replicated
+GET_CLIENT = 21 # non-replicated
+GET_CLIENTS = 22 # non-replicated
+
+# Users
+GET_USER = 31 # non-replicated
+GET_USERS = 32 # non-replicated
+CREATE_USER = 33 # operation 141
+DELETE_USER = 34 # operation 143
+UPDATE_USER = 35 # operation 142
+UPDATE_PERMISSIONS = 36 # operation 145
+CHANGE_PASSWORD = 37 # operation 144
+LOGIN_USER = 38 # legacy; server rejects with MalformedLogin eviction
+LOGOUT_USER = 39 # operation 3 (Logout)
+LOGIN_REGISTER = 40 # operation 1 (Register)
+LOGIN_REGISTER_WITH_PAT = 45 # operation 1 (Register)
+
+# Personal access tokens
+GET_PERSONAL_ACCESS_TOKENS = 41 # non-replicated
+CREATE_PERSONAL_ACCESS_TOKEN = 42 # operation 146
+DELETE_PERSONAL_ACCESS_TOKEN = 43 # operation 147
+LOGIN_WITH_PERSONAL_ACCESS_TOKEN = 44 # legacy; server rejects with MalformedLogin eviction
+
+# Messages
+POLL_MESSAGES = 100 # non-replicated
+SEND_MESSAGES = 101 # operation 160
+FLUSH_UNSAVED_BUFFER = 102 # parses, but always fails with FeatureUnavailable
+
+# Consumer offsets
+GET_CONSUMER_OFFSET = 120 # non-replicated
+STORE_CONSUMER_OFFSET = 121 # operation 161
+DELETE_CONSUMER_OFFSET = 122 # operation 162
+
+# Streams
+GET_STREAM = 200 # non-replicated
+GET_STREAMS = 201 # non-replicated
+CREATE_STREAM = 202 # operation 128
+DELETE_STREAM = 203 # operation 130
+UPDATE_STREAM = 204 # operation 129
+PURGE_STREAM = 205 # operation 131
+
+# Topics
+GET_TOPIC = 300 # non-replicated
+GET_TOPICS = 301 # non-replicated
+CREATE_TOPIC = 302 # operation 132
+DELETE_TOPIC = 303 # operation 134
+UPDATE_TOPIC = 304 # operation 133
+PURGE_TOPIC = 305 # operation 135
+
+# Partitions
+CREATE_PARTITIONS = 402 # operation 136
+DELETE_PARTITIONS = 403 # operation 137
+
+# Segments
+DELETE_SEGMENTS = 503 # operation 138
+
+# Consumer groups
+GET_CONSUMER_GROUP = 600 # non-replicated
+GET_CONSUMER_GROUPS = 601 # non-replicated
+CREATE_CONSUMER_GROUP = 602 # operation 139
+DELETE_CONSUMER_GROUP = 603 # operation 140
+JOIN_CONSUMER_GROUP = 604 # operation 148
+LEAVE_CONSUMER_GROUP = 605 # operation 149
+SYNC_CONSUMER_GROUP = 606 # non-replicated
+```
+
+`FLUSH_UNSAVED_BUFFER` still decodes on the wire, but the server has no on-demand flush primitive and answers every call with `FeatureUnavailable`. Per-topic durability is configured with the `enforce_fsync` [topic option](/docs/server/topic-options) instead.
+
+## Payloads
+
+Payloads below are the request **body** (the bytes after the 256-byte header). Types reference the [shared encodings](/docs/binary-protocol/encodings). Empty payload means the body has zero bytes.
+
+### System
+
+**Ping. Code: 1.** Empty payload. The only command accepted before login.
+
+**Get stats. Code: 10.** Empty payload.
+
+**Get snapshot. Code: 11.**
+
+```text
+[compression: u8][types_count: u8][snapshot_type: u8] x types_count
+```
+
+Compression codes: 1 = Stored, 2 = Deflated, 3 = Bzip2, 4 = Zstd, 5 = Lzma, 6 = Xz. Snapshot type codes: 1 = FilesystemOverview, 2 = ProcessList, 3 = ResourceUsage, 4 = Test, 5 = ServerLogs, 6 = ServerConfig, 100 = All.
+
+**Get cluster metadata. Code: 12.** Empty payload.
+
+**Describe options. Code: 13.**
+
+```text
+[scope: u8]
+```
+
+Scope: 1 = topic, 2 = stream, 3 = user. Returns the option catalog for the scope: keys, value kinds, defaults, and descriptions. See [Options block](/docs/binary-protocol/encodings#options-block).
+
+**Get me. Code: 20.** Empty payload.
+
+**Get client. Code: 21.**
+
+```text
+[client_id: u32]
+```
+
+**Get clients. Code: 22.** Empty payload.
+
+### Streams
+
+**Get stream. Code: 200.**
+
+```text
+[stream_id: Identifier]
+```
+
+**Get streams. Code: 201.** Empty payload.
+
+**Create stream. Code: 202.**
+
+```text
+[name_len: u8][name: N][options block to end]
+```
+
+The stream option catalog is empty today, so the block is normally empty (zero bytes).
+
+**Delete stream. Code: 203.**
+
+```text
+[stream_id: Identifier]
+```
+
+**Update stream. Code: 204.**
+
+```text
+[stream_id: Identifier][name_len: u8][name: N][options block to end]
+```
+
+Patch semantics: absent option keys are left unchanged.
+
+**Purge stream. Code: 205.**
+
+```text
+[stream_id: Identifier]
+```
+
+### Topics
+
+**Get topic. Code: 300.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier]
+```
+
+**Get topics. Code: 301.**
+
+```text
+[stream_id: Identifier]
+```
+
+**Create topic. Code: 302.**
+
+```text
+[stream_id: Identifier][partitions_count: u32][name_len: u8][name: N][options block to end]
+```
+
+The fixed fields are the shape of the operation: which stream, how many partitions, what name. Every topic setting (`compression_algorithm`, `message_expiry`, `max_topic_size`, `segment_size`, `enforce_fsync`, `messages_required_to_save`, `size_of_messages_required_to_save`, `preallocate_segments`) rides the options block. See [Topic options](/docs/server/topic-options). `partitions_count` is an argument, not a setting: it is consumed at admission and never persisted as an option.
+
+**Delete topic. Code: 303.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier]
+```
+
+**Update topic. Code: 304.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][name_len: u8][name: N][options block to end]
+```
+
+Only `compression_algorithm`, `message_expiry`, and `max_topic_size` are updatable. Absent keys are left unchanged.
+
+**Purge topic. Code: 305.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier]
+```
+
+### Partitions
+
+**Create partitions. Code: 402.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][partitions_count: u32]
+```
+
+**Delete partitions. Code: 403.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][partitions_count: u32]
+```
+
+### Segments
+
+**Delete segments. Code: 503.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][partition_id: u32][segments_count: u32]
+```
+
+Deletes the `segments_count` oldest sealed segments of the partition.
+
+### Messages
+
+**Poll messages. Code: 100.**
+
+```text
+[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
+[partition_flag: u8][partition_id: u32]
+[strategy: 9 bytes][count: u32][auto_commit: u8]
+```
+
+`strategy` is a [polling strategy](/docs/binary-protocol/encodings#polling-strategy). `count` is the requested number of messages. `auto_commit = 1` stores the consumer offset server-side as part of the poll.
+
+The response body:
+
+```text
+[partition_id: u32][current_offset: u64][messages_count: u32]
+[batch records to end]
+```
+
+The 16-byte prefix is followed by a stream of [batch records](/docs/binary-protocol/messages) served as stored: each record's header carries the stamped `base_offset` and `base_timestamp`, and each frame's deltas resolve against them. A record may be a server-sliced view of a larger stored batch, so the first polled offset is `base_offset + offset_delta` of the first frame, not necessarily `base_offset` itself. `current_offset` is the partition's newest offset at poll time.
+
+**Send messages. Code: 101.**
+
+```text
+[metadata_length: u32]
+[stream_id: Identifier][topic_id: Identifier][partitioning: Partitioning]
+[messages_count: u32]
+[batch: 256-byte batch header + frames]
+```
+
+`metadata_length` counts the bytes from `stream_id` through `messages_count` inclusive, so a reader can skip straight to the batch. The producer leaves `partition_id`, `base_offset`, and `base_timestamp` zero in the [batch header](/docs/binary-protocol/messages), and the server stamps them. Every checksum is producer-computed and verified at admission. The reserved regions must be zero.
+
+**Flush unsaved buffer. Code: 102.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][partition_id: u32][fsync: u8]
+```
+
+Parses, but the server always answers `FeatureUnavailable`: there is no on-demand flush primitive. Use the per-topic `enforce_fsync` option for durability guarantees.
+
+### Consumer offsets
+
+**Get consumer offset. Code: 120.**
+
+```text
+[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
+[partition_flag: u8][partition_id: u32]
+```
+
+Response body (20 bytes):
+
+```text
+[partition_id: u32][current_offset: u64][stored_offset: u64]
+```
+
+**Store consumer offset. Code: 121.**
+
+```text
+[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
+[partition_flag: u8][partition_id: u32][offset: u64][ack: u8]
+```
+
+**Delete consumer offset. Code: 122.**
+
+```text
+[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
+[partition_flag: u8][partition_id: u32][ack: u8]
+```
+
+The trailing `ack` byte is mandatory on both write commands. A payload without it fails to decode. Ack levels:
+
+| Value | Level | Meaning |
+|-------|-------|---------|
+| 0 | `NoAck` | Leader-local write; respond as soon as local state is updated (the fast path `PollMessages` auto-commit uses) |
+| 1 | `Quorum` | Replicate through partition consensus; respond after a quorum commit (default for explicit writes) |
+
+### Consumer groups
+
+**Get consumer group. Code: 600.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
+```
+
+**Get consumer groups. Code: 601.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier]
+```
+
+**Create consumer group. Code: 602.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][name_len: u8][name: N]
+```
+
+**Delete consumer group. Code: 603.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
+```
+
+**Join consumer group. Code: 604.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
+```
+
+**Leave consumer group. Code: 605.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
+```
+
+**Sync consumer group. Code: 606.**
+
+```text
+[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
+```
+
+Read-only: a member asks for its current partition assignment and the group generation, so it can select partitions to poll client-side.
+
+### Users
+
+**Get user. Code: 31.**
+
+```text
+[user_id: Identifier]
+```
+
+**Get users. Code: 32.** Empty payload.
+
+**Create user. Code: 33.**
+
+```text
+[username_len: u8][username: N][password_len: u8][password: N][status: u8]
+[has_permissions: u8][permissions_len: u32, only when has_permissions = 1][permissions: M bytes]
+[options block to end]
+```
+
+`permissions_len` and the permissions bytes are present **only** when `has_permissions = 1`. When it is 0 the options block follows immediately. The user option catalog is empty today, so the block is normally empty. `status`: 1 = active, 2 = inactive.
+
+**Delete user. Code: 34.**
+
+```text
+[user_id: Identifier]
+```
+
+**Update user. Code: 35.**
+
+```text
+[user_id: Identifier]
+[has_username: u8][username_len: u8, only when 1][username: N, only when 1]
+[has_status: u8][status: u8, only when 1]
+[options block to end]
+```
+
+**Update permissions. Code: 36.**
+
+```text
+[user_id: Identifier][has_permissions: u8]
+[permissions_len: u32, only when has_permissions = 1][permissions: M bytes, only when 1]
+```
+
+As in `CreateUser`, the length field is conditional: with `has_permissions = 0` the payload ends at the flag.
+
+**Change password. Code: 37.**
+
+```text
+[user_id: Identifier]
+[current_password_len: u8][current_password: N]
+[new_password_len: u8][new_password: N]
+```
+
+### Authentication
+
+Login and logout are part of the [connection lifecycle](/docs/binary-protocol/connection-lifecycle):
+
+- **Login register. Code: 40.** `Operation::Register`. The body is `ClientVersionInfo` + credentials, described in [Login-register](/docs/binary-protocol/connection-lifecycle#login-register).
+- **Login register with PAT. Code: 45.** `Operation::Register`. The body is `ClientVersionInfo` + token.
+- **Logout user. Code: 39.** `Operation::Logout`. Empty payload.
+- Codes **38** and **44** are the pre-register login shapes. The server rejects both with a `MalformedLogin` eviction.
+
+### Personal access tokens
+
+**Get personal access tokens. Code: 41.** Empty payload.
+
+**Create personal access token. Code: 42.**
+
+```text
+[name_len: u8][name: N][expiry: u64]
+```
+
+`expiry` is a duration in **microseconds**. Two sentinel values exist: `0` (server default, which currently means no expiry) and `u64::MAX` (never expires).
+
+**Delete personal access token. Code: 43.**
+
+```text
+[name_len: u8][name: N]
+```
diff --git a/content/docs/binary-protocol/connection-lifecycle.mdx b/content/docs/binary-protocol/connection-lifecycle.mdx
new file mode 100644
index 00000000..3c6e49d7
--- /dev/null
+++ b/content/docs/binary-protocol/connection-lifecycle.mdx
@@ -0,0 +1,65 @@
+---
+title: Connection lifecycle
+---
+
+## Protocol version
+
+The protocol version is a semver packed into one u32, 10 bits per component (each must be below 1024):
+
+```text
+bits 31..30 reserved (zero)
+bits 29..20 major
+bits 19..10 minor
+bits 9..0 patch
+value = major << 20 | minor << 10 | patch
+```
+
+Integer order equals semver order. The value tracks the `iggy_binary_protocol` crate release. Under 0.x the compatibility gate is minor-scoped: the server accepts a client whose packed version is at least the server's minimum and whose `major.minor` is at most the server's. Patch releases never change the wire, so the upper bound ignores patch. Past 1.0.0 the gate follows strict semver (major bump = incompatible).
+
+## Login-register
+
+The only way to authenticate is the register handshake: command code **40** (`LOGIN_REGISTER`, username and password) or **45** (`LOGIN_REGISTER_WITH_PAT`, personal access token). Both ride `operation = 1` (`Register`) with `session = 0` and a freshly minted non-zero `client` id.
+
+Both request bodies begin with the `ClientVersionInfo` prefix, so the server can gate on the version before touching credentials:
+
+```text
+[protocol_version: u32]
+[sdk_name_len: u8][sdk_name: UTF-8, 1-255 bytes]
+[sdk_version_len: u8][sdk_version: UTF-8, 1-255 bytes]
+```
+
+`protocol_version` is the packed version of the protocol the client was built against. `sdk_name` identifies the SDK (for example `rust-sdk`, `go-sdk`). `sdk_version` is the SDK's own build version.
+
+After the prefix:
+
+**`LOGIN_REGISTER` (code 40):**
+
+```text
+[ClientVersionInfo]
+[username_len: u8][username: N]
+[password_len: u8][password: N]
+[context_len: u32][context: N, only when context_len > 0]
+```
+
+**`LOGIN_REGISTER_WITH_PAT` (code 45):**
+
+```text
+[ClientVersionInfo]
+[token_len: u8][token: N]
+[context_len: u32][context: N, only when context_len > 0]
+```
+
+An incompatible protocol version is answered with a 256-byte `Eviction` frame, reason `14` (`IncompatibleProtocol`), carrying the accepted window at bytes 144 (max) and 148 (min) as packed u32 versions. A body without a decodable `ClientVersionInfo` prefix gets reason `15` (`MalformedLogin`) with a zero window. Bad credentials get reasons 9-11. See [EvictionHeader](/docs/binary-protocol/framing#evictionheader) for the frame layout.
+
+A successful login is a normal `Reply` whose body is:
+
+```text
+[user_id: u32]
+[session: u64]
+[server_protocol_version: u32]
+[server_version_len: u8][server_version: N]
+```
+
+The client stores `session` and echoes it in every subsequent request header. Logout is command code **39** (`LOGOUT_USER`), `operation = 3` (`Logout`), empty body.
+
+The legacy login commands **38** (`LOGIN_USER`) and **44** (`LOGIN_WITH_PERSONAL_ACCESS_TOKEN`) are refused: the server answers them with an `Eviction` frame, reason `15` (`MalformedLogin`). Only `PING` is accepted before login. Every other command on an unauthenticated connection is denied.
diff --git a/content/docs/binary-protocol/encodings.mdx b/content/docs/binary-protocol/encodings.mdx
new file mode 100644
index 00000000..aa55d3de
--- /dev/null
+++ b/content/docs/binary-protocol/encodings.mdx
@@ -0,0 +1,127 @@
+---
+title: Shared encodings
+---
+
+Primitives the [command payloads](/docs/binary-protocol/commands) are built from.
+
+## Name
+
+Length-prefixed UTF-8 string, used for stream, topic, group, user, and token names:
+
+```text
+[length: u8][value: UTF-8, 1-255 bytes]
+```
+
+## Identifier
+
+Polymorphic identifier of a stream, topic, user, or consumer group:
+
+```text
+[kind: u8][length: u8][value: N bytes]
+```
+
+- Numeric: `kind = 1`, `length = 4`, value is a u32.
+- String: `kind = 2`, `length = 1..255`, value is UTF-8 bytes.
+
+## Consumer
+
+Identifies a consumer or a consumer group in message and offset commands:
+
+```text
+[kind: u8][id: Identifier]
+```
+
+- `kind = 1`: consumer.
+- `kind = 2`: consumer group.
+
+## Optional partition id
+
+Several commands take an optional partition id encoded as 5 fixed bytes:
+
+```text
+[flag: u8][partition_id: u32]
+```
+
+`flag = 1` means the id is set. `flag = 0` means none (the value bytes are then zero). For consumer-group consumers the partition id is left unset and the server resolves it.
+
+## Partitioning
+
+How `SendMessages` picks the target partition:
+
+```text
+[kind: u8][length: u8][value: 0-255 bytes]
+```
+
+- `Balanced`: `kind = 1`, `length = 0`, no value. The server resolves the partition at admission with a per-node round-robin counter.
+- `PartitionId`: `kind = 2`, `length = 4`, value is a u32 partition id.
+- `MessagesKey`: `kind = 3`, `length = 1..255`, value is a routing key hashed to a partition.
+
+First-party binary SDKs pre-resolve balanced and key routing client-side and normally send `kind = 2`.
+
+## Polling strategy
+
+Fixed 9 bytes:
+
+```text
+[kind: u8][value: u64]
+```
+
+| Kind | Name | Meaning of `value` |
+|------|------|--------------------|
+| 1 | Offset | Start at this offset |
+| 2 | Timestamp | Start at this timestamp (microseconds) |
+| 3 | First | Start from the beginning (`value` ignored) |
+| 4 | Last | Start from the end (`value` ignored) |
+| 5 | Next | Continue after the stored consumer offset (`value` ignored) |
+
+## Header kinds
+
+Typed values used by user headers and option values. The kind byte:
+
+| Code | Kind | Code | Kind |
+|------|------|------|------|
+| 1 | Raw | 9 | Uint8 |
+| 2 | String | 10 | Uint16 |
+| 3 | Bool | 11 | Uint32 |
+| 4 | Int8 | 12 | Uint64 |
+| 5 | Int16 | 13 | Uint128 |
+| 6 | Int32 | 14 | Float32 |
+| 7 | Int64 | 15 | Float64 |
+| 8 | Int128 | | |
+
+Integers are little-endian. `Bool` is one byte.
+
+## User headers block
+
+A user headers block (inside a [message frame](/docs/binary-protocol/messages)) is a flat run of TLV fields:
+
+```text
+[kind: u8][length: u32][data: length bytes]
+```
+
+Fields pair up: first the key, then the value. Keys are `String` kind. Every `length` must be 1..=255, kind 0 is rejected, and the block must consume its byte range exactly. Unknown value kind codes are preserved and forwarded, so headers survive mixed-version clusters.
+
+## Options block
+
+`CreateStream`, `UpdateStream`, `CreateTopic`, `UpdateTopic`, `CreateUser`, and `UpdateUser` end with a key-value **options block** that reuses the user-headers TLV encoding. The block runs from its start to the end of the payload (no length prefix, and the validator requires exact consumption) and may be empty.
+
+On top of the TLV walk, options enforce: string keys only (kind 2, valid UTF-8), no duplicate keys, at most 1024 entries, and at most 100000 bytes total.
+
+Semantics:
+
+- **Create** requests resolve absent keys to server defaults and persist the effective values, so `GetTopic` always shows what is in force.
+- **Update** requests are patches: keys absent from the block are left alone, never reset. A client built before a key existed cannot erase it.
+- Unknown keys are rejected at the wire edge, never silently skipped.
+
+The catalog is discoverable at runtime with `DESCRIBE_OPTIONS` (code 13), payload `[scope: u8]` with scope `1` = topic, `2` = stream, `3` = user (HTTP: `GET /options/topic`). Today only topics have keys. The stream and user catalogs are empty, and any key sent for them is rejected. The topic catalog (`segment_size`, `enforce_fsync`, `message_expiry`, `max_topic_size`, and the rest) with defaults and constraints is documented on the [Topic options](/docs/server/topic-options) page. `Updat [...]
+
+## Compression
+
+The `compression_algorithm` value used by topic options:
+
+| Code / name | Algorithm |
+|-------------|-----------|
+| 1, `none` | No compression (default) |
+| 2, `gzip` | Gzip |
+
+Any other value is rejected. The option is a **placeholder today**: the value is validated, persisted, and echoed back, but neither the server nor the SDKs compress or decompress payloads yet. To compress today, do it client-side and tag messages via user headers - see the [message headers examples](https://github.com/apache/iggy/tree/master/examples/rust/src/message-headers) in the Iggy repo.
diff --git a/content/docs/binary-protocol/framing.mdx b/content/docs/binary-protocol/framing.mdx
new file mode 100644
index 00000000..ec0fc080
--- /dev/null
+++ b/content/docs/binary-protocol/framing.mdx
@@ -0,0 +1,194 @@
+---
+title: Framing
+---
+
+Every message on a binary transport is a fixed **256-byte header** followed by an optional body. The header is a `#[repr(C)]` struct decoded by pointer cast (zero-copy), so field offsets are fixed and enforced at compile time in the server. Three header shapes cross the client boundary:
+
+- `RequestHeader`: client to server.
+- `ReplyHeader`: server to client, answers one request.
+- `EvictionHeader`: server to client, session-terminal rejection with no body.
+
+The header shape is identified by the `command` byte at offset 60. Values a client sends or receives:
+
+| `command` | Value | Direction | Meaning |
+|-----------|-------|-----------|---------|
+| `Request` | 5 | client to server | A command request |
+| `Reply` | 8 | server to client | Answer to one request |
+| `Eviction` | 13 | server to client | Session is dead, no per-request correlation |
+
+Other `command` values (Prepare, PrepareOk, view-change traffic, and so on) are replica-to-replica consensus messages and never appear on a client connection. They are documented on the [Server-to-server](/docs/binary-protocol/cluster) page.
+
+The `size` field at offset 48 is the total frame length in bytes, header included. A body, when present, immediately follows the 256 header bytes and has length `size - 256`. Readers can decode `size` before typing the header. That offset is a protocol constant.
+
+## RequestHeader
+
+Client to server. 256 bytes.
+
+| Offset | Size | Field | Type | Description |
+|--------|------|-------|------|-------------|
+| 0 | 16 | `checksum` | u128 | Frame seal. Not used on client-facing frames; send zero. |
+| 16 | 16 | `checksum_body` | u128 | Body seal. Not used on client-facing frames; send zero. |
+| 32 | 16 | `cluster` | u128 | Cluster id. Send zero. |
+| 48 | 4 | `size` | u32 | Total frame length: 256 + body length. |
+| 52 | 4 | `view` | u32 | Consensus view. Send zero. |
+| 56 | 4 | `release` | u32 | Must be zero. |
+| 60 | 1 | `command` | u8 | `5` (`Request`). |
+| 61 | 1 | `replica` | u8 | Send zero. |
+| 62 | 66 | reserved | bytes | Zero. |
+| 128 | 16 | `client` | u128 | Client-chosen session identity, non-zero. Minted fresh for each registration. |
+| 144 | 16 | `request_checksum` | u128 | Optional integrity stamp over the request body (the Rust SDK uses XxHash3-64 widened to u128). Lets the server's client table catch a `request` number reused for different arguments. Zero disables the comparison. Stamped only for metadata-plane operations; zero for partition-plane and non-replicated ones. |
+| 160 | 8 | `timestamp` | u64 | Informational; the server echoes it into `ReplyHeader.timestamp`. May be zero. |
+| 168 | 8 | `request` | u64 | Request number, per client. See [request numbering](#request-numbering). |
+| 176 | 1 | `operation` | u8 | The [`Operation`](#operation-discriminants) discriminant. |
+| 177 | 7 | padding | bytes | Zero. |
+| 184 | 8 | `session` | u64 | Session fence epoch from the login reply. Zero on the login-register request itself and on non-replicated operations sent before login (only `PING` is accepted unauthenticated). |
+| 192 | 4 | `user_id` | u32 | Ignored on the wire; the server stamps the authenticated user itself. Send zero. |
+| 196 | 60 | reserved | bytes | Zero, except: for `operation = 2` (`NonReplicated`) bytes 196..200 carry the u32 command code, little-endian. |
+
+## Operation discriminants
+
+The `operation` byte tells the server which state-machine operation the request carries. Replicated operations are identified by `operation` alone. Non-replicated operations (reads, ping) all use `operation = 2` and carry their concrete u32 command code in header bytes 196..200.
+
+Values a client may send:
+
+| Value | Operation | Plane |
+|-------|-----------|-------|
+| 1 | `Register` | session (login handshake, codes 40 and 45) |
+| 2 | `NonReplicated` | reads and ping; the u32 code rides bytes 196..200 |
+| 3 | `Logout` | session (code 39) |
+| 128 | `CreateStream` | metadata |
+| 129 | `UpdateStream` | metadata |
+| 130 | `DeleteStream` | metadata |
+| 131 | `PurgeStream` | metadata |
+| 132 | `CreateTopic` | metadata |
+| 133 | `UpdateTopic` | metadata |
+| 134 | `DeleteTopic` | metadata |
+| 135 | `PurgeTopic` | metadata |
+| 136 | `CreatePartitions` | metadata |
+| 137 | `DeletePartitions` | metadata |
+| 138 | `DeleteSegments` | metadata (resolved to an internal partition truncation) |
+| 139 | `CreateConsumerGroup` | metadata |
+| 140 | `DeleteConsumerGroup` | metadata |
+| 141 | `CreateUser` | metadata |
+| 142 | `UpdateUser` | metadata |
+| 143 | `DeleteUser` | metadata |
+| 144 | `ChangePassword` | metadata |
+| 145 | `UpdatePermissions` | metadata |
+| 146 | `CreatePersonalAccessToken` | metadata |
+| 147 | `DeletePersonalAccessToken` | metadata |
+| 148 | `JoinConsumerGroup` | metadata |
+| 149 | `LeaveConsumerGroup` | metadata |
+| 160 | `SendMessages` | partition |
+| 161 | `StoreConsumerOffset` | partition |
+| 162 | `DeleteConsumerOffset` | partition |
+
+Value 0 is reserved and rejected. The 64..127 range is reserved for server-internal operations (currently 64 through 68 are assigned) and every value in it is refused from clients.
+
+The planes matter for delivery semantics:
+
+- **Metadata operations** replicate through the metadata consensus group. The server deduplicates them by `(client, request)` and caches replies, so a retried request gets the cached answer instead of a double apply (exactly-once).
+- **Partition operations** replicate through their partition's consensus group. They are at-least-once: no reply cache, a replay may apply again.
+- **Non-replicated operations** are reads. They bypass consensus and deduplication entirely.
+
+## Request numbering
+
+`request` is a per-client counter the server's client table tracks for metadata-plane operations:
+
+- Metadata operations must send a strictly increasing `request` (the SDK advances the counter per metadata request).
+- Partition operations and non-replicated operations send the current counter value without advancing it. The server doesn't track theirs.
+- The counter is a watermark, not a contiguous sequence: any value above the last accepted one is admissible.
+
+`session` is the fence epoch: the value handed back by the login reply. Every request after login must echo it. When the same `client` id registers again, the new registration mints a higher epoch and requests carrying the old one are fenced (rejected as zombies).
+
+## ReplyHeader
+
+Server to client. 256 bytes, followed by `size - 256` bytes of body.
+
+| Offset | Size | Field | Type | Description |
+|--------|------|-------|------|-------------|
+| 0 | 16 | `checksum` | u128 | Zero on client-facing frames. |
+| 16 | 16 | `checksum_body` | u128 | Zero on client-facing frames. |
+| 32 | 16 | `cluster` | u128 | Cluster id. |
+| 48 | 4 | `size` | u32 | Total frame length: 256 + body length. |
+| 52 | 4 | `view` | u32 | Consensus view the reply was produced in. |
+| 56 | 4 | `release` | u32 | Zero. |
+| 60 | 1 | `command` | u8 | `8` (`Reply`). |
+| 61 | 1 | `replica` | u8 | Answering replica index. |
+| 62 | 66 | reserved | bytes | Zero. |
+| 128 | 16 | `request_checksum` | u128 | Echoed from the request. |
+| 144 | 16 | `context` | u128 | Server context. |
+| 160 | 16 | `client` | u128 | Echoed client id. |
+| 176 | 8 | `op` | u64 | Log position of the committed operation. |
+| 184 | 8 | `commit` | u64 | Commit point at reply time. |
+| 192 | 8 | `timestamp` | u64 | Echo of the request `timestamp`. |
+| 200 | 8 | `request` | u64 | Echoed request number; correlate replies by this. |
+| 208 | 1 | `operation` | u8 | Echoed operation discriminant. |
+| 209 | 7 | padding | bytes | Zero. |
+| 216 | 4 | `status` | u32 | `0` = accepted. Nonzero = an `IggyError` code for a failure decided **before** commit (authorization denial, admission reject). A nonzero status always comes with an empty body. |
+| 220 | 36 | reserved | bytes | Zero. |
+
+Decode order for a client:
+
+1. Read 256 bytes, check `command`. `13` (`Eviction`) means the session is dead: map the eviction reason to an error and stop. `8` (`Reply`) continues.
+2. Read the remaining `size - 256` body bytes.
+3. If `status` is nonzero, the request failed pre-commit. The status value is the `IggyError` code and the body is empty.
+4. Otherwise decode the body. For result-framed operations, strip the [result section](#result-section) first.
+
+## Result section
+
+Replies for all metadata operations and for the partition-plane consumer-offset writes (`StoreConsumerOffset`, `DeleteConsumerOffset`) are **result-framed**: the body starts with a committed-result section ahead of the typed payload.
+
+```text
+[count: u32]
+count x { index: u32, result: u32 }
+```
+
+- Success: `count = 0`, and the typed response payload (if any) follows the 4 count bytes.
+- Committed business rejection: one entry `{ index: 0, result: error_code }` and no payload.
+
+The header `status` channel and the result section are mutually exclusive by construction: a reply either failed pre-commit (`status` nonzero, empty body) or committed (`status` zero, result section present). A login-register reply carries the result section only when non-empty. On success its body starts directly with the [login response payload](/docs/binary-protocol/connection-lifecycle#login-register).
+
+Replies to non-replicated commands aren't result-framed: after `status = 0` the body is the response payload directly.
+
+## EvictionHeader
+
+Server to client. 256 bytes, never a body. An eviction is session-terminal: it says "this session is dead", carries no per-request correlation, and a client should deinitialize and re-login.
+
+| Offset | Size | Field | Type | Description |
+|--------|------|-------|------|-------------|
+| 0 | 16 | `checksum` | u128 | Zero on client-facing frames. |
+| 16 | 16 | `checksum_body` | u128 | Zero on client-facing frames. |
+| 32 | 16 | `cluster` | u128 | Cluster id. |
+| 48 | 4 | `size` | u32 | Always 256: an eviction has no body. |
+| 52 | 4 | `view` | u32 | |
+| 56 | 4 | `release` | u32 | Zero. |
+| 60 | 1 | `command` | u8 | `13` (`Eviction`). |
+| 61 | 1 | `replica` | u8 | |
+| 62 | 66 | reserved | bytes | Zero. |
+| 128 | 16 | `client` | u128 | The evicted client id. |
+| 144 | 4 | `server_protocol_version` | u32 | Accepted protocol window (max), packed semver. Set only for reason 14, zero otherwise. |
+| 148 | 4 | `server_protocol_version_min` | u32 | Accepted protocol window (min). Set only for reason 14, zero otherwise. |
+| 152 | 103 | reserved | bytes | Zero. |
+| 255 | 1 | `reason` | u8 | Eviction reason, see below. |
+
+Eviction reasons:
+
+| Value | Reason | Meaning |
+|-------|--------|---------|
+| 1 | `NoSession` | No session exists for this client id |
+| 2 | `ClientReleaseTooLow` | Client release below cluster minimum |
+| 3 | `ClientReleaseTooHigh` | Client release above cluster maximum |
+| 4 | `InvalidRequestOperation` | Unknown `operation` discriminant |
+| 5 | `InvalidRequestBody` | Body failed validation |
+| 6 | `InvalidRequestBodySize` | Body size mismatch |
+| 7 | `SessionTooLow` | Session epoch below the cluster's retained minimum |
+| 8 | `SessionReleaseMismatch` | Session bound to a different release |
+| 9 | `InvalidCredentials` | Login refused: bad username or password |
+| 10 | `InvalidToken` | Login refused: bad personal access token |
+| 11 | `UserInactive` | Login refused: user inactive |
+| 12 | `SessionError` | Session-level failure |
+| 13 | `StaleClient` | Missed heartbeats; server evicted the session |
+| 14 | `IncompatibleProtocol` | Client protocol version outside the accepted window (see bytes 144..152) |
+| 15 | `MalformedLogin` | Login body without a decodable version prefix, or a legacy login code |
+
+Value 0 is reserved and never sent.
diff --git a/content/docs/binary-protocol/index.mdx b/content/docs/binary-protocol/index.mdx
new file mode 100644
index 00000000..d37057f8
--- /dev/null
+++ b/content/docs/binary-protocol/index.mdx
@@ -0,0 +1,20 @@
+---
+title: Binary Protocol
+---
+
+Iggy speaks one binary protocol over its three binary transports: TCP, QUIC, and WebSocket. Commands, responses, data models, and status codes are the same on all of them. The HTTP transport is separate: it exposes the same operations as JSON REST endpoints, listed in [server.http](https://github.com/apache/iggy/blob/master/core/server/server.http).
+
+This section describes **binary protocol version 0.11.0**. The protocol version is the semver of the `iggy_binary_protocol` crate, and it's exchanged and checked during login (see [Connection lifecycle](/docs/binary-protocol/connection-lifecycle)). The crate itself is the source of truth: every request and response module carries a `Wire format:` doc comment, and these pages are written against those.
+
+All multi-byte integers are **little-endian** unless stated otherwise.
+
+The same 256-byte header layout carries two traffic planes. The pages below cover both:
+
+- **Server ↔ SDK client**, everything a client sends or receives:
+ - [Framing](/docs/binary-protocol/framing): the 256-byte headers every client-facing message rides in.
+ - [Connection lifecycle](/docs/binary-protocol/connection-lifecycle): protocol versioning and the login-register handshake.
+ - [Commands](/docs/binary-protocol/commands): the command codes and per-command payload layouts.
+ - [Shared encodings](/docs/binary-protocol/encodings): primitives the payloads are built from.
+ - [Message batches](/docs/binary-protocol/messages): the one layout messages travel and rest in.
+- **Server ↔ server**, the replica-to-replica consensus traffic that never appears on a client connection:
+ - [Server-to-server](/docs/binary-protocol/cluster): prepare/ack replication, view changes, and state transfer.
diff --git a/content/docs/binary-protocol/messages.mdx b/content/docs/binary-protocol/messages.mdx
new file mode 100644
index 00000000..12634a88
--- /dev/null
+++ b/content/docs/binary-protocol/messages.mdx
@@ -0,0 +1,41 @@
+---
+title: Message batches
+---
+
+Messages travel and rest in exactly one layout, the **batch**: the `SendMessages` body, the replicated record, the persisted segment record, and the poll reply all share it. There is no other message encoding.
+
+```text
+[batch header: 256 bytes][blob: message frames]
+frame = [frame header: 48 bytes][payload][user_headers]
+```
+
+## Batch header (256 bytes)
+
+| Offset | Size | Field | Type | Description |
+|--------|------|-------|------|-------------|
+| 0 | 8 | `partition_id` | u64 | Server-stamped at admission. Producers send zero. |
+| 8 | 8 | `base_offset` | u64 | Server-stamped when the batch is appended to the partition journal. Producers send zero. |
+| 16 | 8 | `base_timestamp` | u64 | Server-stamped at journal append (microseconds). Producers send zero. |
+| 24 | 8 | `origin_timestamp` | u64 | Producer-set: the earliest `origin_timestamp` among the batch's messages (microseconds). |
+| 32 | 8 | `batch_length` | u64 | Total batch size: 256 + blob length. |
+| 40 | 8 | `batch_checksum` | u64 | XxHash3-64 over the header fields (`partition_id`, `base_offset`, `base_timestamp`, `origin_timestamp`, `batch_length`, `message_count`, in that order, little-endian bytes) followed by each frame's 8 checksum bytes. Recomputed by the server when it stamps the header. |
+| 48 | 4 | `message_count` | u32 | Number of frames in the blob. |
+| 52 | 204 | reserved | bytes | Must be zero; the server rejects a batch with nonzero reserved bytes. |
+
+## Message frame header (48 bytes)
+
+| Offset | Size | Field | Type | Description |
+|--------|------|-------|------|-------------|
+| 0 | 8 | `checksum` | u64 | XxHash3-64 over frame bytes 8..48 followed by `payload` and `user_headers`. Producer-computed; verified at admission. |
+| 8 | 16 | `id` | u128 | Message id (for example a UUID). |
+| 24 | 4 | `offset_delta` | u32 | Message offset = `base_offset + offset_delta`. Producer sends the frame's index in the batch. |
+| 28 | 4 | `timestamp_delta` | u32 | Microsecond delta against the batch `origin_timestamp`. One batch spans at most about 71.6 minutes of producer clock (`u32` microseconds). |
+| 32 | 4 | `user_headers_length` | u32 | Byte length of the user headers block. |
+| 36 | 4 | `payload_length` | u32 | Byte length of the payload. |
+| 40 | 8 | reserved | u64 | Must be zero. |
+
+After the 48 header bytes: `payload` (`payload_length` bytes), then `user_headers` (`user_headers_length` bytes). The payload comes **first**.
+
+A frame's absolute position is derived, not stored: offset = `base_offset + offset_delta`. The server-stamped time is the flat batch `base_timestamp` (the delta does not apply to it); producer time resolves as `origin_timestamp + timestamp_delta`.
+
+The 64-byte `IggyMessageHeader` (checksum, id, offset, timestamp, origin_timestamp, lengths, reserved) that earlier protocol generations put in front of every message **no longer exists on the wire or on disk**. It survives only as an in-memory model.
diff --git a/content/docs/binary-protocol/meta.json b/content/docs/binary-protocol/meta.json
new file mode 100644
index 00000000..45e3f6c9
--- /dev/null
+++ b/content/docs/binary-protocol/meta.json
@@ -0,0 +1,4 @@
+{
+ "title": "Binary Protocol",
+ "pages": ["index", "framing", "connection-lifecycle", "commands", "encodings", "messages", "cluster"]
+}
diff --git a/content/docs/introduction/concepts.mdx b/content/docs/introduction/concepts.mdx
index c0d74223..47ff815f 100644
--- a/content/docs/introduction/concepts.mdx
+++ b/content/docs/introduction/concepts.mdx
@@ -35,7 +35,7 @@ The stream is a logical concept, and you might think of it as a **namespace**. F
The topic is also the logical concept, which is a part of the stream. The topic is identified by its unique ID. You could think of topic as an entity being responsible for storing the specific type of the records. For example, you could have a topic for the user events, and another topic for the order events, etc.
-The messages are not being stored in the topic directly, but rather in the **partitions**, which are assigned to the topic. The topic can have one or more partitions assigned, that could help achieve higher parallelism and throughput. The topic can also have the **retention policy** assigned, which means that the records are being deleted automatically once they are older than the specified retention period. Topics also support configurable compression (`none`, `gzip`), maximum size limi [...]
+The messages are not being stored in the topic directly, but rather in the **partitions**, which are assigned to the topic. The topic can have one or more partitions assigned, that could help achieve higher parallelism and throughput. The topic can also have the **retention policy** assigned, which means that the records are being deleted automatically once they are older than the specified retention period. Topics also support maximum size limits and per-topic durability options (`segme [...]
## Partition
@@ -74,7 +74,7 @@ Consumer groups provide horizontal scaling for message consumption. When multipl
<MessageHeaderDiagram />
-In the SDKs, each message carries a 64-byte in-memory header (little-endian fields). On the wire and on disk, messages travel inside batch records with a compact 48-byte per-message frame. See the [binary schema](/docs/server/schema) for the exact encodings.
+In the SDKs, each message carries a 64-byte in-memory header (little-endian fields). On the wire and on disk, messages travel inside batch records with a compact 48-byte per-message frame. See the [Binary Protocol](/docs/binary-protocol) section for the exact encodings.
| Field | Bytes | Type | Description |
|-------|-------|------|-------------|
diff --git a/content/docs/meta.json b/content/docs/meta.json
index 4737886a..3c2bc42b 100644
--- a/content/docs/meta.json
+++ b/content/docs/meta.json
@@ -5,6 +5,7 @@
"introduction",
"server",
"clustering",
+ "binary-protocol",
"connectors",
"sdk",
"cli",
diff --git a/content/docs/server/meta.json b/content/docs/server/meta.json
index 3d3ec62d..4c9c3bd6 100644
--- a/content/docs/server/meta.json
+++ b/content/docs/server/meta.json
@@ -1,4 +1,4 @@
{
"title": "Server",
- "pages": ["introduction", "configuration", "topic-options", "storage-engine", "networking", "security", "docker", "schema", "benchmarking"]
+ "pages": ["introduction", "configuration", "topic-options", "storage-engine", "networking", "security", "docker", "benchmarking"]
}
diff --git a/content/docs/server/networking.mdx b/content/docs/server/networking.mdx
index a136e68e..7948f9fd 100644
--- a/content/docs/server/networking.mdx
+++ b/content/docs/server/networking.mdx
@@ -17,7 +17,7 @@ Iggy supports four transport protocols simultaneously, each optimized for differ
## Binary protocol
-Every request and reply on the stateful transports starts with a fixed 256-byte header (the same Viewstamped Replication header layout the cluster uses internally), followed by the command payload. The `iggy_binary_protocol` crate provides zero-copy serialization for these headers, and foreign SDKs implement the same contract natively. See the [protocol schema](/docs/server/schema) for the full specification.
+Every request and reply on the stateful transports starts with a fixed 256-byte header (the same Viewstamped Replication header layout the cluster uses internally), followed by the command payload. The `iggy_binary_protocol` crate provides zero-copy serialization for these headers, and foreign SDKs implement the same contract natively. See the [Binary Protocol](/docs/binary-protocol) section for the full specification.
## Connection handling across shards
diff --git a/content/docs/server/schema.mdx b/content/docs/server/schema.mdx
deleted file mode 100644
index 83a3ebac..00000000
--- a/content/docs/server/schema.mdx
+++ /dev/null
@@ -1,857 +0,0 @@
----
-title: Schema
----
-
-Iggy speaks one binary protocol over its three binary transports: TCP, QUIC, and WebSocket. Commands, responses, data models, and status codes are the same on all of them. The HTTP transport is separate: it exposes the same operations as JSON REST endpoints, listed in [server.http](https://github.com/apache/iggy/blob/master/core/server/server.http).
-
-This page describes **binary protocol version 0.11.0**. The protocol version is the semver of the `iggy_binary_protocol` crate, and it's exchanged and checked during login (see [Connection lifecycle](#connection-lifecycle)). The crate itself is the source of truth: every request and response module carries a `Wire format:` doc comment, and this page is generated against those.
-
-All multi-byte integers are **little-endian** unless stated otherwise.
-
-The page is organized in three layers:
-
-1. [Framing](#framing): the 256-byte consensus headers every message rides in.
-2. [Command catalog](#command-catalog): the command codes and how each maps onto the framing.
-3. [Payload encodings](#shared-encodings): shared primitives, the message batch format, and per-command payload layouts.
-
-## Framing
-
-Every message on a binary transport is a fixed **256-byte header** followed by an optional body. The header is a `#[repr(C)]` struct decoded by pointer cast (zero-copy), so field offsets are fixed and enforced at compile time in the server. Three header shapes cross the client boundary:
-
-- `RequestHeader`: client to server.
-- `ReplyHeader`: server to client, answers one request.
-- `EvictionHeader`: server to client, session-terminal rejection with no body.
-
-The header shape is identified by the `command` byte at offset 60. Values a client sends or receives:
-
-| `command` | Value | Direction | Meaning |
-|-----------|-------|-----------|---------|
-| `Request` | 5 | client to server | A command request |
-| `Reply` | 8 | server to client | Answer to one request |
-| `Eviction` | 13 | server to client | Session is dead, no per-request correlation |
-
-Other `command` values (Prepare, PrepareOk, view-change traffic, and so on) are replica-to-replica consensus messages and never appear on a client connection.
-
-The `size` field at offset 48 is the total frame length in bytes, header included. A body, when present, immediately follows the 256 header bytes and has length `size - 256`. Readers can decode `size` before typing the header. That offset is a protocol constant.
-
-### RequestHeader
-
-Client to server. 256 bytes.
-
-| Offset | Size | Field | Type | Description |
-|--------|------|-------|------|-------------|
-| 0 | 16 | `checksum` | u128 | Frame seal. Not used on client-facing frames; send zero. |
-| 16 | 16 | `checksum_body` | u128 | Body seal. Not used on client-facing frames; send zero. |
-| 32 | 16 | `cluster` | u128 | Cluster id. Send zero. |
-| 48 | 4 | `size` | u32 | Total frame length: 256 + body length. |
-| 52 | 4 | `view` | u32 | Consensus view. Send zero. |
-| 56 | 4 | `release` | u32 | Must be zero. |
-| 60 | 1 | `command` | u8 | `5` (`Request`). |
-| 61 | 1 | `replica` | u8 | Send zero. |
-| 62 | 66 | reserved | bytes | Zero. |
-| 128 | 16 | `client` | u128 | Client-chosen session identity, non-zero. Minted fresh for each registration. |
-| 144 | 16 | `request_checksum` | u128 | Optional integrity stamp over the request body (the Rust SDK uses XxHash3-64 widened to u128). Lets the server's client table catch a `request` number reused for different arguments. Zero disables the comparison. Stamped only for metadata-plane operations; zero for partition-plane and non-replicated ones. |
-| 160 | 8 | `timestamp` | u64 | Informational; the server echoes it into `ReplyHeader.timestamp`. May be zero. |
-| 168 | 8 | `request` | u64 | Request number, per client. See [request numbering](#request-numbering). |
-| 176 | 1 | `operation` | u8 | The [`Operation`](#operation-discriminants) discriminant. |
-| 177 | 7 | padding | bytes | Zero. |
-| 184 | 8 | `session` | u64 | Session fence epoch from the login reply. Zero on the login-register request itself and on non-replicated operations sent before login (only `PING` is accepted unauthenticated). |
-| 192 | 4 | `user_id` | u32 | Ignored on the wire; the server stamps the authenticated user itself. Send zero. |
-| 196 | 60 | reserved | bytes | Zero, except: for `operation = 2` (`NonReplicated`) bytes 196..200 carry the u32 command code, little-endian. |
-
-### Operation discriminants
-
-The `operation` byte tells the server which state-machine operation the request carries. Replicated operations are identified by `operation` alone. Non-replicated operations (reads, ping) all use `operation = 2` and carry their concrete u32 command code in header bytes 196..200.
-
-Values a client may send:
-
-| Value | Operation | Plane |
-|-------|-----------|-------|
-| 1 | `Register` | session (login handshake, codes 40 and 45) |
-| 2 | `NonReplicated` | reads and ping; the u32 code rides bytes 196..200 |
-| 3 | `Logout` | session (code 39) |
-| 128 | `CreateStream` | metadata |
-| 129 | `UpdateStream` | metadata |
-| 130 | `DeleteStream` | metadata |
-| 131 | `PurgeStream` | metadata |
-| 132 | `CreateTopic` | metadata |
-| 133 | `UpdateTopic` | metadata |
-| 134 | `DeleteTopic` | metadata |
-| 135 | `PurgeTopic` | metadata |
-| 136 | `CreatePartitions` | metadata |
-| 137 | `DeletePartitions` | metadata |
-| 138 | `DeleteSegments` | metadata (resolved to an internal partition truncation) |
-| 139 | `CreateConsumerGroup` | metadata |
-| 140 | `DeleteConsumerGroup` | metadata |
-| 141 | `CreateUser` | metadata |
-| 142 | `UpdateUser` | metadata |
-| 143 | `DeleteUser` | metadata |
-| 144 | `ChangePassword` | metadata |
-| 145 | `UpdatePermissions` | metadata |
-| 146 | `CreatePersonalAccessToken` | metadata |
-| 147 | `DeletePersonalAccessToken` | metadata |
-| 148 | `JoinConsumerGroup` | metadata |
-| 149 | `LeaveConsumerGroup` | metadata |
-| 160 | `SendMessages` | partition |
-| 161 | `StoreConsumerOffset` | partition |
-| 162 | `DeleteConsumerOffset` | partition |
-
-Value 0 is reserved and rejected. The 64..127 range is reserved for server-internal operations (currently 64 through 68 are assigned) and every value in it is refused from clients.
-
-The planes matter for delivery semantics:
-
-- **Metadata operations** replicate through the metadata consensus group. The server deduplicates them by `(client, request)` and caches replies, so a retried request gets the cached answer instead of a double apply (exactly-once).
-- **Partition operations** replicate through their partition's consensus group. They are at-least-once: no reply cache, a replay may apply again.
-- **Non-replicated operations** are reads. They bypass consensus and deduplication entirely.
-
-### Request numbering
-
-`request` is a per-client counter the server's client table tracks for metadata-plane operations:
-
-- Metadata operations must send a strictly increasing `request` (the SDK advances the counter per metadata request).
-- Partition operations and non-replicated operations send the current counter value without advancing it. The server doesn't track theirs.
-- The counter is a watermark, not a contiguous sequence: any value above the last accepted one is admissible.
-
-`session` is the fence epoch: the value handed back by the login reply. Every request after login must echo it. When the same `client` id registers again, the new registration mints a higher epoch and requests carrying the old one are fenced (rejected as zombies).
-
-### ReplyHeader
-
-Server to client. 256 bytes, followed by `size - 256` bytes of body.
-
-| Offset | Size | Field | Type | Description |
-|--------|------|-------|------|-------------|
-| 0 | 16 | `checksum` | u128 | Zero on client-facing frames. |
-| 16 | 16 | `checksum_body` | u128 | Zero on client-facing frames. |
-| 32 | 16 | `cluster` | u128 | Cluster id. |
-| 48 | 4 | `size` | u32 | Total frame length: 256 + body length. |
-| 52 | 4 | `view` | u32 | Consensus view the reply was produced in. |
-| 56 | 4 | `release` | u32 | Zero. |
-| 60 | 1 | `command` | u8 | `8` (`Reply`). |
-| 61 | 1 | `replica` | u8 | Answering replica index. |
-| 62 | 66 | reserved | bytes | Zero. |
-| 128 | 16 | `request_checksum` | u128 | Echoed from the request. |
-| 144 | 16 | `context` | u128 | Server context. |
-| 160 | 16 | `client` | u128 | Echoed client id. |
-| 176 | 8 | `op` | u64 | Log position of the committed operation. |
-| 184 | 8 | `commit` | u64 | Commit point at reply time. |
-| 192 | 8 | `timestamp` | u64 | Echo of the request `timestamp`. |
-| 200 | 8 | `request` | u64 | Echoed request number; correlate replies by this. |
-| 208 | 1 | `operation` | u8 | Echoed operation discriminant. |
-| 209 | 7 | padding | bytes | Zero. |
-| 216 | 4 | `status` | u32 | `0` = accepted. Nonzero = an `IggyError` code for a failure decided **before** commit (authorization denial, admission reject). A nonzero status always comes with an empty body. |
-| 220 | 36 | reserved | bytes | Zero. |
-
-Decode order for a client:
-
-1. Read 256 bytes, check `command`. `13` (`Eviction`) means the session is dead: map the eviction reason to an error and stop. `8` (`Reply`) continues.
-2. Read the remaining `size - 256` body bytes.
-3. If `status` is nonzero, the request failed pre-commit. The status value is the `IggyError` code and the body is empty.
-4. Otherwise decode the body. For result-framed operations, strip the [result section](#result-section) first.
-
-### Result section
-
-Replies for all metadata operations and for the partition-plane consumer-offset writes (`StoreConsumerOffset`, `DeleteConsumerOffset`) are **result-framed**: the body starts with a committed-result section ahead of the typed payload.
-
-```text
-[count: u32]
-count x { index: u32, result: u32 }
-```
-
-- Success: `count = 0`, and the typed response payload (if any) follows the 4 count bytes.
-- Committed business rejection: one entry `{ index: 0, result: error_code }` and no payload.
-
-The header `status` channel and the result section are mutually exclusive by construction: a reply either failed pre-commit (`status` nonzero, empty body) or committed (`status` zero, result section present). A login-register reply carries the result section only when non-empty. On success its body starts directly with the [login response payload](#login-register).
-
-Replies to non-replicated commands aren't result-framed: after `status = 0` the body is the response payload directly.
-
-### EvictionHeader
-
-Server to client. 256 bytes, never a body. An eviction is session-terminal: it says "this session is dead", carries no per-request correlation, and a client should deinitialize and re-login.
-
-| Offset | Size | Field | Type | Description |
-|--------|------|-------|------|-------------|
-| 0 | 16 | `checksum` | u128 | Zero on client-facing frames. |
-| 16 | 16 | `checksum_body` | u128 | Zero on client-facing frames. |
-| 32 | 16 | `cluster` | u128 | Cluster id. |
-| 48 | 4 | `size` | u32 | Always 256: an eviction has no body. |
-| 52 | 4 | `view` | u32 | |
-| 56 | 4 | `release` | u32 | Zero. |
-| 60 | 1 | `command` | u8 | `13` (`Eviction`). |
-| 61 | 1 | `replica` | u8 | |
-| 62 | 66 | reserved | bytes | Zero. |
-| 128 | 16 | `client` | u128 | The evicted client id. |
-| 144 | 4 | `server_protocol_version` | u32 | Accepted protocol window (max), packed semver. Set only for reason 14, zero otherwise. |
-| 148 | 4 | `server_protocol_version_min` | u32 | Accepted protocol window (min). Set only for reason 14, zero otherwise. |
-| 152 | 103 | reserved | bytes | Zero. |
-| 255 | 1 | `reason` | u8 | Eviction reason, see below. |
-
-Eviction reasons:
-
-| Value | Reason | Meaning |
-|-------|--------|---------|
-| 1 | `NoSession` | No session exists for this client id |
-| 2 | `ClientReleaseTooLow` | Client release below cluster minimum |
-| 3 | `ClientReleaseTooHigh` | Client release above cluster maximum |
-| 4 | `InvalidRequestOperation` | Unknown `operation` discriminant |
-| 5 | `InvalidRequestBody` | Body failed validation |
-| 6 | `InvalidRequestBodySize` | Body size mismatch |
-| 7 | `SessionTooLow` | Session epoch below the cluster's retained minimum |
-| 8 | `SessionReleaseMismatch` | Session bound to a different release |
-| 9 | `InvalidCredentials` | Login refused: bad username or password |
-| 10 | `InvalidToken` | Login refused: bad personal access token |
-| 11 | `UserInactive` | Login refused: user inactive |
-| 12 | `SessionError` | Session-level failure |
-| 13 | `StaleClient` | Missed heartbeats; server evicted the session |
-| 14 | `IncompatibleProtocol` | Client protocol version outside the accepted window (see bytes 144..152) |
-| 15 | `MalformedLogin` | Login body without a decodable version prefix, or a legacy login code |
-
-Value 0 is reserved and never sent.
-
-## Connection lifecycle
-
-### Protocol version
-
-The protocol version is a semver packed into one u32, 10 bits per component (each must be below 1024):
-
-```text
-bits 31..30 reserved (zero)
-bits 29..20 major
-bits 19..10 minor
-bits 9..0 patch
-value = major << 20 | minor << 10 | patch
-```
-
-Integer order equals semver order. The value tracks the `iggy_binary_protocol` crate release. Under 0.x the compatibility gate is minor-scoped: the server accepts a client whose packed version is at least the server's minimum and whose `major.minor` is at most the server's. Patch releases never change the wire, so the upper bound ignores patch. Past 1.0.0 the gate follows strict semver (major bump = incompatible).
-
-### Login-register
-
-The only way to authenticate is the register handshake: command code **40** (`LOGIN_REGISTER`, username and password) or **45** (`LOGIN_REGISTER_WITH_PAT`, personal access token). Both ride `operation = 1` (`Register`) with `session = 0` and a freshly minted non-zero `client` id.
-
-Both request bodies begin with the `ClientVersionInfo` prefix, so the server can gate on the version before touching credentials:
-
-```text
-[protocol_version: u32]
-[sdk_name_len: u8][sdk_name: UTF-8, 1-255 bytes]
-[sdk_version_len: u8][sdk_version: UTF-8, 1-255 bytes]
-```
-
-`protocol_version` is the packed version of the protocol the client was built against. `sdk_name` identifies the SDK (for example `rust-sdk`, `go-sdk`). `sdk_version` is the SDK's own build version.
-
-After the prefix:
-
-**`LOGIN_REGISTER` (code 40):**
-
-```text
-[ClientVersionInfo]
-[username_len: u8][username: N]
-[password_len: u8][password: N]
-[context_len: u32][context: N, only when context_len > 0]
-```
-
-**`LOGIN_REGISTER_WITH_PAT` (code 45):**
-
-```text
-[ClientVersionInfo]
-[token_len: u8][token: N]
-[context_len: u32][context: N, only when context_len > 0]
-```
-
-An incompatible protocol version is answered with a 256-byte `Eviction` frame, reason `14` (`IncompatibleProtocol`), carrying the accepted window at bytes 144 (max) and 148 (min) as packed u32 versions. A body without a decodable `ClientVersionInfo` prefix gets reason `15` (`MalformedLogin`) with a zero window. Bad credentials get reasons 9-11.
-
-A successful login is a normal `Reply` whose body is:
-
-```text
-[user_id: u32]
-[session: u64]
-[server_protocol_version: u32]
-[server_version_len: u8][server_version: N]
-```
-
-The client stores `session` and echoes it in every subsequent request header. Logout is command code **39** (`LOGOUT_USER`), `operation = 3` (`Logout`), empty body.
-
-The legacy login commands **38** (`LOGIN_USER`) and **44** (`LOGIN_WITH_PERSONAL_ACCESS_TOKEN`) are refused: the server answers them with an `Eviction` frame, reason `15` (`MalformedLogin`). Only `PING` is accepted before login. Every other command on an unauthenticated connection is denied.
-
-## Command catalog
-
-The u32 command codes. Replicated commands are identified on the wire by their `operation` byte (see [Operation discriminants](#operation-discriminants)). The code column below is the protocol-level registry and, for non-replicated commands, the value carried in header bytes 196..200.
-
-```bash
-# System
-PING = 1 # non-replicated; works without login
-GET_STATS = 10 # non-replicated
-GET_SNAPSHOT = 11 # non-replicated
-GET_CLUSTER_METADATA = 12 # non-replicated
-DESCRIBE_OPTIONS = 13 # non-replicated
-GET_ME = 20 # non-replicated
-GET_CLIENT = 21 # non-replicated
-GET_CLIENTS = 22 # non-replicated
-
-# Users
-GET_USER = 31 # non-replicated
-GET_USERS = 32 # non-replicated
-CREATE_USER = 33 # operation 141
-DELETE_USER = 34 # operation 143
-UPDATE_USER = 35 # operation 142
-UPDATE_PERMISSIONS = 36 # operation 145
-CHANGE_PASSWORD = 37 # operation 144
-LOGIN_USER = 38 # legacy; server rejects with MalformedLogin eviction
-LOGOUT_USER = 39 # operation 3 (Logout)
-LOGIN_REGISTER = 40 # operation 1 (Register)
-LOGIN_REGISTER_WITH_PAT = 45 # operation 1 (Register)
-
-# Personal access tokens
-GET_PERSONAL_ACCESS_TOKENS = 41 # non-replicated
-CREATE_PERSONAL_ACCESS_TOKEN = 42 # operation 146
-DELETE_PERSONAL_ACCESS_TOKEN = 43 # operation 147
-LOGIN_WITH_PERSONAL_ACCESS_TOKEN = 44 # legacy; server rejects with MalformedLogin eviction
-
-# Messages
-POLL_MESSAGES = 100 # non-replicated
-SEND_MESSAGES = 101 # operation 160
-FLUSH_UNSAVED_BUFFER = 102 # parses, but always fails with FeatureUnavailable
-
-# Consumer offsets
-GET_CONSUMER_OFFSET = 120 # non-replicated
-STORE_CONSUMER_OFFSET = 121 # operation 161
-DELETE_CONSUMER_OFFSET = 122 # operation 162
-
-# Streams
-GET_STREAM = 200 # non-replicated
-GET_STREAMS = 201 # non-replicated
-CREATE_STREAM = 202 # operation 128
-DELETE_STREAM = 203 # operation 130
-UPDATE_STREAM = 204 # operation 129
-PURGE_STREAM = 205 # operation 131
-
-# Topics
-GET_TOPIC = 300 # non-replicated
-GET_TOPICS = 301 # non-replicated
-CREATE_TOPIC = 302 # operation 132
-DELETE_TOPIC = 303 # operation 134
-UPDATE_TOPIC = 304 # operation 133
-PURGE_TOPIC = 305 # operation 135
-
-# Partitions
-CREATE_PARTITIONS = 402 # operation 136
-DELETE_PARTITIONS = 403 # operation 137
-
-# Segments
-DELETE_SEGMENTS = 503 # operation 138
-
-# Consumer groups
-GET_CONSUMER_GROUP = 600 # non-replicated
-GET_CONSUMER_GROUPS = 601 # non-replicated
-CREATE_CONSUMER_GROUP = 602 # operation 139
-DELETE_CONSUMER_GROUP = 603 # operation 140
-JOIN_CONSUMER_GROUP = 604 # operation 148
-LEAVE_CONSUMER_GROUP = 605 # operation 149
-SYNC_CONSUMER_GROUP = 606 # non-replicated
-```
-
-`FLUSH_UNSAVED_BUFFER` still decodes on the wire, but the server has no on-demand flush primitive and answers every call with `FeatureUnavailable`. Per-topic durability is configured with the `enforce_fsync` [topic option](/docs/server/topic-options) instead.
-
-## Shared encodings
-
-### Name
-
-Length-prefixed UTF-8 string, used for stream, topic, group, user, and token names:
-
-```text
-[length: u8][value: UTF-8, 1-255 bytes]
-```
-
-### Identifier
-
-Polymorphic identifier of a stream, topic, user, or consumer group:
-
-```text
-[kind: u8][length: u8][value: N bytes]
-```
-
-- Numeric: `kind = 1`, `length = 4`, value is a u32.
-- String: `kind = 2`, `length = 1..255`, value is UTF-8 bytes.
-
-### Consumer
-
-Identifies a consumer or a consumer group in message and offset commands:
-
-```text
-[kind: u8][id: Identifier]
-```
-
-- `kind = 1`: consumer.
-- `kind = 2`: consumer group.
-
-### Optional partition id
-
-Several commands take an optional partition id encoded as 5 fixed bytes:
-
-```text
-[flag: u8][partition_id: u32]
-```
-
-`flag = 1` means the id is set. `flag = 0` means none (the value bytes are then zero). For consumer-group consumers the partition id is left unset and the server resolves it.
-
-### Partitioning
-
-How `SendMessages` picks the target partition:
-
-```text
-[kind: u8][length: u8][value: 0-255 bytes]
-```
-
-- `Balanced`: `kind = 1`, `length = 0`, no value. The server resolves the partition at admission with a per-node round-robin counter.
-- `PartitionId`: `kind = 2`, `length = 4`, value is a u32 partition id.
-- `MessagesKey`: `kind = 3`, `length = 1..255`, value is a routing key hashed to a partition.
-
-First-party binary SDKs pre-resolve balanced and key routing client-side and normally send `kind = 2`.
-
-### Polling strategy
-
-Fixed 9 bytes:
-
-```text
-[kind: u8][value: u64]
-```
-
-| Kind | Name | Meaning of `value` |
-|------|------|--------------------|
-| 1 | Offset | Start at this offset |
-| 2 | Timestamp | Start at this timestamp (microseconds) |
-| 3 | First | Start from the beginning (`value` ignored) |
-| 4 | Last | Start from the end (`value` ignored) |
-| 5 | Next | Continue after the stored consumer offset (`value` ignored) |
-
-### Header kinds
-
-Typed values used by user headers and option values. The kind byte:
-
-| Code | Kind | Code | Kind |
-|------|------|------|------|
-| 1 | Raw | 9 | Uint8 |
-| 2 | String | 10 | Uint16 |
-| 3 | Bool | 11 | Uint32 |
-| 4 | Int8 | 12 | Uint64 |
-| 5 | Int16 | 13 | Uint128 |
-| 6 | Int32 | 14 | Float32 |
-| 7 | Int64 | 15 | Float64 |
-
-Integers are little-endian. `Bool` is one byte.
-
-### User headers block
-
-A user headers block (inside a [message frame](#message-batch-format)) is a flat run of TLV fields:
-
-```text
-[kind: u8][length: u32][data: length bytes]
-```
-
-Fields pair up: first the key, then the value. Keys are `String` kind. Every `length` must be 1..=255, kind 0 is rejected, and the block must consume its byte range exactly. Unknown value kind codes are preserved and forwarded, so headers survive mixed-version clusters.
-
-### Options block
-
-`CreateStream`, `UpdateStream`, `CreateTopic`, `UpdateTopic`, `CreateUser`, and `UpdateUser` end with a key-value **options block** that reuses the user-headers TLV encoding. The block runs from its start to the end of the payload (no length prefix, and the validator requires exact consumption) and may be empty.
-
-On top of the TLV walk, options enforce: string keys only (kind 2, valid UTF-8), no duplicate keys, at most 1024 entries, and at most 100000 bytes total.
-
-Semantics:
-
-- **Create** requests resolve absent keys to server defaults and persist the effective values, so `GetTopic` always shows what is in force.
-- **Update** requests are patches: keys absent from the block are left alone, never reset. A client built before a key existed cannot erase it.
-- Unknown keys are rejected at the wire edge, never silently skipped.
-
-The catalog is discoverable at runtime with `DESCRIBE_OPTIONS` (code 13), payload `[scope: u8]` with scope `1` = topic, `2` = stream, `3` = user (HTTP: `GET /options/topic`). Today only topics have keys. The stream and user catalogs are empty, and any key sent for them is rejected. The topic catalog (`segment_size`, `enforce_fsync`, `message_expiry`, `max_topic_size`, and the rest) with defaults and constraints is documented on the [Topic options](/docs/server/topic-options) page. `Updat [...]
-
-### Compression
-
-The `compression_algorithm` value used by topic options:
-
-| Code / name | Algorithm |
-|-------------|-----------|
-| 1, `none` | No compression (default) |
-| 2, `gzip` | Gzip |
-
-Other algorithms (lz4, zstd) are planned but not implemented. Any other value is rejected.
-
-## Message batch format
-
-Messages travel and rest in exactly one layout, the **batch**: the `SendMessages` body, the replicated record, the persisted segment record, and the poll reply all share it. There is no other message encoding.
-
-```text
-[batch header: 256 bytes][blob: message frames]
-frame = [frame header: 48 bytes][payload][user_headers]
-```
-
-### Batch header (256 bytes)
-
-| Offset | Size | Field | Type | Description |
-|--------|------|-------|------|-------------|
-| 0 | 8 | `partition_id` | u64 | Server-stamped at ingestion. Producers send zero. |
-| 8 | 8 | `base_offset` | u64 | Server-stamped at persistence. Producers send zero. |
-| 16 | 8 | `base_timestamp` | u64 | Server-stamped at persistence (microseconds). Producers send zero. |
-| 24 | 8 | `origin_timestamp` | u64 | Producer-set: the earliest `origin_timestamp` among the batch's messages (microseconds). |
-| 32 | 8 | `batch_length` | u64 | Total batch size: 256 + blob length. |
-| 40 | 8 | `batch_checksum` | u64 | XxHash3-64 over the header fields (`partition_id`, `base_offset`, `base_timestamp`, `origin_timestamp`, `batch_length`, `message_count`, in that order, little-endian bytes) followed by each frame's 8 checksum bytes. Recomputed by the server when it stamps the header. |
-| 48 | 4 | `message_count` | u32 | Number of frames in the blob. |
-| 52 | 204 | reserved | bytes | Must be zero; the server rejects a batch with nonzero reserved bytes. |
-
-### Message frame header (48 bytes)
-
-| Offset | Size | Field | Type | Description |
-|--------|------|-------|------|-------------|
-| 0 | 8 | `checksum` | u64 | XxHash3-64 over frame bytes 8..48 followed by `payload` and `user_headers`. Producer-computed; verified at admission. |
-| 8 | 16 | `id` | u128 | Message id (for example a UUID). |
-| 24 | 4 | `offset_delta` | u32 | Message offset = `base_offset + offset_delta`. Producer sends the frame's index in the batch. |
-| 28 | 4 | `timestamp_delta` | u32 | Microsecond delta against the batch `origin_timestamp`. One batch spans at most about 71.6 minutes of producer clock (`u32` microseconds). |
-| 32 | 4 | `user_headers_length` | u32 | Byte length of the user headers block. |
-| 36 | 4 | `payload_length` | u32 | Byte length of the payload. |
-| 40 | 8 | reserved | u64 | Must be zero. |
-
-After the 48 header bytes: `payload` (`payload_length` bytes), then `user_headers` (`user_headers_length` bytes). The payload comes **first**.
-
-A frame's absolute position is derived, not stored: offset = `base_offset + offset_delta`, timestamp = `base_timestamp + timestamp_delta` for the server-stamped time (`origin_timestamp + timestamp_delta` for producer time).
-
-The 64-byte `IggyMessageHeader` (checksum, id, offset, timestamp, origin_timestamp, lengths, reserved) that earlier protocol generations put in front of every message **no longer exists on the wire or on disk**. It survives only as an in-memory model inside the SDKs.
-
-## Command payloads
-
-Payloads below are the request **body** (the bytes after the 256-byte header). Types reference the [shared encodings](#shared-encodings). Empty payload means the body has zero bytes.
-
-### System
-
-**Ping. Code: 1.** Empty payload. The only command accepted before login.
-
-**Get stats. Code: 10.** Empty payload.
-
-**Get snapshot. Code: 11.**
-
-```text
-[compression: u8][types_count: u8][snapshot_type: u8] x types_count
-```
-
-Compression codes: 1 = Stored, 2 = Deflated, 3 = Bzip2, 4 = Zstd, 5 = Lzma, 6 = Xz. Snapshot type codes: 1 = FilesystemOverview, 2 = ProcessList, 3 = ResourceUsage, 4 = Test, 5 = ServerLogs, 6 = ServerConfig, 100 = All.
-
-**Get cluster metadata. Code: 12.** Empty payload.
-
-**Describe options. Code: 13.**
-
-```text
-[scope: u8]
-```
-
-Scope: 1 = topic, 2 = stream, 3 = user. Returns the option catalog for the scope: keys, value kinds, defaults, and descriptions. See [Options block](#options-block).
-
-**Get me. Code: 20.** Empty payload.
-
-**Get client. Code: 21.**
-
-```text
-[client_id: u32]
-```
-
-**Get clients. Code: 22.** Empty payload.
-
-### Streams
-
-**Get stream. Code: 200.**
-
-```text
-[stream_id: Identifier]
-```
-
-**Get streams. Code: 201.** Empty payload.
-
-**Create stream. Code: 202.**
-
-```text
-[name_len: u8][name: N][options block to end]
-```
-
-The stream option catalog is empty today, so the block is normally empty (zero bytes).
-
-**Delete stream. Code: 203.**
-
-```text
-[stream_id: Identifier]
-```
-
-**Update stream. Code: 204.**
-
-```text
-[stream_id: Identifier][name_len: u8][name: N][options block to end]
-```
-
-Patch semantics: absent option keys are left unchanged.
-
-**Purge stream. Code: 205.**
-
-```text
-[stream_id: Identifier]
-```
-
-### Topics
-
-**Get topic. Code: 300.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier]
-```
-
-**Get topics. Code: 301.**
-
-```text
-[stream_id: Identifier]
-```
-
-**Create topic. Code: 302.**
-
-```text
-[stream_id: Identifier][partitions_count: u32][name_len: u8][name: N][options block to end]
-```
-
-The fixed fields are the shape of the operation: which stream, how many partitions, what name. Every topic setting (`compression_algorithm`, `message_expiry`, `max_topic_size`, `segment_size`, `enforce_fsync`, `messages_required_to_save`, `size_of_messages_required_to_save`, `preallocate_segments`) rides the options block. See [Topic options](/docs/server/topic-options). `partitions_count` is an argument, not a setting: it is consumed at admission and never persisted as an option.
-
-**Delete topic. Code: 303.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier]
-```
-
-**Update topic. Code: 304.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][name_len: u8][name: N][options block to end]
-```
-
-Only `compression_algorithm`, `message_expiry`, and `max_topic_size` are updatable. Absent keys are left unchanged.
-
-**Purge topic. Code: 305.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier]
-```
-
-### Partitions
-
-**Create partitions. Code: 402.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][partitions_count: u32]
-```
-
-**Delete partitions. Code: 403.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][partitions_count: u32]
-```
-
-### Segments
-
-**Delete segments. Code: 503.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][partition_id: u32][segments_count: u32]
-```
-
-Deletes the `segments_count` oldest sealed segments of the partition.
-
-### Messages
-
-**Poll messages. Code: 100.**
-
-```text
-[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
-[partition_flag: u8][partition_id: u32]
-[strategy: 9 bytes][count: u32][auto_commit: u8]
-```
-
-`strategy` is a [polling strategy](#polling-strategy). `count` is the requested number of messages. `auto_commit = 1` stores the consumer offset server-side as part of the poll.
-
-The response body:
-
-```text
-[partition_id: u32][current_offset: u64][messages_count: u32]
-[batch records to end]
-```
-
-The 16-byte prefix is followed by a stream of [batch records](#message-batch-format) served as stored: each record's header carries the stamped `base_offset` and `base_timestamp`, and each frame's deltas resolve against them. A record may be a server-sliced view of a larger stored batch, so the first polled offset is `base_offset + offset_delta` of the first frame, not necessarily `base_offset` itself. `current_offset` is the partition's newest offset at poll time.
-
-**Send messages. Code: 101.**
-
-```text
-[metadata_length: u32]
-[stream_id: Identifier][topic_id: Identifier][partitioning: Partitioning]
-[messages_count: u32]
-[batch: 256-byte batch header + frames]
-```
-
-`metadata_length` counts the bytes from `stream_id` through `messages_count` inclusive, so a reader can skip straight to the batch. The producer leaves `partition_id`, `base_offset`, and `base_timestamp` zero in the batch header, and the server stamps them. Every checksum is producer-computed and verified at admission. The reserved regions must be zero.
-
-**Flush unsaved buffer. Code: 102.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][partition_id: u32][fsync: u8]
-```
-
-Parses, but the server always answers `FeatureUnavailable`: there is no on-demand flush primitive. Use the per-topic `enforce_fsync` option for durability guarantees.
-
-### Consumer offsets
-
-**Get consumer offset. Code: 120.**
-
-```text
-[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
-[partition_flag: u8][partition_id: u32]
-```
-
-Response body (20 bytes):
-
-```text
-[partition_id: u32][current_offset: u64][stored_offset: u64]
-```
-
-**Store consumer offset. Code: 121.**
-
-```text
-[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
-[partition_flag: u8][partition_id: u32][offset: u64][ack: u8]
-```
-
-**Delete consumer offset. Code: 122.**
-
-```text
-[consumer: Consumer][stream_id: Identifier][topic_id: Identifier]
-[partition_flag: u8][partition_id: u32][ack: u8]
-```
-
-The trailing `ack` byte is mandatory on both write commands. A payload without it fails to decode. Ack levels:
-
-| Value | Level | Meaning |
-|-------|-------|---------|
-| 0 | `NoAck` | Leader-local write; respond as soon as local state is updated (the fast path `PollMessages` auto-commit uses) |
-| 1 | `Quorum` | Replicate through partition consensus; respond after a quorum commit (default for explicit writes) |
-
-### Consumer groups
-
-**Get consumer group. Code: 600.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
-```
-
-**Get consumer groups. Code: 601.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier]
-```
-
-**Create consumer group. Code: 602.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][name_len: u8][name: N]
-```
-
-**Delete consumer group. Code: 603.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
-```
-
-**Join consumer group. Code: 604.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
-```
-
-**Leave consumer group. Code: 605.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
-```
-
-**Sync consumer group. Code: 606.**
-
-```text
-[stream_id: Identifier][topic_id: Identifier][group_id: Identifier]
-```
-
-Read-only: a member asks for its current partition assignment and the group generation, so it can select partitions to poll client-side.
-
-### Users
-
-**Get user. Code: 31.**
-
-```text
-[user_id: Identifier]
-```
-
-**Get users. Code: 32.** Empty payload.
-
-**Create user. Code: 33.**
-
-```text
-[username_len: u8][username: N][password_len: u8][password: N][status: u8]
-[has_permissions: u8][permissions_len: u32, only when has_permissions = 1][permissions: M bytes]
-[options block to end]
-```
-
-`permissions_len` and the permissions bytes are present **only** when `has_permissions = 1`. When it is 0 the options block follows immediately. The user option catalog is empty today, so the block is normally empty. `status`: 1 = active, 2 = inactive.
-
-**Delete user. Code: 34.**
-
-```text
-[user_id: Identifier]
-```
-
-**Update user. Code: 35.**
-
-```text
-[user_id: Identifier]
-[has_username: u8][username_len: u8, only when 1][username: N, only when 1]
-[has_status: u8][status: u8, only when 1]
-[options block to end]
-```
-
-**Update permissions. Code: 36.**
-
-```text
-[user_id: Identifier][has_permissions: u8]
-[permissions_len: u32, only when has_permissions = 1][permissions: M bytes, only when 1]
-```
-
-As in `CreateUser`, the length field is conditional: with `has_permissions = 0` the payload ends at the flag.
-
-**Change password. Code: 37.**
-
-```text
-[user_id: Identifier]
-[current_password_len: u8][current_password: N]
-[new_password_len: u8][new_password: N]
-```
-
-### Authentication
-
-Login and logout are part of the [connection lifecycle](#connection-lifecycle):
-
-- **Login register. Code: 40.** `Operation::Register`. The body is `ClientVersionInfo` + credentials, described [above](#login-register).
-- **Login register with PAT. Code: 45.** `Operation::Register`. The body is `ClientVersionInfo` + token.
-- **Logout user. Code: 39.** `Operation::Logout`. Empty payload.
-- Codes **38** and **44** are the pre-register login shapes. The server rejects both with a `MalformedLogin` eviction.
-
-### Personal access tokens
-
-**Get personal access tokens. Code: 41.** Empty payload.
-
-**Create personal access token. Code: 42.**
-
-```text
-[name_len: u8][name: N][expiry: u64]
-```
-
-`expiry` is a duration in **microseconds**. Two sentinel values exist: `0` (server default, which currently means no expiry) and `u64::MAX` (never expires).
-
-**Delete personal access token. Code: 43.**
-
-```text
-[name_len: u8][name: N]
-```
diff --git a/content/docs/server/storage-engine.mdx b/content/docs/server/storage-engine.mdx
index c56fb160..7d003434 100644
--- a/content/docs/server/storage-engine.mdx
+++ b/content/docs/server/storage-engine.mdx
@@ -59,9 +59,9 @@ Segments store **batch records**, not individual messages. The record is byte-id
frame = [frame header: 48 bytes][payload][user_headers]
```
-The 256-byte batch header carries `partition_id`, `base_offset`, `base_timestamp`, `origin_timestamp`, `batch_length`, `batch_checksum` (XxHash3-64), and `message_count`. The rest is reserved and must be zero. Each 48-byte frame header carries `checksum` (XxHash3-64), `id` (u128), `offset_delta` (u32), `timestamp_delta` (u32), and the two lengths. A message's absolute offset is `base_offset + offset_delta`, and timestamps resolve the same way against the base. All fields are **little-end [...]
+The 256-byte batch header carries `partition_id`, `base_offset`, `base_timestamp`, `origin_timestamp`, `batch_length`, `batch_checksum` (XxHash3-64), and `message_count`. The rest is reserved and must be zero. Each 48-byte frame header carries `checksum` (XxHash3-64), `id` (u128), `offset_delta` (u32), `timestamp_delta` (u32), and the two lengths. A message's absolute offset is `base_offset + offset_delta`; its server timestamp is the flat `base_timestamp`, while `timestamp_delta` resolv [...]
-The full byte-level tables live on the [Schema](/docs/server/schema#message-batch-format) page. What lands on disk is exactly what the producer sent, with the server stamping `partition_id` at ingestion and `base_offset` / `base_timestamp` (plus a `batch_checksum` recompute) at persistence.
+The full byte-level tables live on the [Message batches](/docs/binary-protocol/messages) page. Absent at-rest encryption, what lands on disk is exactly what the producer sent, with the server stamping `partition_id` at admission and `base_offset` / `base_timestamp` (plus a `batch_checksum` recompute) at journal append.
## Indexes
@@ -79,36 +79,37 @@ There is no index caching configuration: the in-memory index cache is an interna
## Write pipeline
-Messages flow through admission, replication, and buffering before reaching a segment:
+Messages are admitted, stamped, and buffered in the partition journal, then replicated; committed batches reach a segment once a flush trigger fires:
```mermaid
graph TD
A["Client SendMessages: batch with producer-computed checksums"] --> B["Admission: verify checksums, stamp partition_id"]
- B --> C["Partition consensus: prepare replicated to the partition's replicas"]
- C --> D{"Quorum reached?"}
- D -->|"Yes"| E["Commit: batch buffered in the partition journal"]
- E --> F{"Flush threshold reached?"}
- F -->|"count >= messages_required_to_save OR bytes >= size_of_messages_required_to_save"| G["Stamp base_offset / base_timestamp, recompute batch_checksum"]
- F -->|"No"| E
- G --> H["MessagesWriter: vectored write to .log + sparse index entry"]
- H --> I{"enforce_fsync?"}
- I -->|"Yes"| J["fsync"]
- I -->|"No"| K["OS writeback"]
- J --> L{"Segment past segment_size?"}
- K --> L
- L -->|"Yes"| M["Seal segment, open next"]
- L -->|"No"| N["Done"]
+ B --> C["Stamp base_offset / base_timestamp, recompute batch_checksum"]
+ C --> D["Append to the in-memory partition journal"]
+ D --> E["Partition consensus: prepare replicated to the partition's replicas"]
+ E --> F{"Quorum reached?"}
+ F -->|"Yes"| G["Commit"]
+ G --> H{"Flush trigger reached?"}
+ H -->|"No"| W["Stay buffered until a later commit trips the gate"]
+ H -->|"Yes"| I["MessagesWriter: vectored write to .log + sparse index entry"]
+ I --> J{"enforce_fsync?"}
+ J -->|"Yes"| K["fdatasync .log + .index"]
+ J -->|"No"| L["OS writeback"]
+ K --> M{"Segment past segment_size?"}
+ L --> M
+ M -->|"Yes"| N["Seal segment, open next"]
+ M -->|"No"| O["Done"]
```
The flush thresholds and fsync policy are **per-topic creation options** (they used to be server-wide config):
- `messages_required_to_save` (default `1024`): count threshold.
- `size_of_messages_required_to_save` (default `1 MiB`): byte threshold.
-- `enforce_fsync` (default `false`): fsync every segment write.
+- `enforce_fsync` (default `false`): fdatasync the `.log` and `.index` on every flush.
-Whichever threshold trips first flushes. These are soft limits: a flush writes whole batches, so the actual flushed count or size can overshoot. The `MessagesWriter` uses **vectored I/O** (`writev`) with up to `MAX_IOV_COUNT = 1024` buffers per syscall, so many buffered batches land in one write.
+A third trigger is independent of these options: when buffered journal bytes reach the active segment's `segment_size`, the flush fires. Whichever trigger trips first flushes. These are soft limits: a flush writes whole batches, so the actual flushed count or size can overshoot. The gate is evaluated on commit events, so a sub-threshold batch stays buffered until a later commit trips it (shutdown flushes unconditionally). The `MessagesWriter` uses **vectored I/O** (`writev`) with up to ` [...]
-Every partition belongs to a consensus group (Viewstamped Replication), and writes are prepared and quorum-acknowledged before commit. At HEAD the partition-plane consensus journal is **in-memory**. Durability of message data comes from segment persistence and, in clusters, from the copies on other replicas. The metadata plane (below) has a durable on-disk journal.
+Every partition belongs to a consensus group (Viewstamped Replication), and writes are prepared and quorum-acknowledged before commit. Stamping happens at journal-append time, before replication acks: the primary stamps `base_offset` from the log position and `base_timestamp` from the prepare timestamp, forwards the stamped bytes verbatim, and each backup re-derives the expected pair and refuses a mismatch, so the batch (and its recomputed `batch_checksum`) is byte-identical across repli [...]
## Boot-time segment recovery
@@ -186,7 +187,7 @@ key = "" # 32-byte base64-encoded key
## Compression
-Per-topic payload compression is set with the `compression_algorithm` topic option. Supported algorithms: `none` (default) and `gzip`. Other algorithms (lz4, zstd) are planned but **not implemented**.
+The `compression_algorithm` topic option accepts `none` (default) and `gzip`, but it is a **placeholder today**: the value is persisted and reported back, and no compression is applied anywhere - segments store payloads exactly as sent. To compress today, do it client-side and tag messages via user headers - see the [message headers examples](https://github.com/apache/iggy/tree/master/examples/rust/src/message-headers) in the Iggy repo.
## Removed and relocated settings
diff --git a/content/docs/server/topic-options.mdx b/content/docs/server/topic-options.mdx
index 9214b347..c1b1b36b 100644
--- a/content/docs/server/topic-options.mdx
+++ b/content/docs/server/topic-options.mdx
@@ -12,7 +12,7 @@ Options are key-value pairs sent with `CreateTopic`. Unknown keys are **rejected
|--------|---------|-------------|-------------|
| `max_topic_size` | unlimited | | Delete the oldest sealed segments once the topic grows past this size. |
| `message_expiry` | none | | Delete sealed segments older than this. |
-| `compression_algorithm` | `none` | `none` or `gzip` | Payload compression. |
+| `compression_algorithm` | `none` | `none` or `gzip` | Placeholder: stored and reported, no compression applied yet. |
| `segment_size` | 1 GiB | 512-byte multiple, at least 1 MiB, at most 1 GiB | Soft size limit per segment: a segment may close one whole batch past it. |
| `enforce_fsync` | `false` | | fsync every write to this topic's partitions. |
| `messages_required_to_save` | 1024 | non-zero, at most 16777216 | Flush the journal once it holds this many messages. |
diff --git a/public/docs/server/schema/index.html b/public/docs/server/schema/index.html
new file mode 100644
index 00000000..6429f748
--- /dev/null
+++ b/public/docs/server/schema/index.html
@@ -0,0 +1,31 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+<!doctype html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8" />
+ <title>Moved: Binary Protocol</title>
+ <meta http-equiv="refresh" content="0; url=/docs/binary-protocol/" />
+ <link rel="canonical" href="/docs/binary-protocol/" />
+ <meta name="robots" content="noindex" />
+ </head>
+ <body>
+ <p>The Schema page moved to <a href="/docs/binary-protocol/">Binary Protocol</a>.</p>
+ </body>
+</html>
diff --git a/src/app/global.css b/src/app/global.css
index 34b74a55..ca6a2802 100644
--- a/src/app/global.css
+++ b/src/app/global.css
@@ -93,6 +93,14 @@ header#nd-subnav {
background-color: var(--color-fd-background) !important;
}
+/* The mobile TOC bar wrapper is sticky at z-10 with a transparent background,
+ so page content that also uses z-10 paints over it while scrolling under.
+ Lift it above content and make it opaque, matching the navbar above it. */
+div[data-toc-popover] {
+ z-index: 30;
+ background-color: var(--color-fd-background);
+}
+
figure.shiki {
border-color: var(--color-fd-border) !important;
box-shadow: none !important;
diff --git a/src/components/architecture-diagrams.tsx b/src/components/architecture-diagrams.tsx
index c4d025aa..99e02330 100644
--- a/src/components/architecture-diagrams.tsx
+++ b/src/components/architecture-diagrams.tsx
@@ -1139,7 +1139,7 @@ export function DocsHero() {
<div className="absolute inset-0 bg-[radial-gradient(ellipse_at_top_right,var(--color-fd-primary)_0%,transparent_60%)] opacity-[0.07]" />
<div className="absolute top-0 right-0 w-64 h-64 bg-fd-primary/5 rounded-full blur-3xl -translate-y-1/2 translate-x-1/2" />
- <div className="relative z-10">
+ <div className="relative">
<div className="flex flex-col items-center text-center mb-8">
<Logo className="mb-5" />
<h2 className="text-2xl md:text-3xl font-bold text-fd-foreground m-0 mb-2">
diff --git a/src/components/mermaid.tsx b/src/components/mermaid.tsx
index 25400a2b..767c07e9 100644
--- a/src/components/mermaid.tsx
+++ b/src/components/mermaid.tsx
@@ -57,11 +57,14 @@ function MermaidSVG({ chart }: { chart: string }) {
}),
);
- const scaled = svg
- .replaceAll('font-size="13"', 'font-size="15"')
- .replaceAll('font-size="12"', 'font-size="14"')
- .replaceAll('font-size="11"', 'font-size="14"')
- .replaceAll('font-size="10"', 'font-size="13"');
+ // Zoom the whole SVG for readability. Bumping font-size attributes instead
+ // would grow text after layout, overflowing node boxes sized for the
+ // original metrics and colliding adjacent nodes.
+ const scaled = svg.replace(
+ / width="([0-9.]+)" height="([0-9.]+)"/,
+ (_, width, height) =>
+ ` width="${Math.round(Number(width) * 1.15)}" height="${Math.round(Number(height) * 1.15)}"`,
+ );
return (
<div