You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by hg...@apache.org on 2026/07/02 22:08:07 UTC
(iggy) branch http-listener-rebased created (now 7c78a3467)
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a change to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
at 7c78a3467 fix(server-ng): reject flush with typed unavailable deny
This branch includes the following new commits:
new a98759593 feat(message_bus): add in-process reply routing for HTTP listener
new d1e862dbd feat(server-ng): add HTTP listener with login and session binding
new 5d09f1ed5 feat(server-ng): route HTTP create-stream through VSR consensus
new 14d26dc82 feat(server-ng): add HTTP stream/topic update, delete, purge routes
new c93619e76 feat(server-ng): add HTTP create-topic route
new bf38ea794 feat(server-ng): gate HTTP write routes to root user
new afc53d8c1 feat(server-ng): add HTTP user-management write routes
new 354ce587e feat(server-ng): add HTTP personal-access-token create/delete routes
new 4c5436e43 feat(server-ng): add HTTP consumer-group create/delete routes
new 19557901e feat(server-ng): add HTTP read routes
new cfa92f96e feat(server-ng): add HTTP cluster-metadata route and view header
new b667b78f6 feat(server-ng): redirect linearizable follower reads to primary
new dd542f0b2 feat(server-ng): add in-process reply slots for HTTP data plane
new 2789a345c feat(server-ng): add HTTP produce route
new d5a479c42 feat(server-ng): add HTTP poll and consumer-offset read routes
new 2195a5c3b feat(server-ng): add HTTP consumer-offset write routes
new fcd001517 feat(server-ng): add HTTP admission control
new f4dcf825c fix(server-ng): make HTTP write id-advance cancellation-safe
new 3b1dddcdd test(server-ng): add HTTP data-plane integration tests
new 15962a5c3 feat(consensus): carry acting user id on Register through VSR
new faaa94b63 feat(metadata): populate permissioner from replicated user ops
new dc246adea feat(metadata): enforce RBAC in the metadata apply path
new 11bf9ccb4 fix(metadata): make acting-user stamping single-site and fail-closed
new 85454492c feat(server-ng): enforce RBAC at dispatch for data plane and reads
new fb89a35f8 test(server-ng): add RBAC permission-matrix integration gate
new d43eb3947 fix(server-ng): harden HTTP session and auth edges
new 4816ae73f fix(server-ng): verify and strip current_password on change
new 77e0e3565 fix(server-ng): omit X-Iggy-View on error responses
new b938f2ef5 docs(integration): refresh stale permissions-scenario gate reason
new 376db025c perf(server-ng): warm dummy password hash at boot
new 6d856ab00 fix(server-ng): stop leaking account status on inactive login
new 66c6f3284 refactor(server-ng): reuse metadata user resolver for password hash
new 097a0884f fix(server-ng): report self-node listener ports in cluster metadata
new 764ff6c61 fix(server-ng): resolve the me alias inside the users/{user_id} route
new c791f09b0 fix(server-ng): guard raw PAT splice on committed result code
new 79434c29d fix(metadata): reject consumer-group create on a missing parent
new 1d4a04634 fix(server-ng): return typed not-found errors on parity reads
new ed2b83fb2 test(integration): delete only the created user in cg dup-name scenario
new 4cadd9e9b fix(server-ng): frame raw PAT reply with a result section
new 45e167528 fix(server-ng): order topic-not-found identifiers per display contract
new 1b9874fdb feat(server-ng): list personal access tokens over HTTP and TCP
new 80be4a9ad fix(partitions): reply typed deny on delete of missing consumer offset
new 7c78a3467 fix(server-ng): reject flush with typed unavailable deny
The 43 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(iggy) 04/43: feat(server-ng): add HTTP stream/topic update, delete, purge routes
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 14d26dc82cfb7b87d1b215ddd2557221e98adb6f
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 13:27:38 2026 +0200
feat(server-ng): add HTTP stream/topic update, delete, purge routes
Extend the server-ng HTTP control plane beyond create-stream with the
data-adjacent stream and topic writes, all sharing one reply path.
Adds PUT/DELETE/DELETE-purge for streams and PUT/DELETE/DELETE-purge
for topics, each parsing the dual-addressed {stream_id}/{topic_id}
path segments into wire identifiers and returning 204 on the empty
committed reply, matching the legacy HTTP status.
The eviction-aware reply interpretation, per-session gate advance, and
result-code discrimination are factored into a single submit_write
helper and a generic request-message builder now shared by every
write route including create-stream, so the logic lives once.
---
core/server-ng/src/http.rs | 311 ++++++++++++++++++++++++++++++++++++---------
1 file changed, 249 insertions(+), 62 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index a2de4b11c..f2f52868b 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -27,16 +27,22 @@ use std::collections::HashMap;
use std::net::SocketAddr;
use std::rc::Rc;
-use axum::extract::State;
+use axum::extract::{Path, State};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
-use axum::routing::{get, post};
+use axum::routing::{delete, get, post, put};
use axum::{Json, Router};
+use bytes::Bytes;
use configs::http::HttpJwtConfig;
use iggy_binary_protocol::consensus::{
Command2, EvictionHeader, EvictionReason, HEADER_SIZE, result_code, result_section_len,
};
-use iggy_binary_protocol::requests::streams::CreateStreamRequest;
+use iggy_binary_protocol::requests::streams::{
+ CreateStreamRequest, DeleteStreamRequest, PurgeStreamRequest, UpdateStreamRequest,
+};
+use iggy_binary_protocol::requests::topics::{
+ DeleteTopicRequest, PurgeTopicRequest, UpdateTopicRequest,
+};
use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
use iggy_binary_protocol::{
@@ -45,7 +51,12 @@ use iggy_binary_protocol::{
use iggy_common::create_stream::CreateStream;
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
-use iggy_common::{IdentityInfo, IggyError, IggyTimestamp, StreamDetails, TokenInfo};
+use iggy_common::update_stream::UpdateStream;
+use iggy_common::update_topic::UpdateTopic;
+use iggy_common::wire_conversions::identifier_to_wire;
+use iggy_common::{
+ Identifier, IdentityInfo, IggyError, IggyTimestamp, StreamDetails, TokenInfo, Validatable,
+};
use message_bus::client_listener;
use secrecy::ExposeSecret;
use send_wrapper::SendWrapper;
@@ -340,6 +351,19 @@ fn router(state: HttpState) -> Router {
)
.route("/users/me", get(get_me))
.route("/streams", post(create_stream))
+ .route(
+ "/streams/{stream_id}",
+ put(update_stream).delete(delete_stream),
+ )
+ .route("/streams/{stream_id}/purge", delete(purge_stream))
+ .route(
+ "/streams/{stream_id}/topics/{topic_id}",
+ put(update_topic).delete(delete_topic),
+ )
+ .route(
+ "/streams/{stream_id}/topics/{topic_id}/purge",
+ delete(purge_topic),
+ )
.with_state(state)
}
@@ -383,8 +407,7 @@ async fn get_me(identity: Authenticated) -> Json<IdentityInfo> {
})
}
-/// `POST /streams`: create a stream through the per-session gate and the
-/// in-process VSR dispatch spine, then render the committed reply as the same
+/// `POST /streams`: create a stream and render the committed reply as the same
/// `StreamDetails` JSON the legacy server returns.
///
/// The accepted body is name-only (`{"name": ...}`), matching the legacy
@@ -399,94 +422,258 @@ async fn create_stream(
name: WireName::new(command.name)
.map_err(|_| WriteError::Rejected(IggyError::InvalidStreamName))?,
};
+ let body = request.to_bytes();
+ let payload = SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::CreateStream,
+ &body,
+ ))
+ .await?;
+ Ok(Json(decode_stream_details(&payload)?))
+}
+
+/// `PUT /streams/{stream_id}`: rename a stream. A committed write returns 204
+/// with no body, matching the legacy server.
+async fn update_stream(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(stream_id): Path<String>,
+ Json(command): Json<UpdateStream>,
+) -> Result<StatusCode, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let request = UpdateStreamRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ name: WireName::new(command.name)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidStreamName))?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::UpdateStream,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `DELETE /streams/{stream_id}`: delete a stream. Returns 204 on commit.
+async fn delete_stream(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(stream_id): Path<String>,
+) -> Result<StatusCode, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let request = DeleteStreamRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::DeleteStream,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `DELETE /streams/{stream_id}/purge`: drop a stream's messages. Returns 204.
+async fn purge_stream(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(stream_id): Path<String>,
+) -> Result<StatusCode, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let request = PurgeStreamRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::PurgeStream,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `PUT /streams/{stream_id}/topics/{topic_id}`: update a topic. Returns 204.
+async fn update_topic(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Json(command): Json<UpdateTopic>,
+) -> Result<StatusCode, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(WriteError::Rejected)?;
+ // Also rejects replication_factor == Some(0), which `WireName` cannot see.
+ command.validate().map_err(WriteError::Rejected)?;
+ let request = UpdateTopicRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(WriteError::Rejected)?,
+ compression_algorithm: command.compression_algorithm.as_code(),
+ message_expiry: command.message_expiry.into(),
+ max_topic_size: command.max_topic_size.into(),
+ replication_factor: command.replication_factor.unwrap_or(0),
+ name: WireName::new(command.name)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidTopicName))?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::UpdateTopic,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `DELETE /streams/{stream_id}/topics/{topic_id}`: delete a topic. Returns 204.
+async fn delete_topic(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+) -> Result<StatusCode, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(WriteError::Rejected)?;
+ let request = DeleteTopicRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(WriteError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::DeleteTopic,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `DELETE /streams/{stream_id}/topics/{topic_id}/purge`: drop a topic's
+/// messages. Returns 204.
+async fn purge_topic(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+) -> Result<StatusCode, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(WriteError::Rejected)?;
+ let request = PurgeTopicRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(WriteError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::PurgeTopic,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
- let client_id = identity.session.client_id;
- let session_id = identity.session.session;
-
- // Serialize this session's writes behind the gate and take the next VSR
- // request id. The gate is held across the commit so ids reach the primary
- // strictly in order, and it advances only once the request commits (a
- // `Reply` frame): a transient failure or an eviction leaves the id free for
- // the caller's retry, which the depth-1 dedup requires (the next accepted
- // request must be `committed_request + 1`, so a consumed-but-uncommitted id
- // would wedge the session on `RequestGap`).
- let mut next_request_id = identity.session.gate.lock().await;
- let message = build_create_stream_message(client_id, session_id, *next_request_id, &request);
- // `submit_client_request_on_owner` is `!Send` (Rc-based); `SendWrapper`
- // bridges it onto axum's `Send` handler future, sound because the whole
- // handler runs pinned to shard 0's compio thread.
- let reply = SendWrapper::new(submit_client_request_on_owner(&state.shard, message)).await;
+/// Run one authenticated control-plane write end to end and return the committed
+/// reply's typed payload. Shared by every write route: `create_stream` decodes
+/// the payload into an entity, the update/delete/purge routes ignore it (it is
+/// empty) and answer 204.
+///
+/// Serializes this session's writes behind its gate and holds it across the
+/// submit so request ids reach the primary strictly in order. The gate advances
+/// only on a committed `Reply`: a transient failure or an eviction leaves the id
+/// free for the caller's retry, which the depth-1 dedup requires (the next
+/// accepted request must be `committed + 1`, so a consumed-but-uncommitted id
+/// would wedge the session on `RequestGap`).
+///
+/// Reply discrimination mirrors the SDK's `split_metadata_result`: the body
+/// leads with a result section (`Some(0)` is success followed by the typed
+/// payload; a nonzero first result is a committed business rejection carrying an
+/// `IggyError` code). An eviction frame carries no result section and means the
+/// session is dead (re-authenticate); a missing or short result section is a
+/// malformed committed reply, mapped to an error rather than a false success.
+async fn submit_write(
+ state: &HttpInner,
+ session: &HttpSession,
+ operation: Operation,
+ body: &[u8],
+) -> Result<Bytes, WriteError> {
+ let mut next_request_id = session.gate.lock().await;
+ let message = build_request_message(
+ operation,
+ session.client_id,
+ session.session,
+ *next_request_id,
+ body,
+ );
+ let reply = submit_client_request_on_owner(&state.shard, message).await;
let Some(reply) = reply else {
return Err(WriteError::Unavailable);
};
- // Peek the reply frame before decoding: an eviction carries no result
- // section and means the session is dead (re-authenticate), so unlike a
- // committed `Reply` it must not consume this request id.
match reply.header().command {
Command2::Reply => {
- // Committed: the cluster consumed this id, so advance the gate before
- // rendering, whether or not the body decodes.
*next_request_id += 1;
drop(next_request_id);
- Ok(Json(decode_committed_stream(&reply)?))
+ let size = reply.header().size as usize;
+ let reply_body = reply.as_slice().get(HEADER_SIZE..size).unwrap_or_default();
+ match result_code(reply_body) {
+ Some(0) => {
+ let payload_start = result_section_len(reply_body)
+ .ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
+ let payload = reply_body
+ .get(payload_start..)
+ .ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Bytes::copy_from_slice(payload))
+ }
+ Some(code) => Err(WriteError::Rejected(IggyError::from_code(code))),
+ None => Err(WriteError::Rejected(IggyError::InvalidCommand)),
+ }
}
Command2::Eviction => Err(WriteError::Rejected(eviction_error(&reply))),
_ => Err(WriteError::Rejected(IggyError::InvalidCommand)),
}
}
-/// Build the `Message<RequestHeader>` for a create-stream by filling a zeroed
-/// `#[repr(C)]` header, mirroring `wire::rewrite_request_body` and the
-/// partition reconciler's prepare builder.
-fn build_create_stream_message(
+/// Build a `Message<RequestHeader>` for a control-plane write by filling a zeroed
+/// `#[repr(C)]` header, mirroring `wire::rewrite_request_body` and the partition
+/// reconciler's prepare builder. `body` is the already-encoded wire request,
+/// copied in after the header.
+fn build_request_message(
+ operation: Operation,
client_id: u128,
session_id: u64,
request_id: u64,
- request: &CreateStreamRequest,
+ body: &[u8],
) -> Message<RequestHeader> {
- let body = request.to_bytes();
let total = HEADER_SIZE + body.len();
let mut message = Message::<RequestHeader>::new(total);
- message.as_mut_slice()[HEADER_SIZE..].copy_from_slice(&body);
+ message.as_mut_slice()[HEADER_SIZE..].copy_from_slice(body);
let header = bytemuck::checked::try_from_bytes_mut::<RequestHeader>(
&mut message.as_mut_slice()[..HEADER_SIZE],
)
.expect("zeroed bytes form a valid RequestHeader");
header.command = Command2::Request;
- header.operation = Operation::CreateStream;
+ header.operation = operation;
header.client = client_id;
header.session = session_id;
header.request = request_id;
- header.size = u32::try_from(total).expect("create-stream message size fits u32");
+ header.size = u32::try_from(total).expect("control-plane message size fits u32");
message
}
-/// Decode a committed `Reply` frame's body into `StreamDetails`, mirroring the
-/// SDK's `split_metadata_result`. A metadata reply body leads with a result
-/// section (`[count][{index, result}]*`): `count == 0` is success followed by
-/// the `GetStreamResponse` payload, and a nonzero first `result` is a committed
-/// business rejection carrying an `IggyError` code (e.g. duplicate name). The
-/// caller has already confirmed the frame is a `Reply`, so a missing result
-/// section is a malformed committed reply, mapped to an error rather than a 200
-/// with a stale body.
-fn decode_committed_stream(reply: &Message<GenericHeader>) -> Result<StreamDetails, WriteError> {
- let size = reply.header().size as usize;
- let body = reply.as_slice().get(HEADER_SIZE..size).unwrap_or_default();
- match result_code(body) {
- Some(0) => {
- let payload_start =
- result_section_len(body).ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
- let payload = body
- .get(payload_start..)
- .ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
- let response = GetStreamResponse::decode_from(payload)
- .map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
- StreamDetails::try_from(response).map_err(WriteError::Rejected)
- }
- Some(code) => Err(WriteError::Rejected(IggyError::from_code(code))),
- None => Err(WriteError::Rejected(IggyError::InvalidCommand)),
- }
+/// Decode the `GetStreamResponse` payload of a committed create-stream reply into
+/// `StreamDetails`. `payload` is the slice past the result section that
+/// [`submit_write`] already validated as a success.
+fn decode_stream_details(payload: &[u8]) -> Result<StreamDetails, WriteError> {
+ let response = GetStreamResponse::decode_from(payload)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
+ StreamDetails::try_from(response).map_err(WriteError::Rejected)
}
/// Map an eviction frame to the same typed [`IggyError`] the SDK's
(iggy) 12/43: feat(server-ng): redirect linearizable follower reads to primary
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit b667b78f6205ac022c2b8939773cd9ce583e9eef
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 20:07:58 2026 +0200
feat(server-ng): redirect linearizable follower reads to primary
When a linearizable read hits a non-primary node, return 307 with a Location pointing at the current primary's HTTP address resolved from the cluster roster, instead of 503, so a client can follow it to the leader. Falls back to 503 when the primary address is unresolvable.
Dormant on a single node (always primary) and client-inert until an SDK follows the redirect; covered by unit tests on the pure location builder, including IPv6 bracketing.
---
core/server-ng/src/http.rs | 232 +++++++++++++++++++++++++----------
core/server-ng/src/http/extractor.rs | 15 ++-
2 files changed, 181 insertions(+), 66 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index c21fe5c85..7d1799f74 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -24,10 +24,11 @@ mod jwt;
use std::cell::RefCell;
use std::collections::HashMap;
-use std::net::SocketAddr;
+use std::net::{IpAddr, SocketAddr};
use std::rc::Rc;
use axum::extract::{Path, Query, State};
+use axum::http::header::LOCATION;
use axum::http::{HeaderName, HeaderValue, StatusCode};
use axum::middleware::map_response;
use axum::response::{IntoResponse, Response};
@@ -279,10 +280,15 @@ enum ReadError {
Rejected(IggyError),
/// Requested entity is absent -> 404 with the legacy not-found body.
NotFound,
- /// A linearizable read reached a follower. Transient from the caller's view
- /// -> 503, retryable; becomes a 307-to-leader once the roster + redirect
- /// land (see [`not_primary_response`]).
+ /// A linearizable read reached a follower and the primary's HTTP address was
+ /// not resolvable from the roster. Fail-closed 503, retryable against the
+ /// leader (see [`not_primary_response`]).
NotPrimary,
+ /// A linearizable read reached a follower and the current VSR primary's HTTP
+ /// address resolved: 307 to that address carrying the original path and
+ /// query, so the caller re-issues the read against the leader (see
+ /// [`primary_redirect_response`]).
+ RedirectToPrimary(String),
}
impl IntoResponse for ReadError {
@@ -293,15 +299,17 @@ impl IntoResponse for ReadError {
// the legacy server's `CustomError::ResourceNotFound` does.
Self::NotFound => CustomError::ResourceNotFound.into_response(),
Self::NotPrimary => not_primary_response(),
+ Self::RedirectToPrimary(location) => primary_redirect_response(&location),
}
}
}
-/// The 503 body for a linearizable read that reached a follower. No node roster
-/// exists at the HTTP layer yet (F7), so this node cannot 307-redirect to the
-/// primary nor emit a `Location` (F3); the caller must retry, ideally against
-/// the leader. Once both land this branch becomes a 307. Rendered as an
-/// `ErrorResponse` so the body shape matches every other HTTP error.
+/// The 503 fail-closed body for a linearizable read that reached a follower
+/// whose primary HTTP address could not be resolved (absent consensus, a roster
+/// with no node at the primary index, or a port-less node). The resolvable case
+/// is a 307 via [`primary_redirect_response`] instead. Rendered as an
+/// `ErrorResponse` so the body shape matches every other HTTP error; the caller
+/// retries against the leader.
fn not_primary_response() -> Response {
(
StatusCode::SERVICE_UNAVAILABLE,
@@ -315,6 +323,45 @@ fn not_primary_response() -> Response {
.into_response()
}
+/// 307 Temporary Redirect to the current VSR primary for a linearizable read
+/// that reached a follower. `Location` is the primary's HTTP base plus the
+/// original path and query, so the caller re-issues the identical read against
+/// the leader. Dormant on a single node (always primary) and followed by no SDK
+/// yet. A `Location` that is not a valid header value falls back to the 503.
+fn primary_redirect_response(location: &str) -> Response {
+ HeaderValue::from_str(location).map_or_else(
+ |_| not_primary_response(),
+ |value| {
+ let mut response = StatusCode::TEMPORARY_REDIRECT.into_response();
+ response.headers_mut().insert(LOCATION, value);
+ response
+ },
+ )
+}
+
+/// Build the `Location` for a 307 redirect of a linearizable read to the VSR
+/// primary: `http://<host>:<http-port><path_and_query>`. `None` when the roster
+/// has no node at `primary_index`, that node exposes no HTTP port, or its `ip`
+/// is not a valid address, so the caller fails closed to a 503 rather than
+/// pointing at an unreachable target. Formats through [`SocketAddr`] so an IPv6
+/// host is bracketed (`http://[::1]:8080/...`) rather than left ambiguous. Pure
+/// (no consensus or axum dependency) so the redirect target is unit-tested in
+/// isolation.
+fn primary_redirect_location(
+ roster: &ClusterRoster,
+ primary_index: u8,
+ path_and_query: &str,
+) -> Option<String> {
+ let node = roster
+ .nodes
+ .iter()
+ .find(|node| node.replica_id == primary_index)?;
+ let http_port = node.ports.http?;
+ let ip = node.ip.parse::<IpAddr>().ok()?;
+ let socket = SocketAddr::new(ip, http_port);
+ Some(format!("http://{socket}{path_and_query}"))
+}
+
/// Owned snapshot of the cluster topology `GET /cluster/metadata` reports.
///
/// The roster lives only in `ServerNgConfig` and is not reachable from
@@ -361,6 +408,25 @@ impl HttpInner {
.is_some_and(VsrConsensus::is_primary)
}
+ /// Grade a linearizable read that reached a follower: redirect (307) to the
+ /// current VSR primary's HTTP address when it resolves from the roster, else
+ /// fail closed to the 503. The target is the roster node whose `replica_id`
+ /// equals `primary_index(view)`; an absent consensus, an unmatched id, or a
+ /// port-less node all fall back to [`ReadError::NotPrimary`].
+ fn not_primary_read_error(&self, path_and_query: &str) -> ReadError {
+ let location = self
+ .shard
+ .plane
+ .metadata()
+ .consensus
+ .as_ref()
+ .and_then(|consensus| {
+ let primary_index = consensus.primary_index(consensus.view());
+ primary_redirect_location(&self.roster, primary_index, path_and_query)
+ });
+ location.map_or(ReadError::NotPrimary, ReadError::RedirectToPrimary)
+ }
+
/// Resolve the VSR session for `key`, minting and Registering one on first
/// use. Every later request bearing the same credential reuses it.
///
@@ -623,7 +689,7 @@ async fn get_streams(
let body = GetStreamsRequest.to_bytes();
let bytes = read_local(
&state,
- identity.user_id,
+ &identity,
query.consistency,
GET_STREAMS_CODE,
&body,
@@ -647,13 +713,7 @@ async fn get_stream(
stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
};
let body = request.to_bytes();
- let bytes = read_local(
- &state,
- identity.user_id,
- query.consistency,
- GET_STREAM_CODE,
- &body,
- )?;
+ let bytes = read_local(&state, &identity, query.consistency, GET_STREAM_CODE, &body)?;
let response = GetStreamResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -675,13 +735,7 @@ async fn get_topics(
stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
};
let body = request.to_bytes();
- let bytes = read_local(
- &state,
- identity.user_id,
- query.consistency,
- GET_TOPICS_CODE,
- &body,
- )?;
+ let bytes = read_local(&state, &identity, query.consistency, GET_TOPICS_CODE, &body)?;
let response = GetTopicsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -705,13 +759,7 @@ async fn get_topic(
topic_id: identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?,
};
let body = request.to_bytes();
- let bytes = read_local(
- &state,
- identity.user_id,
- query.consistency,
- GET_TOPIC_CODE,
- &body,
- )?;
+ let bytes = read_local(&state, &identity, query.consistency, GET_TOPIC_CODE, &body)?;
let response = GetTopicResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -727,13 +775,7 @@ async fn get_users(
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<Vec<UserInfo>>, ReadError> {
let body = GetUsersRequest.to_bytes();
- let bytes = read_local(
- &state,
- identity.user_id,
- query.consistency,
- GET_USERS_CODE,
- &body,
- )?;
+ let bytes = read_local(&state, &identity, query.consistency, GET_USERS_CODE, &body)?;
let response = GetUsersResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -755,13 +797,7 @@ async fn get_user(
user_id: identifier_to_wire(&user_id).map_err(ReadError::Rejected)?,
};
let body = request.to_bytes();
- let bytes = read_local(
- &state,
- identity.user_id,
- query.consistency,
- GET_USER_CODE,
- &body,
- )?;
+ let bytes = read_local(&state, &identity, query.consistency, GET_USER_CODE, &body)?;
let response = UserDetailsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -787,7 +823,7 @@ async fn get_cgs(
let body = request.to_bytes();
let bytes = read_local(
&state,
- identity.user_id,
+ &identity,
query.consistency,
GET_CONSUMER_GROUPS_CODE,
&body,
@@ -818,7 +854,7 @@ async fn get_cg(
let body = request.to_bytes();
let bytes = read_local(
&state,
- identity.user_id,
+ &identity,
query.consistency,
GET_CONSUMER_GROUP_CODE,
&body,
@@ -838,13 +874,7 @@ async fn get_stats(
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<Stats>, ReadError> {
let body = GetStatsRequest.to_bytes();
- let bytes = read_local(
- &state,
- identity.user_id,
- query.consistency,
- GET_STATS_CODE,
- &body,
- )?;
+ let bytes = read_local(&state, &identity, query.consistency, GET_STATS_CODE, &body)?;
let response = StatsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(Stats::from(response)))
@@ -879,7 +909,7 @@ async fn get_clients(
identity: Identity,
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<Vec<ClientInfo>>, ReadError> {
- authorize_read(&state, identity.user_id, query.consistency)?;
+ authorize_read(&state, &identity, query.consistency)?;
let infos = SendWrapper::new(state.shard.list_all_clients()).await;
let response = GetClientsResponse {
clients: infos
@@ -904,7 +934,7 @@ async fn get_client(
Path(client_id): Path<u32>,
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<ClientInfoDetails>, ReadError> {
- authorize_read(&state, identity.user_id, query.consistency)?;
+ authorize_read(&state, &identity, query.consistency)?;
let infos = SendWrapper::new(state.shard.list_all_clients()).await;
// The wire client id is the u32 seq tail of the u128 transport id.
#[allow(clippy::cast_possible_truncation)]
@@ -1505,20 +1535,20 @@ async fn submit_write(
///
/// Root-only: until server-ng has an RBAC permissioner, every read is root-only,
/// mirroring the write gate in [`submit_committed`]. A non-root credential is
-/// authenticated but unprivileged -> 403. Linearizable reads must come from the
-/// primary; on a follower we cannot yet 307-redirect (no roster: F7) nor emit
-/// the Location (F3), so 503 and let the client retry. This becomes a
-/// 307-to-leader once both land.
+/// authenticated but unprivileged -> 403. A linearizable read must come from the
+/// primary; on a follower it redirects (307) to the primary's HTTP address when
+/// resolvable, else fails closed to a 503 (see
+/// [`HttpInner::not_primary_read_error`]).
fn authorize_read(
state: &HttpInner,
- user_id: u32,
+ identity: &Identity,
consistency: Consistency,
) -> Result<(), ReadError> {
- if user_id != DEFAULT_ROOT_USER_ID {
+ if identity.user_id != DEFAULT_ROOT_USER_ID {
return Err(ReadError::Rejected(IggyError::Unauthorized));
}
if consistency == Consistency::Linearizable && !state.is_metadata_primary() {
- return Err(ReadError::NotPrimary);
+ return Err(state.not_primary_read_error(&identity.path_and_query));
}
Ok(())
}
@@ -1537,12 +1567,12 @@ fn authorize_read(
/// whose entity can be missing shares that not-found shape.
fn read_local(
state: &HttpInner,
- user_id: u32,
+ identity: &Identity,
consistency: Consistency,
code: u32,
body: &[u8],
) -> Result<Bytes, ReadError> {
- authorize_read(state, user_id, consistency)?;
+ authorize_read(state, identity, consistency)?;
match build_non_replicated_response(&state.shard, code, body).map_err(ReadError::Rejected)? {
NonReplicatedResponse::Empty => Err(ReadError::NotFound),
NonReplicatedResponse::Bytes(bytes) => Ok(bytes),
@@ -1785,3 +1815,75 @@ const fn login_error_to_iggy(error: &LoginRegisterError) -> IggyError {
_ => IggyError::Unauthenticated,
}
}
+
+#[cfg(test)]
+mod tests {
+ use configs::cluster::TransportPorts;
+
+ use super::*;
+
+ const READ_PATH: &str = "/streams?consistency=linearizable";
+
+ fn node(replica_id: u8, ip: &str, http: Option<u16>) -> ClusterNodeConfig {
+ ClusterNodeConfig {
+ name: format!("node-{replica_id}"),
+ ip: ip.to_owned(),
+ replica_id,
+ ports: TransportPorts {
+ tcp: None,
+ quic: None,
+ http,
+ websocket: None,
+ tcp_replica: None,
+ },
+ }
+ }
+
+ fn roster(nodes: Vec<ClusterNodeConfig>) -> ClusterRoster {
+ ClusterRoster {
+ enabled: true,
+ name: "test-cluster".to_owned(),
+ nodes,
+ http_addr: "127.0.0.1:3000".parse().expect("valid socket addr"),
+ }
+ }
+
+ #[test]
+ fn primary_redirect_location_targets_primary_http_addr_with_path_passthrough() {
+ let roster = roster(vec![
+ node(0, "10.0.0.1", Some(8080)),
+ node(1, "10.0.0.2", Some(8090)),
+ ]);
+ assert_eq!(
+ primary_redirect_location(&roster, 1, READ_PATH),
+ Some("http://10.0.0.2:8090/streams?consistency=linearizable".to_owned())
+ );
+ }
+
+ #[test]
+ fn primary_redirect_location_is_none_when_no_node_matches_primary_index() {
+ let roster = roster(vec![node(0, "10.0.0.1", Some(8080))]);
+ assert_eq!(primary_redirect_location(&roster, 2, READ_PATH), None);
+ }
+
+ #[test]
+ fn primary_redirect_location_is_none_when_primary_has_no_http_port() {
+ let roster = roster(vec![node(0, "10.0.0.1", None)]);
+ assert_eq!(primary_redirect_location(&roster, 0, READ_PATH), None);
+ }
+
+ #[test]
+ fn primary_redirect_location_is_none_for_empty_roster() {
+ let roster = roster(Vec::new());
+ assert_eq!(primary_redirect_location(&roster, 0, READ_PATH), None);
+ }
+
+ #[test]
+ fn primary_redirect_location_brackets_ipv6_host() {
+ let roster = roster(vec![node(0, "::1", Some(8080))]);
+ assert_eq!(
+ primary_redirect_location(&roster, 0, READ_PATH),
+ Some("http://[::1]:8080/streams?consistency=linearizable".to_owned())
+ );
+ }
+}
diff --git a/core/server-ng/src/http/extractor.rs b/core/server-ng/src/http/extractor.rs
index 96815a56b..c3e235370 100644
--- a/core/server-ng/src/http/extractor.rs
+++ b/core/server-ng/src/http/extractor.rs
@@ -90,6 +90,11 @@ impl FromRequestParts<HttpState> for Authenticated {
/// [`resolve_credential`] chokepoint, so a JWT and a PAT are honored identically.
pub struct Identity {
pub user_id: u32,
+ /// Original request path + query (e.g. `/streams?consistency=linearizable`),
+ /// captured so a linearizable read that reaches a follower can build the
+ /// `Location` for its 307 redirect to the primary. Empty only when the URI
+ /// carries neither, which a routed read never is.
+ pub path_and_query: String,
}
impl FromRequestParts<HttpState> for Identity {
@@ -112,7 +117,15 @@ impl FromRequestParts<HttpState> for Identity {
// no `SendWrapper` bridge (contrast [`Authenticated`], which awaits the
// `!Send` `resolve_session`).
let (_key, user_id, _expiry) = resolve_credential(state, bearer)?;
- Ok(Self { user_id })
+ let path_and_query = parts
+ .uri
+ .path_and_query()
+ .map(|value| value.as_str().to_owned())
+ .unwrap_or_default();
+ Ok(Self {
+ user_id,
+ path_and_query,
+ })
}
}
(iggy) 01/43: feat(message_bus): add in-process reply routing for HTTP listener
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit a98759593a2a4217b5e83d8fda48223cbf22a2e9
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 09:12:38 2026 +0200
feat(message_bus): add in-process reply routing for HTTP listener
server-ng routes every client reply through a per-connection
socket in the connection registry, correlating by client id
only. An in-process caller (the upcoming HTTP listener) holds
no consensus socket: it awaits a single reply that must be
matched by request id on the server side, with no way to
target such a waiter today.
Reify the registry entry's reply target as an enum. Socket
keeps today's per-peer queue unchanged; InProcess holds a
request-id-keyed map of oneshot senders. send_to_client now
routes through ReplyRoute so the request-id decode stays off
the socket path, which stays byte-identical for TCP, WS and
QUIC. Only Socket is constructed here; the InProcess arm is
wired when the HTTP data plane lands.
---
core/message_bus/src/lib.rs | 36 ++++-
.../src/lifecycle/connection_registry.rs | 169 +++++++++++++++------
core/message_bus/src/lifecycle/mod.rs | 2 +-
3 files changed, 153 insertions(+), 54 deletions(-)
diff --git a/core/message_bus/src/lib.rs b/core/message_bus/src/lib.rs
index fc96d7afb..ea50e4cd1 100644
--- a/core/message_bus/src/lib.rs
+++ b/core/message_bus/src/lib.rs
@@ -95,13 +95,13 @@ pub use installer::conn_info::{
};
pub use lifecycle::{
BusMessage, BusReceiver, BusSender, ConnectionRegistry, DrainOutcome, FusedShutdown,
- ReplicaRegistry, Shutdown, ShutdownToken,
+ ReplicaRegistry, ReplyRoute, Shutdown, ShutdownToken,
};
pub use transports::tls::TlsServerCredentials;
pub use compio::runtime::JoinHandle;
use configs::server_ng::ServerNgConfig;
-use iggy_binary_protocol::GenericHeader;
+use iggy_binary_protocol::{GenericHeader, ReplyHeader};
use server_common::{MESSAGE_ALIGN, Message, iobuf::Frozen};
use std::array;
use std::cell::{Cell, OnceCell, RefCell};
@@ -1225,13 +1225,21 @@ impl MessageBus for IggyMessageBus {
// Owning shard is encoded in the top 16 bits of client_id.
let owning_shard = client_id_owning_shard(client_id);
if owning_shard == self.shard_id {
- // Fast path: move `message` straight into `try_send`. On no-slot
- // the registry hands it back; we drop it and surface
- // ClientNotFound (matches prior behaviour: SendError did not
- // preserve payload either).
+ // Route by the entry's reply target. Socket is the unchanged
+ // fast path and never decodes. InProcess resolves the waiting
+ // request id from the reply header and fires its oneshot; that
+ // decode stays off the socket path. No slot surfaces
+ // ClientNotFound (prior behaviour: SendError did not preserve
+ // the payload either).
return match self.clients.try_send_or_return(client_id, message) {
- Ok(send_result) => send_result.map_err(map_try_send_err),
- Err(_msg) => Err(SendError::ClientNotFound(client_id)),
+ ReplyRoute::Delivered(send_result) => send_result.map_err(map_try_send_err),
+ ReplyRoute::InProcess(message) => {
+ let request = reply_request_id(&message);
+ self.clients
+ .fire_in_process(client_id, request, message)
+ .map_err(|_| SendError::ClientNotFound(client_id))
+ }
+ ReplyRoute::NoSlot(_message) => Err(SendError::ClientNotFound(client_id)),
};
}
let forward = self
@@ -1325,6 +1333,18 @@ fn map_try_send_err(e: async_channel::TrySendError<Frozen<MESSAGE_ALIGN>>) -> Se
}
}
+/// Peek `ReplyHeader.request` (the originating request id) from a reply
+/// buffer at its fixed header offset. Only the in-process reply path calls
+/// this; the socket path never decodes.
+fn reply_request_id(reply: &Frozen<MESSAGE_ALIGN>) -> u64 {
+ const OFFSET: usize = std::mem::offset_of!(ReplyHeader, request);
+ reply
+ .as_slice()
+ .get(OFFSET..OFFSET + std::mem::size_of::<u64>())
+ .and_then(|bytes| bytes.try_into().ok())
+ .map_or(0, u64::from_ne_bytes)
+}
+
#[cfg(test)]
mod tests {
use super::*;
diff --git a/core/message_bus/src/lifecycle/connection_registry.rs b/core/message_bus/src/lifecycle/connection_registry.rs
index 162d139df..6a01110b8 100644
--- a/core/message_bus/src/lifecycle/connection_registry.rs
+++ b/core/message_bus/src/lifecycle/connection_registry.rs
@@ -33,6 +33,7 @@
//! [`ShutdownToken`]: crate::lifecycle::ShutdownToken
use compio::runtime::JoinHandle;
+use futures::channel::oneshot;
use server_common::{MESSAGE_ALIGN, iobuf::Frozen};
use std::cell::{Cell, RefCell};
use std::collections::HashMap;
@@ -128,9 +129,50 @@ pub struct DrainOutcome {
pub background_force: usize,
}
+/// Where a client reply is delivered.
+///
+/// `Socket` is the wire path taken by every TCP / WS / QUIC client: the
+/// reply is pushed into the per-peer queue exactly as before. `InProcess`
+/// reifies an in-process reply target keyed by request id so a caller
+/// holding the matching [`oneshot::Receiver`] can await its reply without a
+/// socket. Both ends are shard-0-local, hence a single-threaded oneshot.
+#[derive(Debug)]
+enum ReplyTarget {
+ Socket(BusSender),
+ #[allow(dead_code)] // wired in P2 (HTTP data plane)
+ InProcess {
+ by_request: HashMap<u64, oneshot::Sender<BusMessage>>,
+ },
+}
+
+impl ReplyTarget {
+ /// Wire sender for socket targets; `None` for in-process targets.
+ const fn as_socket(&self) -> Option<&BusSender> {
+ match self {
+ Self::Socket(sender) => Some(sender),
+ Self::InProcess { .. } => None,
+ }
+ }
+}
+
+/// Outcome of routing a client reply through an [`Entry`]'s reply target.
+///
+/// `Delivered` is the socket fast path: `try_send` was attempted and its
+/// result is carried through unchanged. `InProcess` hands the message back
+/// so the caller can decode the request id and fire the matching oneshot
+/// via [`ConnectionRegistry::fire_in_process`], keeping that decode off the
+/// socket path. `NoSlot` means no entry exists for the key.
+#[derive(Debug)]
+#[must_use]
+pub enum ReplyRoute {
+ Delivered(Result<(), async_channel::TrySendError<BusMessage>>),
+ InProcess(BusMessage),
+ NoSlot(BusMessage),
+}
+
#[derive(Debug)]
struct Entry {
- sender: BusSender,
+ reply: ReplyTarget,
writer_handle: Option<JoinHandle<()>>,
reader_handle: Option<JoinHandle<()>>,
/// Generation token minted by the registry on insert. Entries that
@@ -215,39 +257,58 @@ where
/// `close_peer`.
pub fn with_sender<R>(&self, key: K, f: impl FnOnce(&BusSender) -> R) -> Option<R> {
let entries = self.entries.borrow();
- entries.get(&key).map(|entry| f(&entry.sender))
+ entries
+ .get(&key)
+ .and_then(|entry| entry.reply.as_socket())
+ .map(f)
}
- /// Try to send `msg` to the per-peer queue, returning the message back
- /// when no slot exists.
+ /// Route `msg` to the entry's reply target, handing it back when no
+ /// live target accepts it.
///
- /// Lets `send_to_*` skip the unconditional `Frozen::clone()` the
- /// `with_sender` shape forced (closure consumes a clone, outer caller
- /// retains the original for the slow path the borrow checker cannot
- /// prove unreachable). The consuming variant moves `msg` into
- /// `try_send` on the fast path; on no-slot the message is handed back
- /// so the caller can route it via the slow path or drop it.
+ /// Lets `send_to_client` skip the unconditional `Frozen::clone()` the
+ /// `with_sender` shape forced. Socket targets move `msg` straight into
+ /// `try_send`; in-process targets hand it back via
+ /// [`ReplyRoute::InProcess`] so the caller can resolve the request id
+ /// and fire the oneshot without decoding on the socket path.
///
/// Returns:
- /// - `Ok(Ok(()))`: slot exists and the queue accepted `msg`.
- /// - `Ok(Err(TrySendError))`: slot exists but `try_send` failed
- /// (`Full` or `Closed`); the inner error carries `msg` back per
- /// `async_channel`'s contract.
- /// - `Err(msg)`: no slot for `key`; `msg` returned to the caller.
+ /// - [`ReplyRoute::Delivered`]: socket target; carries the `try_send`
+ /// result (`Full` / `Closed` errors carry `msg` back per
+ /// `async_channel`'s contract).
+ /// - [`ReplyRoute::InProcess`]: in-process target; `msg` handed back.
+ /// - [`ReplyRoute::NoSlot`]: no entry for `key`; `msg` handed back.
+ pub fn try_send_or_return(&self, key: K, msg: BusMessage) -> ReplyRoute {
+ let entries = self.entries.borrow();
+ let Some(entry) = entries.get(&key) else {
+ return ReplyRoute::NoSlot(msg);
+ };
+ match &entry.reply {
+ ReplyTarget::Socket(sender) => ReplyRoute::Delivered(sender.try_send(msg)),
+ ReplyTarget::InProcess { .. } => ReplyRoute::InProcess(msg),
+ }
+ }
+
+ /// Fire the in-process reply registered under `request` for `key`.
+ ///
+ /// Companion to [`ReplyRoute::InProcess`]; the socket path never reaches
+ /// here. Removes the matching oneshot and delivers `msg` on it.
///
/// # Errors
///
- /// The outer `Err(msg)` reports that no entry exists for `key`. The
- /// inner `Err(TrySendError)` reports that the entry's queue is full
- /// or closed.
- pub fn try_send_or_return(
- &self,
- key: K,
- msg: BusMessage,
- ) -> Result<Result<(), async_channel::TrySendError<BusMessage>>, BusMessage> {
- let entries = self.entries.borrow();
- match entries.get(&key) {
- Some(entry) => Ok(entry.sender.try_send(msg)),
+ /// Hands `msg` back when `key` has no in-process target or no waiter is
+ /// registered for `request`, so the caller surfaces the same not-found
+ /// outcome as a missing socket.
+ pub fn fire_in_process(&self, key: K, request: u64, msg: BusMessage) -> Result<(), BusMessage> {
+ let mut entries = self.entries.borrow_mut();
+ let Some(entry) = entries.get_mut(&key) else {
+ return Err(msg);
+ };
+ let ReplyTarget::InProcess { by_request } = &mut entry.reply else {
+ return Err(msg);
+ };
+ match by_request.remove(&request) {
+ Some(reply_tx) => reply_tx.send(msg),
None => Err(msg),
}
}
@@ -294,7 +355,7 @@ where
entries.insert(
key,
Entry {
- sender,
+ reply: ReplyTarget::Socket(sender),
writer_handle: Some(writer_handle),
reader_handle: Some(reader_handle),
token,
@@ -360,7 +421,9 @@ where
let Some(mut entry) = self.entries.borrow_mut().remove(&key) else {
return;
};
- entry.sender.close();
+ if let Some(sender) = entry.reply.as_socket() {
+ sender.close();
+ }
if let Some(writer_handle) = entry.writer_handle.take() {
let _ = compio::time::timeout(timeout, writer_handle).await;
}
@@ -389,7 +452,9 @@ where
}
slot.remove()
};
- entry.sender.close();
+ if let Some(sender) = entry.reply.as_socket() {
+ sender.close();
+ }
if let Some(writer_handle) = entry.writer_handle.take() {
let _ = compio::time::timeout(timeout, writer_handle).await;
}
@@ -447,7 +512,9 @@ where
// the shutdown token has not been triggered. Writer is
// awaited before the reader handle is dropped so a
// mid-writev cancellation cannot truncate a frame.
- entry.sender.close();
+ if let Some(sender) = entry.reply.as_socket() {
+ sender.close();
+ }
// Writer gets the full shared deadline so a legitimately-
// flushing `write_vectored_all` never loses budget to any
// other entry in the same drain batch.
@@ -581,7 +648,8 @@ impl ReplicaRegistry {
let slots = self.slots.borrow();
slots[usize::from(key)]
.as_ref()
- .map(|entry| f(&entry.sender))
+ .and_then(|entry| entry.reply.as_socket())
+ .map(f)
}
/// See [`ConnectionRegistry::try_send_or_return`].
@@ -595,8 +663,11 @@ impl ReplicaRegistry {
msg: BusMessage,
) -> Result<Result<(), async_channel::TrySendError<BusMessage>>, BusMessage> {
let slots = self.slots.borrow();
- match slots[usize::from(key)].as_ref() {
- Some(entry) => Ok(entry.sender.try_send(msg)),
+ match slots[usize::from(key)]
+ .as_ref()
+ .and_then(|entry| entry.reply.as_socket())
+ {
+ Some(sender) => Ok(sender.try_send(msg)),
None => Err(msg),
}
}
@@ -627,7 +698,7 @@ impl ReplicaRegistry {
}
let token = self.mint_token();
*slot = Some(Entry {
- sender,
+ reply: ReplyTarget::Socket(sender),
writer_handle: Some(writer_handle),
reader_handle: Some(reader_handle),
token,
@@ -672,7 +743,9 @@ impl ReplicaRegistry {
self.len.set(self.len.get() - 1);
entry
};
- entry.sender.close();
+ if let Some(sender) = entry.reply.as_socket() {
+ sender.close();
+ }
if let Some(writer_handle) = entry.writer_handle.take() {
let _ = compio::time::timeout(timeout, writer_handle).await;
}
@@ -697,7 +770,9 @@ impl ReplicaRegistry {
self.len.set(self.len.get() - 1);
removed_entry
};
- entry.sender.close();
+ if let Some(sender) = entry.reply.as_socket() {
+ sender.close();
+ }
if let Some(writer_handle) = entry.writer_handle.take() {
let _ = compio::time::timeout(timeout, writer_handle).await;
}
@@ -1065,7 +1140,7 @@ mod tests {
.expect("insert ok");
let outer = reg.try_send_or_return(1u8, make_bus_msg());
- assert!(matches!(outer, Ok(Ok(()))));
+ assert!(matches!(outer, ReplyRoute::Delivered(Ok(()))));
// Receiver drains the message; closing the writer's rx end via
// close_peer is not necessary for this assertion.
let received = rx.recv().await.expect("queue had message");
@@ -1081,7 +1156,9 @@ mod tests {
let original_len = msg.len();
let outcome = reg.try_send_or_return(99u8, msg);
- let returned = outcome.expect_err("missing slot should return msg");
+ let ReplyRoute::NoSlot(returned) = outcome else {
+ panic!("missing slot should return msg");
+ };
assert_eq!(returned.len(), original_len);
}
@@ -1104,14 +1181,16 @@ mod tests {
.expect("insert ok");
// Saturate the queue.
- reg.try_send_or_return(1u8, make_bus_msg())
- .expect("slot present")
- .expect("first send accepted");
+ assert!(matches!(
+ reg.try_send_or_return(1u8, make_bus_msg()),
+ ReplyRoute::Delivered(Ok(()))
+ ));
// Second send: slot present, queue full.
- let outcome = reg
- .try_send_or_return(1u8, make_bus_msg())
- .expect("slot still present");
- assert!(matches!(outcome, Err(async_channel::TrySendError::Full(_))));
+ let outcome = reg.try_send_or_return(1u8, make_bus_msg());
+ assert!(matches!(
+ outcome,
+ ReplyRoute::Delivered(Err(async_channel::TrySendError::Full(_)))
+ ));
}
/// Mirror of `try_send_or_return_returns_msg_when_slot_missing` for the
diff --git a/core/message_bus/src/lifecycle/mod.rs b/core/message_bus/src/lifecycle/mod.rs
index 4fa15b9e0..e80bec1f7 100644
--- a/core/message_bus/src/lifecycle/mod.rs
+++ b/core/message_bus/src/lifecycle/mod.rs
@@ -26,6 +26,6 @@ pub mod shutdown;
pub use connection_registry::{
BusMessage, BusReceiver, BusSender, ConnectionRegistry, DrainOutcome, InstanceToken,
- RejectedRegistration, ReplicaRegistry,
+ RejectedRegistration, ReplicaRegistry, ReplyRoute,
};
pub use shutdown::{FusedShutdown, Shutdown, ShutdownToken};
(iggy) 18/43: fix(server-ng): make HTTP write id-advance cancellation-safe
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit f4dcf825cbc5983c1c576d01b466eb0207c5da94
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 00:48:46 2026 +0200
fix(server-ng): make HTTP write id-advance cancellation-safe
Axum drops a write handler's future the moment the HTTP client
disconnects, and that drop could land between the pump-driven consensus
commit and the session gate's id advance. The next write then reused the
id, collided into the depth-1 dedup as Duplicate, and silently received
the prior op's cached reply (or wedged the session near RequestGap on a
single node).
submit_committed now spawns the gate-locked mint/rewrite/submit/advance
sequence (submit_gated) as a detached shard-0 compio task and awaits its
outcome over a oneshot, so a disconnect only drops the receiver half; a
failed result send is ignored because the advance already happened.
---
core/server-ng/src/http.rs | 118 ++++++++++++++++++++++++++++++++++-----------
1 file changed, 90 insertions(+), 28 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index e36f4b813..121e37ce0 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -39,6 +39,7 @@ use bytes::{Bytes, BytesMut};
use configs::cluster::{ClusterConfig, ClusterNodeConfig};
use configs::http::HttpConfig;
use consensus::{MetadataHandle, VsrConsensus};
+use futures::channel::oneshot;
use iggy_binary_protocol::codes::{
GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_STATS_CODE, GET_STREAM_CODE,
GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE, GET_USERS_CODE,
@@ -1975,23 +1976,18 @@ async fn delete_pat(
/// reply body for the stream/topic/user routes, while [`create_pat`] needs the
/// raw `Message` + request header to substitute the one-time token.
///
-/// Enforces the root-only admin gate, then serializes this session's writes
-/// behind its gate and holds it across the submit so request ids reach the
-/// primary strictly in order. The gate advances only on a committed `Reply`: a
-/// transient failure or an eviction leaves the id free for the caller's retry,
-/// which the depth-1 dedup requires (the next accepted request must be
-/// `committed + 1`, so a consumed-but-uncommitted id would wedge the session on
-/// `RequestGap`). An eviction means the session is dead, mapped to 401.
-///
-/// Both shard-0 request rewrites run here before consensus, mirroring the TCP
-/// dispatch path: the PAT rewrite mints a raw token and replicates only its hash
-/// (`CreatePersonalAccessToken`), and the user-password rewrite hashes plaintext
-/// (`CreateUser` / `ChangePassword`). Both are no-ops for every other operation,
-/// so plaintext secrets never enter consensus on any write. On either rewrite's
-/// decode failure the gate is released without advancing, leaving the id free.
+/// Enforces the root-only admin gate, then runs the gate-locked submit
+/// ([`submit_gated`]) on a detached shard-0 task and awaits its outcome over a
+/// oneshot. Axum drops this handler future the moment the HTTP client
+/// disconnects; were the gate held here, that drop could land between the
+/// pump-driven consensus commit and the id advance, leaving the committed op's
+/// reply cached under an id the session still considers unused - the next
+/// write would collide into `Duplicate` and silently receive the prior op's
+/// reply. Detaching moves the whole critical section out of the cancellable
+/// future; a disconnect now only drops the receiver half.
async fn submit_committed(
state: &HttpInner,
- session: &HttpSession,
+ session: &Rc<HttpSession>,
operation: Operation,
body: &[u8],
) -> Result<(RequestHeader, Message<GenericHeader>, Option<String>), WriteError> {
@@ -2001,17 +1997,45 @@ async fn submit_committed(
if session.user_id != DEFAULT_ROOT_USER_ID {
return Err(WriteError::Rejected(IggyError::Unauthorized));
}
- // Per-session gate: serialize this session's request ids across the commit
- // await, advancing the id only after an observed Reply so a transient
- // failure leaves it free to retry.
- //
- // TODO(hubcio): cancellation-unsafe. If the client disconnects while parked
- // on the await below, this handler is dropped without advancing the id, yet
- // the prepare still commits pump-driven. The next write on this session then
- // reuses the id, gets the prior op's cached reply (Duplicate), silently loses
- // its own write, and returns a foreign success. Fix by moving the submit into
- // a detached shard-0 task that owns the id advance and signals the handler
- // via a oneshot, so the commit is recorded regardless of client liveness.
+ let (result_slot, committed) = oneshot::channel();
+ let shard = Rc::clone(&state.shard);
+ let session = Rc::clone(session);
+ let body = body.to_vec();
+ // Detached so a client disconnect cannot split a commit from its id
+ // advance; both happen inside the task regardless of handler liveness.
+ compio::runtime::spawn(async move {
+ let result = submit_gated(&shard, &session, operation, &body).await;
+ // A failed send means the handler died mid-await. The id advance
+ // already happened above, which is the invariant that matters.
+ let _ = result_slot.send(result);
+ })
+ .detach();
+ // `Canceled` = the task was dropped before sending (runtime teardown), a
+ // transient server condition like an unanswered submit.
+ committed.await.map_err(|_| WriteError::Unavailable)?
+}
+
+/// Gate-locked core of [`submit_committed`], run on a task the HTTP client
+/// cannot cancel: serializes this session's writes behind its gate and holds
+/// it across the submit so request ids reach the primary strictly in order.
+/// The gate advances only on a committed `Reply`: a transient failure or an
+/// eviction leaves the id free for the caller's retry, which the depth-1
+/// dedup requires (the next accepted request must be `committed + 1`, so a
+/// consumed-but-uncommitted id would wedge the session on `RequestGap`). An
+/// eviction means the session is dead, mapped to 401.
+///
+/// Both shard-0 request rewrites run here before consensus, mirroring the TCP
+/// dispatch path: the PAT rewrite mints a raw token and replicates only its hash
+/// (`CreatePersonalAccessToken`), and the user-password rewrite hashes plaintext
+/// (`CreateUser` / `ChangePassword`). Both are no-ops for every other operation,
+/// so plaintext secrets never enter consensus on any write. On either rewrite's
+/// decode failure the gate is released without advancing, leaving the id free.
+async fn submit_gated(
+ shard: &Rc<ServerNgShard>,
+ session: &HttpSession,
+ operation: Operation,
+ body: &[u8],
+) -> Result<(RequestHeader, Message<GenericHeader>, Option<String>), WriteError> {
let mut next_request_id = session.gate.lock().await;
let message = build_request_message(
operation,
@@ -2024,7 +2048,7 @@ async fn submit_committed(
rewrite_pat_request_for_user(session.user_id, message).map_err(WriteError::Rejected)?;
let message = maybe_rewrite_user_password_request(message).map_err(WriteError::Rejected)?;
let request_header = *message.header();
- let reply = submit_client_request_on_owner(&state.shard, message).await;
+ let reply = submit_client_request_on_owner(shard, message).await;
let Some(reply) = reply else {
return Err(WriteError::Unavailable);
};
@@ -2046,7 +2070,7 @@ async fn submit_committed(
/// the update/delete/purge routes ignore it (it is empty) and answer 204.
async fn submit_write(
state: &HttpInner,
- session: &HttpSession,
+ session: &Rc<HttpSession>,
operation: Operation,
body: &[u8],
) -> Result<Bytes, WriteError> {
@@ -3209,4 +3233,42 @@ mod tests {
assert_eq!(global.get(), MAX_IN_FLIGHT_WRITES_GLOBAL);
drop(readmitted);
}
+
+ // -- detached control-plane submit --
+
+ /// Pins the platform contract `submit_committed`'s cancellation safety
+ /// rests on: a detached compio task keeps running after the handler side
+ /// is gone, the gate advance it performs sticks, and sending the result
+ /// into a dropped oneshot receiver is an ignorable `Err`, never a panic.
+ /// The full hazard window (client disconnect between the consensus commit
+ /// and the id advance) needs a live commit round-trip, so it is covered by
+ /// construction plus the live cancellation smoke, not faked here.
+ #[compio::test]
+ async fn detached_task_advances_gate_and_ignores_dead_receiver() {
+ let session = Rc::new(HttpSession {
+ client_id: 7,
+ session: 1,
+ user_id: DEFAULT_ROOT_USER_ID,
+ expiry: u64::MAX,
+ gate: Mutex::new(FIRST_REQUEST_ID),
+ data_request: Cell::new(FIRST_REQUEST_ID),
+ registry_token: Cell::new(None),
+ in_flight_writes: Cell::new(0),
+ });
+ let (result_slot, committed) = oneshot::channel::<u64>();
+ // The handler future dies (client disconnect) before the task runs.
+ drop(committed);
+ let (done_slot, done) = oneshot::channel::<()>();
+ let task_session = Rc::clone(&session);
+ compio::runtime::spawn(async move {
+ let mut next_request_id = task_session.gate.lock().await;
+ *next_request_id += 1;
+ let _ = result_slot.send(*next_request_id);
+ drop(next_request_id);
+ let _ = done_slot.send(());
+ })
+ .detach();
+ done.await.expect("detached task must run to completion");
+ assert_eq!(*session.gate.lock().await, FIRST_REQUEST_ID + 1);
+ }
}
(iggy) 10/43: feat(server-ng): add HTTP read routes
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 19557901e254a6d43fa7573ce3938de8d3cc83fd
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 18:52:59 2026 +0200
feat(server-ng): add HTTP read routes
The server-ng HTTP listener could create and manage entities but
had no way to read them back; every GET returned 404/405. This
adds the metadata read surface (streams, topics, users, consumer
groups, stats, clients - list and detail).
Reads are served locally from the shard-0 state machine via the
existing build_non_replicated_response path, bypassing consensus
and needing no bound VSR session - only credential verification.
A shared authorize_read gate applies the interim root-only authz
(consistent with the write routes) and the consistency policy:
the default serializable-local read is served from local state,
while ?consistency=linearizable is served only when this node is
the current VSR primary and otherwise returns 503 (a placeholder
for the leader redirect that lands with the node roster).
GET /personal-access-tokens is deferred: listing a user's tokens
needs a new iteration-safe state-machine accessor that server-ng
does not yet expose (the TCP path lacks it too).
---
core/server-ng/src/http.rs | 533 +++++++++++++++++++++++++++++++++--
core/server-ng/src/http/extractor.rs | 37 +++
2 files changed, 551 insertions(+), 19 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 7d3c7b467..dc1de5f62 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -27,37 +27,54 @@ use std::collections::HashMap;
use std::net::SocketAddr;
use std::rc::Rc;
-use axum::extract::{Path, State};
+use axum::extract::{Path, Query, State};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use axum::routing::{delete, get, post, put};
use axum::{Json, Router};
use bytes::Bytes;
use configs::http::HttpJwtConfig;
+use consensus::{MetadataHandle, VsrConsensus};
+use iggy_binary_protocol::codes::{
+ GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_STATS_CODE, GET_STREAM_CODE,
+ GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE, GET_USERS_CODE,
+};
use iggy_binary_protocol::consensus::{
Command2, EvictionHeader, EvictionReason, HEADER_SIZE, result_code, result_section_len,
};
use iggy_binary_protocol::requests::consumer_groups::{
- CreateConsumerGroupRequest, DeleteConsumerGroupRequest,
+ CreateConsumerGroupRequest, DeleteConsumerGroupRequest, GetConsumerGroupRequest,
+ GetConsumerGroupsRequest,
};
use iggy_binary_protocol::requests::personal_access_tokens::{
CreatePersonalAccessTokenRequest, DeletePersonalAccessTokenRequest,
};
use iggy_binary_protocol::requests::streams::{
- CreateStreamRequest, DeleteStreamRequest, PurgeStreamRequest, UpdateStreamRequest,
+ CreateStreamRequest, DeleteStreamRequest, GetStreamRequest, GetStreamsRequest,
+ PurgeStreamRequest, UpdateStreamRequest,
};
+use iggy_binary_protocol::requests::system::GetStatsRequest;
use iggy_binary_protocol::requests::topics::{
- CreateTopicRequest, DeleteTopicRequest, PurgeTopicRequest, UpdateTopicRequest,
+ CreateTopicRequest, DeleteTopicRequest, GetTopicRequest, GetTopicsRequest, PurgeTopicRequest,
+ UpdateTopicRequest,
};
use iggy_binary_protocol::requests::users::{
- ChangePasswordRequest, CreateUserRequest, DeleteUserRequest, UpdatePermissionsRequest,
- UpdateUserRequest,
+ ChangePasswordRequest, CreateUserRequest, DeleteUserRequest, GetUserRequest, GetUsersRequest,
+ UpdatePermissionsRequest, UpdateUserRequest,
};
+use iggy_binary_protocol::responses::clients::client_response::ConsumerGroupInfoResponse;
+use iggy_binary_protocol::responses::clients::get_client::ClientDetailsResponse;
+use iggy_binary_protocol::responses::clients::get_clients::GetClientsResponse;
use iggy_binary_protocol::responses::consumer_groups::get_consumer_group::ConsumerGroupDetailsResponse;
+use iggy_binary_protocol::responses::consumer_groups::get_consumer_groups::GetConsumerGroupsResponse;
use iggy_binary_protocol::responses::personal_access_tokens::RawPersonalAccessTokenResponse;
use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
+use iggy_binary_protocol::responses::streams::get_streams::GetStreamsResponse;
+use iggy_binary_protocol::responses::system::get_stats::StatsResponse;
use iggy_binary_protocol::responses::topics::get_topic::GetTopicResponse;
+use iggy_binary_protocol::responses::topics::get_topics::GetTopicsResponse;
use iggy_binary_protocol::responses::users::get_user::UserDetailsResponse;
+use iggy_binary_protocol::responses::users::get_users::GetUsersResponse;
use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
use iggy_binary_protocol::{
GenericHeader, Operation, RequestHeader, WireDecode, WireEncode, WireName,
@@ -75,14 +92,20 @@ use iggy_common::update_permissions::UpdatePermissions;
use iggy_common::update_stream::UpdateStream;
use iggy_common::update_topic::UpdateTopic;
use iggy_common::update_user::UpdateUser;
-use iggy_common::wire_conversions::{identifier_to_wire, permissions_to_wire};
+use iggy_common::wire_conversions::{
+ clients_from_wire, consumer_groups_from_wire, identifier_to_wire, permissions_to_wire,
+ streams_from_wire, topics_from_wire, users_from_wire,
+};
use iggy_common::{
- ConsumerGroupDetails, Identifier, IdentityInfo, IggyError, IggyTimestamp,
- RawPersonalAccessToken, StreamDetails, TokenInfo, TopicDetails, UserInfoDetails, Validatable,
+ ClientInfo, ClientInfoDetails, ConsumerGroup, ConsumerGroupDetails, Identifier, IdentityInfo,
+ IggyError, IggyTimestamp, RawPersonalAccessToken, Stats, Stream, StreamDetails, TokenInfo,
+ Topic, TopicDetails, UserInfo, UserInfoDetails, Validatable,
};
use message_bus::client_listener;
+use metadata::impls::metadata::StreamsFrontend;
use secrecy::ExposeSecret;
use send_wrapper::SendWrapper;
+use serde::Deserialize;
use server::http::error::{CustomError, ErrorResponse};
use server_common::Message;
use tokio::sync::Mutex;
@@ -91,11 +114,14 @@ use tracing::{error, info, warn};
use crate::auth::{verify_login_credentials, verify_pat_credentials};
use crate::bootstrap::ServerNgShard;
use crate::dispatch::{submit_client_request_on_owner, submit_register_on_owner};
-use crate::http::extractor::Authenticated;
+use crate::http::extractor::{Authenticated, Identity};
use crate::http::jwt::JwtManager;
use crate::login_register::LoginRegisterError;
use crate::pat::rewrite_pat_request_for_user;
-use crate::responses::build_raw_pat_reply;
+use crate::responses::{
+ NonReplicatedResponse, build_non_replicated_response, build_raw_pat_reply,
+ connected_client_to_response,
+};
use crate::server_error::ServerNgError;
use crate::users::maybe_rewrite_user_password_request;
@@ -205,6 +231,75 @@ fn service_unavailable() -> Response {
.into_response()
}
+/// Read consistency selected by the `?consistency=` query param.
+///
+/// `serializable` (the default) serves from this node's local metadata STM:
+/// correct and consensus-free, but may trail the primary by the replication
+/// delay. `linearizable` demands the freshest committed state and is honored
+/// only on the primary; a follower answers 503 (see [`read_local`]).
+#[derive(Clone, Copy, Default, PartialEq, Eq, Deserialize)]
+#[serde(rename_all = "lowercase")]
+enum Consistency {
+ #[default]
+ Serializable,
+ Linearizable,
+}
+
+/// `?consistency=` query wrapper. An absent param defaults to
+/// [`Consistency::Serializable`]; an unrecognized value is a 400 (axum `Query`).
+#[derive(Default, Deserialize)]
+struct ConsistencyQuery {
+ #[serde(default)]
+ consistency: Consistency,
+}
+
+/// Rejection for an authenticated read route (`GET /streams`,
+/// `GET /streams/{id}`, and the reads that follow). Three classes, three
+/// renderings, mirroring [`WriteError`]'s split so error bodies stay
+/// SDK-identical:
+enum ReadError {
+ /// Caller-side or STM rejection (bad identifier, unsupported op, or the
+ /// interim non-root authz denial) graded through the legacy
+ /// `IggyError -> status` map so SDK error bodies stay byte-identical.
+ Rejected(IggyError),
+ /// Requested entity is absent -> 404 with the legacy not-found body.
+ NotFound,
+ /// A linearizable read reached a follower. Transient from the caller's view
+ /// -> 503, retryable; becomes a 307-to-leader once the roster + redirect
+ /// land (see [`not_primary_response`]).
+ NotPrimary,
+}
+
+impl IntoResponse for ReadError {
+ fn into_response(self) -> Response {
+ match self {
+ Self::Rejected(error) => CustomError::from(error).into_response(),
+ // Reuse the legacy 404 body so a missing stream renders exactly as
+ // the legacy server's `CustomError::ResourceNotFound` does.
+ Self::NotFound => CustomError::ResourceNotFound.into_response(),
+ Self::NotPrimary => not_primary_response(),
+ }
+ }
+}
+
+/// The 503 body for a linearizable read that reached a follower. No node roster
+/// exists at the HTTP layer yet (F7), so this node cannot 307-redirect to the
+/// primary nor emit a `Location` (F3); the caller must retry, ideally against
+/// the leader. Once both land this branch becomes a 307. Rendered as an
+/// `ErrorResponse` so the body shape matches every other HTTP error.
+fn not_primary_response() -> Response {
+ (
+ StatusCode::SERVICE_UNAVAILABLE,
+ Json(ErrorResponse {
+ id: StatusCode::SERVICE_UNAVAILABLE.as_u16().into(),
+ code: "not_primary".to_owned(),
+ reason: "linearizable read requires the primary; retry against the leader".to_owned(),
+ field: None,
+ }),
+ )
+ .into_response()
+}
+
/// Shared shard-0 HTTP state.
///
/// Groups the shard handle, the JWT issuer/verifier, and the per-credential VSR
@@ -221,6 +316,21 @@ struct HttpInner {
}
impl HttpInner {
+ /// True when this shard-0 node is the current VSR metadata primary, i.e.
+ /// `primary_index(current_view) == own_replica_id` - the check consensus
+ /// `is_primary` already encapsulates over the live view and this replica's
+ /// id. Absent consensus (never on shard 0 under VSR, only a no-replica
+ /// build) is treated as not-primary so a linearizable read fails closed
+ /// rather than serving possibly-stale local state as authoritative.
+ fn is_metadata_primary(&self) -> bool {
+ self.shard
+ .plane
+ .metadata()
+ .consensus
+ .as_ref()
+ .is_some_and(VsrConsensus::is_primary)
+ }
+
/// Resolve the VSR session for `key`, minting and Registering one on first
/// use. Every later request bearing the same credential reuses it.
///
@@ -375,20 +485,26 @@ fn router(state: HttpState) -> Router {
post(login_with_personal_access_token),
)
.route("/users/me", get(get_me))
- .route("/users", post(create_user))
- .route("/users/{user_id}", put(update_user).delete(delete_user))
+ .route("/users", get(get_users).post(create_user))
+ .route(
+ "/users/{user_id}",
+ get(get_user).put(update_user).delete(delete_user),
+ )
.route("/users/{user_id}/password", put(change_password))
.route("/users/{user_id}/permissions", put(update_permissions))
- .route("/streams", post(create_stream))
+ .route("/streams", get(get_streams).post(create_stream))
.route(
"/streams/{stream_id}",
- put(update_stream).delete(delete_stream),
+ get(get_stream).put(update_stream).delete(delete_stream),
)
.route("/streams/{stream_id}/purge", delete(purge_stream))
- .route("/streams/{stream_id}/topics", post(create_topic))
+ .route(
+ "/streams/{stream_id}/topics",
+ get(get_topics).post(create_topic),
+ )
.route(
"/streams/{stream_id}/topics/{topic_id}",
- put(update_topic).delete(delete_topic),
+ get(get_topic).put(update_topic).delete(delete_topic),
)
.route(
"/streams/{stream_id}/topics/{topic_id}/purge",
@@ -396,14 +512,19 @@ fn router(state: HttpState) -> Router {
)
.route(
"/streams/{stream_id}/topics/{topic_id}/consumer-groups",
- post(create_cg),
+ get(get_cgs).post(create_cg),
)
.route(
"/streams/{stream_id}/topics/{topic_id}/consumer-groups/{group_id}",
- delete(delete_cg),
+ get(get_cg).delete(delete_cg),
)
+ // TODO(hubcio): GET /personal-access-tokens (list) deferred - needs a server-ng STM
+ // list accessor (PATs sit behind the "Never iterate" apply invariant; TCP-ng lacks it too).
.route("/personal-access-tokens", post(create_pat))
.route("/personal-access-tokens/{name}", delete(delete_pat))
+ .route("/stats", get(get_stats))
+ .route("/clients", get(get_clients))
+ .route("/clients/{client_id}", get(get_client))
.with_state(state)
}
@@ -447,6 +568,316 @@ async fn get_me(identity: Authenticated) -> Json<IdentityInfo> {
})
}
+/// `GET /streams`: list every stream as the same `Vec<Stream>` JSON the legacy
+/// server returns. A consensus-free local STM read via [`read_local`].
+async fn get_streams(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<Vec<Stream>>, ReadError> {
+ let body = GetStreamsRequest.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_STREAMS_CODE,
+ &body,
+ )?;
+ let response = GetStreamsResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(streams_from_wire(response)))
+}
+
+/// `GET /streams/{stream_id}`: fetch one stream by numeric id or name as the
+/// same `StreamDetails` JSON the legacy server returns; 404 when absent. A
+/// consensus-free local STM read via [`read_local`].
+async fn get_stream(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path(stream_id): Path<String>,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<StreamDetails>, ReadError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let request = GetStreamRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_STREAM_CODE,
+ &body,
+ )?;
+ let response = GetStreamResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(
+ StreamDetails::try_from(response).map_err(ReadError::Rejected)?,
+ ))
+}
+
+/// `GET /streams/{stream_id}/topics`: list a stream's topics as the same
+/// `Vec<Topic>` JSON the legacy server returns. A consensus-free local STM read
+/// via [`read_local`].
+async fn get_topics(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path(stream_id): Path<String>,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<Vec<Topic>>, ReadError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let request = GetTopicsRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_TOPICS_CODE,
+ &body,
+ )?;
+ let response = GetTopicsResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(
+ topics_from_wire(response).map_err(ReadError::Rejected)?,
+ ))
+}
+
+/// `GET /streams/{stream_id}/topics/{topic_id}`: fetch one topic by numeric id
+/// or name as the same `TopicDetails` JSON the legacy server returns; 404 when
+/// absent. A consensus-free local STM read via [`read_local`].
+async fn get_topic(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<TopicDetails>, ReadError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let request = GetTopicRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_TOPIC_CODE,
+ &body,
+ )?;
+ let response = GetTopicResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(
+ TopicDetails::try_from(response).map_err(ReadError::Rejected)?,
+ ))
+}
+
+/// `GET /users`: list every user as the same `Vec<UserInfo>` JSON the legacy
+/// server returns. A consensus-free local STM read via [`read_local`].
+async fn get_users(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<Vec<UserInfo>>, ReadError> {
+ let body = GetUsersRequest.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_USERS_CODE,
+ &body,
+ )?;
+ let response = GetUsersResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(
+ users_from_wire(response).map_err(ReadError::Rejected)?,
+ ))
+}
+
+/// `GET /users/{user_id}`: fetch one user by numeric id or name as the same
+/// `UserInfoDetails` JSON the legacy server returns; 404 when absent. A
+/// consensus-free local STM read via [`read_local`].
+async fn get_user(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path(user_id): Path<String>,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<UserInfoDetails>, ReadError> {
+ let user_id = Identifier::from_str_value(&user_id).map_err(ReadError::Rejected)?;
+ let request = GetUserRequest {
+ user_id: identifier_to_wire(&user_id).map_err(ReadError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_USER_CODE,
+ &body,
+ )?;
+ let response = UserDetailsResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(
+ UserInfoDetails::try_from(response).map_err(ReadError::Rejected)?,
+ ))
+}
+
+/// `GET /streams/{stream_id}/topics/{topic_id}/consumer-groups`: list a topic's
+/// consumer groups as the same `Vec<ConsumerGroup>` JSON the legacy server
+/// returns. A consensus-free local STM read via [`read_local`].
+async fn get_cgs(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<Vec<ConsumerGroup>>, ReadError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let request = GetConsumerGroupsRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_CONSUMER_GROUPS_CODE,
+ &body,
+ )?;
+ let response = GetConsumerGroupsResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(consumer_groups_from_wire(response)))
+}
+
+/// `GET /streams/{stream_id}/topics/{topic_id}/consumer-groups/{group_id}`:
+/// fetch one consumer group by numeric id or name as the same
+/// `ConsumerGroupDetails` JSON the legacy server returns; 404 when absent. The
+/// wire-to-domain conversion is infallible.
+async fn get_cg(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path((stream_id, topic_id, group_id)): Path<(String, String, String)>,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<ConsumerGroupDetails>, ReadError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let group_id = Identifier::from_str_value(&group_id).map_err(ReadError::Rejected)?;
+ let request = GetConsumerGroupRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?,
+ group_id: identifier_to_wire(&group_id).map_err(ReadError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_CONSUMER_GROUP_CODE,
+ &body,
+ )?;
+ let response = ConsumerGroupDetailsResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(ConsumerGroupDetails::from(response)))
+}
+
+/// `GET /stats`: server + storage counters as the same `Stats` JSON the legacy
+/// server returns. `GET_STATS` is served by `build_non_replicated_response` like
+/// the entity reads, so it flows through [`read_local`] unchanged rather than a
+/// dedicated builder. No entity can be missing, so no 404 branch.
+async fn get_stats(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<Stats>, ReadError> {
+ let body = GetStatsRequest.to_bytes();
+ let bytes = read_local(
+ &state,
+ identity.user_id,
+ query.consistency,
+ GET_STATS_CODE,
+ &body,
+ )?;
+ let response = StatsResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(Stats::from(response)))
+}
+
+/// `GET /clients`: list every connected client across all shards as the same
+/// `Vec<ClientInfo>` JSON the legacy server returns.
+///
+/// Unlike the entity reads, connections live in each shard's session manager,
+/// not the metadata STM, so this scatter-gathers over the shard mesh
+/// (`list_all_clients`) instead of going through [`read_local`]. It still runs
+/// the identical root-only + consistency gate via [`authorize_read`], so its
+/// authorization matches every metadata read. The gather future is `!Send`,
+/// bridged onto shard 0's thread by `SendWrapper` exactly as the write path
+/// bridges its submit.
+async fn get_clients(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<Vec<ClientInfo>>, ReadError> {
+ authorize_read(&state, identity.user_id, query.consistency)?;
+ let infos = SendWrapper::new(state.shard.list_all_clients()).await;
+ let response = GetClientsResponse {
+ clients: infos
+ .iter()
+ .map(|info| connected_client_to_response(&state.shard, info))
+ .collect(),
+ };
+ Ok(Json(clients_from_wire(response)))
+}
+
+/// `GET /clients/{client_id}`: fetch one connected client as the same
+/// `ClientInfoDetails` JSON the legacy server returns; 404 when absent.
+///
+/// The path id is the `u32` wire client id (the seq tail of the u128 transport
+/// id), matching the legacy route's `Path<u32>`. There is no reverse map from
+/// that id to a home shard, so this gathers every shard's clients and filters -
+/// the same fan-out-and-filter as [`get_clients`] and the TCP `get_client`
+/// dispatch. The wire-to-domain conversion is infallible.
+async fn get_client(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path(client_id): Path<u32>,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<ClientInfoDetails>, ReadError> {
+ authorize_read(&state, identity.user_id, query.consistency)?;
+ let infos = SendWrapper::new(state.shard.list_all_clients()).await;
+ // The wire client id is the u32 seq tail of the u128 transport id.
+ #[allow(clippy::cast_possible_truncation)]
+ let info = infos
+ .iter()
+ .find(|info| info.client_id as u32 == client_id)
+ .ok_or(ReadError::NotFound)?;
+ let consumer_groups = info.vsr_client_id.map_or_else(Vec::new, |vsr_client_id| {
+ state
+ .shard
+ .plane
+ .metadata()
+ .mux_stm
+ .streams()
+ .consumer_group_memberships(vsr_client_id)
+ .into_iter()
+ .map(
+ |(stream_id, topic_id, group_id)| ConsumerGroupInfoResponse {
+ stream_id,
+ topic_id,
+ group_id,
+ },
+ )
+ .collect()
+ });
+ let response = ClientDetailsResponse {
+ client: connected_client_to_response(&state.shard, info),
+ consumer_groups,
+ };
+ Ok(Json(ClientInfoDetails::from(response)))
+}
+
/// `POST /streams`: create a stream and render the committed reply as the same
/// `StreamDetails` JSON the legacy server returns.
///
@@ -952,6 +1383,17 @@ async fn submit_committed(
if session.user_id != DEFAULT_ROOT_USER_ID {
return Err(WriteError::Rejected(IggyError::Unauthorized));
}
+ // Per-session gate: serialize this session's request ids across the commit
+ // await, advancing the id only after an observed Reply so a transient
+ // failure leaves it free to retry.
+ //
+ // TODO(hubcio): cancellation-unsafe. If the client disconnects while parked
+ // on the await below, this handler is dropped without advancing the id, yet
+ // the prepare still commits pump-driven. The next write on this session then
+ // reuses the id, gets the prior op's cached reply (Duplicate), silently loses
+ // its own write, and returns a foreign success. Fix by moving the submit into
+ // a detached shard-0 task that owns the id advance and signals the handler
+ // via a oneshot, so the commit is recorded regardless of client liveness.
let mut next_request_id = session.gate.lock().await;
let message = build_request_message(
operation,
@@ -995,6 +1437,59 @@ async fn submit_write(
Ok(Bytes::copy_from_slice(committed_payload(&reply)?))
}
+/// The two cross-cutting gates every authenticated read enforces before it
+/// touches state. Factored out of [`read_local`] so the cross-shard client
+/// reads (`get_clients` / `get_client`) - which serve from the shard session
+/// managers, not the local STM, and so cannot use [`read_local`] - still pass
+/// the identical gate. Keeping it in one place is what guarantees no read route
+/// can silently skip authz or answer a linearizable request on a follower.
+///
+/// Root-only: until server-ng has an RBAC permissioner, every read is root-only,
+/// mirroring the write gate in [`submit_committed`]. A non-root credential is
+/// authenticated but unprivileged -> 403. Linearizable reads must come from the
+/// primary; on a follower we cannot yet 307-redirect (no roster: F7) nor emit
+/// the Location (F3), so 503 and let the client retry. This becomes a
+/// 307-to-leader once both land.
+fn authorize_read(
+ state: &HttpInner,
+ user_id: u32,
+ consistency: Consistency,
+) -> Result<(), ReadError> {
+ if user_id != DEFAULT_ROOT_USER_ID {
+ return Err(ReadError::Rejected(IggyError::Unauthorized));
+ }
+ if consistency == Consistency::Linearizable && !state.is_metadata_primary() {
+ return Err(ReadError::NotPrimary);
+ }
+ Ok(())
+}
+
+/// Serve one authenticated read from the local metadata STM and hand back the
+/// wire response body. Shared chokepoint for every read route whose data lives
+/// in the metadata STM: it runs the shared [`authorize_read`] gate, then
+/// delegates to [`build_non_replicated_response`], the SAME local-read entry the
+/// TCP dispatch spine uses (`handle_default_non_replicated`), so an HTTP read and
+/// a TCP read of the same entity return byte-identical bodies.
+///
+/// Reads never touch consensus or a VSR session: `build_non_replicated_response`
+/// is a pure STM read. It is synchronous, so this helper is too - no submit
+/// await, no gate, no `SendWrapper`. An absent entity surfaces as
+/// [`NonReplicatedResponse::Empty`], mapped to 404 here because every REST read
+/// whose entity can be missing shares that not-found shape.
+fn read_local(
+ state: &HttpInner,
+ user_id: u32,
+ consistency: Consistency,
+ code: u32,
+ body: &[u8],
+) -> Result<Bytes, ReadError> {
+ authorize_read(state, user_id, consistency)?;
+ match build_non_replicated_response(&state.shard, code, body).map_err(ReadError::Rejected)? {
+ NonReplicatedResponse::Empty => Err(ReadError::NotFound),
+ NonReplicatedResponse::Bytes(bytes) => Ok(bytes),
+ }
+}
+
/// Classify a committed reply's leading result section and return the typed
/// payload slice on success. Mirrors the SDK's `split_metadata_result`:
/// `Some(0)` is success and the payload follows the result section; a nonzero
diff --git a/core/server-ng/src/http/extractor.rs b/core/server-ng/src/http/extractor.rs
index 87d8921ff..96815a56b 100644
--- a/core/server-ng/src/http/extractor.rs
+++ b/core/server-ng/src/http/extractor.rs
@@ -79,6 +79,43 @@ impl FromRequestParts<HttpState> for Authenticated {
}
}
+/// Read-only caller identity for a protected read route: the authenticated
+/// user id and nothing else.
+///
+/// Unlike [`Authenticated`], it verifies the bearer WITHOUT minting or
+/// Registering a VSR session. Reads are served from the local metadata STM and
+/// never enter consensus, so a session (and the `Register` commit round-trip it
+/// costs) would be dead weight. Same 401 rejection surface as [`Authenticated`]
+/// (missing/invalid/expired credential), and the verify runs through the same
+/// [`resolve_credential`] chokepoint, so a JWT and a PAT are honored identically.
+pub struct Identity {
+ pub user_id: u32,
+}
+
+impl FromRequestParts<HttpState> for Identity {
+ type Rejection = AuthError;
+
+ async fn from_request_parts(
+ parts: &mut Parts,
+ state: &HttpState,
+ ) -> Result<Self, Self::Rejection> {
+ let bearer = parts
+ .headers
+ .get(AUTHORIZATION)
+ .and_then(|value| value.to_str().ok())
+ .and_then(|value| value.strip_prefix(BEARER))
+ .ok_or(IggyError::AccessTokenMissing)?;
+
+ // Verify only. The session key and expiry `resolve_credential` also
+ // returns feed the write path's session table; a read discards them.
+ // No `.await` and no session borrow here, so the extractor future needs
+ // no `SendWrapper` bridge (contrast [`Authenticated`], which awaits the
+ // `!Send` `resolve_session`).
+ let (_key, user_id, _expiry) = resolve_credential(state, bearer)?;
+ Ok(Self { user_id })
+ }
+}
+
/// Map a bearer credential to its `(session key, user id, expiry secs)`.
///
/// A JWT is the primary credential, keyed by its `jti`. A raw PAT presented as
(iggy) 07/43: feat(server-ng): add HTTP user-management write routes
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit afc53d8c1e4f03b9fd20f29b91604c27a9d5179a
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 15:32:59 2026 +0200
feat(server-ng): add HTTP user-management write routes
Add the five user-management write routes to server-ng's shard-0 HTTP
listener: POST /users, PUT/DELETE /users/{id}, PUT /users/{id}/password,
and PUT /users/{id}/permissions. This brings the REST control plane to
parity with the legacy server's user surface. CreateUser renders the
committed reply as UserInfoDetails (201 Created); the rest answer 204.
CreateUser and ChangePassword carry plaintext passwords in their JSON
bodies. They are hashed on shard 0 through the shared submit_write helper
via maybe_rewrite_user_password_request before the request enters
consensus, mirroring the TCP dispatch path. Because Argon2 salting is
non-deterministic, hashing must happen once on the primary so every
replica receives the identical hash; a raw password must never be
persisted or replicated. Placing the rewrite in submit_write makes the
invariant hold for every current and future user write.
All five routes flow through submit_write and therefore inherit its
interim root-only authorization gate until server-ng grows an RBAC
permissioner; a non-root credential is rejected with 403.
---
core/server-ng/src/http.rs | 168 ++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 166 insertions(+), 2 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index be7dcd9b5..719219c2f 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -43,23 +43,32 @@ use iggy_binary_protocol::requests::streams::{
use iggy_binary_protocol::requests::topics::{
CreateTopicRequest, DeleteTopicRequest, PurgeTopicRequest, UpdateTopicRequest,
};
+use iggy_binary_protocol::requests::users::{
+ ChangePasswordRequest, CreateUserRequest, DeleteUserRequest, UpdatePermissionsRequest,
+ UpdateUserRequest,
+};
use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
use iggy_binary_protocol::responses::topics::get_topic::GetTopicResponse;
+use iggy_binary_protocol::responses::users::get_user::UserDetailsResponse;
use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
use iggy_binary_protocol::{
GenericHeader, Operation, RequestHeader, WireDecode, WireEncode, WireName,
};
+use iggy_common::change_password::ChangePassword;
use iggy_common::create_stream::CreateStream;
use iggy_common::create_topic::CreateTopic;
+use iggy_common::create_user::CreateUser;
use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
+use iggy_common::update_permissions::UpdatePermissions;
use iggy_common::update_stream::UpdateStream;
use iggy_common::update_topic::UpdateTopic;
-use iggy_common::wire_conversions::identifier_to_wire;
+use iggy_common::update_user::UpdateUser;
+use iggy_common::wire_conversions::{identifier_to_wire, permissions_to_wire};
use iggy_common::{
Identifier, IdentityInfo, IggyError, IggyTimestamp, StreamDetails, TokenInfo, TopicDetails,
- Validatable,
+ UserInfoDetails, Validatable,
};
use message_bus::client_listener;
use secrecy::ExposeSecret;
@@ -76,6 +85,7 @@ use crate::http::extractor::Authenticated;
use crate::http::jwt::JwtManager;
use crate::login_register::LoginRegisterError;
use crate::server_error::ServerNgError;
+use crate::users::maybe_rewrite_user_password_request;
/// `GET /ping` response body, matching the legacy HTTP server's health probe.
const PONG: &str = "pong";
@@ -353,6 +363,10 @@ fn router(state: HttpState) -> Router {
post(login_with_personal_access_token),
)
.route("/users/me", get(get_me))
+ .route("/users", post(create_user))
+ .route("/users/{user_id}", put(update_user).delete(delete_user))
+ .route("/users/{user_id}/password", put(change_password))
+ .route("/users/{user_id}/permissions", put(update_permissions))
.route("/streams", post(create_stream))
.route(
"/streams/{stream_id}",
@@ -620,6 +634,142 @@ async fn purge_topic(
Ok(StatusCode::NO_CONTENT)
}
+/// `POST /users`: create a user and render the committed reply as the same
+/// `UserInfoDetails` JSON the legacy server returns.
+///
+/// The plaintext password rides the JSON body; [`submit_write`] hashes it on
+/// shard 0 before the request enters consensus (see
+/// [`maybe_rewrite_user_password_request`]), so no plaintext is ever replicated.
+async fn create_user(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Json(command): Json<CreateUser>,
+) -> Result<Json<UserInfoDetails>, WriteError> {
+ // Rejects empty/oversized username or password before any consensus work.
+ command.validate().map_err(WriteError::Rejected)?;
+ let request = CreateUserRequest {
+ username: WireName::new(&command.username)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidUsername))?,
+ password: command.password.expose_secret().to_string(),
+ status: command.status.as_code(),
+ permissions: command.permissions.as_ref().map(permissions_to_wire),
+ };
+ let body = request.to_bytes();
+ let payload = SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::CreateUser,
+ &body,
+ ))
+ .await?;
+ Ok(Json(decode_user_details(&payload)?))
+}
+
+/// `PUT /users/{user_id}`: update a user's username and/or status. Returns 204.
+async fn update_user(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(user_id): Path<String>,
+ Json(command): Json<UpdateUser>,
+) -> Result<StatusCode, WriteError> {
+ let user_id = Identifier::from_str_value(&user_id).map_err(WriteError::Rejected)?;
+ // Rejects an oversized replacement username; a no-op when username is absent.
+ command.validate().map_err(WriteError::Rejected)?;
+ let request = UpdateUserRequest {
+ user_id: identifier_to_wire(&user_id).map_err(WriteError::Rejected)?,
+ username: command
+ .username
+ .as_deref()
+ .map(WireName::new)
+ .transpose()
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidUsername))?,
+ status: command.status.map(|status| status.as_code()),
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::UpdateUser,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `DELETE /users/{user_id}`: delete a user. Returns 204.
+async fn delete_user(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(user_id): Path<String>,
+) -> Result<StatusCode, WriteError> {
+ let user_id = Identifier::from_str_value(&user_id).map_err(WriteError::Rejected)?;
+ let request = DeleteUserRequest {
+ user_id: identifier_to_wire(&user_id).map_err(WriteError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::DeleteUser,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `PUT /users/{user_id}/password`: change a user's password. Returns 204.
+///
+/// Like [`create_user`], the new password rides the JSON body in plaintext and
+/// is hashed by [`submit_write`] on shard 0 before replication.
+async fn change_password(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(user_id): Path<String>,
+ Json(command): Json<ChangePassword>,
+) -> Result<StatusCode, WriteError> {
+ let user_id = Identifier::from_str_value(&user_id).map_err(WriteError::Rejected)?;
+ // Rejects empty/oversized current or new password before any consensus work.
+ command.validate().map_err(WriteError::Rejected)?;
+ let request = ChangePasswordRequest {
+ user_id: identifier_to_wire(&user_id).map_err(WriteError::Rejected)?,
+ current_password: command.current_password.expose_secret().to_string(),
+ new_password: command.new_password.expose_secret().to_string(),
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::ChangePassword,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `PUT /users/{user_id}/permissions`: replace a user's permissions. Returns 204.
+async fn update_permissions(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(user_id): Path<String>,
+ Json(command): Json<UpdatePermissions>,
+) -> Result<StatusCode, WriteError> {
+ let user_id = Identifier::from_str_value(&user_id).map_err(WriteError::Rejected)?;
+ command.validate().map_err(WriteError::Rejected)?;
+ let request = UpdatePermissionsRequest {
+ user_id: identifier_to_wire(&user_id).map_err(WriteError::Rejected)?,
+ permissions: command.permissions.as_ref().map(permissions_to_wire),
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::UpdatePermissions,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
/// Run one authenticated control-plane write end to end and return the committed
/// reply's typed payload. Shared by every write route: `create_stream` decodes
/// the payload into an entity, the update/delete/purge routes ignore it (it is
@@ -658,6 +808,11 @@ async fn submit_write(
*next_request_id,
body,
);
+ // Hash raw passwords on shard 0 before replication (CreateUser /
+ // ChangePassword); a no-op for every other operation. Mirrors the TCP
+ // dispatch path so plaintext never enters consensus. On decode failure the
+ // gate is released without advancing, leaving the request id free to retry.
+ let message = maybe_rewrite_user_password_request(message).map_err(WriteError::Rejected)?;
let reply = submit_client_request_on_owner(&state.shard, message).await;
let Some(reply) = reply else {
return Err(WriteError::Unavailable);
@@ -732,6 +887,15 @@ fn decode_topic_details(payload: &[u8]) -> Result<TopicDetails, WriteError> {
TopicDetails::try_from(response).map_err(WriteError::Rejected)
}
+/// Decode the `UserDetailsResponse` payload of a committed create-user reply into
+/// `UserInfoDetails`. `payload` is the slice past the result section that
+/// [`submit_write`] already validated as a success.
+fn decode_user_details(payload: &[u8]) -> Result<UserInfoDetails, WriteError> {
+ let response = UserDetailsResponse::decode_from(payload)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
+ UserInfoDetails::try_from(response).map_err(WriteError::Rejected)
+}
+
/// Map an eviction frame to the same typed [`IggyError`] the SDK's
/// `decode_eviction` produces, so an HTTP caller sees the identical status a TCP
/// caller would (session-terminal reasons render as 401 -> re-authenticate).
(iggy) 32/43: refactor(server-ng): reuse metadata user resolver for password hash
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 66c6f3284fd6ebac1087c54e6103f054531a779b
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 21:23:23 2026 +0200
refactor(server-ng): reuse metadata user resolver for password hash
resolve_password_hash re-implemented the items/index resolution that
metadata's resolve_user_id already owns; two copies would drift. Expose
password_hash_of on UsersInner and drop the duplicate.
---
core/metadata/src/stm/user.rs | 9 +++++++++
core/server-ng/src/users.rs | 29 ++++++++---------------------
2 files changed, 17 insertions(+), 21 deletions(-)
diff --git a/core/metadata/src/stm/user.rs b/core/metadata/src/stm/user.rs
index 72ea4eff7..dafd4e812 100644
--- a/core/metadata/src/stm/user.rs
+++ b/core/metadata/src/stm/user.rs
@@ -139,6 +139,15 @@ impl UsersInner {
}
}
+ /// Stored password hash of the user named by `identifier`, `None` when
+ /// the user does not resolve.
+ #[must_use]
+ pub fn password_hash_of(&self, identifier: &WireIdentifier) -> Option<Arc<str>> {
+ self.resolve_user_id(identifier)
+ .and_then(|user_id| self.items.get(user_id))
+ .map(|user| Arc::clone(&user.password_hash))
+ }
+
/// Collect `(user_id, name)` for every expired personal access token.
///
/// Walks the expiry-ordered `personal_access_token_expiry_index`,
diff --git a/core/server-ng/src/users.rs b/core/server-ng/src/users.rs
index b8d9af4ae..c4de5816c 100644
--- a/core/server-ng/src/users.rs
+++ b/core/server-ng/src/users.rs
@@ -38,12 +38,11 @@ use bytes::Bytes;
use consensus::MetadataHandle;
use iggy_binary_protocol::codec::{WireDecode, WireEncode};
use iggy_binary_protocol::requests::users::{ChangePasswordRequest, CreateUserRequest};
-use iggy_binary_protocol::{Operation, RequestHeader, WireIdentifier};
+use iggy_binary_protocol::{Operation, RequestHeader};
use iggy_common::IggyError;
use metadata::impls::metadata::StreamsFrontend;
use server_common::{Message, crypto};
use std::rc::Rc;
-use std::sync::Arc;
/// Rewrite a raw wire password request before replication.
///
@@ -69,7 +68,12 @@ pub(crate) fn maybe_rewrite_user_password_request(
Operation::ChangePassword => {
let mut wire =
ChangePasswordRequest::decode_from(body).map_err(|_| IggyError::InvalidCommand)?;
- let stored_hash = resolve_password_hash(shard, &wire.user_id);
+ let stored_hash = shard
+ .plane
+ .metadata()
+ .mux_stm
+ .users()
+ .read(|users| users.password_hash_of(&wire.user_id));
verify_and_rewrite_change_password(&mut wire, stored_hash.as_deref())?
}
_ => return Ok(request),
@@ -102,27 +106,10 @@ fn verify_and_rewrite_change_password(
Ok(wire.to_bytes())
}
-/// Stored Argon2 hash of the user named by `user_id`, read from the local
-/// metadata STM (the same replica login verifies against). `None` when the
-/// user does not resolve.
-fn resolve_password_hash(shard: &Rc<ServerNgShard>, user_id: &WireIdentifier) -> Option<Arc<str>> {
- shard.plane.metadata().mux_stm.users().read(|users| {
- let resolved = match user_id {
- WireIdentifier::Numeric(id) => {
- let id = *id as usize;
- users.items.contains(id).then_some(id)
- }
- WireIdentifier::String(name) => users.index.get(name.as_str()).map(|&id| id as usize),
- };
- resolved
- .and_then(|id| users.items.get(id))
- .map(|user| Arc::clone(&user.password_hash))
- })
-}
-
#[cfg(test)]
mod tests {
use super::*;
+ use iggy_binary_protocol::WireIdentifier;
const CURRENT_PASSWORD: &str = "current-password";
const NEW_PASSWORD: &str = "brand-new-password";
(iggy) 31/43: fix(server-ng): stop leaking account status on inactive login
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 6d856ab001ce2862f554faa58b679aec0f648539
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 21:22:12 2026 +0200
fix(server-ng): stop leaking account status on inactive login
Login against an inactive user returned a distinct UserInactive error
without paying the Argon2 verify, so an attacker could tell inactive
accounts apart from nonexistent ones by error shape and timing. Verify
the password first and return InvalidCredentials either way; the PAT
path keeps UserInactive since a valid token already proves existence.
---
core/server-ng/src/auth.rs | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/core/server-ng/src/auth.rs b/core/server-ng/src/auth.rs
index 9dd5d5413..7fa03af06 100644
--- a/core/server-ng/src/auth.rs
+++ b/core/server-ng/src/auth.rs
@@ -83,10 +83,13 @@ pub(crate) fn verify_login_credentials(
let _ = crypto::verify_password(password, DUMMY_PASSWORD_HASH.as_str());
return Err(LoginRegisterError::InvalidCredentials);
};
- if user.status != UserStatus::Active {
- return Err(LoginRegisterError::UserInactive);
- }
- if !crypto::verify_password(password, user.password_hash.as_ref()) {
+ // Verify before the status check and collapse inactive to
+ // InvalidCredentials: an inactive account must answer exactly like a
+ // wrong password (same error, same Argon2 cost), or login could probe
+ // which accounts exist but are disabled.
+ if !crypto::verify_password(password, user.password_hash.as_ref())
+ || user.status != UserStatus::Active
+ {
return Err(LoginRegisterError::InvalidCredentials);
}
Ok(user.id)
(iggy) 24/43: feat(server-ng): enforce RBAC at dispatch for data plane and reads
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 85454492ce71ab2edeec50426e202ea43ca1bf6f
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 15:59:31 2026 +0200
feat(server-ng): enforce RBAC at dispatch for data plane and reads
The in-apply RBAC gate covers replicated metadata ops only. Partition
ops and non-replicated reads never reach apply, so TCP had no authz on
them at all and HTTP hid the gap behind interim root-only gates that
blocked every non-root user.
Gate them at dispatch time on the connection's own shard: resolve the
acting user from the session, resolve the target ids, and check the
live permissioner rules before enqueueing or serving. HTTP swaps its
root-only gates for the same per-op rules and drops the pre-submit
gate on control writes entirely, since apply already authorizes those.
PAT routes become self-scoped, matching the legacy server.
Denials need a reply vehicle that partition and non-replicated frames
lacked, so ReplyHeader gains a status field: nonzero only on a
dispatch-time denial, which always carries an empty body. Committed
results keep using the metadata result section, so the two channels
are mutually exclusive by construction. The SDK peeks status before
any body decode and surfaces a typed error.
Permission changes reach the partition plane once the owning shard
applies the revoking commit; this apply-lag window is bounded by
replication lag and accepted, control ops stay exact.
---
core/binary_protocol/src/consensus/header.rs | 48 ++-
core/integration/tests/server/http_vsr.rs | 125 ++++++++
core/metadata/src/stm/user.rs | 18 +-
core/sdk/src/vsr.rs | 53 ++++
core/server-ng/src/dispatch.rs | 432 +++++++++++++++++++++++++--
core/server-ng/src/http.rs | 419 ++++++++++++++++++++++----
core/server-ng/src/responses.rs | 26 +-
7 files changed, 1019 insertions(+), 102 deletions(-)
diff --git a/core/binary_protocol/src/consensus/header.rs b/core/binary_protocol/src/consensus/header.rs
index 230684211..bc0d7b2fd 100644
--- a/core/binary_protocol/src/consensus/header.rs
+++ b/core/binary_protocol/src/consensus/header.rs
@@ -279,7 +279,20 @@ pub struct ReplyHeader {
pub operation: Operation,
pub operation_padding: [u8; 7],
pub namespace: u64,
- pub reserved: [u8; 32],
+ /// Request-level status: 0 = ok; nonzero = the `IggyError` code for a
+ /// failure decided at dispatch time (authorization denial today; other
+ /// pre-commit gate failures may adopt it later).
+ ///
+ /// Contract: this is nonzero ONLY on a dispatch-time denial, and a deny
+ /// reply always carries an EMPTY body. So this header channel and the
+ /// committed per-sub-op results in the metadata result section are mutually
+ /// exclusive by construction: a reply either commits (status 0, result
+ /// section present) or is denied before commit (status set, no body), and a
+ /// consumer never reconciles the two. Carved from `reserved` exactly like
+ /// `user_id` in `RequestHeader` / `PrepareHeader`; no existing field offset
+ /// moves and `validate` does not inspect it.
+ pub status: u32,
+ pub reserved: [u8; 28],
}
const _: () = {
assert!(size_of::<ReplyHeader>() == HEADER_SIZE);
@@ -287,7 +300,10 @@ const _: () = {
offset_of!(ReplyHeader, request_checksum)
== offset_of!(ReplyHeader, reserved_frame) + size_of::<[u8; 66]>()
);
- assert!(offset_of!(ReplyHeader, reserved) + size_of::<[u8; 32]>() == HEADER_SIZE);
+ assert!(
+ offset_of!(ReplyHeader, status) == offset_of!(ReplyHeader, namespace) + size_of::<u64>()
+ );
+ assert!(offset_of!(ReplyHeader, reserved) + size_of::<[u8; 28]>() == HEADER_SIZE);
};
impl Default for ReplyHeader {
@@ -312,7 +328,8 @@ impl Default for ReplyHeader {
operation: Operation::Reserved,
operation_padding: [0; 7],
namespace: 0,
- reserved: [0; 32],
+ status: 0,
+ reserved: [0; 28],
}
}
}
@@ -1128,6 +1145,31 @@ mod tests {
assert!(header.validate().is_ok());
}
+ // `status` is carved from the reserved tail; the SDK reply funnel peeks it
+ // at this offset before any body decode, so a layout drift must trip here.
+ #[test]
+ fn reply_header_status_offset_and_size_pinned() {
+ use std::mem::offset_of;
+ assert_eq!(size_of::<ReplyHeader>(), 256);
+ assert_eq!(
+ offset_of!(ReplyHeader, status),
+ offset_of!(ReplyHeader, namespace) + size_of::<u64>()
+ );
+ assert_eq!(offset_of!(ReplyHeader, reserved) + 28, 256);
+ }
+
+ // A nonzero status rides the reserved region, which reply `validate` does
+ // not inspect: a status-bearing reply stays valid so the SDK can peek it.
+ #[test]
+ fn reply_header_nonzero_status_still_validates() {
+ let header = ReplyHeader {
+ command: Command2::Reply,
+ status: 41,
+ ..ReplyHeader::default()
+ };
+ assert!(header.validate().is_ok());
+ }
+
// Wire-discriminant pin: any change breaks SDK decoders.
#[test]
fn eviction_reason_discriminants_pinned() {
diff --git a/core/integration/tests/server/http_vsr.rs b/core/integration/tests/server/http_vsr.rs
index eb68dd355..cbd203b70 100644
--- a/core/integration/tests/server/http_vsr.rs
+++ b/core/integration/tests/server/http_vsr.rs
@@ -221,6 +221,69 @@ impl HttpSession {
assert_eq!(response.status(), StatusCode::OK, "poll must answer 200");
response.json().await.expect("decode PolledMessages")
}
+
+ /// Log in as an already-created user, reusing this session's client and
+ /// base URL but swapping the bearer for the new credential. Single attempt:
+ /// the server is already warm once the root session exists.
+ async fn login(&self, username: &str, password: &str) -> Self {
+ let body = serde_json::json!({ "username": username, "password": password });
+ let response = self
+ .client
+ .post(self.url("/users/login"))
+ .json(&body)
+ .send()
+ .await
+ .expect("login request");
+ assert!(
+ response.status().is_success(),
+ "login for {username} failed: {}",
+ response.status()
+ );
+ let identity: IdentityInfo = response.json().await.expect("decode IdentityInfo");
+ let token = identity
+ .access_token
+ .expect("login must return an access token")
+ .token;
+ Self {
+ client: self.client.clone(),
+ base_url: self.base_url.clone(),
+ token,
+ }
+ }
+
+ /// Create an ungranted user (no permissions -> the permissioner holds no
+ /// entry, so every RBAC rule denies). Runs as this session's user.
+ async fn create_user(&self, username: &str, password: &str) {
+ let body = serde_json::json!({
+ "username": username,
+ "password": password,
+ "status": "active",
+ "permissions": null,
+ });
+ let response = self
+ .client
+ .post(self.url("/users"))
+ .bearer_auth(&self.token)
+ .json(&body)
+ .send()
+ .await
+ .expect("create user request");
+ assert!(
+ response.status().is_success(),
+ "create user failed: {}",
+ response.status()
+ );
+ }
+
+ /// GET `path` with this session's bearer.
+ async fn get(&self, path: &str) -> Response {
+ self.client
+ .get(self.url(path))
+ .bearer_auth(&self.token)
+ .send()
+ .await
+ .expect("get request")
+ }
}
fn durability(response: &Response) -> &str {
@@ -544,3 +607,65 @@ async fn given_ack_none_when_producing_should_return_202_and_commit(harness: &Te
sleep(ASYNC_COMMIT_RETRY_INTERVAL).await;
}
}
+
+/// End-to-end RBAC proof: an ungranted user is 403 on a metadata read and on a
+/// data-plane produce, root stays 200/201, and the auth-only cluster-metadata
+/// route is never gated. Exercises the HTTP per-op gates (read + partition
+/// write) and the in-apply control gate (user creation as root) in one path.
+#[iggy_harness]
+async fn given_ungranted_user_when_reading_or_producing_should_be_forbidden(harness: &TestHarness) {
+ let root = HttpSession::login_root(harness).await;
+ root.create_stream_and_topic("rbac-stream", "rbac-topic", 1)
+ .await;
+ // Root provisions a user with no permissions, then we log in as them.
+ root.create_user("rbac-nobody", "rbac-nobody-pass").await;
+ let nobody = root.login("rbac-nobody", "rbac-nobody-pass").await;
+
+ // A metadata read: 403 for the ungranted user, 200 for root.
+ assert_eq!(
+ nobody.get("/streams").await.status(),
+ StatusCode::FORBIDDEN,
+ "ungranted read must be 403"
+ );
+ assert_eq!(
+ root.get("/streams").await.status(),
+ StatusCode::OK,
+ "root read must be 200"
+ );
+
+ // A data-plane produce: 403 for the ungranted user, 201 for root.
+ let denied = nobody
+ .produce(
+ "rbac-stream",
+ "rbac-topic",
+ PARTITION_ID,
+ vec![text_message(1, "denied".to_string())],
+ )
+ .await;
+ assert_eq!(
+ denied.status(),
+ StatusCode::FORBIDDEN,
+ "ungranted produce must be 403"
+ );
+ let allowed = root
+ .produce(
+ "rbac-stream",
+ "rbac-topic",
+ PARTITION_ID,
+ vec![text_message(2, "allowed".to_string())],
+ )
+ .await;
+ assert_eq!(
+ allowed.status(),
+ StatusCode::CREATED,
+ "root produce must commit"
+ );
+
+ // The auth-only cluster-metadata route carries no rule: any authenticated
+ // user reaches it, ungranted included.
+ assert_eq!(
+ nobody.get("/cluster/metadata").await.status(),
+ StatusCode::OK,
+ "cluster metadata is auth-only, never RBAC-gated"
+ );
+}
diff --git a/core/metadata/src/stm/user.rs b/core/metadata/src/stm/user.rs
index a13a50c1b..72ea4eff7 100644
--- a/core/metadata/src/stm/user.rs
+++ b/core/metadata/src/stm/user.rs
@@ -37,7 +37,7 @@ use iggy_binary_protocol::responses::users::user_response::UserResponse;
use iggy_binary_protocol::{WireIdentifier, WireName};
use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
use iggy_common::{
- GlobalPermissions, IggyExpiry, IggyTimestamp, Permissions, PersonalAccessToken,
+ GlobalPermissions, IggyError, IggyExpiry, IggyTimestamp, Permissions, PersonalAccessToken,
StreamPermissions, UserId, UserStatus,
};
use serde::{Deserialize, Serialize};
@@ -167,6 +167,22 @@ impl Users {
self.inner.read(f)
}
+ /// Run a permissioner rule against the committed user permissions and
+ /// return its decision. The dispatch-time authorization surface for the
+ /// checks the in-apply RBAC gate cannot cover: non-replicated reads and
+ /// partition-plane ops, both decided on the connection's own shard rather
+ /// than in a replicated apply. The permissioner stays private; callers pass
+ /// the rule closure.
+ ///
+ /// # Errors
+ /// Propagates the rule's `IggyError` (always `Unauthorized`) on denial.
+ pub fn authorize(
+ &self,
+ rule: impl FnOnce(&Permissioner) -> Result<(), IggyError>,
+ ) -> Result<(), IggyError> {
+ self.read(|inner| rule(&inner.permissioner))
+ }
+
/// Ensures a root user exists in an empty user set.
///
/// # Panics
diff --git a/core/sdk/src/vsr.rs b/core/sdk/src/vsr.rs
index f62b67b63..cb821dcc3 100644
--- a/core/sdk/src/vsr.rs
+++ b/core/sdk/src/vsr.rs
@@ -179,6 +179,9 @@ pub(crate) fn decode_response(response: Bytes) -> Result<Bytes, IggyError> {
if response.len() < total_size {
return Err(IggyError::InvalidCommand);
}
+ if let Some(error) = read_reply_status(header_bytes) {
+ return Err(error);
+ }
let operation = read_operation(header_bytes)?;
split_metadata_result(operation, response.slice(HEADER_SIZE..total_size))
}
@@ -206,6 +209,9 @@ pub(crate) fn decode_response_split(
if body.len() < expected_body {
return Err(IggyError::InvalidCommand);
}
+ if let Some(error) = read_reply_status(header_bytes) {
+ return Err(error);
+ }
let operation = read_operation(header_bytes)?;
split_metadata_result(operation, body.slice(..expected_body))
}
@@ -213,6 +219,21 @@ pub(crate) fn decode_response_split(
}
}
+/// Peek `ReplyHeader.status`, the request-level error channel decided at
+/// dispatch time (authorization denial today). Read before any body decode:
+/// a nonzero status maps to its [`IggyError`] and fails the request, an empty
+/// or partial body notwithstanding. `None` (status 0) lets the reply flow to
+/// the body/result-section decode. Read by wire offset for the same
+/// misalignment reason as [`read_operation`]; metadata replies always stamp 0,
+/// so this is inert for them.
+fn read_reply_status(header_bytes: &[u8; HEADER_SIZE]) -> Option<IggyError> {
+ const STATUS_OFFSET: usize = std::mem::offset_of!(ReplyHeader, status);
+ let mut bytes = [0u8; 4];
+ bytes.copy_from_slice(&header_bytes[STATUS_OFFSET..STATUS_OFFSET + 4]);
+ let status = u32::from_le_bytes(bytes);
+ (status != 0).then(|| IggyError::from_code(status))
+}
+
/// Read the [`Operation`] discriminant from a reply header by wire offset.
/// Mirrors [`peek_command`]: a full `ReplyHeader` struct cast is avoided
/// because the response buffers are not 16-aligned, so the leading `u128`
@@ -523,6 +544,38 @@ mod tests {
}
}
+ #[test]
+ fn reply_with_nonzero_status_surfaces_as_typed_error() {
+ // A dispatch-time authorization denial rides `ReplyHeader.status`; the
+ // decode funnel surfaces it as the typed error before any body decode,
+ // even though the deny body is empty.
+ let header = ReplyHeader {
+ command: Command2::Reply,
+ size: HEADER_SIZE as u32,
+ status: IggyError::Unauthorized.as_code(),
+ ..Default::default()
+ };
+ let mut buf = [0u8; HEADER_SIZE];
+ buf.copy_from_slice(bytemuck::bytes_of(&header));
+ let result = decode_response_split(&buf, Bytes::new());
+ assert!(matches!(result, Err(IggyError::Unauthorized)));
+ }
+
+ #[test]
+ fn reply_with_zero_status_passes_body_through() {
+ // status 0 is the ok channel: a non-metadata reply returns its body.
+ let header = ReplyHeader {
+ command: Command2::Reply,
+ operation: Operation::NonReplicated,
+ size: (HEADER_SIZE + 3) as u32,
+ ..Default::default()
+ };
+ let mut buf = [0u8; HEADER_SIZE];
+ buf.copy_from_slice(bytemuck::bytes_of(&header));
+ let out = decode_response_split(&buf, Bytes::from_static(b"abc")).unwrap();
+ assert_eq!(&out[..], b"abc");
+ }
+
#[test]
fn replicated_request_increments_request_counter() {
let mut session = ConsensusSession::with_client_id(42);
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index 4bf29a025..e27f27ceb 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -33,10 +33,11 @@ use crate::consumer_group::{
use crate::login_register::LoginRegisterError;
use crate::pat::maybe_rewrite_pat_request;
use crate::responses::{
- NonReplicatedResponse, build_consumer_offset_body, build_empty_reply, build_get_me_response,
- build_non_replicated_response, build_polled_messages_body, build_raw_pat_reply,
- connected_client_to_response, current_metadata_commit, resolve_partition_namespace,
- resolve_partition_request_namespace,
+ NonReplicatedResponse, build_consumer_offset_body, build_deny_reply, build_empty_reply,
+ build_get_me_response, build_non_replicated_response, build_polled_messages_body,
+ build_raw_pat_reply, connected_client_to_response, current_metadata_commit,
+ resolve_partition_namespace, resolve_partition_request_namespace, resolve_stream_id,
+ resolve_topic_id,
};
use crate::session_manager::SessionManager;
use crate::users::maybe_rewrite_user_password_request;
@@ -47,16 +48,22 @@ use consensus::{
build_incompatible_protocol_eviction_message,
};
use iggy_binary_protocol::codes::{
- GET_CLIENT_CODE, GET_CLIENTS_CODE, GET_CLUSTER_METADATA_CODE, GET_CONSUMER_OFFSET_CODE,
- GET_ME_CODE, PING_CODE, POLL_MESSAGES_CODE, SYNC_CONSUMER_GROUP_CODE,
+ GET_CLIENT_CODE, GET_CLIENTS_CODE, GET_CLUSTER_METADATA_CODE, GET_CONSUMER_GROUP_CODE,
+ GET_CONSUMER_GROUPS_CODE, GET_CONSUMER_OFFSET_CODE, GET_ME_CODE, GET_STATS_CODE,
+ GET_STREAM_CODE, GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE,
+ GET_USERS_CODE, PING_CODE, POLL_MESSAGES_CODE, SYNC_CONSUMER_GROUP_CODE,
};
use iggy_binary_protocol::primitives::consumer::WireConsumer;
use iggy_binary_protocol::primitives::polling_strategy::WirePollingStrategy;
-use iggy_binary_protocol::requests::consumer_groups::SyncConsumerGroupRequest;
+use iggy_binary_protocol::requests::consumer_groups::{
+ GetConsumerGroupRequest, GetConsumerGroupsRequest, SyncConsumerGroupRequest,
+};
use iggy_binary_protocol::requests::consumer_offsets::GetConsumerOffsetRequest;
use iggy_binary_protocol::requests::messages::PollMessagesRequest;
use iggy_binary_protocol::requests::segments::DeleteSegmentsRequest;
+use iggy_binary_protocol::requests::streams::GetStreamRequest;
use iggy_binary_protocol::requests::system::get_client::GetClientRequest;
+use iggy_binary_protocol::requests::topics::{GetTopicRequest, GetTopicsRequest};
use iggy_binary_protocol::requests::users::{LoginRegisterRequest, LoginRegisterWithPatRequest};
use iggy_binary_protocol::responses::clients::client_response::ConsumerGroupInfoResponse;
use iggy_binary_protocol::responses::clients::get_client::ClientDetailsResponse;
@@ -64,7 +71,7 @@ use iggy_binary_protocol::responses::clients::get_clients::GetClientsResponse;
use iggy_binary_protocol::responses::consumer_groups::SyncConsumerGroupResponse;
use iggy_binary_protocol::{
ClientVersionInfo, EvictionReason, GenericHeader, KIND_CONSUMER_GROUP, Operation,
- ProtocolVersion, RequestHeader, WireDecode, WireEncode, is_protocol_compatible,
+ ProtocolVersion, RequestHeader, WireDecode, WireEncode, WireIdentifier, is_protocol_compatible,
};
use iggy_common::{IggyError, PollingStrategy};
use message_bus::client_listener::RequestHandler;
@@ -73,6 +80,7 @@ use message_bus::{IggyMessageBus, MessageBus};
use metadata::impls::metadata::{
MetadataSubmitError, StreamsFrontend, build_truncate_partition_client_message,
};
+use metadata::permissioner::Permissioner;
use partitions::{PollPlan, PollingArgs, PollingConsumer};
use secrecy::ExposeSecret;
use server_common::Message;
@@ -552,12 +560,17 @@ async fn handle_client_request(
if header.operation.is_partition() {
// `bound` is Some here (unbound transports returned above).
let (vsr_client_id, bound_session) = bound.unwrap_or((0, 0));
+ // `get_session` discards the acting user id the partition gate needs;
+ // resolve it from the same bound connection. A bound transport always
+ // has one, but the gate fails closed on `None` rather than trust that.
+ let acting_user_id = sessions.borrow().get_user_id(transport_client_id);
dispatch_partition_request(
shard,
request,
vsr_client_id,
bound_session,
transport_client_id,
+ acting_user_id,
)
.await;
return;
@@ -682,6 +695,7 @@ pub(crate) async fn dispatch_partition_request(
vsr_client_id: u128,
bound_session: u64,
transport_client_id: u128,
+ acting_user_id: Option<u32>,
) {
let header = *request.header();
let namespace = match resolve_partition_request_namespace(
@@ -706,6 +720,37 @@ pub(crate) async fn dispatch_partition_request(
return;
}
};
+ // Dispatch-time RBAC. The partition plane is not replicated through the
+ // metadata STM, so the in-apply gate cannot cover it; authorize here, on
+ // the connection's own shard, before burning the routable wait or touching
+ // the plane. The namespace resolved above, so its stream/topic are the
+ // committed slab ids the permissioner keys on directly. A denial replies
+ // the op's frame with an empty body and a nonzero `status` the SDK peeks.
+ //
+ // Consistency: this reads THIS shard's local committed permissioner. On a
+ // peer shard that is a replicated read-mirror, so a permission revocation
+ // takes effect on the partition plane only once this shard applies the
+ // revoking commit -- an apply-lag window bounded by replication lag.
+ // Control-plane ops are exact (gated in-apply, in the same committed order
+ // on every replica); this local-read relaxation on the data plane is the
+ // accepted trade for keeping partition ops off the metadata consensus.
+ let scope = IggyNamespace::from_raw(namespace);
+ if let Some(status) = authorize_partition_op(
+ shard,
+ header.operation,
+ acting_user_id,
+ scope.stream_id(),
+ scope.topic_id(),
+ ) {
+ warn!(
+ transport_client_id,
+ status,
+ operation = ?header.operation,
+ "partition request denied by authorization; replying with status"
+ );
+ send_partition_deny_reply(shard, transport_client_id, &header, status).await;
+ return;
+ }
// Convergence wait: a CreateTopic commit returns to the client
// before the per-shard reconcilers seed routing rows and
// materialise the partition (next wake/periodic tick). The SDK
@@ -756,7 +801,7 @@ pub(crate) async fn dispatch_partition_request(
shard.dispatch(request.into_generic());
}
-#[allow(clippy::future_not_send)]
+#[allow(clippy::future_not_send, clippy::too_many_lines)]
async fn handle_non_replicated_request(
shard: &Rc<ServerNgShard>,
sessions: &Rc<RefCell<SessionManager>>,
@@ -765,6 +810,10 @@ async fn handle_non_replicated_request(
) {
const CODE_RANGE: std::ops::Range<usize> = 0..4;
let code = u32::from_le_bytes(request.header().reserved[CODE_RANGE].try_into().unwrap());
+ // Acting user for the read gates below, resolved once. `None` only on the
+ // pre-auth path (PING / GET_CLUSTER_METADATA), which serves ungated codes;
+ // the gated arms fail closed on it.
+ let user_id = sessions.borrow().get_user_id(transport_client_id);
match code {
PING_CODE => {
// A ping is the client's liveness proof; reset its staleness clock
@@ -805,6 +854,11 @@ async fn handle_non_replicated_request(
.await;
}
GET_CLIENTS_CODE => {
+ if let Err(error) = authorize_uid(shard, user_id, Permissioner::get_clients) {
+ send_non_replicated_deny(shard, &request, transport_client_id, error.as_code())
+ .await;
+ return;
+ }
// Shared-nothing: each shard knows only its own connections, so
// gather across all shards (scatter-gather over the mesh).
let infos = shard.list_all_clients().await;
@@ -824,6 +878,11 @@ async fn handle_non_replicated_request(
.await;
}
GET_CLIENT_CODE => {
+ if let Err(error) = authorize_uid(shard, user_id, Permissioner::get_client) {
+ send_non_replicated_deny(shard, &request, transport_client_id, error.as_code())
+ .await;
+ return;
+ }
// No reverse map from the wire u32 id to a u128 transport id /
// home shard (the u32 is just the seq tail), so gather all and
// filter -- same fan-out as `get_clients`.
@@ -862,15 +921,20 @@ async fn handle_non_replicated_request(
.await;
}
POLL_MESSAGES_CODE => {
- handle_poll_messages(shard, transport_client_id, &request).await;
+ handle_poll_messages(shard, transport_client_id, &request, user_id).await;
}
GET_CONSUMER_OFFSET_CODE => {
- handle_get_consumer_offset(shard, transport_client_id, &request).await;
+ handle_get_consumer_offset(shard, transport_client_id, &request, user_id).await;
}
SYNC_CONSUMER_GROUP_CODE => {
+ // Self-scoped: serves the caller's own assignment keyed by the
+ // header client id, so it carries no permissioner rule.
handle_sync_consumer_group(shard, transport_client_id, &request).await;
}
- _ => handle_default_non_replicated(shard, transport_client_id, code, &request).await,
+ _ => {
+ handle_default_non_replicated(shard, transport_client_id, code, &request, user_id)
+ .await;
+ }
}
}
@@ -880,7 +944,15 @@ async fn handle_default_non_replicated(
transport_client_id: u128,
code: u32,
request: &Message<RequestHeader>,
+ user_id: Option<u32>,
) {
+ // Gate by command code before the shared builder runs. The builder stays
+ // authz-free (it is byte-shared with the HTTP read path, which gates
+ // separately); a denial replies status!=0 with an empty body.
+ if let Err(error) = authorize_default_read(shard, code, request_body(request), user_id) {
+ send_non_replicated_deny(shard, request, transport_client_id, error.as_code()).await;
+ return;
+ }
match build_non_replicated_response(shard, code, request_body(request)) {
Ok(response) => {
let commit = current_metadata_commit(shard);
@@ -1081,8 +1153,37 @@ async fn handle_poll_messages(
shard: &Rc<ServerNgShard>,
transport_client_id: u128,
request: &Message<RequestHeader>,
+ user_id: Option<u32>,
) {
- let body = match decode_poll_request(shard, request) {
+ let Ok(wire) = PollMessagesRequest::decode_from(request_body(request)) else {
+ // Undecodable poll: keep the fail-fast empty-poll shape.
+ send_non_replicated_bytes(
+ shard,
+ request,
+ transport_client_id,
+ empty_polled_messages_body(0),
+ "poll_messages",
+ )
+ .await;
+ return;
+ };
+ // Gate on (stream, topic) before touching the partition plane. A resolution
+ // miss falls through to the resolve path below (empty-poll / not-found); a
+ // denial replies status!=0 with an empty body, distinct from the empty-poll
+ // "0 messages" shape.
+ if let Some(status) = authorize_partition_read(
+ shard,
+ &wire.stream_id,
+ &wire.topic_id,
+ user_id,
+ |permissioner, uid, stream_id, topic_id| {
+ permissioner.poll_messages(uid, stream_id, topic_id)
+ },
+ ) {
+ send_non_replicated_deny(shard, request, transport_client_id, status).await;
+ return;
+ }
+ let body = match resolve_poll_request(shard, &wire, request.header().client) {
Ok((namespace, partition_id, consumer, args)) => {
match shard
.partition_read(namespace, PartitionRead::Poll { consumer, args })
@@ -1140,8 +1241,33 @@ async fn handle_get_consumer_offset(
shard: &Rc<ServerNgShard>,
transport_client_id: u128,
request: &Message<RequestHeader>,
+ user_id: Option<u32>,
) {
- let body = match decode_consumer_offset_request(shard, request) {
+ let Ok(wire) = GetConsumerOffsetRequest::decode_from(request_body(request)) else {
+ // Undecodable: an empty body decodes as None (no offset) on the SDK.
+ send_non_replicated_bytes(
+ shard,
+ request,
+ transport_client_id,
+ Bytes::new(),
+ "get_consumer_offset",
+ )
+ .await;
+ return;
+ };
+ if let Some(status) = authorize_partition_read(
+ shard,
+ &wire.stream_id,
+ &wire.topic_id,
+ user_id,
+ |permissioner, uid, stream_id, topic_id| {
+ permissioner.get_consumer_offset(uid, stream_id, topic_id)
+ },
+ ) {
+ send_non_replicated_deny(shard, request, transport_client_id, status).await;
+ return;
+ }
+ let body = match resolve_consumer_offset_request(shard, &wire) {
Ok((namespace, partition_id, consumer)) => {
match shard
.partition_read(namespace, PartitionRead::ConsumerOffset { consumer })
@@ -1247,6 +1373,266 @@ async fn send_empty_partition_reply(
}
}
+// Dispatch-time authorization
+//
+// The metadata STM enforces RBAC in-apply for replicated control ops. What
+// that gate cannot see -- partition-plane ops and non-replicated reads, both
+// decided on the connection's own shard without a replicated apply -- is gated
+// here against the live permissioner via `Users::authorize`. A denial rides
+// `ReplyHeader.status` (empty body), the request-level error channel the SDK
+// peeks before body decode. Root holds every grant in the permissioner, so the
+// rules pass for it without any user-id short-circuit.
+
+/// Authorize a partition-plane op on its resolved (stream, topic) for the
+/// acting user, returning the deny status code or `None` to proceed. The
+/// namespace already resolved, so the entity exists; a `None` user id (which
+/// the bound-session gate should preclude) fails closed with `Unauthenticated`
+/// rather than allow an unattributed write.
+fn authorize_partition_op(
+ shard: &Rc<ServerNgShard>,
+ operation: Operation,
+ user_id: Option<u32>,
+ stream_id: usize,
+ topic_id: usize,
+) -> Option<u32> {
+ let Some(user_id) = user_id else {
+ return Some(IggyError::Unauthenticated.as_code());
+ };
+ let decision =
+ shard
+ .plane
+ .metadata()
+ .mux_stm
+ .users()
+ .authorize(|permissioner| match operation {
+ Operation::SendMessages => {
+ permissioner.append_messages(user_id, stream_id, topic_id)
+ }
+ Operation::StoreConsumerOffset | Operation::StoreConsumerOffset2 => {
+ permissioner.store_consumer_offset(user_id, stream_id, topic_id)
+ }
+ Operation::DeleteConsumerOffset | Operation::DeleteConsumerOffset2 => {
+ permissioner.delete_consumer_offset(user_id, stream_id, topic_id)
+ }
+ // The caller only routes the five partition ops above here; any
+ // other op is an allow, never a surprise denial.
+ _ => Ok(()),
+ });
+ decision.err().map(|error| error.as_code())
+}
+
+/// Reply to a denied partition op with the op's frame: empty body + nonzero
+/// `status`. Distinct from [`send_empty_partition_reply`] (status 0, the
+/// fail-fast "not routable" ack); here the SDK peeks the status and surfaces
+/// the typed authorization error. Same lockstep reasoning: a silent drop would
+/// wedge every later request on the connection.
+#[allow(clippy::future_not_send)]
+async fn send_partition_deny_reply(
+ shard: &Rc<ServerNgShard>,
+ transport_client_id: u128,
+ request_header: &RequestHeader,
+ status: u32,
+) {
+ let commit = current_metadata_commit(shard);
+ let reply = build_deny_reply(request_header, transport_client_id, 0, commit, status);
+ if let Err(error) = shard
+ .bus
+ .send_to_client(transport_client_id, reply.into_generic().into_frozen())
+ .await
+ {
+ warn!(
+ transport_client_id,
+ status,
+ error = %error,
+ operation = ?request_header.operation,
+ "failed to surface partition authz denial"
+ );
+ }
+}
+
+/// Run an unscoped non-replicated-read rule for the acting user. A `None` user
+/// id (only the pre-auth path, which serves ungated codes) fails closed.
+fn authorize_uid(
+ shard: &Rc<ServerNgShard>,
+ user_id: Option<u32>,
+ rule: impl FnOnce(&Permissioner, u32) -> Result<(), IggyError>,
+) -> Result<(), IggyError> {
+ let user_id = user_id.ok_or(IggyError::Unauthenticated)?;
+ shard
+ .plane
+ .metadata()
+ .mux_stm
+ .users()
+ .authorize(|permissioner| rule(permissioner, user_id))
+}
+
+/// Authorize a partition-plane non-replicated read (poll / consumer-offset) on
+/// (stream, topic). `None` proceeds (allowed, or a resolution miss the caller's
+/// own not-found path handles); `Some(status)` denies. A `None` user id fails
+/// closed.
+fn authorize_partition_read(
+ shard: &Rc<ServerNgShard>,
+ stream_id: &WireIdentifier,
+ topic_id: &WireIdentifier,
+ user_id: Option<u32>,
+ rule: impl FnOnce(&Permissioner, u32, usize, usize) -> Result<(), IggyError>,
+) -> Option<u32> {
+ let Some(user_id) = user_id else {
+ return Some(IggyError::Unauthenticated.as_code());
+ };
+ let (stream_id, topic_id) = resolve_topic_scope(shard, stream_id, topic_id)?;
+ shard
+ .plane
+ .metadata()
+ .mux_stm
+ .users()
+ .authorize(|permissioner| rule(permissioner, user_id, stream_id, topic_id))
+ .err()
+ .map(|error| error.as_code())
+}
+
+/// Authorize a non-replicated read routed through `build_non_replicated_response`
+/// (`handle_default_non_replicated`). `Ok(())` allows -- including a resolution
+/// miss, which falls through to the builder's own not-found reply so the legacy
+/// notfound-before-permission ordering holds. `Err` denies with that code.
+/// Unscoped rules gate directly; identifier-scoped rules resolve (stream[,
+/// topic]) against committed state first. `GET_CLUSTER_METADATA` is auth-only
+/// (bootstrap / leader discovery) and every other code the builder serves is
+/// ungated here.
+fn authorize_default_read(
+ shard: &Rc<ServerNgShard>,
+ code: u32,
+ body: &[u8],
+ user_id: Option<u32>,
+) -> Result<(), IggyError> {
+ match code {
+ GET_STATS_CODE => authorize_uid(shard, user_id, Permissioner::get_stats),
+ GET_USERS_CODE => authorize_uid(shard, user_id, Permissioner::get_users),
+ GET_USER_CODE => authorize_uid(shard, user_id, Permissioner::get_user),
+ GET_STREAMS_CODE => authorize_uid(shard, user_id, Permissioner::get_streams),
+ GET_STREAM_CODE => {
+ let Ok(request) = GetStreamRequest::decode_from(body) else {
+ return Ok(());
+ };
+ let Some(stream_id) = resolve_stream_scope(shard, &request.stream_id) else {
+ return Ok(());
+ };
+ authorize_uid(shard, user_id, |permissioner, uid| {
+ permissioner.get_stream(uid, stream_id)
+ })
+ }
+ GET_TOPICS_CODE => {
+ let Ok(request) = GetTopicsRequest::decode_from(body) else {
+ return Ok(());
+ };
+ let Some(stream_id) = resolve_stream_scope(shard, &request.stream_id) else {
+ return Ok(());
+ };
+ authorize_uid(shard, user_id, |permissioner, uid| {
+ permissioner.get_topics(uid, stream_id)
+ })
+ }
+ GET_TOPIC_CODE => {
+ let Ok(request) = GetTopicRequest::decode_from(body) else {
+ return Ok(());
+ };
+ let Some((stream_id, topic_id)) =
+ resolve_topic_scope(shard, &request.stream_id, &request.topic_id)
+ else {
+ return Ok(());
+ };
+ authorize_uid(shard, user_id, |permissioner, uid| {
+ permissioner.get_topic(uid, stream_id, topic_id)
+ })
+ }
+ GET_CONSUMER_GROUP_CODE => {
+ let Ok(request) = GetConsumerGroupRequest::decode_from(body) else {
+ return Ok(());
+ };
+ let Some((stream_id, topic_id)) =
+ resolve_topic_scope(shard, &request.stream_id, &request.topic_id)
+ else {
+ return Ok(());
+ };
+ authorize_uid(shard, user_id, |permissioner, uid| {
+ permissioner.get_consumer_group(uid, stream_id, topic_id)
+ })
+ }
+ GET_CONSUMER_GROUPS_CODE => {
+ let Ok(request) = GetConsumerGroupsRequest::decode_from(body) else {
+ return Ok(());
+ };
+ let Some((stream_id, topic_id)) =
+ resolve_topic_scope(shard, &request.stream_id, &request.topic_id)
+ else {
+ return Ok(());
+ };
+ authorize_uid(shard, user_id, |permissioner, uid| {
+ permissioner.get_consumer_groups(uid, stream_id, topic_id)
+ })
+ }
+ _ => Ok(()),
+ }
+}
+
+/// Resolve a wire stream identifier to its committed slab id, or `None` on a
+/// miss (the gate then falls through to the builder's not-found reply).
+fn resolve_stream_scope(shard: &Rc<ServerNgShard>, stream_id: &WireIdentifier) -> Option<usize> {
+ shard
+ .plane
+ .metadata()
+ .mux_stm
+ .streams()
+ .read(|inner| resolve_stream_id(inner, stream_id))
+}
+
+/// Resolve a wire (stream, topic) pair to committed slab ids, or `None` if
+/// either misses.
+fn resolve_topic_scope(
+ shard: &Rc<ServerNgShard>,
+ stream_id: &WireIdentifier,
+ topic_id: &WireIdentifier,
+) -> Option<(usize, usize)> {
+ shard.plane.metadata().mux_stm.streams().read(|inner| {
+ let stream_id = resolve_stream_id(inner, stream_id)?;
+ let topic_id = resolve_topic_id(inner, stream_id, topic_id)?;
+ Some((stream_id, topic_id))
+ })
+}
+
+/// Reply to a denied non-replicated read with the request's reply frame: empty
+/// body + nonzero `status`. The SDK peeks the status before body decode and
+/// surfaces the typed error, so a poll denial never reaches the empty-poll
+/// "0 messages" body path.
+#[allow(clippy::future_not_send)]
+async fn send_non_replicated_deny(
+ shard: &Rc<ServerNgShard>,
+ request: &Message<RequestHeader>,
+ transport_client_id: u128,
+ status: u32,
+) {
+ let commit = current_metadata_commit(shard);
+ let reply = build_deny_reply(
+ request.header(),
+ request.header().client,
+ request.header().session,
+ commit,
+ status,
+ );
+ if let Err(error) = shard
+ .bus
+ .send_to_client(transport_client_id, reply.into_generic().into_frozen())
+ .await
+ {
+ warn!(
+ transport_client_id,
+ status,
+ error = %error,
+ "failed to surface non-replicated authz denial"
+ );
+ }
+}
+
/// Wait (bounded) until `namespace` is routable: this shard's routing row
/// exists and the owning shard answers a probe read (partition
/// materialised). Fast path: row already present -> no probe, no wait.
@@ -1308,15 +1694,6 @@ fn empty_polled_messages_body(partition_id: u32) -> Bytes {
pub(crate) type DecodedPollRequest = (IggyNamespace, u32, PollingConsumer, PollingArgs);
-fn decode_poll_request(
- shard: &Rc<ServerNgShard>,
- request: &Message<RequestHeader>,
-) -> Result<DecodedPollRequest, IggyError> {
- let wire = PollMessagesRequest::decode_from(request_body(request))
- .map_err(|_| IggyError::InvalidCommand)?;
- resolve_poll_request(shard, &wire, request.header().client)
-}
-
/// Resolve a decoded poll request into its owning-shard read: namespace,
/// partition, polling consumer, and args. Shared by the TCP dispatch (client
/// id = the connection's bound VSR client) and the HTTP route (client id 0,
@@ -1375,15 +1752,6 @@ pub(crate) fn resolve_poll_request(
Ok((namespace, partition_id, consumer, args))
}
-fn decode_consumer_offset_request(
- shard: &Rc<ServerNgShard>,
- request: &Message<RequestHeader>,
-) -> Result<(IggyNamespace, u32, PollingConsumer), IggyError> {
- let wire = GetConsumerOffsetRequest::decode_from(request_body(request))
- .map_err(|_| IggyError::InvalidCommand)?;
- resolve_consumer_offset_request(shard, &wire)
-}
-
/// Resolve a decoded consumer-offset read into its owning-shard read:
/// namespace, partition, and polling consumer. Shared by the TCP dispatch and
/// the HTTP route; needs no client id because offset reads are not fenced
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 2e9f60e54..3b8a940d6 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -91,7 +91,7 @@ use iggy_binary_protocol::responses::users::get_users::GetUsersResponse;
use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
use iggy_binary_protocol::{
AckLevel, GenericHeader, Operation, ReplyHeader, RequestHeader, WireDecode, WireEncode,
- WireName,
+ WireIdentifier, WireName,
};
use iggy_common::change_password::ChangePassword;
use iggy_common::create_consumer_group::CreateConsumerGroup;
@@ -99,9 +99,10 @@ use iggy_common::create_personal_access_token::CreatePersonalAccessToken;
use iggy_common::create_stream::CreateStream;
use iggy_common::create_topic::CreateTopic;
use iggy_common::create_user::CreateUser;
-use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
use iggy_common::delete_consumer_offset::DeleteConsumerOffset;
use iggy_common::get_consumer_offset::GetConsumerOffset;
+// `DEFAULT_ROOT_USER_ID` is only referenced by the test module now that the
+// interim root-only gates are gone; its import lives there.
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
use iggy_common::poll_messages::DEFAULT_PARTITION_ID;
@@ -125,6 +126,7 @@ use iggy_common::{
};
use message_bus::{BusMessage, InstanceToken, client_listener};
use metadata::impls::metadata::StreamsFrontend;
+use metadata::permissioner::Permissioner;
use secrecy::ExposeSecret;
use send_wrapper::SendWrapper;
use serde::Deserialize;
@@ -146,7 +148,7 @@ use crate::login_register::LoginRegisterError;
use crate::pat::rewrite_pat_request_for_user;
use crate::responses::{
NonReplicatedResponse, build_non_replicated_response, build_polled_messages_body,
- build_raw_pat_reply, connected_client_to_response,
+ build_raw_pat_reply, connected_client_to_response, resolve_stream_id, resolve_topic_id,
};
use crate::server_error::ServerNgError;
use crate::users::maybe_rewrite_user_password_request;
@@ -335,9 +337,9 @@ impl IntoResponse for WriteError {
/// unknown rather than failed.
#[derive(Debug)]
enum PartitionWriteError {
- /// Caller-side rejection (bad identifier, oversized batch, the interim
- /// non-root denial) or a malformed reply frame, rendered through the
- /// legacy `IggyError -> status` map for SDK-identical bodies.
+ /// Caller-side rejection (bad identifier, oversized batch, an authorization
+ /// denial) or a malformed reply frame, rendered through the legacy
+ /// `IggyError -> status` map for SDK-identical bodies.
Rejected(IggyError),
/// The dispatch gates could not route the write: the stream, topic, or
/// partition does not resolve (or never materialised within the routable
@@ -509,9 +511,9 @@ struct ProduceQuery {
/// renderings, mirroring [`WriteError`]'s split so error bodies stay
/// SDK-identical:
enum ReadError {
- /// Caller-side or STM rejection (bad identifier, unsupported op, or the
- /// interim non-root authz denial) graded through the legacy
- /// `IggyError -> status` map so SDK error bodies stay byte-identical.
+ /// Caller-side or STM rejection (bad identifier, unsupported op, or an
+ /// authorization denial) graded through the legacy `IggyError -> status`
+ /// map so SDK error bodies stay byte-identical.
Rejected(IggyError),
/// Requested entity is absent -> 404 with the legacy not-found body.
NotFound,
@@ -981,6 +983,7 @@ async fn get_streams(
query.consistency,
GET_STREAMS_CODE,
&body,
+ Permissioner::get_streams,
)?;
let response = GetStreamsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
@@ -997,11 +1000,24 @@ async fn get_stream(
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<StreamDetails>, ReadError> {
let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let wire_stream_id = identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?;
+ // Resolve for the gate; a miss leaves it a pass-through so the read renders
+ // the existing 404 rather than a 403.
+ let scope = resolve_gate_stream(&state, &wire_stream_id);
let request = GetStreamRequest {
- stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
+ stream_id: wire_stream_id,
};
let body = request.to_bytes();
- let bytes = read_local(&state, &identity, query.consistency, GET_STREAM_CODE, &body)?;
+ let bytes = read_local(
+ &state,
+ &identity,
+ query.consistency,
+ GET_STREAM_CODE,
+ &body,
+ |permissioner, uid| {
+ scope.map_or(Ok(()), |stream_id| permissioner.get_stream(uid, stream_id))
+ },
+ )?;
let response = GetStreamResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -1019,11 +1035,22 @@ async fn get_topics(
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<Vec<Topic>>, ReadError> {
let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let wire_stream_id = identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?;
+ let scope = resolve_gate_stream(&state, &wire_stream_id);
let request = GetTopicsRequest {
- stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
+ stream_id: wire_stream_id,
};
let body = request.to_bytes();
- let bytes = read_local(&state, &identity, query.consistency, GET_TOPICS_CODE, &body)?;
+ let bytes = read_local(
+ &state,
+ &identity,
+ query.consistency,
+ GET_TOPICS_CODE,
+ &body,
+ |permissioner, uid| {
+ scope.map_or(Ok(()), |stream_id| permissioner.get_topics(uid, stream_id))
+ },
+ )?;
let response = GetTopicsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -1042,12 +1069,26 @@ async fn get_topic(
) -> Result<Json<TopicDetails>, ReadError> {
let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let wire_stream_id = identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?;
+ let wire_topic_id = identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?;
+ let scope = resolve_gate_topic(&state, &wire_stream_id, &wire_topic_id);
let request = GetTopicRequest {
- stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
- topic_id: identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?,
+ stream_id: wire_stream_id,
+ topic_id: wire_topic_id,
};
let body = request.to_bytes();
- let bytes = read_local(&state, &identity, query.consistency, GET_TOPIC_CODE, &body)?;
+ let bytes = read_local(
+ &state,
+ &identity,
+ query.consistency,
+ GET_TOPIC_CODE,
+ &body,
+ |permissioner, uid| {
+ scope.map_or(Ok(()), |(stream_id, topic_id)| {
+ permissioner.get_topic(uid, stream_id, topic_id)
+ })
+ },
+ )?;
let response = GetTopicResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -1063,7 +1104,14 @@ async fn get_users(
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<Vec<UserInfo>>, ReadError> {
let body = GetUsersRequest.to_bytes();
- let bytes = read_local(&state, &identity, query.consistency, GET_USERS_CODE, &body)?;
+ let bytes = read_local(
+ &state,
+ &identity,
+ query.consistency,
+ GET_USERS_CODE,
+ &body,
+ Permissioner::get_users,
+ )?;
let response = GetUsersResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -1085,7 +1133,14 @@ async fn get_user(
user_id: identifier_to_wire(&user_id).map_err(ReadError::Rejected)?,
};
let body = request.to_bytes();
- let bytes = read_local(&state, &identity, query.consistency, GET_USER_CODE, &body)?;
+ let bytes = read_local(
+ &state,
+ &identity,
+ query.consistency,
+ GET_USER_CODE,
+ &body,
+ Permissioner::get_user,
+ )?;
let response = UserDetailsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(
@@ -1104,9 +1159,12 @@ async fn get_cgs(
) -> Result<Json<Vec<ConsumerGroup>>, ReadError> {
let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let wire_stream_id = identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?;
+ let wire_topic_id = identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?;
+ let scope = resolve_gate_topic(&state, &wire_stream_id, &wire_topic_id);
let request = GetConsumerGroupsRequest {
- stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
- topic_id: identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?,
+ stream_id: wire_stream_id,
+ topic_id: wire_topic_id,
};
let body = request.to_bytes();
let bytes = read_local(
@@ -1115,6 +1173,11 @@ async fn get_cgs(
query.consistency,
GET_CONSUMER_GROUPS_CODE,
&body,
+ |permissioner, uid| {
+ scope.map_or(Ok(()), |(stream_id, topic_id)| {
+ permissioner.get_consumer_groups(uid, stream_id, topic_id)
+ })
+ },
)?;
let response = GetConsumerGroupsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
@@ -1134,9 +1197,12 @@ async fn get_cg(
let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
let group_id = Identifier::from_str_value(&group_id).map_err(ReadError::Rejected)?;
+ let wire_stream_id = identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?;
+ let wire_topic_id = identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?;
+ let scope = resolve_gate_topic(&state, &wire_stream_id, &wire_topic_id);
let request = GetConsumerGroupRequest {
- stream_id: identifier_to_wire(&stream_id).map_err(ReadError::Rejected)?,
- topic_id: identifier_to_wire(&topic_id).map_err(ReadError::Rejected)?,
+ stream_id: wire_stream_id,
+ topic_id: wire_topic_id,
group_id: identifier_to_wire(&group_id).map_err(ReadError::Rejected)?,
};
let body = request.to_bytes();
@@ -1146,6 +1212,11 @@ async fn get_cg(
query.consistency,
GET_CONSUMER_GROUP_CODE,
&body,
+ |permissioner, uid| {
+ scope.map_or(Ok(()), |(stream_id, topic_id)| {
+ permissioner.get_consumer_group(uid, stream_id, topic_id)
+ })
+ },
)?;
let response = ConsumerGroupDetailsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
@@ -1162,7 +1233,14 @@ async fn get_stats(
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<Stats>, ReadError> {
let body = GetStatsRequest.to_bytes();
- let bytes = read_local(&state, &identity, query.consistency, GET_STATS_CODE, &body)?;
+ let bytes = read_local(
+ &state,
+ &identity,
+ query.consistency,
+ GET_STATS_CODE,
+ &body,
+ Permissioner::get_stats,
+ )?;
let response = StatsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
Ok(Json(Stats::from(response)))
@@ -1172,9 +1250,10 @@ async fn get_stats(
/// `ClusterMetadata` JSON the legacy server returns.
///
/// Auth-only: any valid token serves. Unlike the entity reads it bypasses both
-/// the root gate and the consistency gate, and serves from the roster captured
-/// at listener start plus the sync consensus getters, so it never touches the
-/// metadata STM, consensus, or a VSR session and stays fully synchronous.
+/// the per-op authorization gate and the consistency gate, and serves from the
+/// roster captured at listener start plus the sync consensus getters, so it
+/// never touches the metadata STM, consensus, or a VSR session and stays fully
+/// synchronous.
async fn get_cluster_metadata(
State(state): State<HttpState>,
_identity: Identity,
@@ -1188,7 +1267,7 @@ async fn get_cluster_metadata(
/// Unlike the entity reads, connections live in each shard's session manager,
/// not the metadata STM, so this scatter-gathers over the shard mesh
/// (`list_all_clients`) instead of going through [`read_local`]. It still runs
-/// the identical root-only + consistency gate via [`authorize_read`], so its
+/// the identical per-op + consistency gate via [`authorize_read`], so its
/// authorization matches every metadata read. The gather future is `!Send`,
/// bridged onto shard 0's thread by `SendWrapper` exactly as the write path
/// bridges its submit.
@@ -1197,7 +1276,9 @@ async fn get_clients(
identity: Identity,
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<Vec<ClientInfo>>, ReadError> {
- authorize_read(&state, &identity, query.consistency)?;
+ authorize_read(&state, &identity, query.consistency, |permissioner, uid| {
+ permissioner.get_clients(uid)
+ })?;
let infos = SendWrapper::new(state.shard.list_all_clients()).await;
let response = GetClientsResponse {
clients: infos
@@ -1222,7 +1303,9 @@ async fn get_client(
Path(client_id): Path<u32>,
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<ClientInfoDetails>, ReadError> {
- authorize_read(&state, &identity, query.consistency)?;
+ authorize_read(&state, &identity, query.consistency, |permissioner, uid| {
+ permissioner.get_client(uid)
+ })?;
let infos = SendWrapper::new(state.shard.list_all_clients()).await;
// The wire client id is the u32 seq tail of the u128 transport id.
#[allow(clippy::cast_possible_truncation)]
@@ -1483,9 +1566,19 @@ async fn poll_messages(
Query(query): Query<PollMessages>,
Query(consistency): Query<ConsistencyQuery>,
) -> Result<Json<PolledMessages>, ReadError> {
- authorize_read(&state, &identity, consistency.consistency)?;
let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let scope = resolve_gate_topic_ids(&state, &stream_id, &topic_id);
+ authorize_read(
+ &state,
+ &identity,
+ consistency.consistency,
+ |permissioner, uid| {
+ scope.map_or(Ok(()), |(stream_id, topic_id)| {
+ permissioner.poll_messages(uid, stream_id, topic_id)
+ })
+ },
+ )?;
let wire = poll_wire_request(&stream_id, &topic_id, &query).map_err(ReadError::Rejected)?;
let (namespace, partition_id, consumer, args) =
match resolve_poll_request(&state.shard, &wire, HTTP_READ_CLIENT_ID) {
@@ -1540,9 +1633,19 @@ async fn get_consumer_offset(
Query(query): Query<GetConsumerOffset>,
Query(consistency): Query<ConsistencyQuery>,
) -> Result<Json<ConsumerOffsetInfo>, ReadError> {
- authorize_read(&state, &identity, consistency.consistency)?;
let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let scope = resolve_gate_topic_ids(&state, &stream_id, &topic_id);
+ authorize_read(
+ &state,
+ &identity,
+ consistency.consistency,
+ |permissioner, uid| {
+ scope.map_or(Ok(()), |(stream_id, topic_id)| {
+ permissioner.get_consumer_offset(uid, stream_id, topic_id)
+ })
+ },
+ )?;
let wire =
consumer_offset_wire_request(&stream_id, &topic_id, &query).map_err(ReadError::Rejected)?;
let (namespace, partition_id, consumer) =
@@ -1589,14 +1692,19 @@ async fn send_messages(
Query(query): Query<ProduceQuery>,
Json(command): Json<SendMessages>,
) -> Result<Response, PartitionWriteError> {
- // Interim authorization: root-only until server-ng has an RBAC
- // permissioner, mirroring the control-plane gate in `submit_committed`.
- if identity.session.user_id != DEFAULT_ROOT_USER_ID {
- return Err(PartitionWriteError::Rejected(IggyError::Unauthorized));
- }
let stream_id =
Identifier::from_str_value(&stream_id).map_err(PartitionWriteError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(PartitionWriteError::Rejected)?;
+ // RBAC: authorize the produce on (stream, topic) before any consensus work.
+ // Handler-side so an HTTP denial never enters the partition plane.
+ authorize_data_plane(
+ &state,
+ identity.session.user_id,
+ &stream_id,
+ &topic_id,
+ Permissioner::append_messages,
+ )
+ .map_err(PartitionWriteError::Rejected)?;
// Rejects an oversized partitioning key and an empty or oversized batch.
command.validate().map_err(PartitionWriteError::Rejected)?;
let body = encode_send_messages(&stream_id, &topic_id, &command)
@@ -1648,14 +1756,18 @@ async fn store_consumer_offset(
Path((stream_id, topic_id)): Path<(String, String)>,
Json(command): Json<StoreConsumerOffset>,
) -> Result<StatusCode, PartitionWriteError> {
- // Interim authorization: root-only until server-ng has an RBAC
- // permissioner, mirroring the control-plane gate in `submit_committed`.
- if identity.session.user_id != DEFAULT_ROOT_USER_ID {
- return Err(PartitionWriteError::Rejected(IggyError::Unauthorized));
- }
let stream_id =
Identifier::from_str_value(&stream_id).map_err(PartitionWriteError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(PartitionWriteError::Rejected)?;
+ // RBAC: authorize the offset write on (stream, topic) handler-side.
+ authorize_data_plane(
+ &state,
+ identity.session.user_id,
+ &stream_id,
+ &topic_id,
+ Permissioner::store_consumer_offset,
+ )
+ .map_err(PartitionWriteError::Rejected)?;
let request = store_offset_wire_request(&stream_id, &topic_id, &command)
.map_err(PartitionWriteError::Rejected)?;
let body = request.to_bytes();
@@ -1680,14 +1792,18 @@ async fn delete_consumer_offset(
Path((stream_id, topic_id, consumer_id)): Path<(String, String, String)>,
Query(query): Query<DeleteConsumerOffset>,
) -> Result<StatusCode, PartitionWriteError> {
- // Interim authorization: root-only until server-ng has an RBAC
- // permissioner, mirroring the control-plane gate in `submit_committed`.
- if identity.session.user_id != DEFAULT_ROOT_USER_ID {
- return Err(PartitionWriteError::Rejected(IggyError::Unauthorized));
- }
let stream_id =
Identifier::from_str_value(&stream_id).map_err(PartitionWriteError::Rejected)?;
let topic_id = Identifier::from_str_value(&topic_id).map_err(PartitionWriteError::Rejected)?;
+ // RBAC: authorize the offset delete on (stream, topic) handler-side.
+ authorize_data_plane(
+ &state,
+ identity.session.user_id,
+ &stream_id,
+ &topic_id,
+ Permissioner::delete_consumer_offset,
+ )
+ .map_err(PartitionWriteError::Rejected)?;
// `Consumer::new` fixes the kind to `Consumer`, exactly as the legacy
// handler does; HTTP cannot express a group-kind offset op.
let consumer = Consumer::new(
@@ -1949,7 +2065,8 @@ async fn create_pat(
}
/// `DELETE /personal-access-tokens/{name}`: delete one of the caller's tokens by
-/// name. Returns 204 on commit. Inherits the root-only gate via [`submit_write`].
+/// name. Returns 204 on commit. Self-scoped: any authenticated user may delete
+/// their own tokens (the in-apply gate skips PAT ops), matching legacy parity.
async fn delete_pat(
State(state): State<HttpState>,
identity: Authenticated,
@@ -1976,7 +2093,8 @@ async fn delete_pat(
/// reply body for the stream/topic/user routes, while [`create_pat`] needs the
/// raw `Message` + request header to substitute the one-time token.
///
-/// Enforces the root-only admin gate, then runs the gate-locked submit
+/// Control-plane writes are authorized in-apply on the metadata STM, so this
+/// runs no pre-submit gate; it drives the gate-locked submit
/// ([`submit_gated`]) on a detached shard-0 task and awaits its outcome over a
/// oneshot. Axum drops this handler future the moment the HTTP client
/// disconnects; were the gate held here, that drop could land between the
@@ -1991,12 +2109,12 @@ async fn submit_committed(
operation: Operation,
body: &[u8],
) -> Result<(RequestHeader, Message<GenericHeader>, Option<String>), WriteError> {
- // Interim authorization: until server-ng has an RBAC permissioner, every
- // control-plane write is root-only. A non-root credential is authenticated
- // but unprivileged, rejected before any consensus work is spent.
- if session.user_id != DEFAULT_ROOT_USER_ID {
- return Err(WriteError::Rejected(IggyError::Unauthorized));
- }
+ // Control writes are authorized in-apply on the metadata STM: a denial
+ // comes back as `Unauthorized` in the committed result section, which
+ // `committed_payload` maps to a 403. No pre-submit gate here, so the
+ // in-apply check is the single source of truth. Self-scoped PAT ops (which
+ // the in-apply gate skips) reach here as any authenticated user, matching
+ // legacy parity.
let (result_slot, committed) = oneshot::channel();
let shard = Rc::clone(&state.shard);
let session = Rc::clone(session);
@@ -2307,6 +2425,7 @@ async fn partition_write_replicated(
session.client_id,
session.session,
session.client_id,
+ Some(session.user_id),
)
.await;
let outcome = compio::time::timeout(PARTITION_WRITE_REPLY_TIMEOUT, receiver).await;
@@ -2339,6 +2458,7 @@ async fn produce_unacked(state: &HttpInner, session: &HttpSession, body: &[u8])
session.client_id,
session.session,
session.client_id,
+ Some(session.user_id),
)
.await;
}
@@ -2394,20 +2514,27 @@ fn classify_partition_reply(reply: &BusMessage) -> Result<(), PartitionWriteErro
/// the identical gate. Keeping it in one place is what guarantees no read route
/// can silently skip authz or answer a linearizable request on a follower.
///
-/// Root-only: until server-ng has an RBAC permissioner, every read is root-only,
-/// mirroring the write gate in [`submit_committed`]. A non-root credential is
-/// authenticated but unprivileged -> 403. A linearizable read must come from the
-/// primary; on a follower it redirects (307) to the primary's HTTP address when
-/// resolvable, else fails closed to a 503 (see
+/// Per-op RBAC: run the route's `rule` against the caller's committed
+/// permissions via the live permissioner. A denial (always `Unauthorized`)
+/// renders 403 through the legacy `IggyError -> status` map; root holds every
+/// grant, so its reads pass without a user-id short-circuit. A linearizable
+/// read must come from the primary; on a follower it redirects (307) to the
+/// primary's HTTP address when resolvable, else fails closed to a 503 (see
/// [`HttpInner::not_primary_read_error`]).
fn authorize_read(
state: &HttpInner,
identity: &Identity,
consistency: Consistency,
+ rule: impl FnOnce(&Permissioner, u32) -> Result<(), IggyError>,
) -> Result<(), ReadError> {
- if identity.user_id != DEFAULT_ROOT_USER_ID {
- return Err(ReadError::Rejected(IggyError::Unauthorized));
- }
+ state
+ .shard
+ .plane
+ .metadata()
+ .mux_stm
+ .users()
+ .authorize(|permissioner| rule(permissioner, identity.user_id))
+ .map_err(ReadError::Rejected)?;
if consistency == Consistency::Linearizable && !state.is_metadata_primary() {
return Err(state.not_primary_read_error(&identity.path_and_query));
}
@@ -2432,14 +2559,92 @@ fn read_local(
consistency: Consistency,
code: u32,
body: &[u8],
+ rule: impl FnOnce(&Permissioner, u32) -> Result<(), IggyError>,
) -> Result<Bytes, ReadError> {
- authorize_read(state, identity, consistency)?;
+ authorize_read(state, identity, consistency, rule)?;
match build_non_replicated_response(&state.shard, code, body).map_err(ReadError::Rejected)? {
NonReplicatedResponse::Empty => Err(ReadError::NotFound),
NonReplicatedResponse::Bytes(bytes) => Ok(bytes),
}
}
+/// Resolve a wire stream identifier to its committed slab id for a read/write
+/// gate, or `None` on a miss (the gate is then a pass-through, so the existing
+/// not-found path renders the 404). Mirrors the TCP dispatch resolvers.
+fn resolve_gate_stream(state: &HttpInner, stream_id: &WireIdentifier) -> Option<usize> {
+ state
+ .shard
+ .plane
+ .metadata()
+ .mux_stm
+ .streams()
+ .read(|inner| resolve_stream_id(inner, stream_id))
+}
+
+/// Resolve a wire (stream, topic) pair to committed slab ids, or `None` if
+/// either misses.
+fn resolve_gate_topic(
+ state: &HttpInner,
+ stream_id: &WireIdentifier,
+ topic_id: &WireIdentifier,
+) -> Option<(usize, usize)> {
+ state
+ .shard
+ .plane
+ .metadata()
+ .mux_stm
+ .streams()
+ .read(|inner| {
+ let stream_id = resolve_stream_id(inner, stream_id)?;
+ let topic_id = resolve_topic_id(inner, stream_id, topic_id)?;
+ Some((stream_id, topic_id))
+ })
+}
+
+/// Resolve an (`Identifier`, `Identifier`) pair to committed (stream, topic)
+/// slab ids for a gate, or `None` on any conversion or resolution miss. For the
+/// poll / consumer-offset read routes, which carry domain identifiers rather
+/// than the pre-converted wire form the entity reads hold.
+fn resolve_gate_topic_ids(
+ state: &HttpInner,
+ stream_id: &Identifier,
+ topic_id: &Identifier,
+) -> Option<(usize, usize)> {
+ let wire_stream = identifier_to_wire(stream_id).ok()?;
+ let wire_topic = identifier_to_wire(topic_id).ok()?;
+ resolve_gate_topic(state, &wire_stream, &wire_topic)
+}
+
+/// Authorize an HTTP data-plane write (produce / consumer-offset) on (stream,
+/// topic). A resolution miss returns `Ok(())` so the write proceeds to the
+/// dispatch gates, which render the existing 404; a resolved entity whose rule
+/// rejects returns the `Unauthorized` error for a 403. Kept handler-side so an
+/// HTTP denial never enters the partition plane (the plane's empty replies
+/// carry no status a slot-waiting handler could read as a 403).
+fn authorize_data_plane(
+ state: &HttpInner,
+ user_id: u32,
+ stream_id: &Identifier,
+ topic_id: &Identifier,
+ rule: impl FnOnce(&Permissioner, u32, usize, usize) -> Result<(), IggyError>,
+) -> Result<(), IggyError> {
+ let (Ok(wire_stream), Ok(wire_topic)) =
+ (identifier_to_wire(stream_id), identifier_to_wire(topic_id))
+ else {
+ return Ok(());
+ };
+ let Some((stream_id, topic_id)) = resolve_gate_topic(state, &wire_stream, &wire_topic) else {
+ return Ok(());
+ };
+ state
+ .shard
+ .plane
+ .metadata()
+ .mux_stm
+ .users()
+ .authorize(|permissioner| rule(permissioner, user_id, stream_id, topic_id))
+}
+
/// Build the live [`ClusterMetadata`] for `GET /cluster/metadata`.
///
/// Mirrors the legacy `get_cluster_metadata`: with a configured, non-empty
@@ -2695,10 +2900,96 @@ mod tests {
};
use super::*;
- use crate::responses::build_empty_reply;
+ use crate::responses::{
+ build_deny_reply, build_empty_reply, build_reply_from_bytes, build_reply_with_body,
+ };
+ use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
const READ_PATH: &str = "/streams?consistency=linearizable";
+ // The dispatch-time deny frame: an empty body plus `ReplyHeader.status` set
+ // to the rule's error code, the request-level channel the SDK peeks before
+ // body decode. `build_empty_reply` (status 0) is the ok-shaped counterpart.
+ #[test]
+ fn deny_reply_stamps_status_over_an_empty_body() {
+ let request = build_request_message(Operation::SendMessages, 42, 7, 3, &[]);
+ let status = IggyError::Unauthorized.as_code();
+ let reply = build_deny_reply(request.header(), 42, 0, 9, status);
+ assert_eq!(
+ reply.header().size as usize,
+ HEADER_SIZE,
+ "deny body is empty (header-only)"
+ );
+ assert_eq!(
+ reply.header().status,
+ status,
+ "status carries the deny code"
+ );
+ assert_ne!(status, 0, "a deny status is nonzero so the SDK peek fires");
+ assert_eq!(reply.header().command, Command2::Reply);
+ // Echoes the request so the SDK routes it back to the waiting slot.
+ assert_eq!(reply.header().request, request.header().request);
+ assert_eq!(reply.header().operation, request.header().operation);
+ }
+
+ // Enforces the status contract: every reply builder EXCEPT build_deny_reply
+ // leaves status 0, so a future builder that leaks a nonzero reserved tail
+ // (or sets status by mistake) trips here rather than silently shipping a
+ // fake denial. build_deny_reply is the sole nonzero-status writer.
+ #[test]
+ fn only_the_deny_builder_emits_a_nonzero_status() {
+ let request = build_request_message(Operation::CreateStream, 42, 7, 3, &[]);
+ let header = request.header();
+ let commit = 9;
+ let body = Bytes::from_static(b"body");
+
+ // server-ng reply builders, all funnelled through build_reply_with_body.
+ for status in [
+ build_reply_with_body(header, 42, 7, commit, 0, |_| {})
+ .header()
+ .status,
+ build_empty_reply(header, 42, 7, commit).header().status,
+ build_reply_from_bytes(header, 42, 7, commit, &body)
+ .header()
+ .status,
+ NonReplicatedResponse::Empty
+ .into_reply(header, 42, 7, commit)
+ .header()
+ .status,
+ NonReplicatedResponse::Bytes(body.clone())
+ .into_reply(header, 42, 7, commit)
+ .header()
+ .status,
+ ] {
+ assert_eq!(status, 0, "ok-path reply builders must leave status 0");
+ }
+
+ // The consensus plane's committed-reply builder (plane_helpers path).
+ let prepare = PrepareHeader {
+ command: Command2::Prepare,
+ operation: Operation::SendMessages,
+ client: 42,
+ op: 1,
+ request: 1,
+ ..Default::default()
+ };
+ let committed = consensus::build_reply_message(&prepare, &Bytes::new());
+ assert_eq!(
+ committed.header().status,
+ 0,
+ "committed replies carry status 0"
+ );
+
+ // The one intentional writer stamps nonzero.
+ assert_ne!(
+ build_deny_reply(header, 42, 7, commit, IggyError::Unauthorized.as_code())
+ .header()
+ .status,
+ 0,
+ "the deny builder is the sole nonzero-status writer"
+ );
+ }
+
fn node(replica_id: u8, ip: &str, http: Option<u16>) -> ClusterNodeConfig {
ClusterNodeConfig {
name: format!("node-{replica_id}"),
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index bd11b9915..16198ca5b 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -680,7 +680,7 @@ fn build_get_topics_response(
})
}
-fn resolve_stream_id(
+pub(crate) fn resolve_stream_id(
streams: &metadata::stm::stream::StreamsInner,
identifier: &WireIdentifier,
) -> Option<usize> {
@@ -693,7 +693,7 @@ fn resolve_stream_id(
}
}
-fn resolve_topic_id(
+pub(crate) fn resolve_topic_id(
streams: &metadata::stm::stream::StreamsInner,
stream_id: usize,
identifier: &WireIdentifier,
@@ -778,6 +778,28 @@ pub(crate) fn build_empty_reply(
build_reply_with_body(request_header, client_id, session, commit, 0, |_| {})
}
+/// Build an empty reply that denies a dispatch-time authorization check: the
+/// same request echo as [`build_empty_reply`] but with `ReplyHeader.status`
+/// set to the rule's error code -- the request-level failure channel the SDK
+/// peeks before any body decode. The single writer of a nonzero status, so
+/// every deny path stamps it one way.
+pub(crate) fn build_deny_reply(
+ request_header: &RequestHeader,
+ client_id: u128,
+ session: u64,
+ commit: u64,
+ status: u32,
+) -> Message<ReplyHeader> {
+ let mut reply = build_empty_reply(request_header, client_id, session, commit);
+ let header_len = std::mem::size_of::<ReplyHeader>();
+ let header = bytemuck::checked::try_from_bytes_mut::<ReplyHeader>(
+ &mut reply.as_mut_slice()[..header_len],
+ )
+ .expect("empty reply header is a valid ReplyHeader");
+ header.status = status;
+ reply
+}
+
/// Server build version advertised in the login-register response.
const SERVER_VERSION: &str = env!("CARGO_PKG_VERSION");
(iggy) 28/43: fix(server-ng): omit X-Iggy-View on error responses
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 77e0e3565ad143a7984233c7a5c76afdd3bce025
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 18:23:40 2026 +0200
fix(server-ng): omit X-Iggy-View on error responses
The view header was attached by the outermost response layer
unconditionally, so pre-auth rejections leaked the cluster-internal
view number to anonymous callers. Attach it only to success and
redirect responses; the 307 primary-redirect contract is unaffected.
---
core/server-ng/src/http.rs | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 53931ca46..c98318633 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -2876,10 +2876,16 @@ fn self_node_metadata(roster: &ClusterRoster) -> ClusterMetadata {
}
}
-/// Set the [`VIEW_HEADER`] to the current VSR view on `response`. Omits the
-/// header when this node has no live consensus: a missing header is
-/// unambiguous, whereas a fabricated view number would mislead.
+/// Set the [`VIEW_HEADER`] to the current VSR view on a successful or redirect
+/// `response`. Omits the header on error responses and when this node has no
+/// live consensus: a missing header is unambiguous, whereas a fabricated view
+/// number would mislead.
fn insert_view_header(state: &HttpInner, mut response: Response) -> Response {
+ // The view is cluster-internal; error responses (notably pre-auth 401s)
+ // must not leak it. Success and the 307 primary-redirect still carry it.
+ if !(response.status().is_success() || response.status().is_redirection()) {
+ return response;
+ }
if let Some(consensus) = state.shard.plane.metadata().consensus.as_ref() {
response
.headers_mut()
(iggy) 25/43: test(server-ng): add RBAC permission-matrix integration gate
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit fb89a35f889c8502495a551c9e352c61c7794513
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 16:44:37 2026 +0200
test(server-ng): add RBAC permission-matrix integration gate
Per-op RBAC now spans the apply path and dispatch time, replacing the
interim root-only gates. Prove the whole surface end-to-end with a
persona matrix over HTTP plus a TCP typed-deny case: ungranted user,
stream-scoped grant, global-bits-only grant (exercising the
all-streams indexes), revocation and user-deletion visibility,
non-root user management with the root guards, PAT identity binding,
and the SDK surfacing a dispatch-time denial through the reply status
field. Every deny is paired with a matching positive control so the
suite fails on an allow-all server and on a deny-all server alike.
---
core/integration/tests/server/http_rbac.rs | 916 +++++++++++++++++++++++++++++
core/integration/tests/server/mod.rs | 3 +
2 files changed, 919 insertions(+)
diff --git a/core/integration/tests/server/http_rbac.rs b/core/integration/tests/server/http_rbac.rs
new file mode 100644
index 000000000..7fcd89f89
--- /dev/null
+++ b/core/integration/tests/server/http_rbac.rs
@@ -0,0 +1,916 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! End-to-end RBAC matrix for server-ng, exercised over raw `reqwest` against
+//! the shard-0 HTTP listener plus one SDK/TCP case. Proves the whole
+//! authorization surface a client can reach:
+//!
+//! - dispatch-time per-op gates on reads (`get_streams`/`get_stream`/... ) and
+//! the data plane (produce, poll, consumer offsets) -> HTTP 403;
+//! - the in-apply control gate whose `Unauthorized` denial rides the metadata
+//! result section back to an HTTP 4xx with a typed error body;
+//! - business-rule rejections (`CannotDeleteUser`/`CannotChangePermissions`)
+//! that fire only AFTER the RBAC gate admits the acting user;
+//! - the denormalized "all-streams" indexes populated from a user's GLOBAL
+//! `poll_messages`/`send_messages` bits (the trap case);
+//! - live revocation visibility (re-permission + user delete) on a single node;
+//! - self-scoped PAT ops that bypass RBAC, and PAT-bound identity;
+//! - the TCP typed-`Unauthorized` deny surfaced through the SDK.
+//!
+//! Personas are provisioned by root over HTTP; each test owns its own server
+//! (one spawn per `#[iggy_harness]` fn) so the mutating cases stay isolated.
+
+use iggy::prelude::*;
+use iggy_common::create_stream::CreateStream;
+use iggy_common::create_topic::CreateTopic;
+use iggy_common::store_consumer_offset::StoreConsumerOffset;
+use iggy_common::{GlobalPermissions, IggyMessagesBatch, Permissions, StreamPermissions};
+use integration::harness::TestHarness;
+use integration::iggy_harness;
+use reqwest::{Response, StatusCode};
+use serde_json::{Value, json};
+use std::collections::BTreeMap;
+use std::time::{Duration, Instant};
+use tokio::time::sleep;
+
+// server-ng partition ids are 0-based (CreateTopic assigns them from 0).
+const PARTITION_ID: u32 = 0;
+// Explicit consumer id shared by the offset body and query so both address the
+// same offset-table key (`Consumer::default()` carries numeric id 0, not 1).
+const CONSUMER_ID: u32 = 1;
+
+// `IggyError` discriminants (== the numeric `id` in the JSON error body).
+const UNAUTHORIZED_ID: u64 = 41;
+const CANNOT_DELETE_USER_ID: u64 = 48;
+const CANNOT_CHANGE_PERMISSIONS_ID: u64 = 49;
+// Snake-case `IggyError` names (== the `code` string in the JSON error body).
+const UNAUTHORIZED_CODE: &str = "unauthorized";
+const CANNOT_DELETE_USER_CODE: &str = "cannot_delete_user";
+const CANNOT_CHANGE_PERMISSIONS_CODE: &str = "cannot_change_permissions";
+
+/// Root is the first user (slab id 0) and is undeletable / permission-locked.
+const ROOT_USER_PATH: &str = "0";
+
+/// The harness readiness gate waits for the dumped runtime config, which the
+/// server writes before binding the HTTP listener; a bounded login retry
+/// bridges that gap plus single-node consensus warmup.
+const LOGIN_TIMEOUT: Duration = Duration::from_secs(15);
+const LOGIN_RETRY_INTERVAL: Duration = Duration::from_millis(50);
+const REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// One authenticated HTTP session: a `reqwest` client plus the bearer to send.
+/// The bearer is either a JWT (from a login) or a raw personal access token
+/// (server-ng resolves either on the `Authorization: Bearer` header).
+struct Client {
+ http: reqwest::Client,
+ base_url: String,
+ token: String,
+}
+
+impl Client {
+ /// Log in as root, retrying while the listener binds and consensus warms up.
+ async fn login_root(harness: &TestHarness) -> Self {
+ let addr = harness
+ .server()
+ .http_addr()
+ .expect("HTTP transport not configured on test server");
+ let base_url = format!("http://{addr}");
+ let http = reqwest::Client::builder()
+ .timeout(REQUEST_TIMEOUT)
+ .build()
+ .expect("build reqwest client");
+
+ let body = json!({
+ "username": DEFAULT_ROOT_USERNAME,
+ "password": DEFAULT_ROOT_PASSWORD,
+ });
+ let deadline = Instant::now() + LOGIN_TIMEOUT;
+ loop {
+ let attempt = http
+ .post(format!("{base_url}/users/login"))
+ .json(&body)
+ .send()
+ .await;
+ match attempt {
+ Ok(response) if response.status().is_success() => {
+ return Self {
+ http,
+ base_url,
+ token: access_token(response).await,
+ };
+ }
+ // Listener not bound yet or consensus still warming up; any
+ // non-5xx rejection (e.g. 401) is a real failure.
+ Ok(response) if response.status().is_server_error() => {}
+ Ok(response) => panic!("root login rejected: {}", response.status()),
+ Err(error) if error.is_connect() => {}
+ Err(error) => panic!("root login request failed: {error}"),
+ }
+ assert!(
+ Instant::now() < deadline,
+ "HTTP root login did not succeed within {LOGIN_TIMEOUT:?}"
+ );
+ sleep(LOGIN_RETRY_INTERVAL).await;
+ }
+ }
+
+ /// Log in as an already-created user; single attempt (the server is warm
+ /// once root exists). Reuses this session's client and base URL.
+ async fn login(&self, username: &str, password: &str) -> Self {
+ let body = json!({ "username": username, "password": password });
+ let response = self
+ .http
+ .post(self.url("/users/login"))
+ .json(&body)
+ .send()
+ .await
+ .expect("login request");
+ assert!(
+ response.status().is_success(),
+ "login for {username} failed: {}",
+ response.status()
+ );
+ Self {
+ http: self.http.clone(),
+ base_url: self.base_url.clone(),
+ token: access_token(response).await,
+ }
+ }
+
+ /// A sibling session authenticating with an arbitrary bearer (a raw PAT),
+ /// reusing this session's client and base URL.
+ fn with_bearer(&self, token: String) -> Self {
+ Self {
+ http: self.http.clone(),
+ base_url: self.base_url.clone(),
+ token,
+ }
+ }
+
+ fn url(&self, path: &str) -> String {
+ format!("{}{path}", self.base_url)
+ }
+
+ async fn get(&self, path: &str) -> Response {
+ self.http
+ .get(self.url(path))
+ .bearer_auth(&self.token)
+ .send()
+ .await
+ .expect("get request")
+ }
+
+ /// Unauthenticated GET (no bearer) for the public `/ping` probe.
+ async fn get_anonymous(&self, path: &str) -> Response {
+ self.http
+ .get(self.url(path))
+ .send()
+ .await
+ .expect("anonymous get request")
+ }
+
+ async fn post_json(&self, path: &str, body: &Value) -> Response {
+ self.http
+ .post(self.url(path))
+ .bearer_auth(&self.token)
+ .json(body)
+ .send()
+ .await
+ .expect("post request")
+ }
+
+ async fn put_json(&self, path: &str, body: &Value) -> Response {
+ self.http
+ .put(self.url(path))
+ .bearer_auth(&self.token)
+ .json(body)
+ .send()
+ .await
+ .expect("put request")
+ }
+
+ async fn delete(&self, path: &str) -> Response {
+ self.http
+ .delete(self.url(path))
+ .bearer_auth(&self.token)
+ .send()
+ .await
+ .expect("delete request")
+ }
+
+ /// Create a stream and return its server-assigned numeric id, which is both
+ /// the id clients address and the key the permissioner uses for per-stream
+ /// grants (`resolve_stream_id` maps name -> that same id).
+ async fn create_stream(&self, name: &str) -> u64 {
+ let response = self
+ .post_json(
+ "/streams",
+ &serde_json::to_value(CreateStream { name: name.into() }).unwrap(),
+ )
+ .await;
+ assert!(
+ response.status().is_success(),
+ "root create stream {name} failed: {}",
+ response.status()
+ );
+ let details: Value = response.json().await.expect("stream details json");
+ details["id"]
+ .as_u64()
+ .expect("stream id in create response")
+ }
+
+ async fn create_topic(&self, stream: &str, topic: &str, partitions: u32) {
+ let command = CreateTopic {
+ stream_id: Identifier::default(),
+ partitions_count: partitions,
+ compression_algorithm: CompressionAlgorithm::None,
+ message_expiry: IggyExpiry::NeverExpire,
+ max_topic_size: MaxTopicSize::ServerDefault,
+ replication_factor: None,
+ name: topic.to_string(),
+ };
+ let response = self
+ .post_json(
+ &format!("/streams/{stream}/topics"),
+ &serde_json::to_value(command).unwrap(),
+ )
+ .await;
+ assert!(
+ response.status().is_success(),
+ "root create topic {topic} failed: {}",
+ response.status()
+ );
+ }
+
+ /// Create a user with the given permissions (`Value::Null` = no grants).
+ async fn create_user(&self, username: &str, password: &str, permissions: Value) -> Response {
+ self.post_json(
+ "/users",
+ &json!({
+ "username": username,
+ "password": password,
+ "status": "active",
+ "permissions": permissions,
+ }),
+ )
+ .await
+ }
+
+ async fn produce(
+ &self,
+ stream: &str,
+ topic: &str,
+ partition_id: u32,
+ messages: Vec<IggyMessage>,
+ ) -> Response {
+ let body = SendMessages {
+ metadata_length: 0,
+ stream_id: Identifier::default(),
+ topic_id: Identifier::default(),
+ partitioning: Partitioning::partition_id(partition_id),
+ batch: IggyMessagesBatch::from(&messages),
+ };
+ self.http
+ .post(self.url(&format!("/streams/{stream}/topics/{topic}/messages")))
+ .bearer_auth(&self.token)
+ .json(&body)
+ .send()
+ .await
+ .expect("produce request")
+ }
+
+ async fn poll(&self, stream: &str, topic: &str, partition_id: u32) -> Response {
+ let path = format!(
+ "/streams/{stream}/topics/{topic}/messages\
+ ?consumer_id={CONSUMER_ID}&partition_id={partition_id}\
+ &kind=offset&value=0&count=10&auto_commit=false"
+ );
+ self.get(&path).await
+ }
+
+ async fn store_offset(&self, stream: &str, topic: &str) -> Response {
+ let store = StoreConsumerOffset {
+ consumer: Consumer::new(Identifier::numeric(CONSUMER_ID).expect("consumer id")),
+ partition_id: Some(PARTITION_ID),
+ offset: 0,
+ };
+ self.put_json(
+ &format!("/streams/{stream}/topics/{topic}/consumer-offsets"),
+ &serde_json::to_value(store).unwrap(),
+ )
+ .await
+ }
+
+ async fn get_offset(&self, stream: &str, topic: &str) -> Response {
+ self.get(&format!(
+ "/streams/{stream}/topics/{topic}/consumer-offsets\
+ ?consumer_id={CONSUMER_ID}&partition_id={PARTITION_ID}"
+ ))
+ .await
+ }
+
+ async fn delete_offset(&self, stream: &str, topic: &str) -> Response {
+ self.delete(&format!(
+ "/streams/{stream}/topics/{topic}/consumer-offsets/{CONSUMER_ID}\
+ ?partition_id={PARTITION_ID}"
+ ))
+ .await
+ }
+}
+
+/// Extract the JWT from a successful login / PAT-login response.
+async fn access_token(response: Response) -> String {
+ let identity: IdentityInfo = response.json().await.expect("decode IdentityInfo");
+ identity
+ .access_token
+ .expect("login must return an access token")
+ .token
+}
+
+fn text_message(id: u128, payload: &str) -> IggyMessage {
+ IggyMessage::builder()
+ .id(id)
+ .payload(payload.to_string().into())
+ .build()
+ .expect("message build")
+}
+
+// --- permission-body builders (the legacy-parity common types) --------------
+
+/// No permissions at all: the permissioner holds no entry, so every rule denies.
+fn no_permissions() -> Value {
+ Value::Null
+}
+
+/// Global-only grants (no per-stream map).
+fn global_permissions(global: GlobalPermissions) -> Value {
+ serde_json::to_value(Permissions {
+ global,
+ streams: None,
+ })
+ .expect("serialize permissions")
+}
+
+/// A single per-stream grant keyed by the stream's numeric id, no global grants.
+fn stream_scoped_permissions(stream_id: u64, stream: StreamPermissions) -> Value {
+ serde_json::to_value(Permissions {
+ global: GlobalPermissions::default(),
+ streams: Some(BTreeMap::from([(stream_id as usize, stream)])),
+ })
+ .expect("serialize permissions")
+}
+
+/// Streams S1/S2 (each with one topic) provisioned by root for every test.
+struct Fixture {
+ s1: &'static str,
+ s2: &'static str,
+ // S1's id keys bob's per-stream grant; S2 is only ever addressed by name.
+ s1_id: u64,
+ topic: &'static str,
+}
+
+/// Root login + the shared S1/S2 + topic fixture. Returns the root session and
+/// S1's id for keying per-stream grants.
+async fn setup(harness: &TestHarness) -> (Client, Fixture) {
+ let root = Client::login_root(harness).await;
+ let (s1, s2, topic) = ("rbac-s1", "rbac-s2", "rbac-t");
+ let s1_id = root.create_stream(s1).await;
+ root.create_topic(s1, topic, 1).await;
+ root.create_stream(s2).await;
+ root.create_topic(s2, topic, 1).await;
+ (
+ root,
+ Fixture {
+ s1,
+ s2,
+ s1_id,
+ topic,
+ },
+ )
+}
+
+/// Assert a JSON error body carries the expected typed `id` + `code`. Proves the
+/// error crossed the wire as a real `IggyError` code, not a bare status.
+async fn assert_error_body(response: Response, expected_id: u64, expected_code: &str, ctx: &str) {
+ let body: Value = response.json().await.expect("error body json");
+ assert_eq!(
+ body["id"].as_u64(),
+ Some(expected_id),
+ "{ctx}: error id (body={body})"
+ );
+ assert_eq!(
+ body["code"].as_str(),
+ Some(expected_code),
+ "{ctx}: error code (body={body})"
+ );
+}
+
+// === Case family 1: alice, NO permissions ===================================
+
+#[iggy_harness]
+async fn given_ungranted_user_when_touching_any_surface_should_deny_except_self_scoped(
+ harness: &TestHarness,
+) {
+ let (root, fx) = setup(harness).await;
+ root.create_user("alice", "alice-pass", no_permissions())
+ .await;
+ let alice = root.login("alice", "alice-pass").await;
+
+ // Reads: every metadata read is 403 for the ungranted user.
+ let reads = [
+ ("GET /streams", "/streams".to_string()),
+ ("GET /streams/{s}", format!("/streams/{}", fx.s1)),
+ (
+ "GET /streams/{s}/topics/{t}",
+ format!("/streams/{}/topics/{}", fx.s1, fx.topic),
+ ),
+ ("GET /users", "/users".to_string()),
+ ("GET /stats", "/stats".to_string()),
+ ("GET /clients", "/clients".to_string()),
+ ];
+ for (label, path) in &reads {
+ assert_eq!(
+ alice.get(path).await.status(),
+ StatusCode::FORBIDDEN,
+ "alice {label} must be 403"
+ );
+ }
+
+ // Data plane: produce, poll, and all three consumer-offset ops are 403.
+ assert_eq!(
+ alice
+ .produce(fx.s1, fx.topic, PARTITION_ID, vec![text_message(1, "x")])
+ .await
+ .status(),
+ StatusCode::FORBIDDEN,
+ "alice produce must be 403"
+ );
+ assert_eq!(
+ alice.poll(fx.s1, fx.topic, PARTITION_ID).await.status(),
+ StatusCode::FORBIDDEN,
+ "alice poll must be 403"
+ );
+ assert_eq!(
+ alice.store_offset(fx.s1, fx.topic).await.status(),
+ StatusCode::FORBIDDEN,
+ "alice store-offset must be 403"
+ );
+ assert_eq!(
+ alice.get_offset(fx.s1, fx.topic).await.status(),
+ StatusCode::FORBIDDEN,
+ "alice get-offset must be 403"
+ );
+ assert_eq!(
+ alice.delete_offset(fx.s1, fx.topic).await.status(),
+ StatusCode::FORBIDDEN,
+ "alice delete-offset must be 403"
+ );
+
+ // Control write via the in-apply gate: the denial rides the result section
+ // back as a typed `Unauthorized` body under a 403.
+ let denied = alice
+ .post_json("/streams", &json!({ "name": "alice-stream" }))
+ .await;
+ assert_eq!(
+ denied.status(),
+ StatusCode::FORBIDDEN,
+ "alice create-stream must be 403"
+ );
+ assert_error_body(
+ denied,
+ UNAUTHORIZED_ID,
+ UNAUTHORIZED_CODE,
+ "alice create-stream",
+ )
+ .await;
+
+ // Self-scoped ops bypass RBAC: alice manages her own PAT and reads herself.
+ let pat = alice
+ .post_json(
+ "/personal-access-tokens",
+ &json!({ "name": "alice-pat", "expiry": 0 }),
+ )
+ .await;
+ assert_eq!(
+ pat.status(),
+ StatusCode::OK,
+ "alice create own PAT must be 200"
+ );
+ assert_eq!(
+ alice
+ .delete("/personal-access-tokens/alice-pat")
+ .await
+ .status(),
+ StatusCode::NO_CONTENT,
+ "alice delete own PAT must be 204"
+ );
+ assert_eq!(
+ alice.get("/users/me").await.status(),
+ StatusCode::OK,
+ "alice GET /users/me must be 200"
+ );
+
+ // Auth-only cluster metadata (no rule) and the public ping.
+ assert_eq!(
+ alice.get("/cluster/metadata").await.status(),
+ StatusCode::OK,
+ "cluster metadata is auth-only, never RBAC-gated"
+ );
+ assert_eq!(
+ alice.get_anonymous("/ping").await.status(),
+ StatusCode::OK,
+ "ping is public"
+ );
+}
+
+// === Case family 2: bob, per-stream grant on S1 only ========================
+
+#[iggy_harness]
+async fn given_stream_scoped_user_when_crossing_the_stream_boundary_should_deny(
+ harness: &TestHarness,
+) {
+ let (root, fx) = setup(harness).await;
+ // read_stream + poll_messages on S1 only; no send, no global read_streams.
+ let permissions = stream_scoped_permissions(
+ fx.s1_id,
+ StreamPermissions {
+ read_stream: true,
+ poll_messages: true,
+ ..Default::default()
+ },
+ );
+ root.create_user("bob", "bob-pass", permissions).await;
+ let bob = root.login("bob", "bob-pass").await;
+
+ assert_eq!(
+ bob.poll(fx.s1, fx.topic, PARTITION_ID).await.status(),
+ StatusCode::OK,
+ "bob poll on granted S1 must be 200"
+ );
+ assert_eq!(
+ bob.poll(fx.s2, fx.topic, PARTITION_ID).await.status(),
+ StatusCode::FORBIDDEN,
+ "bob poll on ungranted S2 must be 403 (cross-stream deny)"
+ );
+ assert_eq!(
+ bob.produce(fx.s1, fx.topic, PARTITION_ID, vec![text_message(1, "x")])
+ .await
+ .status(),
+ StatusCode::FORBIDDEN,
+ "bob produce on S1 must be 403 (read grant does not imply send)"
+ );
+ assert_eq!(
+ bob.get(&format!("/streams/{}", fx.s1)).await.status(),
+ StatusCode::OK,
+ "bob GET single granted stream S1 must be 200"
+ );
+ assert_eq!(
+ bob.get("/streams").await.status(),
+ StatusCode::FORBIDDEN,
+ "bob GET /streams must be 403 (no global read_streams)"
+ );
+}
+
+// === Case family 3: carol, GLOBAL message bits, ZERO per-stream (the trap) ===
+
+#[iggy_harness]
+async fn given_global_message_bits_without_stream_entries_when_polling_or_producing_should_allow(
+ harness: &TestHarness,
+) {
+ let (root, fx) = setup(harness).await;
+ // Global poll+send only; the all-streams indexes are the ONLY thing that can
+ // admit her (her global bits set no read_streams/manage_*).
+ let permissions = global_permissions(GlobalPermissions {
+ poll_messages: true,
+ send_messages: true,
+ ..Default::default()
+ });
+ root.create_user("carol", "carol-pass", permissions).await;
+ let carol = root.login("carol", "carol-pass").await;
+
+ for (label, stream) in [("S1", fx.s1), ("S2", fx.s2)] {
+ assert_eq!(
+ carol
+ .produce(stream, fx.topic, PARTITION_ID, vec![text_message(1, "x")])
+ .await
+ .status(),
+ StatusCode::CREATED,
+ "carol produce on {label} must be 201 (global send_messages index)"
+ );
+ assert_eq!(
+ carol.poll(stream, fx.topic, PARTITION_ID).await.status(),
+ StatusCode::OK,
+ "carol poll on {label} must be 200 (global poll_messages index)"
+ );
+ }
+}
+
+// === Case family 4: revocation takes effect immediately (single node) =======
+
+#[iggy_harness]
+async fn given_permission_revocation_when_reapplied_should_deny_next_request(
+ harness: &TestHarness,
+) {
+ let (root, fx) = setup(harness).await;
+
+ // carol: global poll+send. bob: global read_streams+poll. Both start allowed.
+ root.create_user(
+ "carol",
+ "carol-pass",
+ global_permissions(GlobalPermissions {
+ poll_messages: true,
+ send_messages: true,
+ ..Default::default()
+ }),
+ )
+ .await;
+ root.create_user(
+ "bob",
+ "bob-pass",
+ global_permissions(GlobalPermissions {
+ read_streams: true,
+ poll_messages: true,
+ ..Default::default()
+ }),
+ )
+ .await;
+ let carol = root.login("carol", "carol-pass").await;
+ let bob = root.login("bob", "bob-pass").await;
+
+ assert_eq!(
+ carol
+ .produce(fx.s1, fx.topic, PARTITION_ID, vec![text_message(1, "x")])
+ .await
+ .status(),
+ StatusCode::CREATED,
+ "carol produce must be 201 before revocation"
+ );
+ assert_eq!(
+ bob.get("/streams").await.status(),
+ StatusCode::OK,
+ "bob read must be 200 before deletion"
+ );
+
+ // Strip carol's send (keep poll): her all-streams send index entry clears.
+ let reperm = root
+ .put_json(
+ "/users/carol/permissions",
+ &json!({
+ "permissions": global_permissions(GlobalPermissions {
+ poll_messages: true,
+ send_messages: false,
+ ..Default::default()
+ })
+ }),
+ )
+ .await;
+ assert_eq!(
+ reperm.status(),
+ StatusCode::NO_CONTENT,
+ "root re-permission must be 204"
+ );
+ assert_eq!(
+ carol
+ .produce(fx.s1, fx.topic, PARTITION_ID, vec![text_message(2, "y")])
+ .await
+ .status(),
+ StatusCode::FORBIDDEN,
+ "carol produce must be 403 after send stripped"
+ );
+ assert_eq!(
+ carol.poll(fx.s1, fx.topic, PARTITION_ID).await.status(),
+ StatusCode::OK,
+ "carol poll must still be 200 (poll grant retained)"
+ );
+
+ // Delete bob entirely: his permissioner indexes clear though his JWT is live.
+ assert_eq!(
+ root.delete("/users/bob").await.status(),
+ StatusCode::NO_CONTENT,
+ "root delete bob must be 204"
+ );
+ assert_eq!(
+ bob.get("/streams").await.status(),
+ StatusCode::FORBIDDEN,
+ "deleted bob read must be 403 (indexes cleared)"
+ );
+ assert_eq!(
+ bob.poll(fx.s1, fx.topic, PARTITION_ID).await.status(),
+ StatusCode::FORBIDDEN,
+ "deleted bob poll must be 403 (indexes cleared)"
+ );
+}
+
+// === Case family 5: dave, global manage_users admin (non-root) ==============
+
+#[iggy_harness]
+async fn given_non_root_manage_users_admin_when_managing_should_honor_business_rules(
+ harness: &TestHarness,
+) {
+ let (root, _fx) = setup(harness).await;
+ root.create_user(
+ "dave",
+ "dave-pass",
+ global_permissions(GlobalPermissions {
+ manage_users: true,
+ ..Default::default()
+ }),
+ )
+ .await;
+ let dave = root.login("dave", "dave-pass").await;
+
+ // F8: a non-root admin may create users (in-apply gate admits manage_users).
+ let created = dave
+ .create_user("dave-made", "made-pass", no_permissions())
+ .await;
+ assert_eq!(
+ created.status(),
+ StatusCode::OK,
+ "dave create user must be 200 (non-root manage_users admin)"
+ );
+
+ // RBAC admits dave, then the business invariant rejects touching root. The
+ // typed code rides the body; the status is whatever the IggyError map yields
+ // (400 for these, not 403).
+ let delete_root = dave.delete(&format!("/users/{ROOT_USER_PATH}")).await;
+ assert_eq!(
+ delete_root.status(),
+ StatusCode::BAD_REQUEST,
+ "dave delete root status"
+ );
+ assert_error_body(
+ delete_root,
+ CANNOT_DELETE_USER_ID,
+ CANNOT_DELETE_USER_CODE,
+ "dave delete root",
+ )
+ .await;
+
+ let change_root = dave
+ .put_json(
+ &format!("/users/{ROOT_USER_PATH}/permissions"),
+ &json!({ "permissions": no_permissions() }),
+ )
+ .await;
+ assert_eq!(
+ change_root.status(),
+ StatusCode::BAD_REQUEST,
+ "dave change root permissions status"
+ );
+ assert_error_body(
+ change_root,
+ CANNOT_CHANGE_PERMISSIONS_ID,
+ CANNOT_CHANGE_PERMISSIONS_CODE,
+ "dave change root permissions",
+ )
+ .await;
+
+ // dave holds no stream grant: creating a stream is denied by the in-apply gate.
+ let denied = dave
+ .post_json("/streams", &json!({ "name": "dave-stream" }))
+ .await;
+ assert_eq!(
+ denied.status(),
+ StatusCode::FORBIDDEN,
+ "dave create stream must be 403 (no manage_streams)"
+ );
+ assert_error_body(
+ denied,
+ UNAUTHORIZED_ID,
+ UNAUTHORIZED_CODE,
+ "dave create stream",
+ )
+ .await;
+}
+
+// === Case family 6: root full-surface canary ================================
+
+#[iggy_harness]
+async fn given_root_when_exercising_the_full_surface_should_allow(harness: &TestHarness) {
+ let root = Client::login_root(harness).await;
+ let root_id = root.create_stream("root-canary").await;
+ assert!(root_id < u64::MAX, "root create stream must return an id");
+ root.create_topic("root-canary", "canary-topic", 1).await;
+
+ assert_eq!(
+ root.produce(
+ "root-canary",
+ "canary-topic",
+ PARTITION_ID,
+ vec![text_message(1, "root-msg")],
+ )
+ .await
+ .status(),
+ StatusCode::CREATED,
+ "root produce must be 201"
+ );
+ assert_eq!(
+ root.poll("root-canary", "canary-topic", PARTITION_ID)
+ .await
+ .status(),
+ StatusCode::OK,
+ "root poll must be 200"
+ );
+ assert_eq!(
+ root.get("/stats").await.status(),
+ StatusCode::OK,
+ "root GET /stats must be 200"
+ );
+}
+
+// === Case family 7: PAT-bound identity ======================================
+
+#[iggy_harness]
+async fn given_personal_access_token_when_authenticating_should_bind_owner_rules(
+ harness: &TestHarness,
+) {
+ let (root, fx) = setup(harness).await;
+ root.create_user("alice", "alice-pass", no_permissions())
+ .await;
+ let alice = root.login("alice", "alice-pass").await;
+
+ // alice mints a PAT (self-scoped) and authenticates WITH it (raw PAT bearer).
+ let pat_response = alice
+ .post_json(
+ "/personal-access-tokens",
+ &json!({ "name": "alice-id", "expiry": 0 }),
+ )
+ .await;
+ assert_eq!(
+ pat_response.status(),
+ StatusCode::OK,
+ "alice create PAT must be 200"
+ );
+ let raw: Value = pat_response.json().await.expect("PAT json");
+ let token = raw["token"].as_str().expect("raw PAT token").to_string();
+ let via_pat = alice.with_bearer(token);
+
+ // The PAT resolves to alice's user id, so it inherits her (empty) rules: the
+ // same denials she gets via her JWT.
+ assert_eq!(
+ via_pat.get("/streams").await.status(),
+ StatusCode::FORBIDDEN,
+ "PAT-authenticated read must be 403 (bound to alice's rules)"
+ );
+ assert_eq!(
+ via_pat
+ .produce(fx.s1, fx.topic, PARTITION_ID, vec![text_message(1, "x")])
+ .await
+ .status(),
+ StatusCode::FORBIDDEN,
+ "PAT-authenticated produce must be 403 (bound to alice's rules)"
+ );
+}
+
+// === Case family 8: TCP typed Unauthorized through the SDK ==================
+
+#[iggy_harness(server(tcp.socket.override_defaults = true, tcp.socket.nodelay = true))]
+async fn given_non_root_tcp_session_when_polling_denied_should_surface_typed_unauthorized(
+ harness: &TestHarness,
+) {
+ // Provision alice + the stream over HTTP-as-root (same single-node metadata
+ // STM the TCP dispatch reads).
+ let (root, fx) = setup(harness).await;
+ root.create_user("alice", "alice-pass", no_permissions())
+ .await;
+
+ // A fresh TCP SDK client, logged in as the non-root alice.
+ let client = harness.tcp_new_client().await.expect("tcp client");
+ client
+ .login_user("alice", "alice-pass")
+ .await
+ .expect("alice tcp login");
+
+ let stream_id = Identifier::from_str_value(fx.s1).expect("stream identifier");
+ let topic_id = Identifier::from_str_value(fx.topic).expect("topic identifier");
+ let result = client
+ .poll_messages(
+ &stream_id,
+ &topic_id,
+ Some(PARTITION_ID),
+ &Consumer::default(),
+ &PollingStrategy::offset(0),
+ 1,
+ false,
+ )
+ .await;
+
+ assert!(
+ matches!(&result, Err(error) if error.as_code() == IggyError::Unauthorized.as_code()),
+ "non-root TCP poll must surface Err(Unauthorized) via the reply status peek, got {result:?}"
+ );
+}
diff --git a/core/integration/tests/server/mod.rs b/core/integration/tests/server/mod.rs
index b2f1d8351..eeb536de9 100644
--- a/core/integration/tests/server/mod.rs
+++ b/core/integration/tests/server/mod.rs
@@ -27,6 +27,9 @@ mod cg_vsr;
// Raw-HTTP data-plane contract against server-ng's shard-0 listener.
#[cfg(feature = "vsr")]
mod http_vsr;
+// End-to-end RBAC permission matrix against server-ng (HTTP + one TCP case).
+#[cfg(feature = "vsr")]
+mod http_rbac;
// 80-case race matrix with hardcoded HTTP variants (test_matrix bypasses
// the harness transport filter); revisit under vsr once basics are green.
#[cfg(not(feature = "vsr"))]
(iggy) 36/43: fix(metadata): reject consumer-group create on a missing parent
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 79434c29dd43b3c6ffc549d46b233bafb40d6695
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 22:05:06 2026 +0200
fix(metadata): reject consumer-group create on a missing parent
CreateConsumerGroup's apply committed a silent ok-empty no-op when the
parent stream or topic did not exist, so native TCP reported success
and HTTP surfaced a misleading 400 from the empty reply body. Resolve
the parent level by level and commit the typed StreamNotFound /
TopicNotFound rejection instead, mirroring CreateTopic. The simulator
workload now also targets both not-found outcomes.
---
core/metadata/src/stm/consumer_group.rs | 52 +++++++++++++++++++++-
core/metadata/src/stm/result.rs | 14 +++++-
.../src/workload/ops/create_consumer_group.rs | 29 ++++++++++--
3 files changed, 88 insertions(+), 7 deletions(-)
diff --git a/core/metadata/src/stm/consumer_group.rs b/core/metadata/src/stm/consumer_group.rs
index b05afb873..fa9e07b48 100644
--- a/core/metadata/src/stm/consumer_group.rs
+++ b/core/metadata/src/stm/consumer_group.rs
@@ -505,8 +505,21 @@ impl StateHandler for CreateConsumerGroupRequest {
state: &mut StreamsInner,
_timestamp: iggy_common::IggyTimestamp,
) -> ApplyReply {
- let Some(topic) = state.topic_mut(&self.stream_id, &self.topic_id) else {
- return ApplyReply::ok(Bytes::new());
+ // A missing parent is a committed rejection, mirroring `CreateTopic`
+ // on a missing stream. Resolve level by level so the error names the
+ // level that missed.
+ let Some(stream_id) = state.resolve_stream_id(&self.stream_id) else {
+ return ApplyReply::err(CreateConsumerGroupResult::StreamNotFound);
+ };
+ let Some(topic_id) = state.resolve_topic_id(stream_id, &self.topic_id) else {
+ return ApplyReply::err(CreateConsumerGroupResult::TopicNotFound);
+ };
+ let Some(topic) = state
+ .items
+ .get_mut(stream_id)
+ .and_then(|stream| stream.topics.get_mut(topic_id))
+ else {
+ return ApplyReply::err(CreateConsumerGroupResult::TopicNotFound);
};
let name: Arc<str> = Arc::from(self.name.as_str());
// Per-(stream,topic) name uniqueness. A same-name group already exists:
@@ -957,6 +970,41 @@ mod tests {
assert!(apply.body.is_empty());
}
+ #[test]
+ fn given_missing_parent_when_apply_create_consumer_group_should_return_not_found() {
+ let mut state = streams_with_topic();
+
+ let missing_stream = StateHandler::apply(
+ &CreateConsumerGroupRequest {
+ stream_id: WireIdentifier::numeric(999),
+ topic_id: WireIdentifier::numeric(0),
+ name: WireName::new("group").unwrap(),
+ },
+ &mut state,
+ IggyTimestamp::now(),
+ );
+ assert_eq!(
+ missing_stream.code,
+ u32::from(CreateConsumerGroupResult::StreamNotFound)
+ );
+ assert!(missing_stream.body.is_empty());
+
+ let missing_topic = StateHandler::apply(
+ &CreateConsumerGroupRequest {
+ stream_id: WireIdentifier::numeric(0),
+ topic_id: WireIdentifier::numeric(999),
+ name: WireName::new("group").unwrap(),
+ },
+ &mut state,
+ IggyTimestamp::now(),
+ );
+ assert_eq!(
+ missing_topic.code,
+ u32::from(CreateConsumerGroupResult::TopicNotFound)
+ );
+ assert!(missing_topic.body.is_empty());
+ }
+
#[test]
fn given_missing_group_when_apply_delete_consumer_group_should_return_not_found() {
let mut state = streams_with_topic();
diff --git a/core/metadata/src/stm/result.rs b/core/metadata/src/stm/result.rs
index 9a3eef484..979c18bc2 100644
--- a/core/metadata/src/stm/result.rs
+++ b/core/metadata/src/stm/result.rs
@@ -210,7 +210,11 @@ result_enum!(CreatePersonalAccessTokenResult {
result_enum!(DeletePersonalAccessTokenResult { NotFound = 20 });
// Consumer groups.
-result_enum!(CreateConsumerGroupResult { NameAlreadyExists = 5004 });
+result_enum!(CreateConsumerGroupResult {
+ StreamNotFound = 1009,
+ TopicNotFound = 2010,
+ NameAlreadyExists = 5004,
+});
result_enum!(DeleteConsumerGroupResult { NotFound = 5000 });
/// `IggyError::Unauthorized`. Any control-plane op can commit as an in-apply
@@ -408,6 +412,10 @@ mod tests {
u32::from(DeletePartitionsResult::StreamNotFound),
stream_not_found
);
+ assert_eq!(
+ u32::from(CreateConsumerGroupResult::StreamNotFound),
+ stream_not_found
+ );
// StreamNameAlreadyExists (1012).
let stream_name_exists = IggyError::StreamNameAlreadyExists(String::new()).as_code();
@@ -433,6 +441,10 @@ mod tests {
u32::from(DeletePartitionsResult::TopicNotFound),
topic_not_found
);
+ assert_eq!(
+ u32::from(CreateConsumerGroupResult::TopicNotFound),
+ topic_not_found
+ );
// TopicNameAlreadyExists (2013).
let topic_name_exists = IggyError::TopicNameAlreadyExists(String::new(), id()).as_code();
diff --git a/core/simulator/src/workload/ops/create_consumer_group.rs b/core/simulator/src/workload/ops/create_consumer_group.rs
index d8d6b9e5f..88c66a4fb 100644
--- a/core/simulator/src/workload/ops/create_consumer_group.rs
+++ b/core/simulator/src/workload/ops/create_consumer_group.rs
@@ -15,8 +15,11 @@
// specific language governing permissions and limitations
// under the License.
-//! `CreateConsumerGroup` op. Targets `Ok` (fresh group under a live
-//! stream/topic) or `NameAlreadyExists` (an existing group name).
+//! `CreateConsumerGroup` op.
+//!
+//! Targets `Ok` (fresh group under a live stream/topic), `StreamNotFound`
+//! (a fabricated parent stream), `TopicNotFound` (a live stream with a
+//! fabricated topic), or `NameAlreadyExists` (an existing group name).
use iggy_binary_protocol::RequestHeader;
use rand_xoshiro::Xoshiro256Plus;
@@ -36,7 +39,12 @@ pub struct Input {
pub name: String,
}
-pub const OUTCOMES: &[Outcome] = &[Outcome::Ok, Outcome::NameAlreadyExists];
+pub const OUTCOMES: &[Outcome] = &[
+ Outcome::Ok,
+ Outcome::StreamNotFound,
+ Outcome::TopicNotFound,
+ Outcome::NameAlreadyExists,
+];
pub fn sample(
shadow: &mut Shadow,
@@ -54,6 +62,19 @@ pub fn sample(
name,
})
}
+ Outcome::StreamNotFound => Some(Input {
+ stream: shadow.fabricate_absent_name("stream"),
+ topic: shadow.fabricate_absent_name("topic"),
+ name: shadow.fresh_name("cg"),
+ }),
+ Outcome::TopicNotFound => {
+ let stream = shadow.pick_stream_name(prng)?;
+ Some(Input {
+ stream,
+ topic: shadow.fabricate_absent_name("topic"),
+ name: shadow.fresh_name("cg"),
+ })
+ }
Outcome::NameAlreadyExists => {
let (stream, topic, name) = shadow.pick_consumer_group_triple(prng)?;
Some(Input {
@@ -87,6 +108,6 @@ pub fn predicted_effect(input: &Input, outcome: Outcome) -> Effect {
topic: input.topic.clone(),
name: input.name.clone(),
},
- Outcome::NameAlreadyExists => Effect::None,
+ _ => Effect::None,
}
}
(iggy) 17/43: feat(server-ng): add HTTP admission control
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit fcd0015176eac033412f7effc233bcccea1da6b0
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 00:30:21 2026 +0200
feat(server-ng): add HTTP admission control
HTTP handlers run on shard 0's single-threaded compio runtime, so
unbounded concurrent writes park unbounded handlers whose buffered
bodies and per-request CPU compete with the consensus pump. Three
bounds close that: the legacy DefaultBodyLimit (413 past
http.max_request_size) caps the per-request term, a per-session
in-flight cap (429) keeps one credential from monopolizing the shared
budget, and a global in-flight budget (503 server_busy) bounds the
multiplier, so budget x max_request_size caps worst-case buffered
bytes and budget x per-request CPU caps pump starvation. One RAII
guard owns both counter releases, so success, errors, the reply
timeout, and client disconnect all release admission through the same
Drop. ack=none produces stay outside admission: they never park a
handler, so their whole cost is the synchronous handler body.
---
core/server-ng/src/bootstrap.rs | 2 +-
core/server-ng/src/http.rs | 245 ++++++++++++++++++++++++++++++++++++++--
2 files changed, 237 insertions(+), 10 deletions(-)
diff --git a/core/server-ng/src/bootstrap.rs b/core/server-ng/src/bootstrap.rs
index a3910964f..87ba615da 100644
--- a/core/server-ng/src/bootstrap.rs
+++ b/core/server-ng/src/bootstrap.rs
@@ -1946,7 +1946,7 @@ async fn start_tcp_runtime(
// caller of this function.
if config.http.enabled {
let http_addr = parse_socket_addr("http.address", &config.http.address)?;
- http::start(shard, http_addr, &config.http.jwt, &config.cluster).await?;
+ http::start(shard, http_addr, &config.http, &config.cluster).await?;
}
Ok(())
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 596f00a94..e36f4b813 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -28,7 +28,7 @@ use std::net::{IpAddr, SocketAddr};
use std::rc::Rc;
use std::time::Duration;
-use axum::extract::{Path, Query, State};
+use axum::extract::{DefaultBodyLimit, Path, Query, State};
use axum::http::header::LOCATION;
use axum::http::{HeaderName, HeaderValue, StatusCode};
use axum::middleware::map_response;
@@ -37,7 +37,7 @@ use axum::routing::{delete, get, post, put};
use axum::{Json, Router};
use bytes::{Bytes, BytesMut};
use configs::cluster::{ClusterConfig, ClusterNodeConfig};
-use configs::http::HttpJwtConfig;
+use configs::http::HttpConfig;
use consensus::{MetadataHandle, VsrConsensus};
use iggy_binary_protocol::codes::{
GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_STATS_CODE, GET_STREAM_CODE,
@@ -173,6 +173,25 @@ const FIRST_REQUEST_ID: u64 = 1;
/// caller's behalf.
const PARTITION_WRITE_REPLY_TIMEOUT: Duration = Duration::from_secs(10);
+/// Per-session cap on concurrently awaited partition writes (produce /
+/// consumer-offset). Bounds how much of [`MAX_IN_FLIGHT_WRITES_GLOBAL`] one
+/// credential can occupy, so a session that outruns its own commits saturates
+/// itself (429, its own backpressure signal) before it can starve every other
+/// session out of the shared budget.
+const MAX_IN_FLIGHT_WRITES_PER_SESSION: u32 = 32;
+
+/// Shard-0-global budget for concurrently awaited partition writes, across all
+/// sessions. Every admitted write parks a handler for up to
+/// [`PARTITION_WRITE_REPLY_TIMEOUT`] while pinning its buffered body, and its
+/// decode/encode/HS256 CPU runs on the same single-threaded core that pumps
+/// consensus. The budget therefore bounds both starvation terms: budget x
+/// `max_request_size` bounds the worst-case buffered bytes, and budget x
+/// per-request CPU bounds how far admitted HTTP work can delay the consensus
+/// pump. `?ack=none` produces sit outside the budget by design: they never
+/// park a handler, so their whole cost is the synchronous handler body itself,
+/// which admission could not shed anyway.
+const MAX_IN_FLIGHT_WRITES_GLOBAL: u32 = 128;
+
/// VSR client id stamped on HTTP data-plane reads (poll / consumer-offset).
/// HTTP reads never Register a VSR client, and shard-0 client ids are minted
/// from 1, so 0 can never name a live consumer-group member: a group-kind
@@ -234,6 +253,10 @@ struct HttpSession {
/// session eviction can tear the registry entry down fenced by the same
/// token.
registry_token: Cell<Option<InstanceToken>>,
+ /// Awaited partition writes currently in flight on this session, gated by
+ /// [`MAX_IN_FLIGHT_WRITES_PER_SESSION`]. Only [`InFlightWriteGuard`]
+ /// touches it, so every admission is paired with exactly one release.
+ in_flight_writes: Cell<u32>,
}
impl HttpSession {
@@ -309,6 +332,7 @@ impl IntoResponse for WriteError {
/// reply distinguishable only by header (see [`classify_partition_reply`]),
/// and an unanswered write is a distinct outcome the caller must treat as
/// unknown rather than failed.
+#[derive(Debug)]
enum PartitionWriteError {
/// Caller-side rejection (bad identifier, oversized batch, the interim
/// non-root denial) or a malformed reply frame, rendered through the
@@ -321,6 +345,15 @@ enum PartitionWriteError {
/// The in-process reply slot could not be installed. Transient server
/// condition -> the shared 503, retryable.
Unavailable,
+ /// This session is already at [`MAX_IN_FLIGHT_WRITES_PER_SESSION`]
+ /// awaited writes. 429: the caller's own concurrency is the problem, so
+ /// it must drain its outstanding writes before submitting more.
+ TooManyInFlight,
+ /// Shard 0 is already at [`MAX_IN_FLIGHT_WRITES_GLOBAL`] awaited writes
+ /// across all sessions. 503 with its own code (distinct from the shared
+ /// consensus-unavailable body) so an operator can tell admission shedding
+ /// from a consensus outage.
+ ServerBusy,
/// No committed reply within [`PARTITION_WRITE_REPLY_TIMEOUT`], or the
/// session's reply target was torn down mid-wait. 504: the commit may
/// still land (at-least-once), so this is a hard "outcome unknown", not a
@@ -335,6 +368,8 @@ impl IntoResponse for PartitionWriteError {
Self::Rejected(error) => CustomError::from(error).into_response(),
Self::NotFound => CustomError::ResourceNotFound.into_response(),
Self::Unavailable => service_unavailable(),
+ Self::TooManyInFlight => too_many_in_flight_response(),
+ Self::ServerBusy => server_busy_response(),
Self::Timeout(operation) => partition_write_timeout_response(operation),
}
}
@@ -389,6 +424,41 @@ fn service_unavailable() -> Response {
.into_response()
}
+/// 429 for a session at [`MAX_IN_FLIGHT_WRITES_PER_SESSION`] awaited partition
+/// writes. Shaped like every other HTTP error (`ErrorResponse`) so clients
+/// parse one error schema; the remedy is the caller's own: let outstanding
+/// writes finish, then retry.
+fn too_many_in_flight_response() -> Response {
+ (
+ StatusCode::TOO_MANY_REQUESTS,
+ Json(ErrorResponse {
+ id: StatusCode::TOO_MANY_REQUESTS.as_u16().into(),
+ code: "too_many_in_flight_writes".to_owned(),
+ reason: "session reached its in-flight write cap; await outstanding writes and retry"
+ .to_owned(),
+ field: None,
+ }),
+ )
+ .into_response()
+}
+
+/// 503 for shard 0 at [`MAX_IN_FLIGHT_WRITES_GLOBAL`] awaited partition writes
+/// across all sessions. A distinct `server_busy` code (unlike the shared
+/// consensus-unavailable 503) so admission shedding is tellable from a
+/// consensus outage; retry with backoff.
+fn server_busy_response() -> Response {
+ (
+ StatusCode::SERVICE_UNAVAILABLE,
+ Json(ErrorResponse {
+ id: StatusCode::SERVICE_UNAVAILABLE.as_u16().into(),
+ code: "server_busy".to_owned(),
+ reason: "shard is at its in-flight write budget; retry with backoff".to_owned(),
+ field: None,
+ }),
+ )
+ .into_response()
+}
+
/// Read consistency selected by the `?consistency=` query param.
///
/// `serializable` (the default) serves from this node's local metadata STM:
@@ -562,6 +632,10 @@ struct HttpInner {
/// across an `.await` (see [`HttpInner::resolve_session`]).
sessions: RefCell<HashMap<String, Rc<HttpSession>>>,
roster: ClusterRoster,
+ /// Awaited partition writes currently in flight across all sessions, gated
+ /// by [`MAX_IN_FLIGHT_WRITES_GLOBAL`]. Only [`InFlightWriteGuard`] touches
+ /// it, so every admission is paired with exactly one release.
+ in_flight_writes: Cell<u32>,
}
impl HttpInner {
@@ -696,6 +770,7 @@ impl HttpInner {
gate: Mutex::new(FIRST_REQUEST_ID),
data_request: Cell::new(FIRST_REQUEST_ID),
registry_token: Cell::new(None),
+ in_flight_writes: Cell::new(0),
}))
}
}
@@ -728,14 +803,14 @@ type HttpState = SendWrapper<Rc<HttpInner>>;
/// # Errors
///
/// Returns [`ServerNgError`] if the JWT manager cannot be built from
-/// `jwt_config` or the listener cannot bind to `addr`.
+/// `http_config.jwt` or the listener cannot bind to `addr`.
pub async fn start(
shard: &Rc<ServerNgShard>,
addr: SocketAddr,
- jwt_config: &HttpJwtConfig,
+ http_config: &HttpConfig,
cluster: &ClusterConfig,
) -> Result<(), ServerNgError> {
- let jwt = JwtManager::build(jwt_config)?;
+ let jwt = JwtManager::build(&http_config.jwt)?;
let (listener, bound_addr) = client_listener::tcp::bind(addr).await?;
info!(address = %bound_addr, "server-ng HTTP listener started");
@@ -749,8 +824,13 @@ pub async fn start(
nodes: cluster.nodes.clone(),
http_addr: bound_addr,
},
+ in_flight_writes: Cell::new(0),
}));
- let router = router(state);
+ // Saturating: a configured limit past the pointer width (32-bit target,
+ // >4 GiB value) clamps to the largest enforceable cap instead of wrapping.
+ let max_request_size =
+ usize::try_from(http_config.max_request_size.as_bytes_u64()).unwrap_or(usize::MAX);
+ let router = router(state, max_request_size);
let shutdown = shard.bus.token();
let handle = compio::runtime::spawn(async move {
@@ -768,7 +848,12 @@ pub async fn start(
/// Assemble the shard-0 router: unauthenticated health + login routes, plus
/// one authenticated probe that exercises the [`Authenticated`] extractor.
-fn router(state: HttpState) -> Router {
+///
+/// `max_request_size` becomes the router-wide `DefaultBodyLimit` (413 past
+/// it), exactly like the legacy server: it bounds the per-request term of the
+/// admission math - what one body may cost in bytes and decode CPU - while
+/// the in-flight caps bound the multiplier.
+fn router(state: HttpState, max_request_size: usize) -> Router {
// Cloned for the response layer so `X-Iggy-View` reads the live view per
// response; the original `state` is moved into `with_state` below.
let view_source = state.clone();
@@ -834,6 +919,7 @@ fn router(state: HttpState) -> Router {
.route("/clients", get(get_clients))
.route("/clients/{client_id}", get(get_client))
.with_state(state)
+ .layer(DefaultBodyLimit::max(max_request_size))
.layer(map_response(move |response: Response| {
let view_source = view_source.clone();
async move { insert_view_header(&view_source, response) }
@@ -2109,9 +2195,47 @@ const fn resync_required_polled_messages() -> PolledMessages {
}
}
+/// In-flight admission token for one awaited partition write. One guard owns
+/// both releases (session + global) so success, every error return, the reply
+/// timeout, and handler cancellation (the client hanging up mid-await drops
+/// this future) all decrement through the same `Drop`.
+struct InFlightWriteGuard<'a> {
+ session_in_flight: &'a Cell<u32>,
+ global_in_flight: &'a Cell<u32>,
+}
+
+impl Drop for InFlightWriteGuard<'_> {
+ fn drop(&mut self) {
+ self.session_in_flight.set(self.session_in_flight.get() - 1);
+ self.global_in_flight.set(self.global_in_flight.get() - 1);
+ }
+}
+
+/// Admit one awaited partition write against the per-session cap and the
+/// shard-0 global budget, incrementing both counters only when both pass. The
+/// session cap is checked first so a session that saturates itself reads as
+/// its own 429 rather than as server-wide pressure.
+fn admit_partition_write<'a>(
+ session_in_flight: &'a Cell<u32>,
+ global_in_flight: &'a Cell<u32>,
+) -> Result<InFlightWriteGuard<'a>, PartitionWriteError> {
+ if session_in_flight.get() >= MAX_IN_FLIGHT_WRITES_PER_SESSION {
+ return Err(PartitionWriteError::TooManyInFlight);
+ }
+ if global_in_flight.get() >= MAX_IN_FLIGHT_WRITES_GLOBAL {
+ return Err(PartitionWriteError::ServerBusy);
+ }
+ session_in_flight.set(session_in_flight.get() + 1);
+ global_in_flight.set(global_in_flight.get() + 1);
+ Ok(InFlightWriteGuard {
+ session_in_flight,
+ global_in_flight,
+ })
+}
+
/// Run one awaited partition write (produce / consumer-offset write) end to
-/// end: install the reply slot, dispatch into the partition plane, and wait
-/// (bounded) for the committed reply.
+/// end: admit against the in-flight caps, install the reply slot, dispatch
+/// into the partition plane, and wait (bounded) for the committed reply.
///
/// Slot-before-dispatch is load-bearing: every pre-dispatch gate failure
/// inside [`dispatch_partition_request`] replies through `send_to_client`,
@@ -2124,6 +2248,13 @@ async fn partition_write_replicated(
operation: Operation,
body: &[u8],
) -> Result<(), PartitionWriteError> {
+ // Admission sits here rather than before body decode: axum's extractors
+ // already buffered and deserialized the body (bounded by the router-wide
+ // `DefaultBodyLimit`) before the handler ran, so the caps gate what is
+ // actually unbounded - the slot install, the dispatch, and the parked
+ // reply await that pins this request's buffers for up to the reply
+ // timeout. Held across every exit below; released by `Drop`.
+ let _in_flight = admit_partition_write(&session.in_flight_writes, &state.in_flight_writes)?;
ensure_in_process_reply_target(state, session);
let request_id = session.next_data_request_id();
let message = build_request_message(
@@ -2982,4 +3113,100 @@ mod tests {
Err(PartitionWriteError::Rejected(IggyError::InvalidCommand))
));
}
+
+ #[test]
+ fn in_flight_write_guard_decrements_both_counters_on_drop() {
+ let session = Cell::new(0);
+ let global = Cell::new(0);
+ let guard = admit_partition_write(&session, &global).expect("below both caps");
+ assert_eq!(session.get(), 1);
+ assert_eq!(global.get(), 1);
+ drop(guard);
+ assert_eq!(session.get(), 0);
+ assert_eq!(global.get(), 0);
+ }
+
+ #[test]
+ fn admission_at_session_cap_rejects_with_too_many_in_flight() {
+ let session = Cell::new(MAX_IN_FLIGHT_WRITES_PER_SESSION);
+ let global = Cell::new(0);
+ assert!(matches!(
+ admit_partition_write(&session, &global),
+ Err(PartitionWriteError::TooManyInFlight)
+ ));
+ // A refusal must not leak a partial increment on either counter.
+ assert_eq!(session.get(), MAX_IN_FLIGHT_WRITES_PER_SESSION);
+ assert_eq!(global.get(), 0);
+ }
+
+ #[test]
+ fn admission_at_global_budget_rejects_with_server_busy() {
+ let session = Cell::new(0);
+ let global = Cell::new(MAX_IN_FLIGHT_WRITES_GLOBAL);
+ assert!(matches!(
+ admit_partition_write(&session, &global),
+ Err(PartitionWriteError::ServerBusy)
+ ));
+ assert_eq!(session.get(), 0);
+ assert_eq!(global.get(), MAX_IN_FLIGHT_WRITES_GLOBAL);
+ }
+
+ #[test]
+ fn interleaved_admission_reopens_exactly_released_session_slots() {
+ let session = Cell::new(0);
+ let global = Cell::new(0);
+ let mut guards = Vec::new();
+ for _ in 0..MAX_IN_FLIGHT_WRITES_PER_SESSION {
+ guards.push(admit_partition_write(&session, &global).expect("below both caps"));
+ }
+ assert!(matches!(
+ admit_partition_write(&session, &global),
+ Err(PartitionWriteError::TooManyInFlight)
+ ));
+ let released = 3;
+ guards.truncate((MAX_IN_FLIGHT_WRITES_PER_SESSION - released) as usize);
+ assert_eq!(session.get(), MAX_IN_FLIGHT_WRITES_PER_SESSION - released);
+ for _ in 0..released {
+ guards.push(admit_partition_write(&session, &global).expect("released slots"));
+ }
+ assert!(matches!(
+ admit_partition_write(&session, &global),
+ Err(PartitionWriteError::TooManyInFlight)
+ ));
+ drop(guards);
+ assert_eq!(session.get(), 0);
+ assert_eq!(global.get(), 0);
+ }
+
+ #[test]
+ fn global_budget_spans_sessions_and_reopens_after_release() {
+ let global = Cell::new(0);
+ let session_count =
+ MAX_IN_FLIGHT_WRITES_GLOBAL.div_ceil(MAX_IN_FLIGHT_WRITES_PER_SESSION) as usize;
+ let sessions: Vec<Cell<u32>> = (0..session_count).map(|_| Cell::new(0)).collect();
+ let mut guards = Vec::new();
+ 'fill: for session in &sessions {
+ for _ in 0..MAX_IN_FLIGHT_WRITES_PER_SESSION {
+ match admit_partition_write(session, &global) {
+ Ok(guard) => guards.push(guard),
+ Err(PartitionWriteError::ServerBusy) => break 'fill,
+ Err(other) => {
+ panic!("only the global budget may refuse this fill, got {other:?}")
+ }
+ }
+ }
+ }
+ assert_eq!(global.get(), MAX_IN_FLIGHT_WRITES_GLOBAL);
+ // A fresh session is refused on the shared budget, not its own cap.
+ let fresh = Cell::new(0);
+ assert!(matches!(
+ admit_partition_write(&fresh, &global),
+ Err(PartitionWriteError::ServerBusy)
+ ));
+ drop(guards.pop());
+ let readmitted = admit_partition_write(&fresh, &global).expect("budget slot released");
+ assert_eq!(fresh.get(), 1);
+ assert_eq!(global.get(), MAX_IN_FLIGHT_WRITES_GLOBAL);
+ drop(readmitted);
+ }
}
(iggy) 11/43: feat(server-ng): add HTTP cluster-metadata route and view header
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit cfa92f96e649ea73fe729e241b3c2236de576f45
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 19:40:40 2026 +0200
feat(server-ng): add HTTP cluster-metadata route and view header
Serve GET /cluster/metadata from the live config roster and consensus view instead of the hardcoded shared builder, and stamp X-Iggy-View on responses so clients can observe the current VSR view.
Auth-only like legacy but bypassing the consistency gate so followers still answer for leader discovery. The roster is captured from config at http::start; a single self-node is synthesized when the cluster is disabled, and the leader is marked per primary_index(view).
---
core/server-ng/src/bootstrap.rs | 2 +-
core/server-ng/src/http.rs | 136 ++++++++++++++++++++++++++++++++++++++--
2 files changed, 133 insertions(+), 5 deletions(-)
diff --git a/core/server-ng/src/bootstrap.rs b/core/server-ng/src/bootstrap.rs
index aa3d4fc0b..a3910964f 100644
--- a/core/server-ng/src/bootstrap.rs
+++ b/core/server-ng/src/bootstrap.rs
@@ -1946,7 +1946,7 @@ async fn start_tcp_runtime(
// caller of this function.
if config.http.enabled {
let http_addr = parse_socket_addr("http.address", &config.http.address)?;
- http::start(shard, http_addr, &config.http.jwt).await?;
+ http::start(shard, http_addr, &config.http.jwt, &config.cluster).await?;
}
Ok(())
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index dc1de5f62..c21fe5c85 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -28,11 +28,13 @@ use std::net::SocketAddr;
use std::rc::Rc;
use axum::extract::{Path, Query, State};
-use axum::http::StatusCode;
+use axum::http::{HeaderName, HeaderValue, StatusCode};
+use axum::middleware::map_response;
use axum::response::{IntoResponse, Response};
use axum::routing::{delete, get, post, put};
use axum::{Json, Router};
use bytes::Bytes;
+use configs::cluster::{ClusterConfig, ClusterNodeConfig};
use configs::http::HttpJwtConfig;
use consensus::{MetadataHandle, VsrConsensus};
use iggy_binary_protocol::codes::{
@@ -97,9 +99,10 @@ use iggy_common::wire_conversions::{
streams_from_wire, topics_from_wire, users_from_wire,
};
use iggy_common::{
- ClientInfo, ClientInfoDetails, ConsumerGroup, ConsumerGroupDetails, Identifier, IdentityInfo,
- IggyError, IggyTimestamp, RawPersonalAccessToken, Stats, Stream, StreamDetails, TokenInfo,
- Topic, TopicDetails, UserInfo, UserInfoDetails, Validatable,
+ ClientInfo, ClientInfoDetails, ClusterMetadata, ClusterNode, ClusterNodeRole,
+ ClusterNodeStatus, ConsumerGroup, ConsumerGroupDetails, Identifier, IdentityInfo, IggyError,
+ IggyTimestamp, RawPersonalAccessToken, Stats, Stream, StreamDetails, TokenInfo, Topic,
+ TopicDetails, TransportEndpoints, UserInfo, UserInfoDetails, Validatable,
};
use message_bus::client_listener;
use metadata::impls::metadata::StreamsFrontend;
@@ -139,6 +142,18 @@ const MAX_HTTP_SESSIONS: usize = 100_000;
/// are 1-based and strictly increasing within a session.
const FIRST_REQUEST_ID: u64 = 1;
+/// Node name reported for the synthesized single self-node, matching the legacy
+/// server's single-node synthesis.
+const SELF_NODE_NAME: &str = "iggy-node";
+
+/// Cluster name reported when no roster is configured, matching the legacy
+/// server's single-node synthesis.
+const SINGLE_NODE_CLUSTER_NAME: &str = "single-node";
+
+/// Response header carrying the current VSR view number, set on every response
+/// while this shard-0 node has live consensus.
+const VIEW_HEADER: HeaderName = HeaderName::from_static("x-iggy-view");
+
/// One VSR session established for a single login credential (a JWT `jti` or a
/// PAT). Shared via `Rc` by every concurrent request bearing that credential,
/// so the session granularity is per-login.
@@ -300,6 +315,20 @@ fn not_primary_response() -> Response {
.into_response()
}
+/// Owned snapshot of the cluster topology `GET /cluster/metadata` reports.
+///
+/// The roster lives only in `ServerNgConfig` and is not reachable from
+/// [`ServerNgShard`], so [`start`] copies the minimal pieces here at listener
+/// start. Owned so the handler stays synchronous and never borrows config.
+struct ClusterRoster {
+ enabled: bool,
+ name: String,
+ nodes: Vec<ClusterNodeConfig>,
+ /// This node's bound HTTP address, used to synthesize the sole self-node
+ /// when no roster is configured (the only transport port known here).
+ http_addr: SocketAddr,
+}
+
/// Shared shard-0 HTTP state.
///
/// Groups the shard handle, the JWT issuer/verifier, and the per-credential VSR
@@ -313,6 +342,7 @@ struct HttpInner {
/// bridge tolerates the `!Sync` interior - but the guard must never be held
/// across an `.await` (see [`HttpInner::resolve_session`]).
sessions: RefCell<HashMap<String, Rc<HttpSession>>>,
+ roster: ClusterRoster,
}
impl HttpInner {
@@ -448,6 +478,7 @@ pub async fn start(
shard: &Rc<ServerNgShard>,
addr: SocketAddr,
jwt_config: &HttpJwtConfig,
+ cluster: &ClusterConfig,
) -> Result<(), ServerNgError> {
let jwt = JwtManager::build(jwt_config)?;
let (listener, bound_addr) = client_listener::tcp::bind(addr).await?;
@@ -457,6 +488,12 @@ pub async fn start(
shard: Rc::clone(shard),
jwt,
sessions: RefCell::new(HashMap::new()),
+ roster: ClusterRoster {
+ enabled: cluster.enabled,
+ name: cluster.name.clone(),
+ nodes: cluster.nodes.clone(),
+ http_addr: bound_addr,
+ },
}));
let router = router(state);
@@ -477,6 +514,9 @@ pub async fn start(
/// Assemble the shard-0 router: unauthenticated health + login routes, plus
/// one authenticated probe that exercises the [`Authenticated`] extractor.
fn router(state: HttpState) -> Router {
+ // Cloned for the response layer so `X-Iggy-View` reads the live view per
+ // response; the original `state` is moved into `with_state` below.
+ let view_source = state.clone();
Router::new()
.route("/ping", get(ping))
.route("/users/login", post(login_user))
@@ -523,9 +563,14 @@ fn router(state: HttpState) -> Router {
.route("/personal-access-tokens", post(create_pat))
.route("/personal-access-tokens/{name}", delete(delete_pat))
.route("/stats", get(get_stats))
+ .route("/cluster/metadata", get(get_cluster_metadata))
.route("/clients", get(get_clients))
.route("/clients/{client_id}", get(get_client))
.with_state(state)
+ .layer(map_response(move |response: Response| {
+ let view_source = view_source.clone();
+ async move { insert_view_header(&view_source, response) }
+ }))
}
/// Extracting the state here proves at compile time that the `!Send` state
@@ -805,6 +850,20 @@ async fn get_stats(
Ok(Json(Stats::from(response)))
}
+/// `GET /cluster/metadata`: report the live cluster topology as the same
+/// `ClusterMetadata` JSON the legacy server returns.
+///
+/// Auth-only: any valid token serves. Unlike the entity reads it bypasses both
+/// the root gate and the consistency gate, and serves from the roster captured
+/// at listener start plus the sync consensus getters, so it never touches the
+/// metadata STM, consensus, or a VSR session and stays fully synchronous.
+async fn get_cluster_metadata(
+ State(state): State<HttpState>,
+ _identity: Identity,
+) -> Json<ClusterMetadata> {
+ Json(build_cluster_metadata(&state))
+}
+
/// `GET /clients`: list every connected client across all shards as the same
/// `Vec<ClientInfo>` JSON the legacy server returns.
///
@@ -1490,6 +1549,75 @@ fn read_local(
}
}
+/// Build the live [`ClusterMetadata`] for `GET /cluster/metadata`.
+///
+/// Mirrors the legacy `get_cluster_metadata`: with a configured, non-empty
+/// roster and live consensus, emit one node per roster entry and derive each
+/// role from the current VSR view - the node at `primary_index(view)` leads, the
+/// rest follow. Otherwise (cluster disabled, empty roster, or absent consensus)
+/// fail closed to a synthesized self-node as the sole leader.
+fn build_cluster_metadata(state: &HttpInner) -> ClusterMetadata {
+ let roster = &state.roster;
+ match state.shard.plane.metadata().consensus.as_ref() {
+ Some(consensus) if roster.enabled && !roster.nodes.is_empty() => {
+ let primary = consensus.primary_index(consensus.view());
+ let nodes = roster
+ .nodes
+ .iter()
+ .map(|node| ClusterNode {
+ name: node.name.clone(),
+ ip: node.ip.clone(),
+ endpoints: TransportEndpoints::new(
+ node.ports.tcp.unwrap_or(0),
+ node.ports.quic.unwrap_or(0),
+ node.ports.http.unwrap_or(0),
+ node.ports.websocket.unwrap_or(0),
+ ),
+ role: if node.replica_id == primary {
+ ClusterNodeRole::Leader
+ } else {
+ ClusterNodeRole::Follower
+ },
+ status: ClusterNodeStatus::Healthy,
+ })
+ .collect();
+ ClusterMetadata {
+ name: roster.name.clone(),
+ nodes,
+ }
+ }
+ _ => self_node_metadata(roster),
+ }
+}
+
+/// Synthesize a single self-node [`ClusterMetadata`], mirroring the legacy
+/// single-node path. Only this node's bound HTTP port is known at the HTTP layer
+/// (the roster is not threaded onto the shard), so the other transports report 0.
+fn self_node_metadata(roster: &ClusterRoster) -> ClusterMetadata {
+ ClusterMetadata {
+ name: SINGLE_NODE_CLUSTER_NAME.to_owned(),
+ nodes: vec![ClusterNode {
+ name: SELF_NODE_NAME.to_owned(),
+ ip: roster.http_addr.ip().to_string(),
+ endpoints: TransportEndpoints::new(0, 0, roster.http_addr.port(), 0),
+ role: ClusterNodeRole::Leader,
+ status: ClusterNodeStatus::Healthy,
+ }],
+ }
+}
+
+/// Set the [`VIEW_HEADER`] to the current VSR view on `response`. Omits the
+/// header when this node has no live consensus: a missing header is
+/// unambiguous, whereas a fabricated view number would mislead.
+fn insert_view_header(state: &HttpInner, mut response: Response) -> Response {
+ if let Some(consensus) = state.shard.plane.metadata().consensus.as_ref() {
+ response
+ .headers_mut()
+ .insert(VIEW_HEADER, HeaderValue::from(consensus.view()));
+ }
+ response
+}
+
/// Classify a committed reply's leading result section and return the typed
/// payload slice on success. Mirrors the SDK's `split_metadata_result`:
/// `Some(0)` is success and the payload follows the result section; a nonzero
(iggy) 30/43: perf(server-ng): warm dummy password hash at boot
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 376db025c1fbdb8d8100331c4bfef95c4670f87d
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 21:21:20 2026 +0200
perf(server-ng): warm dummy password hash at boot
The unknown-user dummy verify hash is a LazyLock, so the first login
with a missing username paid the one-time Argon2 init inside the
request. Force it during bootstrap instead.
---
core/server-ng/src/auth.rs | 6 ++++++
core/server-ng/src/bootstrap.rs | 2 ++
2 files changed, 8 insertions(+)
diff --git a/core/server-ng/src/auth.rs b/core/server-ng/src/auth.rs
index cc1c25412..9dd5d5413 100644
--- a/core/server-ng/src/auth.rs
+++ b/core/server-ng/src/auth.rs
@@ -50,6 +50,12 @@ use tracing::warn;
static DUMMY_PASSWORD_HASH: LazyLock<String> =
LazyLock::new(|| crypto::hash_password("http-login-timing-guard"));
+/// Pay the one-time Argon2 cost of [`DUMMY_PASSWORD_HASH`] at boot instead of
+/// inside the first unknown-username login request.
+pub(crate) fn warm_dummy_password_hash() {
+ LazyLock::force(&DUMMY_PASSWORD_HASH);
+}
+
pub(crate) fn verify_login_credentials(
shard: &Rc<ServerNgShard>,
username: &str,
diff --git a/core/server-ng/src/bootstrap.rs b/core/server-ng/src/bootstrap.rs
index 87ba615da..d346699da 100644
--- a/core/server-ng/src/bootstrap.rs
+++ b/core/server-ng/src/bootstrap.rs
@@ -15,6 +15,7 @@
// specific language governing permissions and limitations
// under the License.
+use crate::auth::warm_dummy_password_hash;
use crate::config_writer::write_current_config;
use crate::dispatch::{
make_client_request_handler, make_deferred_client_request_handler,
@@ -513,6 +514,7 @@ pub fn bootstrap(
config: ServerNgConfig,
current_replica_id: Option<u8>,
) -> Result<ShardHandles, ServerNgError> {
+ warm_dummy_password_hash();
let allocator = ShardAllocator::new(&config.system.sharding.cpu_allocation)
.map_err(ServerNgError::ShardAllocator)?;
let assignments = allocator
(iggy) 41/43: feat(server-ng): list personal access tokens over HTTP and TCP
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 1b9874fdb3c23c0ae6e92e52ad2f6efc8197a56f
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 22:43:05 2026 +0200
feat(server-ng): list personal access tokens over HTTP and TCP
GET_PERSONAL_ACCESS_TOKENS was the one read the ng server did not
serve: native TCP fell through to an empty reply (a false success)
and the HTTP route was parked. PATs sit in AHashMaps guarded by the
deterministic-apply invariant, so the users STM gains a read-only,
name-sorted accessor instead of exposing raw map iteration order.
The list is caller-scoped, so the shared read builder now carries
the authenticated user id; the transport gates require only
authentication, matching the legacy handler (no permissioner rule).
---
core/integration/tests/server/http_rbac.rs | 92 ++++++++++++++++++++++++++++++
core/metadata/src/stm/user.rs | 64 +++++++++++++++++++++
core/server-ng/src/dispatch.rs | 15 +++--
core/server-ng/src/http.rs | 54 +++++++++++++-----
core/server-ng/src/responses.rs | 91 ++++++++++++++++++++++++++---
5 files changed, 291 insertions(+), 25 deletions(-)
diff --git a/core/integration/tests/server/http_rbac.rs b/core/integration/tests/server/http_rbac.rs
index f408bc5e7..09ff99351 100644
--- a/core/integration/tests/server/http_rbac.rs
+++ b/core/integration/tests/server/http_rbac.rs
@@ -946,6 +946,98 @@ async fn given_non_root_tcp_session_when_polling_denied_should_surface_typed_una
);
}
+// === Case family 9: PAT list is caller-scoped ================================
+
+/// Listing is self-scoped like the other PAT ops: any authenticated user sees
+/// exactly their own tokens (name + expiry, name-sorted), never another
+/// user's. Asserted over HTTP and via the SDK on TCP, which share the read
+/// builder.
+#[iggy_harness]
+async fn given_two_users_with_tokens_when_listing_should_return_only_callers(
+ harness: &TestHarness,
+) {
+ let root = Client::login_root(harness).await;
+ root.create_user("alice", "alice-pass", no_permissions())
+ .await;
+ root.create_user("bob", "bob-pass", no_permissions()).await;
+ let alice = root.login("alice", "alice-pass").await;
+ let bob = root.login("bob", "bob-pass").await;
+
+ // Created in reverse name order to prove the listing sorts by name.
+ for name in ["alice-zeta", "alice-alpha"] {
+ let created = alice
+ .post_json(
+ "/personal-access-tokens",
+ &json!({ "name": name, "expiry": 0 }),
+ )
+ .await;
+ assert_eq!(
+ created.status(),
+ StatusCode::OK,
+ "alice create PAT {name} must be 200"
+ );
+ }
+ let created = bob
+ .post_json(
+ "/personal-access-tokens",
+ &json!({ "name": "bob-pat", "expiry": 0 }),
+ )
+ .await;
+ assert_eq!(
+ created.status(),
+ StatusCode::OK,
+ "bob create PAT must be 200"
+ );
+
+ assert_eq!(
+ pat_names(alice.get("/personal-access-tokens").await).await,
+ ["alice-alpha", "alice-zeta"],
+ "alice must list exactly her own tokens, name-sorted"
+ );
+ assert_eq!(
+ pat_names(bob.get("/personal-access-tokens").await).await,
+ ["bob-pat"],
+ "bob must never see alice's tokens"
+ );
+
+ // Same caller-scoped view over native TCP through the SDK.
+ let client = harness.tcp_new_client().await.expect("tcp client");
+ client
+ .login_user("alice", "alice-pass")
+ .await
+ .expect("alice tcp login");
+ let tokens = client
+ .get_personal_access_tokens()
+ .await
+ .expect("tcp PAT list");
+ let tcp_names: Vec<&str> = tokens.iter().map(|token| token.name.as_str()).collect();
+ assert_eq!(
+ tcp_names,
+ ["alice-alpha", "alice-zeta"],
+ "TCP list must match the HTTP caller-scoped view"
+ );
+}
+
+/// Decode a PAT-list response body into its token names, asserting 200 first.
+async fn pat_names(response: Response) -> Vec<String> {
+ assert_eq!(
+ response.status(),
+ StatusCode::OK,
+ "PAT list must be 200 for an authenticated caller"
+ );
+ let body: Value = response.json().await.expect("PAT list body must be JSON");
+ body.as_array()
+ .expect("PAT list must be a JSON array")
+ .iter()
+ .map(|token| {
+ token["name"]
+ .as_str()
+ .expect("every token must carry a name")
+ .to_owned()
+ })
+ .collect()
+}
+
/// The current-password check gates a self-service rotation: a wrong current
/// password is rejected pre-consensus as 400 `InvalidCredentials` (never
/// reaching the replicated apply), and only the correct one rotates the
diff --git a/core/metadata/src/stm/user.rs b/core/metadata/src/stm/user.rs
index dafd4e812..7a5ee16eb 100644
--- a/core/metadata/src/stm/user.rs
+++ b/core/metadata/src/stm/user.rs
@@ -165,6 +165,34 @@ impl UsersInner {
.map(|(_, user_id, name)| (*user_id, Arc::clone(name)))
.collect()
}
+
+ /// Collect `(name, expiry_at)` for every live personal access token of
+ /// one user, sorted by name.
+ ///
+ /// Read-only accessor for the non-replicated list path (the caller lists
+ /// its own tokens). The backing per-user map is an `AHashMap`, whose
+ /// iteration order is seeded per process, so the collected list is
+ /// sorted before it is returned. Never lift this iteration into an apply
+ /// handler: apply must stay deterministic across replicas (see the
+ /// invariant on `personal_access_token_index`).
+ #[must_use]
+ pub fn personal_access_tokens_of(
+ &self,
+ user_id: UserId,
+ ) -> Vec<(Arc<str>, Option<IggyTimestamp>)> {
+ let mut tokens: Vec<_> = self
+ .personal_access_tokens
+ .get(&user_id)
+ .map(|user_tokens| {
+ user_tokens
+ .values()
+ .map(|pat| (Arc::clone(&pat.name), pat.expiry_at))
+ .collect()
+ })
+ .unwrap_or_default();
+ tokens.sort_by(|(left, _), (right, _)| left.cmp(right));
+ tokens
+ }
}
impl Users {
@@ -971,6 +999,42 @@ mod tests {
);
}
+ #[test]
+ fn personal_access_tokens_of_lists_only_callers_sorted_by_name() {
+ let mut users = UsersInner::new();
+ // Insertion order deliberately not name order; a token for user 9
+ // proves the listing is caller-scoped.
+ for (user_id, name, expiry, fill) in [
+ (5u32, "zeta", 0u64, b'a'),
+ (5u32, "alpha", 1, b'b'),
+ (9u32, "other", 0, b'c'),
+ ] {
+ CreatePersonalAccessTokenRequest {
+ user_id,
+ name: WireName::new(name).unwrap(),
+ expiry,
+ token_hash: [fill; PAT_TOKEN_HASH_BYTES],
+ }
+ .apply(&mut users, IggyTimestamp::zero());
+ }
+
+ let tokens = users.personal_access_tokens_of(5);
+ let names: Vec<&str> = tokens.iter().map(|(name, _)| name.as_ref()).collect();
+ assert_eq!(names, ["alpha", "zeta"], "own tokens only, sorted by name");
+ assert!(
+ tokens[0].1.is_some(),
+ "expiring token must carry its expiry_at"
+ );
+ assert!(
+ tokens[1].1.is_none(),
+ "never-expiring token must carry no expiry_at"
+ );
+ assert!(
+ users.personal_access_tokens_of(42).is_empty(),
+ "a user without tokens lists empty"
+ );
+ }
+
fn create_user(users: &mut UsersInner, username: &str) {
let request = CreateUserRequest {
username: WireName::new(username).unwrap(),
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index ce4e64195..b13088f29 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -49,9 +49,10 @@ use consensus::{
};
use iggy_binary_protocol::codes::{
GET_CLIENT_CODE, GET_CLIENTS_CODE, GET_CLUSTER_METADATA_CODE, GET_CONSUMER_GROUP_CODE,
- GET_CONSUMER_GROUPS_CODE, GET_CONSUMER_OFFSET_CODE, GET_ME_CODE, GET_STATS_CODE,
- GET_STREAM_CODE, GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE,
- GET_USERS_CODE, PING_CODE, POLL_MESSAGES_CODE, SYNC_CONSUMER_GROUP_CODE,
+ GET_CONSUMER_GROUPS_CODE, GET_CONSUMER_OFFSET_CODE, GET_ME_CODE,
+ GET_PERSONAL_ACCESS_TOKENS_CODE, GET_STATS_CODE, GET_STREAM_CODE, GET_STREAMS_CODE,
+ GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE, GET_USERS_CODE, PING_CODE, POLL_MESSAGES_CODE,
+ SYNC_CONSUMER_GROUP_CODE,
};
use iggy_binary_protocol::primitives::consumer::WireConsumer;
use iggy_binary_protocol::primitives::polling_strategy::WirePollingStrategy;
@@ -971,7 +972,7 @@ async fn handle_default_non_replicated(
send_non_replicated_deny(shard, request, transport_client_id, error.as_code()).await;
return;
}
- match build_non_replicated_response(shard, code, request_body(request)) {
+ match build_non_replicated_response(shard, code, request_body(request), user_id) {
Ok(response) => {
let commit = current_metadata_commit(shard);
let reply = response.into_reply(
@@ -1519,7 +1520,8 @@ fn authorize_partition_read(
/// miss, which falls through to the builder's own not-found reply so the legacy
/// notfound-before-permission ordering holds. `Err` denies with that code.
/// Unscoped rules gate directly; identifier-scoped rules resolve (stream[,
-/// topic]) against committed state first. `GET_CLUSTER_METADATA` is auth-only
+/// topic]) against committed state first. The PAT list is self-scoped, so
+/// authentication is its whole rule. `GET_CLUSTER_METADATA` is auth-only
/// (bootstrap / leader discovery) and every other code the builder serves is
/// ungated here.
fn authorize_default_read(
@@ -1532,6 +1534,9 @@ fn authorize_default_read(
GET_STATS_CODE => authorize_uid(shard, user_id, Permissioner::get_stats),
GET_USERS_CODE => authorize_uid(shard, user_id, Permissioner::get_users),
GET_USER_CODE => authorize_uid(shard, user_id, Permissioner::get_user),
+ // Self-scoped: lists only the caller's own tokens, so there is no
+ // permissioner rule to run (legacy runs none either).
+ GET_PERSONAL_ACCESS_TOKENS_CODE => user_id.map(|_| ()).ok_or(IggyError::Unauthenticated),
GET_STREAMS_CODE => authorize_uid(shard, user_id, Permissioner::get_streams),
GET_STREAM_CODE => {
let Ok(request) = GetStreamRequest::decode_from(body) else {
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 0fd2b846b..02ffc1f52 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -42,8 +42,9 @@ use configs::http::HttpConfig;
use consensus::{CLIENTS_TABLE_MAX, MetadataHandle, VsrConsensus};
use futures::channel::oneshot;
use iggy_binary_protocol::codes::{
- GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_STATS_CODE, GET_STREAM_CODE,
- GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE, GET_USERS_CODE,
+ GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_PERSONAL_ACCESS_TOKENS_CODE,
+ GET_STATS_CODE, GET_STREAM_CODE, GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE,
+ GET_USER_CODE, GET_USERS_CODE,
};
use iggy_binary_protocol::consensus::{
Command2, EvictionHeader, EvictionReason, HEADER_SIZE, result_code, result_section_len,
@@ -62,6 +63,7 @@ use iggy_binary_protocol::requests::messages::{
};
use iggy_binary_protocol::requests::personal_access_tokens::{
CreatePersonalAccessTokenRequest, DeletePersonalAccessTokenRequest,
+ GetPersonalAccessTokensRequest,
};
use iggy_binary_protocol::requests::streams::{
CreateStreamRequest, DeleteStreamRequest, GetStreamRequest, GetStreamsRequest,
@@ -81,7 +83,9 @@ use iggy_binary_protocol::responses::clients::get_client::ClientDetailsResponse;
use iggy_binary_protocol::responses::clients::get_clients::GetClientsResponse;
use iggy_binary_protocol::responses::consumer_groups::get_consumer_group::ConsumerGroupDetailsResponse;
use iggy_binary_protocol::responses::consumer_groups::get_consumer_groups::GetConsumerGroupsResponse;
-use iggy_binary_protocol::responses::personal_access_tokens::RawPersonalAccessTokenResponse;
+use iggy_binary_protocol::responses::personal_access_tokens::{
+ GetPersonalAccessTokensResponse, RawPersonalAccessTokenResponse,
+};
use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
use iggy_binary_protocol::responses::streams::get_streams::GetStreamsResponse;
use iggy_binary_protocol::responses::system::get_stats::StatsResponse;
@@ -114,16 +118,16 @@ use iggy_common::update_topic::UpdateTopic;
use iggy_common::update_user::UpdateUser;
use iggy_common::wire_conversions::{
clients_from_wire, consumer_groups_from_wire, consumer_to_wire, identifier_to_wire,
- partitioning_to_wire, permissions_to_wire, streams_from_wire, topics_from_wire,
- users_from_wire,
+ partitioning_to_wire, permissions_to_wire, personal_access_tokens_from_wire, streams_from_wire,
+ topics_from_wire, users_from_wire,
};
use iggy_common::{
ClientInfo, ClientInfoDetails, ClusterMetadata, ClusterNode, ClusterNodeRole,
ClusterNodeStatus, Consumer, ConsumerGroup, ConsumerGroupDetails, ConsumerOffsetInfo,
- Identifier, IdentityInfo, IggyError, IggyMessageView, IggyTimestamp, PollMessages,
- PolledMessages, RESYNC_REQUIRED_PARTITION_SENTINEL, RawPersonalAccessToken, SendMessages,
- Stats, Stream, StreamDetails, TokenInfo, Topic, TopicDetails, TransportEndpoints, UserInfo,
- UserInfoDetails, Validatable,
+ Identifier, IdentityInfo, IggyError, IggyMessageView, IggyTimestamp, PersonalAccessTokenInfo,
+ PollMessages, PolledMessages, RESYNC_REQUIRED_PARTITION_SENTINEL, RawPersonalAccessToken,
+ SendMessages, Stats, Stream, StreamDetails, TokenInfo, Topic, TopicDetails, TransportEndpoints,
+ UserInfo, UserInfoDetails, Validatable,
};
use message_bus::{BusMessage, InstanceToken, client_listener};
use metadata::impls::metadata::StreamsFrontend;
@@ -1083,9 +1087,7 @@ fn router(state: HttpState, max_request_size: usize) -> Router {
"/streams/{stream_id}/topics/{topic_id}/consumer-groups/{group_id}",
get(get_cg).delete(delete_cg),
)
- // TODO(hubcio): GET /personal-access-tokens (list) deferred - needs a server-ng STM
- // list accessor (PATs sit behind the "Never iterate" apply invariant; TCP-ng lacks it too).
- .route("/personal-access-tokens", post(create_pat))
+ .route("/personal-access-tokens", get(get_pats).post(create_pat))
.route("/personal-access-tokens/{name}", delete(delete_pat))
.route("/stats", get(get_stats))
.route("/cluster/metadata", get(get_cluster_metadata))
@@ -2193,6 +2195,30 @@ async fn update_permissions(
Ok(StatusCode::NO_CONTENT)
}
+/// `GET /personal-access-tokens`: list the caller's own tokens (name +
+/// expiry, never the secrets) as the same `Vec<PersonalAccessTokenInfo>` JSON
+/// the legacy server returns. Self-scoped, so authentication is the whole
+/// rule - no permissioner check, matching legacy parity. A consensus-free
+/// local STM read via [`read_local`].
+async fn get_pats(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Query(query): Query<ConsistencyQuery>,
+) -> Result<Json<Vec<PersonalAccessTokenInfo>>, ReadError> {
+ let body = GetPersonalAccessTokensRequest.to_bytes();
+ let bytes = read_local(
+ &state,
+ &identity,
+ query.consistency,
+ GET_PERSONAL_ACCESS_TOKENS_CODE,
+ &body,
+ |_, _| Ok(()),
+ )?;
+ let response = GetPersonalAccessTokensResponse::decode_from(&bytes)
+ .map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
+ Ok(Json(personal_access_tokens_from_wire(response)))
+}
+
/// `POST /personal-access-tokens`: mint a personal access token for the caller
/// and return its one-time raw secret as the same `{"token": ...}` JSON the
/// legacy server returns, with HTTP 200.
@@ -2749,7 +2775,9 @@ fn read_local(
rule: impl FnOnce(&Permissioner, u32) -> Result<(), IggyError>,
) -> Result<Bytes, ReadError> {
authorize_read(state, identity, consistency, rule)?;
- match build_non_replicated_response(&state.shard, code, body).map_err(ReadError::Rejected)? {
+ match build_non_replicated_response(&state.shard, code, body, Some(identity.user_id))
+ .map_err(ReadError::Rejected)?
+ {
NonReplicatedResponse::Empty => Err(ReadError::NotFound),
NonReplicatedResponse::Bytes(bytes) => Ok(bytes),
}
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index ed7e3225a..27f014812 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -18,8 +18,8 @@
//! Wire-response builders for the non-replicated read path.
//!
//! Assemble `get_me` / `get_clients` / `get_stream(s)` / `get_topic(s)` /
-//! `get_user(s)` / stats / cluster-metadata responses from per-shard
-//! session state and the metadata state machine, plus the
+//! `get_user(s)` / `get_personal_access_tokens` / stats / cluster-metadata
+//! responses from per-shard session state and the metadata state machine, plus the
//! [`NonReplicatedResponse`] dispatch shim and the partition-namespace
//! resolvers.
@@ -29,9 +29,9 @@ use crate::wire::{transport_kind_to_wire, usize_to_u32};
use bytes::{BufMut, Bytes, BytesMut};
use consensus::{MetadataHandle, VsrConsensus};
use iggy_binary_protocol::codes::{
- GET_CLUSTER_METADATA_CODE, GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_STATS_CODE,
- GET_STREAM_CODE, GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE,
- GET_USERS_CODE,
+ GET_CLUSTER_METADATA_CODE, GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE,
+ GET_PERSONAL_ACCESS_TOKENS_CODE, GET_STATS_CODE, GET_STREAM_CODE, GET_STREAMS_CODE,
+ GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE, GET_USERS_CODE,
};
use iggy_binary_protocol::consensus::{RESULT_COUNT_LEN, result_code};
use iggy_binary_protocol::primitives::consumer::WireConsumer;
@@ -43,6 +43,7 @@ use iggy_binary_protocol::requests::consumer_offsets::{
StoreConsumerOffsetRequest,
};
use iggy_binary_protocol::requests::messages::SendMessagesHeader;
+use iggy_binary_protocol::requests::personal_access_tokens::GetPersonalAccessTokensRequest;
use iggy_binary_protocol::requests::segments::DeleteSegmentsRequest;
use iggy_binary_protocol::requests::streams::{GetStreamRequest, GetStreamsRequest};
use iggy_binary_protocol::requests::topics::{GetTopicRequest, GetTopicsRequest};
@@ -51,7 +52,9 @@ use iggy_binary_protocol::responses::clients::client_response::ClientResponse;
use iggy_binary_protocol::responses::clients::client_response::ConsumerGroupInfoResponse;
use iggy_binary_protocol::responses::clients::get_client::ClientDetailsResponse;
use iggy_binary_protocol::responses::consumer_groups::GetConsumerGroupsResponse;
-use iggy_binary_protocol::responses::personal_access_tokens::RawPersonalAccessTokenResponse;
+use iggy_binary_protocol::responses::personal_access_tokens::{
+ GetPersonalAccessTokensResponse, PersonalAccessTokenResponse, RawPersonalAccessTokenResponse,
+};
use iggy_binary_protocol::responses::streams::StreamResponse;
use iggy_binary_protocol::responses::streams::get_stream::{
GetStreamResponse, TopicHeader as StreamTopicHeader,
@@ -71,7 +74,7 @@ use iggy_binary_protocol::{
Command2, GenericHeader, IGGY_PROTOCOL_VERSION, KIND_CONSUMER_GROUP, Operation, ReplyHeader,
RequestHeader, WireDecode, WireEncode, WireIdentifier, WireName, WirePartitioning,
};
-use iggy_common::IggyError;
+use iggy_common::{IggyError, IggyTimestamp};
use metadata::impls::metadata::StreamsFrontend;
use partitions::PollFragments;
use server::binary::dispatch::wire_id_to_identifier;
@@ -81,6 +84,7 @@ use server_common::sharding::IggyNamespace;
use shard::ConnectedClientInfo;
use std::cell::RefCell;
use std::rc::Rc;
+use std::sync::Arc;
/// Build the `get_me` reply for the requesting connection. Identity
/// (`user_id`, transport kind, peer address) comes from the per-shard
@@ -372,10 +376,14 @@ pub(crate) fn resolve_partition_namespace(
.ok_or(IggyError::InvalidIdentifier)
}
+/// `user_id` is the authenticated caller, used only by the identity-scoped
+/// reads (currently the PAT list); every other arm ignores it. Authorization
+/// stays with the per-transport gates that run before this builder.
pub(crate) fn build_non_replicated_response(
shard: &Rc<ServerNgShard>,
code: u32,
body: &[u8],
+ user_id: Option<u32>,
) -> Result<NonReplicatedResponse, IggyError> {
match code {
GET_CLUSTER_METADATA_CODE => Ok(NonReplicatedResponse::Bytes(
@@ -427,6 +435,17 @@ pub(crate) fn build_non_replicated_response(
})
})
}
+ GET_PERSONAL_ACCESS_TOKENS_CODE => {
+ let _ = GetPersonalAccessTokensRequest::decode_from(body)
+ .map_err(|_| IggyError::InvalidCommand)?;
+ // Caller-scoped: both transport gates reject unauthenticated
+ // callers before this builder runs, so a missing id is a gate
+ // bug; fail closed rather than serve another scope.
+ let user_id = user_id.ok_or(IggyError::Unauthenticated)?;
+ Ok(NonReplicatedResponse::Bytes(
+ build_get_personal_access_tokens_response(shard, user_id)?.to_bytes(),
+ ))
+ }
GET_CONSUMER_GROUP_CODE => {
let request = GetConsumerGroupRequest::decode_from(body)
.map_err(|_| IggyError::InvalidCommand)?;
@@ -632,6 +651,39 @@ fn build_get_user_response(
})
}
+/// The caller's own tokens, never another user's - the one identity-scoped
+/// read in this module, which is why [`build_non_replicated_response`]
+/// carries a `user_id`.
+fn build_get_personal_access_tokens_response(
+ shard: &Rc<ServerNgShard>,
+ user_id: u32,
+) -> Result<GetPersonalAccessTokensResponse, IggyError> {
+ let tokens = shard
+ .plane
+ .metadata()
+ .mux_stm
+ .users()
+ .read(|users| users.personal_access_tokens_of(user_id));
+ personal_access_tokens_response(tokens)
+}
+
+fn personal_access_tokens_response(
+ tokens: Vec<(Arc<str>, Option<IggyTimestamp>)>,
+) -> Result<GetPersonalAccessTokensResponse, IggyError> {
+ let tokens = tokens
+ .into_iter()
+ .map(|(name, expiry_at)| {
+ Ok(PersonalAccessTokenResponse {
+ name: WireName::new(name.as_ref()).map_err(|_| IggyError::InvalidFormat)?,
+ // 0 is the wire encoding for a never-expiring token, matching
+ // the legacy handler and the SDK-side decode.
+ expiry_at: expiry_at.map_or(0, |expiry_at| expiry_at.as_micros()),
+ })
+ })
+ .collect::<Result<Vec<_>, IggyError>>()?;
+ Ok(GetPersonalAccessTokensResponse { tokens })
+}
+
fn build_get_topic_response(
shard: &Rc<ServerNgShard>,
stream_id: &WireIdentifier,
@@ -1182,4 +1234,29 @@ mod tests {
build_raw_pat_reply(&pat_request_header(), success, None).expect("pass-through");
assert_eq!(reply.as_slice(), original.as_slice());
}
+
+ #[test]
+ fn personal_access_tokens_response_preserves_order_and_encodes_never_as_zero() {
+ let expiry = IggyTimestamp::from(123_456u64);
+ let tokens: Vec<(Arc<str>, Option<IggyTimestamp>)> = vec![
+ (Arc::from("alpha"), Some(expiry)),
+ (Arc::from("zeta"), None),
+ ];
+
+ let response = personal_access_tokens_response(tokens).expect("mapping succeeds");
+
+ assert_eq!(response.tokens.len(), 2);
+ assert_eq!(response.tokens[0].name.as_str(), "alpha");
+ assert_eq!(response.tokens[0].expiry_at, expiry.as_micros());
+ assert_eq!(response.tokens[1].name.as_str(), "zeta");
+ assert_eq!(response.tokens[1].expiry_at, 0);
+ }
+
+ #[test]
+ fn personal_access_tokens_response_encodes_empty_list_as_empty_body() {
+ let response = personal_access_tokens_response(Vec::new()).expect("mapping succeeds");
+ // An empty body is the wire shape the SDK decodes as "no tokens"; it
+ // must stay `Bytes` (not the not-found `Empty` variant) end to end.
+ assert!(response.to_bytes().is_empty());
+ }
}
(iggy) 21/43: feat(metadata): populate permissioner from replicated user ops
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit faaa94b63d3d2986edeab277a6b3fb84852f152e
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 11:07:46 2026 +0200
feat(metadata): populate permissioner from replicated user ops
The permissioner rule fns and index maps existed but were dead:
nothing populated them, so every check would deny. Its snapshot
also iterated AHash containers, which would serialize in
per-replica-random order the moment the maps held data.
Wire the index into the replicated user ops: create-user,
update-permissions and delete-user applies maintain it from the
just-stored user so it cannot drift from User.permissions; the
root user inherits via ensure_root_user's ordinary CreateUser
path. Snapshots now serialize intentionally-empty vecs (kept for
positional-msgpack layout compat) and restore rebuilds the index
from User.permissions, making snapshot bytes deterministic.
Add the get_snapshot rule missing from the port (same
read_servers/manage_servers semantics as its system siblings).
---
.../src/permissioner/permissioner_rules/system.rs | 4 +
core/metadata/src/stm/user.rs | 349 +++++++++++++++++----
2 files changed, 285 insertions(+), 68 deletions(-)
diff --git a/core/metadata/src/permissioner/permissioner_rules/system.rs b/core/metadata/src/permissioner/permissioner_rules/system.rs
index 720067afb..4a11c8c96 100644
--- a/core/metadata/src/permissioner/permissioner_rules/system.rs
+++ b/core/metadata/src/permissioner/permissioner_rules/system.rs
@@ -32,6 +32,10 @@ impl Permissioner {
self.get_server_info(user_id)
}
+ pub fn get_snapshot(&self, user_id: u32) -> Result<(), IggyError> {
+ self.get_server_info(user_id)
+ }
+
fn get_server_info(&self, user_id: u32) -> Result<(), IggyError> {
if let Some(global_permissions) = self.users_permissions.get(&user_id)
&& (global_permissions.manage_servers || global_permissions.read_servers)
diff --git a/core/metadata/src/stm/user.rs b/core/metadata/src/stm/user.rs
index cda74d0b1..51e51e60c 100644
--- a/core/metadata/src/stm/user.rs
+++ b/core/metadata/src/stm/user.rs
@@ -360,6 +360,16 @@ impl StateHandler for CreateUserRequest {
.personal_access_tokens
.insert(id as UserId, AHashMap::default());
+ // The permissioner is a denormalized index of `User.permissions`;
+ // derive it from the just-stored user so the two never drift.
+ let user_permissions = state
+ .items
+ .get(id)
+ .and_then(|user| user.permissions.as_deref().cloned());
+ state
+ .permissioner
+ .init_permissions_for_user(id as UserId, user_permissions);
+
// Reply body: the SDK `create_user` decodes a `UserDetailsResponse`.
// Serialization local to this state machine.
ApplyReply::ok(
@@ -435,6 +445,9 @@ impl StateHandler for DeleteUserRequest {
}
}
}
+ state
+ .permissioner
+ .delete_permissions_for_user(user_id as UserId);
}
ApplyReply::ok(Bytes::new())
}
@@ -456,6 +469,7 @@ impl StateHandler for ChangePasswordRequest {
impl StateHandler for UpdatePermissionsRequest {
type State = UsersInner;
+ #[allow(clippy::cast_possible_truncation)]
fn apply(&self, state: &mut UsersInner, _timestamp: IggyTimestamp) -> ApplyReply {
let Some(user_id) = state.resolve_user_id(&self.user_id) else {
return ApplyReply::err(UpdatePermissionsResult::UserNotFound);
@@ -467,6 +481,13 @@ impl StateHandler for UpdatePermissionsRequest {
.as_ref()
.map(|p| Arc::new(Permissions::from(p.clone())));
}
+ let user_permissions = state
+ .items
+ .get(user_id)
+ .and_then(|user| user.permissions.as_deref().cloned());
+ state
+ .permissioner
+ .update_permissions_for_user(user_id as UserId, user_permissions);
ApplyReply::ok(Bytes::new())
}
}
@@ -650,43 +671,18 @@ impl Snapshotable for Users {
})
.collect();
+ // The permissioner is a denormalized index of `User.permissions`.
+ // Its `AHashMap`/`AHashSet` fields iterate in a per-replica-random
+ // order, so serializing them would make the snapshot bytes differ
+ // across replicas. Persist empty vecs (kept only for the positional
+ // msgpack layout) and rebuild the index from the users on restore.
let permissioner = PermissionerSnapshot {
- users_permissions: inner
- .permissioner
- .users_permissions
- .iter()
- .map(|(&k, v)| (k, v.clone()))
- .collect(),
- users_streams_permissions: inner
- .permissioner
- .users_streams_permissions
- .iter()
- .map(|(&k, v)| (k, v.clone()))
- .collect(),
- users_that_can_poll_messages_from_all_streams: inner
- .permissioner
- .users_that_can_poll_messages_from_all_streams
- .iter()
- .copied()
- .collect(),
- users_that_can_send_messages_to_all_streams: inner
- .permissioner
- .users_that_can_send_messages_to_all_streams
- .iter()
- .copied()
- .collect(),
- users_that_can_poll_messages_from_specific_streams: inner
- .permissioner
- .users_that_can_poll_messages_from_specific_streams
- .iter()
- .copied()
- .collect(),
- users_that_can_send_messages_to_specific_streams: inner
- .permissioner
- .users_that_can_send_messages_to_specific_streams
- .iter()
- .copied()
- .collect(),
+ users_permissions: Vec::new(),
+ users_streams_permissions: Vec::new(),
+ users_that_can_poll_messages_from_all_streams: Vec::new(),
+ users_that_can_send_messages_to_all_streams: Vec::new(),
+ users_that_can_poll_messages_from_specific_streams: Vec::new(),
+ users_that_can_send_messages_to_specific_streams: Vec::new(),
};
UsersSnapshot {
@@ -751,38 +747,14 @@ impl Snapshotable for Users {
personal_access_tokens.insert(user_id, token_map);
}
- let permissioner = Permissioner {
- users_permissions: snapshot
- .permissioner
- .users_permissions
- .into_iter()
- .collect(),
- users_streams_permissions: snapshot
- .permissioner
- .users_streams_permissions
- .into_iter()
- .collect(),
- users_that_can_poll_messages_from_all_streams: snapshot
- .permissioner
- .users_that_can_poll_messages_from_all_streams
- .into_iter()
- .collect(),
- users_that_can_send_messages_to_all_streams: snapshot
- .permissioner
- .users_that_can_send_messages_to_all_streams
- .into_iter()
- .collect(),
- users_that_can_poll_messages_from_specific_streams: snapshot
- .permissioner
- .users_that_can_poll_messages_from_specific_streams
- .into_iter()
- .collect(),
- users_that_can_send_messages_to_specific_streams: snapshot
- .permissioner
- .users_that_can_send_messages_to_specific_streams
- .into_iter()
- .collect(),
- };
+ // Rebuild the permissioner from the restored users rather than the
+ // snapshot vecs, which are intentionally empty (see `to_snapshot`).
+ // Slab iteration is ascending by key, so the build is deterministic.
+ let mut permissioner = Permissioner::new();
+ for (user_id, user) in &items {
+ permissioner
+ .init_permissions_for_user(user_id as UserId, user.permissions.as_deref().cloned());
+ }
let inner = UsersInner {
index,
@@ -802,6 +774,8 @@ impl_fill_restore!(Users, users);
#[cfg(test)]
mod tests {
use super::*;
+ use iggy_binary_protocol::primitives::permissions::WireStreamPermissions;
+ use iggy_common::IggyError;
#[test]
fn create_pat_request_roundtrip_preserves_user_id() {
@@ -989,4 +963,243 @@ mod tests {
let apply = StateHandler::apply(&request, &mut users, IggyTimestamp::now());
assert_eq!(apply.code, u32::from(DeleteUserResult::UserNotFound));
}
+
+ fn empty_global() -> WireGlobalPermissions {
+ WireGlobalPermissions {
+ manage_servers: false,
+ read_servers: false,
+ manage_users: false,
+ read_users: false,
+ manage_streams: false,
+ read_streams: false,
+ manage_topics: false,
+ read_topics: false,
+ poll_messages: false,
+ send_messages: false,
+ }
+ }
+
+ fn create_user_with(
+ users: &mut UsersInner,
+ username: &str,
+ permissions: Option<WirePermissions>,
+ ) -> UserId {
+ let request = CreateUserRequest {
+ username: WireName::new(username).unwrap(),
+ password: "hash".to_owned(),
+ status: 1,
+ permissions,
+ };
+ let reply = StateHandler::apply(&request, users, IggyTimestamp::now());
+ assert_eq!(reply.code, 0);
+ *users.index.get(username).expect("user created")
+ }
+
+ #[test]
+ fn given_global_poll_grant_when_apply_create_user_should_allow_poll_from_any_stream() {
+ let mut users = UsersInner::new();
+ let mut global = empty_global();
+ global.poll_messages = true;
+ let uid = create_user_with(
+ &mut users,
+ "poller",
+ Some(WirePermissions {
+ global,
+ streams: Vec::new(),
+ }),
+ );
+
+ // The global poll bit denormalizes into the can-poll-all set, so the
+ // user may poll any stream/topic without a stream-specific grant.
+ assert!(users.permissioner.poll_messages(uid, 7, 3).is_ok());
+ assert!(
+ users
+ .permissioner
+ .users_that_can_poll_messages_from_all_streams
+ .contains(&uid)
+ );
+ }
+
+ #[test]
+ fn given_poll_grant_when_apply_update_permissions_removes_it_should_return_unauthorized() {
+ let mut users = UsersInner::new();
+ let mut global = empty_global();
+ global.poll_messages = true;
+ let uid = create_user_with(
+ &mut users,
+ "poller",
+ Some(WirePermissions {
+ global,
+ streams: Vec::new(),
+ }),
+ );
+ assert!(users.permissioner.poll_messages(uid, 1, 1).is_ok());
+
+ let update = UpdatePermissionsRequest {
+ user_id: WireIdentifier::numeric(uid),
+ permissions: None,
+ };
+ let reply = StateHandler::apply(&update, &mut users, IggyTimestamp::now());
+ assert_eq!(reply.code, 0);
+
+ assert!(matches!(
+ users.permissioner.poll_messages(uid, 1, 1),
+ Err(IggyError::Unauthorized)
+ ));
+ assert!(
+ !users
+ .permissioner
+ .users_that_can_poll_messages_from_all_streams
+ .contains(&uid)
+ );
+ }
+
+ #[test]
+ fn given_permissioned_user_when_apply_delete_user_should_clear_all_indexes() {
+ let mut users = UsersInner::new();
+ let mut global = empty_global();
+ global.poll_messages = true;
+ global.send_messages = true;
+ let stream = WireStreamPermissions {
+ stream_id: 4,
+ manage_stream: false,
+ read_stream: false,
+ manage_topics: false,
+ read_topics: false,
+ poll_messages: true,
+ send_messages: true,
+ topics: Vec::new(),
+ };
+ let uid = create_user_with(
+ &mut users,
+ "victim",
+ Some(WirePermissions {
+ global,
+ streams: vec![stream],
+ }),
+ );
+ assert!(!users.permissioner.users_permissions.is_empty());
+
+ let delete = DeleteUserRequest {
+ user_id: WireIdentifier::numeric(uid),
+ };
+ let reply = StateHandler::apply(&delete, &mut users, IggyTimestamp::now());
+ assert_eq!(reply.code, 0);
+
+ let permissioner = &users.permissioner;
+ assert!(permissioner.users_permissions.is_empty());
+ assert!(permissioner.users_streams_permissions.is_empty());
+ assert!(
+ permissioner
+ .users_that_can_poll_messages_from_all_streams
+ .is_empty()
+ );
+ assert!(
+ permissioner
+ .users_that_can_send_messages_to_all_streams
+ .is_empty()
+ );
+ assert!(
+ permissioner
+ .users_that_can_poll_messages_from_specific_streams
+ .is_empty()
+ );
+ assert!(
+ permissioner
+ .users_that_can_send_messages_to_specific_streams
+ .is_empty()
+ );
+ }
+
+ #[test]
+ fn given_populated_permissions_when_snapshot_round_trip_should_rebuild_index_and_persist_empty_vecs()
+ {
+ let mut inner = UsersInner::new();
+ let mut global = empty_global();
+ global.poll_messages = true;
+ global.send_messages = true;
+ let stream = WireStreamPermissions {
+ stream_id: 2,
+ manage_stream: false,
+ read_stream: false,
+ manage_topics: false,
+ read_topics: false,
+ poll_messages: true,
+ send_messages: true,
+ topics: Vec::new(),
+ };
+ let uid = create_user_with(
+ &mut inner,
+ "alice",
+ Some(WirePermissions {
+ global,
+ streams: vec![stream],
+ }),
+ );
+ let pre_poll = inner.permissioner.poll_messages(uid, 9, 9).is_ok();
+ let pre_send = inner.permissioner.append_messages(uid, 2, 0).is_ok();
+
+ let users: Users = inner.into();
+ let snapshot = users.to_snapshot();
+
+ // The derived index is not serialized: every permissioner vec is empty.
+ let snap = &snapshot.permissioner;
+ assert!(snap.users_permissions.is_empty());
+ assert!(snap.users_streams_permissions.is_empty());
+ assert!(
+ snap.users_that_can_poll_messages_from_all_streams
+ .is_empty()
+ );
+ assert!(snap.users_that_can_send_messages_to_all_streams.is_empty());
+ assert!(
+ snap.users_that_can_poll_messages_from_specific_streams
+ .is_empty()
+ );
+ assert!(
+ snap.users_that_can_send_messages_to_specific_streams
+ .is_empty()
+ );
+
+ let restored = Users::from_snapshot(snapshot).expect("snapshot restore");
+ let (poll_ok, send_ok, in_poll_all, in_send_specific) = restored.read(|inner| {
+ let permissioner = &inner.permissioner;
+ (
+ permissioner.poll_messages(uid, 9, 9).is_ok(),
+ permissioner.append_messages(uid, 2, 0).is_ok(),
+ permissioner
+ .users_that_can_poll_messages_from_all_streams
+ .contains(&uid),
+ permissioner
+ .users_that_can_send_messages_to_specific_streams
+ .contains(&(uid, 2)),
+ )
+ });
+
+ // Rule results survive the round-trip...
+ assert_eq!(poll_ok, pre_poll);
+ assert_eq!(send_ok, pre_send);
+ // ...because the index was rebuilt from the restored user: the global
+ // poll bit repopulated the can-poll-all set and the stream send bit
+ // repopulated the specific set.
+ assert!(in_poll_all);
+ assert!(in_send_specific);
+ }
+
+ #[test]
+ fn given_root_user_when_ensure_root_after_bootstrap_should_have_root_grants() {
+ let users = Users::default();
+ users.ensure_root_user("iggy", "hash");
+ // Root carries `Permissions::root()`: full grants across every rule.
+ let (can_create_stream, can_get_stats, can_poll) = users.read(|inner| {
+ let root = *inner.index.get("iggy").expect("root user created");
+ (
+ inner.permissioner.create_stream(root).is_ok(),
+ inner.permissioner.get_stats(root).is_ok(),
+ inner.permissioner.poll_messages(root, 1, 1).is_ok(),
+ )
+ });
+ assert!(can_create_stream);
+ assert!(can_get_stats);
+ assert!(can_poll);
+ }
}
(iggy) 37/43: fix(server-ng): return typed not-found errors on parity reads
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 1d4a046343434826023ce9940e3e985f11d2892d
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 22:07:16 2026 +0200
fix(server-ng): return typed not-found errors on parity reads
Two read builders diverged from the legacy typed errors: topics-list on
a missing stream answered a success-shaped empty list, and consumer-
group list/detail on a missing parent collapsed to the generic
not-found reply. Resolve the parent in the shared read builders and
answer StreamIdNotFound / TopicIdNotFound instead, so HTTP renders the
legacy typed 404 and native TCP surfaces the same code. The TCP
dispatch now sends builder errors on the deny channel instead of
dropping the request, which used to wedge the client until its read
timeout.
---
core/server-ng/src/dispatch.rs | 7 ++++++-
core/server-ng/src/responses.rs | 43 +++++++++++++++++++++++++++++++++++++----
2 files changed, 45 insertions(+), 5 deletions(-)
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index 9f628a710..ce4e64195 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -994,12 +994,17 @@ async fn handle_default_non_replicated(
}
}
Err(error) => {
+ // Surface the builder's typed error (unsupported op, undecodable
+ // body, or a not-found parity read) on the same deny channel the
+ // authz gate uses; a silent drop would wedge the client until its
+ // read timeout.
warn!(
transport_client_id,
code,
error = %error,
- "dropping unsupported non-replicated VSR request"
+ "denying non-replicated VSR request"
);
+ send_non_replicated_deny(shard, request, transport_client_id, error.as_code()).await;
}
}
}
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index d5c9544d6..e41659bd5 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -74,6 +74,7 @@ use iggy_binary_protocol::{
use iggy_common::IggyError;
use metadata::impls::metadata::StreamsFrontend;
use partitions::PollFragments;
+use server::binary::dispatch::wire_id_to_identifier;
use server_common::Message;
use server_common::send_messages2::{COMMAND_HEADER_SIZE, SendMessages2Header};
use server_common::sharding::IggyNamespace;
@@ -429,6 +430,7 @@ pub(crate) fn build_non_replicated_response(
GET_CONSUMER_GROUP_CODE => {
let request = GetConsumerGroupRequest::decode_from(body)
.map_err(|_| IggyError::InvalidCommand)?;
+ ensure_topic_exists(shard, &request.stream_id, &request.topic_id)?;
let response = shard
.plane
.metadata()
@@ -442,6 +444,7 @@ pub(crate) fn build_non_replicated_response(
GET_CONSUMER_GROUPS_CODE => {
let request = GetConsumerGroupsRequest::decode_from(body)
.map_err(|_| IggyError::InvalidCommand)?;
+ ensure_topic_exists(shard, &request.stream_id, &request.topic_id)?;
let groups = shard
.plane
.metadata()
@@ -665,12 +668,11 @@ fn build_get_topics_response(
stream_id: &WireIdentifier,
) -> Result<GetTopicsResponse, IggyError> {
shard.plane.metadata().mux_stm.streams().read(|streams| {
- let Some(stream_id) = resolve_stream_id(streams, stream_id) else {
- return Ok(GetTopicsResponse { topics: Vec::new() });
- };
+ let resolved_stream =
+ resolve_stream_id(streams, stream_id).ok_or_else(|| stream_not_found(stream_id))?;
let stream = streams
.items
- .get(stream_id)
+ .get(resolved_stream)
.ok_or(IggyError::InvalidIdentifier)?;
stream
.topics
@@ -681,6 +683,39 @@ fn build_get_topics_response(
})
}
+/// Reject a consumer-group read whose parent stream/topic is absent with the
+/// legacy typed error naming the level that missed; the group itself missing
+/// stays the shared not-found reply (empty over TCP, 404 over HTTP).
+fn ensure_topic_exists(
+ shard: &Rc<ServerNgShard>,
+ stream_id: &WireIdentifier,
+ topic_id: &WireIdentifier,
+) -> Result<(), IggyError> {
+ shard.plane.metadata().mux_stm.streams().read(|streams| {
+ let resolved_stream =
+ resolve_stream_id(streams, stream_id).ok_or_else(|| stream_not_found(stream_id))?;
+ resolve_topic_id(streams, resolved_stream, topic_id)
+ .ok_or_else(|| topic_not_found(stream_id, topic_id))?;
+ Ok(())
+ })
+}
+
+/// Typed miss for a read's parent stream, matching the legacy servers' error
+/// shape. The identifier only feeds the error message; a wire form with no
+/// domain equivalent (numeric 0 is a live slab id here but not a legacy id)
+/// falls back to the default identifier.
+fn stream_not_found(stream_id: &WireIdentifier) -> IggyError {
+ IggyError::StreamIdNotFound(wire_id_to_identifier(stream_id).unwrap_or_default())
+}
+
+/// Typed miss for a read's parent topic; see [`stream_not_found`].
+fn topic_not_found(stream_id: &WireIdentifier, topic_id: &WireIdentifier) -> IggyError {
+ IggyError::TopicIdNotFound(
+ wire_id_to_identifier(stream_id).unwrap_or_default(),
+ wire_id_to_identifier(topic_id).unwrap_or_default(),
+ )
+}
+
pub(crate) fn resolve_stream_id(
streams: &metadata::stm::stream::StreamsInner,
identifier: &WireIdentifier,
(iggy) 02/43: feat(server-ng): add HTTP listener with login and session binding
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit d1e862dbd5ebc29b892faf2d5fda306fc33556b7
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 11:18:19 2026 +0200
feat(server-ng): add HTTP listener with login and session binding
server-ng had no HTTP/REST surface; management and data access went
through the native binary transports only.
This adds a cyper-axum listener on shard 0 (mirroring the existing
QUIC/TLS placement) serving GET /ping plus JWT and personal-access-
token login. The dispatch spine drops any request without a bound VSR
session, so an authenticated caller cannot submit until a Register
commit binds one. The listener establishes that session lazily on the
first authenticated request and caches it per credential (JWT jti /
PAT hash) rather than per connection, because the physical HTTP socket
is the wrong unit under connection pooling and HTTP/2 multiplexing.
Credential verification is stateless; the Rc-based, !Send session
future is bridged to axum's Send bound with SendWrapper, sound because
compio pins every handler to shard 0's thread. Control writes and the
data plane are not wired yet.
---
core/server-ng/Cargo.toml | 1 +
core/server-ng/src/auth.rs | 18 +-
core/server-ng/src/bootstrap.rs | 33 ++-
core/server-ng/src/http.rs | 379 +++++++++++++++++++++++++++++++++++
core/server-ng/src/http/extractor.rs | 111 ++++++++++
core/server-ng/src/http/jwt.rs | 175 ++++++++++++++++
core/server-ng/src/lib.rs | 1 +
7 files changed, 706 insertions(+), 12 deletions(-)
diff --git a/core/server-ng/Cargo.toml b/core/server-ng/Cargo.toml
index bf42650ed..fd9193c5b 100644
--- a/core/server-ng/Cargo.toml
+++ b/core/server-ng/Cargo.toml
@@ -155,6 +155,7 @@ strum = { workspace = true }
sysinfo = { workspace = true }
tempfile = { workspace = true }
thiserror = { workspace = true }
+tokio = { workspace = true }
toml = { workspace = true }
tower-http = { workspace = true }
tracing = { workspace = true }
diff --git a/core/server-ng/src/auth.rs b/core/server-ng/src/auth.rs
index c02b2ca23..a177b263d 100644
--- a/core/server-ng/src/auth.rs
+++ b/core/server-ng/src/auth.rs
@@ -74,6 +74,17 @@ pub(crate) fn verify_pat_credentials(
shard: &Rc<ServerNgShard>,
token: &str,
) -> Result<u32, LoginRegisterError> {
+ verify_pat_credentials_with_expiry(shard, token).map(|(user_id, _)| user_id)
+}
+
+/// Like [`verify_pat_credentials`] but also surfaces the token's expiry (unix
+/// seconds, `u64::MAX` when the PAT never expires). The HTTP extractor keys a
+/// per-token VSR session table on this expiry for lazy eviction; the wire and
+/// login paths only need the user id and go through [`verify_pat_credentials`].
+pub(crate) fn verify_pat_credentials_with_expiry(
+ shard: &Rc<ServerNgShard>,
+ token: &str,
+) -> Result<(u32, u64), LoginRegisterError> {
let token_hash = PersonalAccessToken::hash_token(token);
let now = IggyTimestamp::now();
shard.plane.metadata().mux_stm.users().read(|users| {
@@ -98,7 +109,12 @@ pub(crate) fn verify_pat_credentials(
if user.status != UserStatus::Active {
return Err(LoginRegisterError::UserInactive);
}
- Ok(user.id)
+ // `expiry_at == None` is a never-expiring PAT; map it to `u64::MAX` so
+ // the HTTP session table never expiry-evicts its entry.
+ let expiry = pat
+ .expiry_at
+ .map_or(u64::MAX, |expiry_at| expiry_at.to_secs());
+ Ok((user.id, expiry))
})
}
diff --git a/core/server-ng/src/bootstrap.rs b/core/server-ng/src/bootstrap.rs
index ab82265cc..aa3d4fc0b 100644
--- a/core/server-ng/src/bootstrap.rs
+++ b/core/server-ng/src/bootstrap.rs
@@ -21,6 +21,7 @@ use crate::dispatch::{
make_deferred_replica_message_handler, make_list_clients_handler, make_metadata_submit_handler,
make_partition_read_handler,
};
+use crate::http;
use crate::partition_helpers::{
configure_consumer_offsets, ensure_initial_segment, validate_namespace_bounds,
};
@@ -1919,7 +1920,7 @@ async fn start_tcp_runtime(
accepted_clients: LocalClientAcceptFns,
) -> Result<(), ServerNgError> {
if config.tcp.enabled && !config.tcp.tls.enabled {
- return start_via_replica_io(
+ start_via_replica_io(
shard,
config,
topology,
@@ -1927,18 +1928,28 @@ async fn start_tcp_runtime(
dialed_replica,
accepted_clients,
)
- .await;
+ .await?;
+ } else {
+ start_manual_runtime(
+ shard,
+ config,
+ topology,
+ accepted_replica,
+ dialed_replica,
+ accepted_clients,
+ )
+ .await?;
}
- start_manual_runtime(
- shard,
- config,
- topology,
- accepted_replica,
- dialed_replica,
- accepted_clients,
- )
- .await
+ // HTTP is served over TCP but sits outside the replica_io / manual client
+ // reactor, so it binds independently. Shard-0 gating comes from the sole
+ // caller of this function.
+ if config.http.enabled {
+ let http_addr = parse_socket_addr("http.address", &config.http.address)?;
+ http::start(shard, http_addr, &config.http.jwt).await?;
+ }
+
+ Ok(())
}
async fn start_via_replica_io(
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
new file mode 100644
index 000000000..e4f2d58f4
--- /dev/null
+++ b/core/server-ng/src/http.rs
@@ -0,0 +1,379 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Shard-0 HTTP/REST listener: state bridge, router, and the auth-issuance
+//! half (login + bearer extraction). Management and data-plane routes land
+//! later; this module owns only health, login, and the authenticated probe.
+
+mod extractor;
+mod jwt;
+
+use std::cell::RefCell;
+use std::collections::HashMap;
+use std::net::SocketAddr;
+use std::rc::Rc;
+
+use axum::extract::State;
+use axum::http::StatusCode;
+use axum::response::{IntoResponse, Response};
+use axum::routing::{get, post};
+use axum::{Json, Router};
+use configs::http::HttpJwtConfig;
+use iggy_common::login_user::LoginUser;
+use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
+use iggy_common::{IdentityInfo, IggyError, IggyTimestamp, TokenInfo};
+use message_bus::client_listener;
+use secrecy::ExposeSecret;
+use send_wrapper::SendWrapper;
+use server::http::error::{CustomError, ErrorResponse};
+use tokio::sync::Mutex;
+use tracing::{error, info, warn};
+
+use crate::auth::{verify_login_credentials, verify_pat_credentials};
+use crate::bootstrap::ServerNgShard;
+use crate::dispatch::submit_register_on_owner;
+use crate::http::extractor::Authenticated;
+use crate::http::jwt::JwtManager;
+use crate::login_register::LoginRegisterError;
+use crate::server_error::ServerNgError;
+
+/// `GET /ping` response body, matching the legacy HTTP server's health probe.
+const PONG: &str = "pong";
+
+/// Hard cap on live per-credential sessions. A leak-guard, not a tuning knob:
+/// reaching it means this many distinct live tokens are in flight at once
+/// (pathological at P1 volume). New sessions past the cap are refused with a
+/// transient 503 rather than evicting a live one; the client retries. Expired
+/// entries are dropped first, so the cap only bites on live oversubscription.
+const MAX_HTTP_SESSIONS: usize = 100_000;
+
+/// First per-session request id the write path (2c) will hand out. VSR request
+/// numbers are 1-based and strictly increasing within a session.
+const FIRST_REQUEST_ID: u64 = 1;
+
+/// One VSR session established for a single login credential (a JWT `jti` or a
+/// PAT). Shared via `Rc` by every concurrent request bearing that credential,
+/// so the session granularity is per-login.
+struct HttpSession {
+ /// Shard-0 client id minted for this credential; its top 16 bits are 0, so
+ /// it shares the shard-0 id space with TCP virtual clients without
+ /// colliding. Consumed by the write path (2c).
+ #[expect(
+ dead_code,
+ reason = "read by the write path (2c) to fill RequestHeader.client"
+ )]
+ client_id: u128,
+ /// Cluster session number returned by the VSR `Register` commit. Consumed
+ /// by the write path (2c).
+ #[expect(
+ dead_code,
+ reason = "read by the write path (2c) to fill RequestHeader.session"
+ )]
+ session: u64,
+ /// User the credential authenticated as. Consumed by the write path (2c).
+ #[expect(dead_code, reason = "read by the write path (2c) for authorization")]
+ user_id: u32,
+ /// Credential expiry in unix seconds (`u64::MAX` = never). Drives lazy
+ /// eviction of stale table entries.
+ expiry: u64,
+ /// Serializes the write path (2c): the guarded value is the NEXT request
+ /// id. A `tokio::sync::Mutex` because 2c holds it across the submit `.await`
+ /// so each session's request numbers stay monotonic and gap-free.
+ #[expect(
+ dead_code,
+ reason = "locked by the write path (2c) to serialize per-session writes"
+ )]
+ gate: Mutex<u64>,
+}
+
+/// Rejection for protected routes.
+///
+/// Two failure classes get two statuses: a missing, invalid, or expired
+/// credential is the caller's fault (401, rendered exactly like the legacy
+/// server so SDK error bodies match), while a VSR `Register` that cannot commit
+/// right now is a transient server condition (503) and must never masquerade as
+/// an auth failure.
+pub enum AuthError {
+ Unauthenticated(IggyError),
+ SessionUnavailable,
+}
+
+impl From<IggyError> for AuthError {
+ fn from(error: IggyError) -> Self {
+ Self::Unauthenticated(error)
+ }
+}
+
+impl IntoResponse for AuthError {
+ fn into_response(self) -> Response {
+ match self {
+ // Reuse the legacy `IggyError -> CustomError` rendering so 401
+ // bodies stay byte-identical to what the SDKs are tested against.
+ Self::Unauthenticated(error) => CustomError::from(error).into_response(),
+ // Register could not commit (no caught-up primary, pipeline full, or
+ // a view-change cancel). Transient server condition -> 503, retryable.
+ Self::SessionUnavailable => (
+ StatusCode::SERVICE_UNAVAILABLE,
+ Json(ErrorResponse::from_error(
+ IggyError::CannotEstablishConnection,
+ )),
+ )
+ .into_response(),
+ }
+ }
+}
+
+/// Shared shard-0 HTTP state.
+///
+/// Groups the shard handle, the JWT issuer/verifier, and the per-credential VSR
+/// session table so every handler and the [`Authenticated`] extractor reach
+/// them through one axum `State`.
+struct HttpInner {
+ shard: Rc<ServerNgShard>,
+ jwt: JwtManager,
+ /// Per-credential VSR sessions keyed by JWT `jti` / PAT hash. `RefCell` is
+ /// sound here - shard 0 is single-threaded and the `SendWrapper` state
+ /// bridge tolerates the `!Sync` interior - but the guard must never be held
+ /// across an `.await` (see [`HttpInner::resolve_session`]).
+ sessions: RefCell<HashMap<String, Rc<HttpSession>>>,
+}
+
+impl HttpInner {
+ /// Resolve the VSR session for `key`, minting and Registering one on first
+ /// use. Every later request bearing the same credential reuses it.
+ ///
+ /// Borrow discipline: the `RefCell` table guard is taken, read, and dropped
+ /// WITHOUT crossing the `.await`. Holding it across the Register suspend
+ /// would panic the moment a sibling shard-0 task borrowed the table while
+ /// this one is parked (single-threaded `RefCell` + cooperative scheduling).
+ async fn resolve_session(
+ &self,
+ key: String,
+ user_id: u32,
+ expiry: u64,
+ ) -> Result<Rc<HttpSession>, AuthError> {
+ let now = IggyTimestamp::now().to_secs();
+ if let Some(session) = self.live_session(&key, now) {
+ return Ok(session);
+ }
+
+ // Miss: mint + Register with no borrow held (an async VSR commit).
+ let fresh = self.register_session(user_id, expiry).await?;
+
+ let mut table = self.sessions.borrow_mut();
+ // A concurrent first request for the same key may have Registered and
+ // inserted while we were parked. Last-writer-wins: reuse the installed
+ // entry and drop `fresh`, whose client id is then orphaned on the peers
+ // until they evict it - an accepted rare cost at P1 volume.
+ if let Some(session) = live_entry(&table, &key, now) {
+ return Ok(session);
+ }
+ table.retain(|_, session| session.expiry > now);
+ if table.len() >= MAX_HTTP_SESSIONS {
+ // Still full after dropping expired entries: too many genuinely live
+ // sessions. Refuse rather than evict a live one; the client retries.
+ return Err(AuthError::SessionUnavailable);
+ }
+ table.insert(key, Rc::clone(&fresh));
+ Ok(fresh)
+ }
+
+ /// Clone the live (non-expired) entry for `key`, if present. Confines the
+ /// shared `RefCell` borrow to this call so it can never span an `.await`.
+ fn live_session(&self, key: &str, now_secs: u64) -> Option<Rc<HttpSession>> {
+ live_entry(&self.sessions.borrow(), key, now_secs)
+ }
+
+ /// Mint a shard-0 client id and run the VSR `Register` for a fresh session.
+ /// Holds no table borrow; the caller inserts the result.
+ async fn register_session(
+ &self,
+ user_id: u32,
+ expiry: u64,
+ ) -> Result<Rc<HttpSession>, AuthError> {
+ let coordinator = self
+ .shard
+ .coordinator()
+ .ok_or(AuthError::SessionUnavailable)?;
+ // Reuse the TCP accept path's minter: it draws from the same shard-0
+ // `client_seq`, so an HTTP session id can never collide with a TCP
+ // virtual client's and the shard-0 tag (top 16 bits == 0) is preserved.
+ let client_id = coordinator.mint_shard_zero_client_id();
+ // The minter seeds at 1, so 0 is only reachable after a 2^112 wrap.
+ // Guard anyway: `submit_register_in_process` asserts `client_id != 0`,
+ // and an assert on this request path would be a panic.
+ if client_id == 0 {
+ return Err(AuthError::SessionUnavailable);
+ }
+ // Shared Register entry point; on shard 0 (always, for HTTP) it runs
+ // `submit_register_in_process` directly on the metadata owner.
+ let session = submit_register_on_owner(&self.shard, client_id)
+ .await
+ .map_err(|error| {
+ warn!(?error, "server-ng HTTP: VSR Register submit failed");
+ AuthError::SessionUnavailable
+ })?;
+ Ok(Rc::new(HttpSession {
+ client_id,
+ session,
+ user_id,
+ expiry,
+ gate: Mutex::new(FIRST_REQUEST_ID),
+ }))
+ }
+}
+
+/// Borrow-and-clone a live table entry, or `None` if missing or expired. Shared
+/// by the fast path and the post-Register re-check so neither leaks a guard.
+fn live_entry(
+ table: &HashMap<String, Rc<HttpSession>>,
+ key: &str,
+ now_secs: u64,
+) -> Option<Rc<HttpSession>> {
+ table
+ .get(key)
+ .filter(|session| session.expiry > now_secs)
+ .map(Rc::clone)
+}
+
+/// Axum router state: shard-0's [`HttpInner`] behind an `Rc`, `!Send` yet
+/// bridged into axum's `Send + Sync` requirement by `SendWrapper`. Sound
+/// because the listener and every handler run on shard 0's compio thread - the
+/// same thread that builds this state. Never touch it off that thread.
+type HttpState = SendWrapper<Rc<HttpInner>>;
+
+/// Bind the shard-0 HTTP listener and spawn the `cyper-axum` serve loop as a
+/// background task on shard 0's compio runtime.
+///
+/// The caller gates this to shard 0 and to `http.enabled`; the listener stops
+/// when the bus shutdown token fires.
+///
+/// # Errors
+///
+/// Returns [`ServerNgError`] if the JWT manager cannot be built from
+/// `jwt_config` or the listener cannot bind to `addr`.
+pub async fn start(
+ shard: &Rc<ServerNgShard>,
+ addr: SocketAddr,
+ jwt_config: &HttpJwtConfig,
+) -> Result<(), ServerNgError> {
+ let jwt = JwtManager::build(jwt_config)?;
+ let (listener, bound_addr) = client_listener::tcp::bind(addr).await?;
+ info!(address = %bound_addr, "server-ng HTTP listener started");
+
+ let state: HttpState = SendWrapper::new(Rc::new(HttpInner {
+ shard: Rc::clone(shard),
+ jwt,
+ sessions: RefCell::new(HashMap::new()),
+ }));
+ let router = router(state);
+
+ let shutdown = shard.bus.token();
+ let handle = compio::runtime::spawn(async move {
+ if let Err(error) = cyper_axum::serve(listener, router)
+ .with_graceful_shutdown(async move { shutdown.wait().await })
+ .await
+ {
+ error!(%error, "server-ng HTTP listener terminated with error");
+ }
+ });
+ shard.bus.track_background(handle);
+
+ Ok(())
+}
+
+/// Assemble the shard-0 router: unauthenticated health + login routes, plus
+/// one authenticated probe that exercises the [`Authenticated`] extractor.
+fn router(state: HttpState) -> Router {
+ Router::new()
+ .route("/ping", get(ping))
+ .route("/users/login", post(login_user))
+ .route(
+ "/personal-access-tokens/login",
+ post(login_with_personal_access_token),
+ )
+ .route("/users/me", get(get_me))
+ .with_state(state)
+}
+
+/// Extracting the state here proves at compile time that the `!Send` state
+/// bridges into axum's `Send + Sync` router state on shard 0's compio thread.
+/// Ping needs no state, so it is discarded.
+async fn ping(State(_state): State<HttpState>) -> &'static str {
+ PONG
+}
+
+async fn login_user(
+ State(state): State<HttpState>,
+ Json(command): Json<LoginUser>,
+) -> Result<Json<IdentityInfo>, CustomError> {
+ let user_id = verify_login_credentials(
+ &state.shard,
+ &command.username,
+ command.password.expose_secret(),
+ )
+ .map_err(|error| login_error_to_iggy(&error))?;
+ issue_identity(&state, user_id)
+}
+
+async fn login_with_personal_access_token(
+ State(state): State<HttpState>,
+ Json(command): Json<LoginWithPersonalAccessToken>,
+) -> Result<Json<IdentityInfo>, CustomError> {
+ let user_id = verify_pat_credentials(&state.shard, command.token.expose_secret())
+ .map_err(|error| login_error_to_iggy(&error))?;
+ issue_identity(&state, user_id)
+}
+
+/// Interim authenticated probe: echoes the caller's resolved `user_id` so the
+/// [`Authenticated`] extractor is covered end to end. Iggy exposes no "current
+/// identity" route to reuse. This path also shadows a future
+/// `GET /users/{user_id}` and must be reconciled when that route is added.
+async fn get_me(identity: Authenticated) -> Json<IdentityInfo> {
+ Json(IdentityInfo {
+ user_id: identity.user_id,
+ access_token: None,
+ })
+}
+
+/// Issue a fresh access token for `user_id` and wrap it in the exact
+/// `IdentityInfo` shape the SDKs pin: numeric `user_id` plus an `access_token`
+/// carrying the token string and its unix-seconds expiry.
+fn issue_identity(inner: &HttpInner, user_id: u32) -> Result<Json<IdentityInfo>, CustomError> {
+ let generated = inner.jwt.generate(user_id)?;
+ Ok(Json(IdentityInfo {
+ user_id: generated.user_id,
+ access_token: Some(TokenInfo {
+ token: generated.access_token,
+ expiry: generated.access_token_expiry,
+ }),
+ }))
+}
+
+/// Grade a credential-verification failure onto the `IggyError` the legacy
+/// HTTP error map already maps to a status + body, so `CustomError` renders
+/// what the SDKs are tested against. `verify_*` only yield the first three
+/// variants; the tail is unreachable but kept terminal (401) for the
+/// `#[non_exhaustive]` enum.
+const fn login_error_to_iggy(error: &LoginRegisterError) -> IggyError {
+ match error {
+ LoginRegisterError::InvalidCredentials => IggyError::InvalidCredentials,
+ LoginRegisterError::InvalidToken => IggyError::InvalidPersonalAccessToken,
+ LoginRegisterError::UserInactive => IggyError::UserInactive,
+ _ => IggyError::Unauthenticated,
+ }
+}
diff --git a/core/server-ng/src/http/extractor.rs b/core/server-ng/src/http/extractor.rs
new file mode 100644
index 000000000..fd2e90051
--- /dev/null
+++ b/core/server-ng/src/http/extractor.rs
@@ -0,0 +1,111 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Bearer-credential extractor for protected shard-0 HTTP routes.
+
+use std::rc::Rc;
+
+use axum::extract::FromRequestParts;
+use axum::http::header::AUTHORIZATION;
+use axum::http::request::Parts;
+use iggy_common::{IggyError, PersonalAccessToken};
+use send_wrapper::SendWrapper;
+
+use super::{AuthError, HttpSession, HttpState};
+use crate::auth::verify_pat_credentials_with_expiry;
+
+/// Bearer scheme prefix in the `Authorization` header.
+const BEARER: &str = "Bearer ";
+
+/// Key-space prefixes: a JWT `jti` and a PAT hash live in disjoint namespaces
+/// so the two credential kinds can never collide on the same table key.
+const JWT_KEY_PREFIX: &str = "jwt:";
+const PAT_KEY_PREFIX: &str = "pat:";
+
+/// Resolved caller identity for a protected route: the numeric user id plus the
+/// shared VSR session established once per presenting credential.
+pub struct Authenticated {
+ pub user_id: u32,
+ /// Per-credential VSR session. The write path (2c) reads its client id and
+ /// session number and serializes writes through its gate; `get_me` and
+ /// other read-only probes ignore it.
+ ///
+ /// Wrapped because axum requires every handler argument to be `Send` and
+ /// `Rc` is not; this mirrors the `HttpState` bridge. Sound for the same
+ /// reason: the extractor mints it on shard 0's compio thread and every
+ /// handler runs on that same thread, so the wrapper is never crossed.
+ #[expect(
+ dead_code,
+ reason = "read by the write path (2c) via client id, session, and gate"
+ )]
+ pub session: SendWrapper<Rc<HttpSession>>,
+}
+
+impl FromRequestParts<HttpState> for Authenticated {
+ type Rejection = AuthError;
+
+ async fn from_request_parts(
+ parts: &mut Parts,
+ state: &HttpState,
+ ) -> Result<Self, Self::Rejection> {
+ let bearer = parts
+ .headers
+ .get(AUTHORIZATION)
+ .and_then(|value| value.to_str().ok())
+ .and_then(|value| value.strip_prefix(BEARER))
+ .ok_or(IggyError::AccessTokenMissing)?;
+
+ let (key, user_id, expiry) = resolve_credential(state, bearer)?;
+
+ // `resolve_session` is `Rc`-based and `!Send`, yet axum requires this
+ // extractor future to be `Send`. `SendWrapper` bridges the gap: sound
+ // because compio pins the future to shard 0's thread - the only thread
+ // that ever touches the session table (mirrors legacy `HttpSafeShard`).
+ let session = SendWrapper::new(state.resolve_session(key, user_id, expiry)).await?;
+ Ok(Self {
+ user_id,
+ session: SendWrapper::new(session),
+ })
+ }
+}
+
+/// Map a bearer credential to its `(session key, user id, expiry secs)`.
+///
+/// A JWT is the primary credential, keyed by its `jti`. A raw PAT presented as
+/// the bearer is the documented fallback, keyed by its SHA-256 hash - the same
+/// digest Iggy already indexes PATs by, so the key is stable and collision-free
+/// while the raw secret never enters the table. Both checks are local and
+/// synchronous, so no borrow is held across an await here.
+fn resolve_credential(state: &HttpState, bearer: &str) -> Result<(String, u32, u64), AuthError> {
+ if let Ok(claims) = state.jwt.decode(bearer)
+ && let Ok(user_id) = claims.sub.parse::<u32>()
+ {
+ return Ok((
+ format!("{JWT_KEY_PREFIX}{}", claims.jti),
+ user_id,
+ claims.exp,
+ ));
+ }
+
+ let (user_id, expiry) = verify_pat_credentials_with_expiry(&state.shard, bearer)
+ .map_err(|_| IggyError::Unauthenticated)?;
+ let key = format!(
+ "{PAT_KEY_PREFIX}{}",
+ PersonalAccessToken::hash_token(bearer)
+ );
+ Ok((key, user_id, expiry))
+}
diff --git a/core/server-ng/src/http/jwt.rs b/core/server-ng/src/http/jwt.rs
new file mode 100644
index 000000000..1304ef692
--- /dev/null
+++ b/core/server-ng/src/http/jwt.rs
@@ -0,0 +1,175 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Minimal JWT issuer/verifier for the shard-0 HTTP listener.
+//!
+//! Ported from the legacy `server::http::jwt::jwt_manager::JwtManager`, reduced
+//! to the issue + verify half: no revoked-token persistence, no JWKS /
+//! trusted-issuer path, no refresh. The claim set (`JwtClaims`, including the
+//! `jti`) and the `IggyError -> HTTP` grading are reused verbatim so issued
+//! tokens and error bodies stay identical to the legacy server.
+
+use std::ops::Range;
+
+use configs::http::HttpJwtConfig;
+use iggy_common::{IggyDuration, IggyError, IggyExpiry, IggyTimestamp, UserId};
+use jsonwebtoken::{Algorithm, DecodingKey, EncodingKey, Header, Validation, decode, encode};
+use server::http::jwt::json_web_token::{Audience, GeneratedToken, JwtClaims};
+use server_common::crypto;
+use tracing::warn;
+use uuid::Uuid;
+
+/// Length window for the random secret minted when no secret is configured.
+/// Matches the legacy server so both behave identically on an empty secret.
+const GENERATED_SECRET_LEN: Range<usize> = 32..64;
+
+/// Expiry stamp used for a non-expiring token: far enough out to never trip
+/// `exp` validation, small enough to fit `u32`. Mirrors the legacy server.
+const NEVER_EXPIRE_SECS: u32 = 1_000_000_000;
+
+/// Fixed-lifetime issuer/verifier for HTTP bearer tokens on shard 0.
+pub struct JwtManager {
+ algorithm: Algorithm,
+ issuer: String,
+ audience: String,
+ access_token_expiry: IggyExpiry,
+ not_before: IggyDuration,
+ encoding_key: EncodingKey,
+ decoding_key: DecodingKey,
+ validation: Validation,
+}
+
+impl JwtManager {
+ /// Build the manager from the `[http.jwt]` config, normalizing an
+ /// unconfigured secret the same way the legacy server does (mint a random
+ /// ephemeral secret, or mirror whichever half is set).
+ ///
+ /// # Errors
+ ///
+ /// Returns [`IggyError`] if the configured algorithm is unsupported or a
+ /// secret cannot be turned into an encoding/decoding key.
+ pub fn build(config: &HttpJwtConfig) -> Result<Self, IggyError> {
+ let config = normalize_secrets(config.clone());
+ let algorithm = config.get_algorithm()?;
+ let encoding_key = config.get_encoding_key()?;
+ let decoding_key = config.get_decoding_key()?;
+ let mut validation = Validation::new(algorithm);
+ validation.set_issuer(&config.valid_issuers);
+ validation.set_audience(&config.valid_audiences);
+ validation.leeway = u64::from(config.clock_skew.as_secs());
+ Ok(Self {
+ algorithm,
+ issuer: config.issuer,
+ audience: config.audience,
+ access_token_expiry: config.access_token_expiry,
+ not_before: config.not_before,
+ encoding_key,
+ decoding_key,
+ validation,
+ })
+ }
+
+ /// Issue a signed access token for `user_id` carrying a unique `jti` plus
+ /// issued-at / not-before / expiry stamps derived from the config.
+ ///
+ /// # Errors
+ ///
+ /// Returns [`IggyError::CannotGenerateJwt`] if signing fails.
+ pub fn generate(&self, user_id: UserId) -> Result<GeneratedToken, IggyError> {
+ let header = Header::new(self.algorithm);
+ let iat = IggyTimestamp::now().to_secs();
+ let expiry_secs = match self.access_token_expiry {
+ IggyExpiry::NeverExpire => NEVER_EXPIRE_SECS,
+ IggyExpiry::ServerDefault => 0,
+ IggyExpiry::ExpireDuration(duration) => duration.as_secs(),
+ };
+ let exp = iat + u64::from(expiry_secs);
+ let nbf = iat + u64::from(self.not_before.as_secs());
+ let claims = JwtClaims {
+ jti: Uuid::now_v7().to_string(),
+ sub: user_id.to_string(),
+ aud: Audience::from(self.audience.clone()),
+ iss: self.issuer.clone(),
+ iat,
+ exp,
+ nbf,
+ };
+ let access_token = encode::<JwtClaims>(&header, &claims, &self.encoding_key)
+ .map_err(|_| IggyError::CannotGenerateJwt)?;
+ Ok(GeneratedToken {
+ user_id,
+ access_token,
+ access_token_expiry: exp,
+ })
+ }
+
+ /// Verify a token's signature and registered claims, returning its claim
+ /// set (the `jti` is what a session table keys on).
+ ///
+ /// # Errors
+ ///
+ /// Returns [`IggyError::Unauthenticated`] if the token is malformed,
+ /// expired, or fails issuer/audience/signature validation.
+ pub fn decode(&self, token: &str) -> Result<JwtClaims, IggyError> {
+ decode::<JwtClaims>(token, &self.decoding_key, &self.validation)
+ .map(|data| data.claims)
+ .map_err(|_| IggyError::Unauthenticated)
+ }
+}
+
+/// Fill in an unconfigured JWT secret exactly like the legacy HTTP server:
+/// both empty -> one random ephemeral secret; one empty -> mirror the other;
+/// both set but different under an HMAC algorithm -> warn (they must match).
+fn normalize_secrets(mut config: HttpJwtConfig) -> HttpJwtConfig {
+ match (
+ config.encoding_secret.is_empty(),
+ config.decoding_secret.is_empty(),
+ ) {
+ (true, true) => {
+ let secret = crypto::generate_secret(GENERATED_SECRET_LEN);
+ let redacted: String = secret.chars().take(3).collect();
+ warn!(
+ "JWT encoding and decoding secrets are not configured - generated a random secret: {redacted}***. JWT tokens will be invalidated on server restart. Set 'encoding_secret' and 'decoding_secret' in the config to use persistent secrets."
+ );
+ config.encoding_secret.clone_from(&secret);
+ config.decoding_secret = secret;
+ }
+ (true, false) => {
+ warn!(
+ "JWT encoding secret is not configured but decoding secret is set - using decoding secret for both. Set 'encoding_secret' in the config to avoid this warning."
+ );
+ config.encoding_secret = config.decoding_secret.clone();
+ }
+ (false, true) => {
+ warn!(
+ "JWT decoding secret is not configured but encoding secret is set - using encoding secret for both. Set 'decoding_secret' in the config to avoid this warning."
+ );
+ config.decoding_secret = config.encoding_secret.clone();
+ }
+ (false, false) => {
+ if config.encoding_secret != config.decoding_secret
+ && config.algorithm.starts_with("HS")
+ {
+ warn!(
+ "JWT encoding and decoding secrets are different but algorithm is {} (HMAC) - both secrets must be identical for symmetric algorithms.",
+ config.algorithm
+ );
+ }
+ }
+ }
+ config
+}
diff --git a/core/server-ng/src/lib.rs b/core/server-ng/src/lib.rs
index 78d9847b8..cd212e98b 100644
--- a/core/server-ng/src/lib.rs
+++ b/core/server-ng/src/lib.rs
@@ -22,6 +22,7 @@ pub mod bootstrap;
pub mod config_writer;
pub mod consumer_group;
pub mod dispatch;
+pub(crate) mod http;
pub mod login_register;
pub mod partition_helpers;
pub mod partition_reconciler;
(iggy) 23/43: fix(metadata): make acting-user stamping single-site and fail-closed
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 11bf9ccb46cb21b42c1e024029cdff7d45e51892
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 14:41:01 2026 +0200
fix(metadata): make acting-user stamping single-site and fail-closed
The RBAC apply gate trusts PrepareHeader.user_id, but only
submit_request_in_process stamped it. The wire ingresses (on_request,
drain_request_queue_into_prepares) forwarded the wire-supplied value,
and a missing ClientTable session defaulted to 0, which the gate's
root short-circuit treats as allow-all. Both paths failed open.
Move the stamp into prepare_request, the funnel every client-op
prepare passes through on the primary, so no ingress can skip it.
Register and Logout are exempt: Register carries the login identity
itself and neither op is gated. A missing session now rejects with
Unauthenticated instead of defaulting to root. A new test drives the
real prepare_request and asserts the built prepare carries the
ClientTable identity, not the wire value.
---
core/metadata/src/impls/metadata.rs | 280 +++++++++++++++++++++++++++++++-----
1 file changed, 248 insertions(+), 32 deletions(-)
diff --git a/core/metadata/src/impls/metadata.rs b/core/metadata/src/impls/metadata.rs
index 8667818f8..d638c420b 100644
--- a/core/metadata/src/impls/metadata.rs
+++ b/core/metadata/src/impls/metadata.rs
@@ -1294,7 +1294,7 @@ where
#[allow(clippy::future_not_send)]
pub async fn submit_request_in_process(
&self,
- mut message: Message<RequestHeader>,
+ message: Message<RequestHeader>,
) -> Result<Message<GenericHeader>, MetadataSubmitError> {
let request_header = *message.header();
let client_id = request_header.client;
@@ -1343,33 +1343,9 @@ where
return Err(MetadataSubmitError::PipelineFull);
}
- // Stamp the acting user id into the request header for the in-apply RBAC
- // gate, overwriting any wire-supplied value (never trusted). The cluster
- // resolves session -> user at Register (f1) and the gate must run
- // identically on every replica, so identity rides the replicated prepare
- // rather than a per-node session table. `request_preflight` above proved
- // the session is live, so `get_user_id` is `Some`; the `unwrap_or(0)` is
- // an unreachable guard, not a root grant (0 is only root's genuine id).
- //
- // Kept here rather than hoisted into the shared `prepare_request`:
- // `Register` and `Logout` funnel through it too, and a mid-`Register`
- // client has no committed session yet (`get_user_id` is `None`), so a
- // blanket stamp there would zero `Register`'s f1 user id. The wire
- // ingresses (`on_request`, `drain_request_queue_into_prepares`) are not
- // client-reachable today; closing that latent gap needs an op-guarded
- // stamp, tracked separately.
- let acting_user_id = self
- .client_table
- .borrow()
- .get_user_id(client_id)
- .unwrap_or(0);
- {
- let header = bytemuck::checked::from_bytes_mut::<RequestHeader>(
- &mut message.as_mut_slice()[..size_of::<RequestHeader>()],
- );
- header.user_id = acting_user_id;
- }
-
+ // The acting-user RBAC stamp lives in the shared `prepare_request`
+ // (op-guarded, fail-closed on an unknown session). `request_preflight`
+ // above proved this client's session is live, so it resolves there.
let Ok(prepare) = self.prepare_request(message) else {
// Structurally-invalid request. Return an eviction frame (relayed to
// the socket by the home shard) rather than `Canceled`, which leaves
@@ -1552,10 +1528,11 @@ where
#[allow(clippy::too_many_lines)]
fn prepare_request(
&self,
- message: Message<RequestHeader>,
+ mut message: Message<RequestHeader>,
) -> Result<Message<PrepareHeader>, iggy_common::IggyError> {
let consensus = self.consensus.as_ref().unwrap();
- let header = *message.header();
+ let operation = message.header().operation;
+ let client_id = message.header().client;
// `TruncatePartition` is server-originated (the owning shard resolves a
// client `DeleteSegments` count to a concrete offset) but replicated AS
// the client's own request, so the commit records the client's request
@@ -1563,10 +1540,38 @@ where
// maps to it, so a client cannot construct one directly -- hence the
// `is_client_allowed` gate excludes it; admit it explicitly. The default
// match arm below projects it through unchanged.
- if !header.operation.is_client_allowed() && header.operation != Operation::TruncatePartition
- {
+ if !operation.is_client_allowed() && operation != Operation::TruncatePartition {
return Err(IggyError::InvalidCommand);
}
+
+ // Stamp the acting user id into the replicated header so the in-apply
+ // RBAC gate (`crate::stm::authz`) resolves the same identity on every
+ // replica, WAL replay included (no session table there). Every client-op
+ // prepare funnels through here, primary-only by construction, so stamping
+ // here -- not in the in-process client path alone -- also covers the
+ // wire-plane ingresses (`on_request` and the request-queue drain). The
+ // wire `user_id` is never trusted: it is overwritten for every gated
+ // client op, and a client whose session is unknown is denied here
+ // (fail-closed), never defaulted to root. `Register` / `Logout` are exempt
+ // (see `resolve_acting_user_id`); server-originated internal ops
+ // (`CompleteConsumerGroupRevocation`, the PAT-cleaner delete) build their
+ // prepare directly, bypassing this path, and keep `user_id` 0 (gate skips).
+ if let Some(acting_user_id) =
+ resolve_acting_user_id(operation, client_id, &self.client_table)?
+ {
+ let request_header = bytemuck::checked::from_bytes_mut::<RequestHeader>(
+ &mut message.as_mut_slice()[..size_of::<RequestHeader>()],
+ );
+ request_header.user_id = acting_user_id;
+ }
+
+ // Must be read AFTER the stamp: the `CreateTopic` / `CreatePartitions`
+ // arms hand this `header` copy to `build_prepare_message`, so it has to
+ // carry the stamped acting user. Reading it before the stamp would ship
+ // those prepares with `user_id` 0 = root, silently bypassing the authz
+ // gate. The default arm projects the mutated buffer directly and is
+ // order-independent.
+ let header = *message.header();
let body = &message.as_slice()[size_of::<RequestHeader>()..header.size as usize];
match header.operation {
@@ -2052,12 +2057,47 @@ const fn eviction_reason_for_invalid(operation: Operation) -> EvictionReason {
}
}
+/// Resolve the acting user id to stamp into a client op's replicated
+/// `RequestHeader`, so the in-apply RBAC gate (`crate::stm::authz`) reads the
+/// same identity on every replica (WAL replay has no session table).
+///
+/// - `Ok(Some(id))`: overwrite the header's `user_id` with the committed
+/// session's user; the wire-supplied value is never trusted.
+/// - `Ok(None)`: leave it untouched. `Register` carries the credential-verified
+/// user from login (a mid-`Register` client has no session yet, so a stamp
+/// would zero it) and `Logout` is not gated -- both keep their builder value.
+/// - `Err(Unauthenticated)`: fail-closed. A client op whose `client_id` has no
+/// session cannot be attributed, so it is denied rather than defaulted to
+/// root (`user_id` 0 is the gate's all-allow short-circuit). Every caller
+/// preflights a live session first (`request_preflight` dispatches only on
+/// `New`), so this guards a future ingress that reaches here without one.
+fn resolve_acting_user_id(
+ operation: Operation,
+ client_id: u128,
+ client_table: &RefCell<ClientTable>,
+) -> Result<Option<u32>, IggyError> {
+ if matches!(operation, Operation::Register | Operation::Logout) {
+ return Ok(None);
+ }
+ client_table
+ .borrow()
+ .get_user_id(client_id)
+ .ok_or(IggyError::Unauthenticated)
+ .map(Some)
+}
+
#[cfg(test)]
mod tests {
use super::*;
use crate::stm::stream::Streams;
use crate::stm::user::Users;
+ use consensus::LocalPipeline;
+ use iggy_binary_protocol::requests::topics::CreateTopicRequest;
use iggy_common::variadic;
+ use journal::prepare_journal::PrepareJournal;
+ use message_bus::{ClientForwardFn, ConnectionLostFn, JoinHandle, ReplicaForwardFn, SendError};
+ use server_common::MESSAGE_ALIGN;
+ use server_common::iobuf::Frozen;
use std::cell::RefCell;
use std::rc::Rc;
@@ -2153,4 +2193,180 @@ mod tests {
"cleared notifier must stay quiet"
);
}
+
+ /// Minimal committed `Register` reply for `ClientTable::commit_register`,
+ /// which reads only `client` and `commit` (the assigned session).
+ fn register_reply(client: u128, session: u64) -> Message<ReplyHeader> {
+ let header_size = size_of::<ReplyHeader>();
+ let mut reply = Message::<ReplyHeader>::new(header_size);
+ let header = bytemuck::checked::try_from_bytes_mut::<ReplyHeader>(
+ &mut reply.as_mut_slice()[..header_size],
+ )
+ .expect("zeroed bytes are a valid ReplyHeader");
+ *header = ReplyHeader {
+ client,
+ request: 0,
+ commit: session,
+ command: Command2::Reply,
+ operation: Operation::Register,
+ ..Default::default()
+ };
+ reply
+ }
+
+ #[test]
+ fn resolve_acting_user_id_skips_register_and_logout() {
+ // Session-lifecycle ops keep the user id their request builder set
+ // (Register's login identity, Logout's ungated value); the ClientTable
+ // is never consulted, so an empty table still yields `Ok(None)`.
+ let client_table = RefCell::new(ClientTable::new(CLIENTS_TABLE_MAX));
+ for operation in [Operation::Register, Operation::Logout] {
+ assert!(
+ matches!(
+ resolve_acting_user_id(operation, 1, &client_table),
+ Ok(None)
+ ),
+ "{operation:?} must not be stamped"
+ );
+ }
+ }
+
+ #[test]
+ fn resolve_acting_user_id_stamps_from_client_table() {
+ // A gated client op takes the acting user from the committed session,
+ // independent of any wire-supplied header value.
+ const CLIENT: u128 = 1;
+ const SESSION: u64 = 10;
+ const ACTING_USER: u32 = 7;
+ let mut table = ClientTable::new(CLIENTS_TABLE_MAX);
+ table.commit_register(CLIENT, ACTING_USER, register_reply(CLIENT, SESSION), |_| {
+ false
+ });
+ let client_table = RefCell::new(table);
+
+ match resolve_acting_user_id(Operation::CreateStream, CLIENT, &client_table) {
+ Ok(Some(user_id)) => assert_eq!(user_id, ACTING_USER),
+ other => panic!("expected Ok(Some({ACTING_USER})), got {other:?}"),
+ }
+ }
+
+ #[test]
+ fn resolve_acting_user_id_fails_closed_for_unknown_session() {
+ // A gated client op with no committed session is denied, never
+ // defaulted to root (user id 0).
+ let client_table = RefCell::new(ClientTable::new(CLIENTS_TABLE_MAX));
+ match resolve_acting_user_id(Operation::CreateStream, 999, &client_table) {
+ Err(IggyError::Unauthenticated) => {}
+ other => panic!("expected Err(Unauthenticated), got {other:?}"),
+ }
+ }
+
+ /// No-op bus: `prepare_request` builds a prepare without ever sending, so
+ /// every method is an unused stub.
+ #[derive(Debug, Default)]
+ struct NoopBus;
+
+ impl MessageBus for NoopBus {
+ fn track_background(&self, _handle: JoinHandle<()>) {}
+ async fn send_to_client(
+ &self,
+ _client_id: u128,
+ _data: Frozen<MESSAGE_ALIGN>,
+ ) -> Result<(), SendError> {
+ Ok(())
+ }
+ async fn send_to_replica(
+ &self,
+ _replica: u8,
+ _data: Frozen<MESSAGE_ALIGN>,
+ ) -> Result<(), SendError> {
+ Ok(())
+ }
+ fn set_connection_lost_fn(&self, _f: ConnectionLostFn) {}
+ fn set_replica_forward_fn(&self, _f: ReplicaForwardFn) {}
+ fn set_client_forward_fn(&self, _f: ClientForwardFn) {}
+ }
+
+ /// Single-node metadata plane whose `prepare_request` is callable. `J` is
+ /// `PrepareJournal` in name only (the value is `None`) to satisfy the
+ /// impl-block bound; no journal or snapshot is constructed.
+ fn metadata_plane() -> IggyMetadata<VsrConsensus<NoopBus>, PrepareJournal, (), TestMux> {
+ let consensus = VsrConsensus::new(
+ 1,
+ 0,
+ 1,
+ server_common::sharding::METADATA_CONSENSUS_NAMESPACE,
+ NoopBus,
+ LocalPipeline::new(),
+ );
+ consensus.init();
+ IggyMetadata::new(Some(consensus), None, None, TestMux::default(), None)
+ }
+
+ fn create_topic_request(client: u128, wire_user_id: u32) -> Message<RequestHeader> {
+ let body = CreateTopicRequest {
+ stream_id: WireIdentifier::numeric(1),
+ partitions_count: 1,
+ compression_algorithm: 0,
+ message_expiry: 0,
+ max_topic_size: 0,
+ replication_factor: 1,
+ name: WireName::new("t").unwrap(),
+ }
+ .to_bytes();
+ let header_size = size_of::<RequestHeader>();
+ let total = header_size + body.len();
+ let mut message = Message::<RequestHeader>::new(total);
+ {
+ let slice = message.as_mut_slice();
+ slice[header_size..total].copy_from_slice(&body);
+ let header =
+ bytemuck::checked::from_bytes_mut::<RequestHeader>(&mut slice[..header_size]);
+ *header = RequestHeader {
+ command: Command2::Request,
+ operation: Operation::CreateTopic,
+ size: u32::try_from(total).unwrap(),
+ client,
+ session: 1,
+ request: 1,
+ user_id: wire_user_id,
+ namespace: server_common::sharding::METADATA_CONSENSUS_NAMESPACE,
+ ..Default::default()
+ };
+ }
+ message
+ }
+
+ #[test]
+ fn prepare_request_stamps_create_topic_from_client_table_not_wire() {
+ // `CreateTopic` is the projection that reaches `build_prepare_message`
+ // through the post-stamp `header` copy, so the built prepare must carry
+ // the ClientTable identity, not the (bogus) wire value. This pins the
+ // stamp-then-re-read ordering: a hoist would ship `user_id` 0 = root.
+ const CLIENT: u128 = 1;
+ const SESSION: u64 = 10;
+ const ACTING_USER: u32 = 7;
+ const WIRE_USER: u32 = 999;
+ let plane = metadata_plane();
+ plane.client_table.borrow_mut().commit_register(
+ CLIENT,
+ ACTING_USER,
+ register_reply(CLIENT, SESSION),
+ |_| false,
+ );
+
+ let prepare = plane
+ .prepare_request(create_topic_request(CLIENT, WIRE_USER))
+ .expect("CreateTopic is client-allowed");
+ assert_eq!(
+ prepare.header().operation,
+ Operation::CreateTopicWithAssignments,
+ "CreateTopic projects to the enriched form"
+ );
+ assert_eq!(
+ prepare.header().user_id,
+ ACTING_USER,
+ "prepare must carry the ClientTable identity, not the wire value"
+ );
+ }
}
(iggy) 34/43: fix(server-ng): resolve the me alias inside the users/{user_id} route
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 764ff6c614a1d872b42ee9b2f85c0387bbb29578
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 21:44:06 2026 +0200
fix(server-ng): resolve the me alias inside the users/{user_id} route
The static GET /users/me probe always beat the {user_id} matcher, so a
user literally named "me" was unreachable by name. Fold the alias into
get_user: "me" resolves to the caller without the read_users gate
(legacy self-read bypass) and returns the same UserInfoDetails as any
other lookup, replacing the interim IdentityInfo echo. Authenticated
loses its now-unread user_id field.
---
core/integration/tests/server/http_rbac.rs | 8 +++++-
core/server-ng/src/http.rs | 44 ++++++++++++++++++------------
core/server-ng/src/http/extractor.rs | 10 +++----
3 files changed, 37 insertions(+), 25 deletions(-)
diff --git a/core/integration/tests/server/http_rbac.rs b/core/integration/tests/server/http_rbac.rs
index d8e5df831..f408bc5e7 100644
--- a/core/integration/tests/server/http_rbac.rs
+++ b/core/integration/tests/server/http_rbac.rs
@@ -543,11 +543,17 @@ async fn given_ungranted_user_when_touching_any_surface_should_deny_except_self_
StatusCode::NO_CONTENT,
"alice delete own PAT must be 204"
);
+ let me = alice.get("/users/me").await;
assert_eq!(
- alice.get("/users/me").await.status(),
+ me.status(),
StatusCode::OK,
"alice GET /users/me must be 200"
);
+ let me_body: Value = me.json().await.expect("users/me body must be JSON");
+ assert_eq!(
+ me_body["username"], "alice",
+ "the me alias must resolve to the caller's own user"
+ );
// Auth-only cluster metadata (no rule) and the public ping.
assert_eq!(
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 8fb2a40c8..1f5579505 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -229,6 +229,12 @@ const SELF_NODE_NAME: &str = "iggy-node";
/// server's single-node synthesis.
const SINGLE_NODE_CLUSTER_NAME: &str = "single-node";
+/// `{user_id}` segment alias resolving to the caller in `GET /users/{user_id}`.
+/// Handled inside the handler rather than as a static `/users/me` route so it
+/// cannot shadow `{user_id}` matching (a user literally named "me" stays
+/// reachable by numeric id).
+const CURRENT_USER_ALIAS: &str = "me";
+
/// Response header carrying the current VSR view number, set on every response
/// while this shard-0 node has live consensus.
const VIEW_HEADER: HeaderName = HeaderName::from_static("x-iggy-view");
@@ -1014,8 +1020,8 @@ pub async fn start(
Ok(())
}
-/// Assemble the shard-0 router: unauthenticated health + login routes, plus
-/// one authenticated probe that exercises the [`Authenticated`] extractor.
+/// Assemble the shard-0 router: unauthenticated health + login routes plus the
+/// authenticated REST surface.
///
/// `max_request_size` becomes the router-wide `DefaultBodyLimit` (413 past
/// it), exactly like the legacy server: it bounds the per-request term of the
@@ -1032,7 +1038,6 @@ fn router(state: HttpState, max_request_size: usize) -> Router {
"/personal-access-tokens/login",
post(login_with_personal_access_token),
)
- .route("/users/me", get(get_me))
.route("/users", get(get_users).post(create_user))
.route(
"/users/{user_id}",
@@ -1123,17 +1128,6 @@ async fn login_with_personal_access_token(
issue_identity(&state, user_id)
}
-/// Interim authenticated probe: echoes the caller's resolved `user_id` so the
-/// [`Authenticated`] extractor is covered end to end. Iggy exposes no "current
-/// identity" route to reuse. This path also shadows a future
-/// `GET /users/{user_id}` and must be reconciled when that route is added.
-async fn get_me(identity: Authenticated) -> Json<IdentityInfo> {
- Json(IdentityInfo {
- user_id: identity.user_id,
- access_token: None,
- })
-}
-
/// `GET /streams`: list every stream as the same `Vec<Stream>` JSON the legacy
/// server returns. A consensus-free local STM read via [`read_local`].
async fn get_streams(
@@ -1286,16 +1280,24 @@ async fn get_users(
/// `GET /users/{user_id}`: fetch one user by numeric id or name as the same
/// `UserInfoDetails` JSON the legacy server returns; 404 when absent. A
-/// consensus-free local STM read via [`read_local`].
+/// consensus-free local STM read via [`read_local`]. [`CURRENT_USER_ALIAS`]
+/// resolves to the caller and skips the `read_users` gate, mirroring the
+/// legacy self-read bypass.
async fn get_user(
State(state): State<HttpState>,
identity: Identity,
Path(user_id): Path<String>,
Query(query): Query<ConsistencyQuery>,
) -> Result<Json<UserInfoDetails>, ReadError> {
- let user_id = Identifier::from_str_value(&user_id).map_err(ReadError::Rejected)?;
+ let is_self = user_id == CURRENT_USER_ALIAS;
+ let wire_user_id = if is_self {
+ WireIdentifier::numeric(identity.user_id)
+ } else {
+ let user_id = Identifier::from_str_value(&user_id).map_err(ReadError::Rejected)?;
+ identifier_to_wire(&user_id).map_err(ReadError::Rejected)?
+ };
let request = GetUserRequest {
- user_id: identifier_to_wire(&user_id).map_err(ReadError::Rejected)?,
+ user_id: wire_user_id,
};
let body = request.to_bytes();
let bytes = read_local(
@@ -1304,7 +1306,13 @@ async fn get_user(
query.consistency,
GET_USER_CODE,
&body,
- Permissioner::get_user,
+ |permissioner, uid| {
+ if is_self {
+ Ok(())
+ } else {
+ permissioner.get_user(uid)
+ }
+ },
)?;
let response = UserDetailsResponse::decode_from(&bytes)
.map_err(|_| ReadError::Rejected(IggyError::InvalidCommand))?;
diff --git a/core/server-ng/src/http/extractor.rs b/core/server-ng/src/http/extractor.rs
index c3e235370..22e9e9e34 100644
--- a/core/server-ng/src/http/extractor.rs
+++ b/core/server-ng/src/http/extractor.rs
@@ -36,13 +36,12 @@ const BEARER: &str = "Bearer ";
const JWT_KEY_PREFIX: &str = "jwt:";
const PAT_KEY_PREFIX: &str = "pat:";
-/// Resolved caller identity for a protected route: the numeric user id plus the
-/// shared VSR session established once per presenting credential.
+/// Resolved caller identity for a protected write route: the shared VSR
+/// session established once per presenting credential (it carries the
+/// authenticated user id).
pub struct Authenticated {
- pub user_id: u32,
/// Per-credential VSR session. The write path reads its client id and
- /// session number and serializes writes through its gate; `get_me` and
- /// other read-only probes ignore it.
+ /// session number and serializes writes through its gate.
///
/// Wrapped because axum requires every handler argument to be `Send` and
/// `Rc` is not; this mirrors the `HttpState` bridge. Sound for the same
@@ -73,7 +72,6 @@ impl FromRequestParts<HttpState> for Authenticated {
// that ever touches the session table (mirrors legacy `HttpSafeShard`).
let session = SendWrapper::new(state.resolve_session(key, user_id, expiry)).await?;
Ok(Self {
- user_id,
session: SendWrapper::new(session),
})
}
(iggy) 33/43: fix(server-ng): report self-node listener ports in cluster metadata
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 097a0884f0dd23be82ccd183ee0a28ff56853f63
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 21:32:01 2026 +0200
fix(server-ng): report self-node listener ports in cluster metadata
In cluster-disabled mode GET /cluster/metadata synthesized the self
node with tcp/quic/ws ports hardcoded to 0 even though bootstrap has
the resolved listener addresses. Thread them into the HTTP roster so
the fallback reports real ports (0 only for disabled transports).
---
core/server-ng/src/bootstrap.rs | 11 ++++++++++-
core/server-ng/src/http.rs | 21 ++++++++++++++++-----
2 files changed, 26 insertions(+), 6 deletions(-)
diff --git a/core/server-ng/src/bootstrap.rs b/core/server-ng/src/bootstrap.rs
index d346699da..751b3795f 100644
--- a/core/server-ng/src/bootstrap.rs
+++ b/core/server-ng/src/bootstrap.rs
@@ -1948,7 +1948,16 @@ async fn start_tcp_runtime(
// caller of this function.
if config.http.enabled {
let http_addr = parse_socket_addr("http.address", &config.http.address)?;
- http::start(shard, http_addr, &config.http, &config.cluster).await?;
+ let self_ports = configs::cluster::TransportPorts {
+ tcp: config
+ .tcp
+ .enabled
+ .then(|| topology.client_listen_addr.port()),
+ quic: topology.quic_listen_addr.map(|addr| addr.port()),
+ websocket: topology.ws_listen_addr.map(|addr| addr.port()),
+ ..Default::default()
+ };
+ http::start(shard, http_addr, &config.http, &config.cluster, self_ports).await?;
}
Ok(())
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index c98318633..8fb2a40c8 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -37,7 +37,7 @@ use axum::response::{IntoResponse, Response};
use axum::routing::{delete, get, post, put};
use axum::{Json, Router};
use bytes::{Bytes, BytesMut};
-use configs::cluster::{ClusterConfig, ClusterNodeConfig};
+use configs::cluster::{ClusterConfig, ClusterNodeConfig, TransportPorts};
use configs::http::HttpConfig;
use consensus::{CLIENTS_TABLE_MAX, MetadataHandle, VsrConsensus};
use futures::channel::oneshot;
@@ -640,8 +640,11 @@ struct ClusterRoster {
name: String,
nodes: Vec<ClusterNodeConfig>,
/// This node's bound HTTP address, used to synthesize the sole self-node
- /// when no roster is configured (the only transport port known here).
+ /// when no roster is configured.
http_addr: SocketAddr,
+ /// This node's other client listener ports for the same self-node
+ /// synthesis (`None` = transport disabled; HTTP comes from `http_addr`).
+ self_ports: TransportPorts,
}
/// Serializes first-use VSR registration per credential key so a herd of
@@ -971,6 +974,7 @@ pub async fn start(
addr: SocketAddr,
http_config: &HttpConfig,
cluster: &ClusterConfig,
+ self_ports: TransportPorts,
) -> Result<(), ServerNgError> {
let jwt = JwtManager::build(&http_config.jwt)?;
let (listener, bound_addr) = client_listener::tcp::bind(addr).await?;
@@ -986,6 +990,7 @@ pub async fn start(
name: cluster.name.clone(),
nodes: cluster.nodes.clone(),
http_addr: bound_addr,
+ self_ports,
},
in_flight_writes: Cell::new(0),
}));
@@ -2861,15 +2866,20 @@ fn build_cluster_metadata(state: &HttpInner) -> ClusterMetadata {
}
/// Synthesize a single self-node [`ClusterMetadata`], mirroring the legacy
-/// single-node path. Only this node's bound HTTP port is known at the HTTP layer
-/// (the roster is not threaded onto the shard), so the other transports report 0.
+/// single-node path: this node's own listener ports, with the HTTP port taken
+/// from the live bound address and disabled transports reporting 0.
fn self_node_metadata(roster: &ClusterRoster) -> ClusterMetadata {
ClusterMetadata {
name: SINGLE_NODE_CLUSTER_NAME.to_owned(),
nodes: vec![ClusterNode {
name: SELF_NODE_NAME.to_owned(),
ip: roster.http_addr.ip().to_string(),
- endpoints: TransportEndpoints::new(0, 0, roster.http_addr.port(), 0),
+ endpoints: TransportEndpoints::new(
+ roster.self_ports.tcp.unwrap_or(0),
+ roster.self_ports.quic.unwrap_or(0),
+ roster.http_addr.port(),
+ roster.self_ports.websocket.unwrap_or(0),
+ ),
role: ClusterNodeRole::Leader,
status: ClusterNodeStatus::Healthy,
}],
@@ -3191,6 +3201,7 @@ mod tests {
name: "test-cluster".to_owned(),
nodes,
http_addr: "127.0.0.1:3000".parse().expect("valid socket addr"),
+ self_ports: TransportPorts::default(),
}
}
(iggy) 29/43: docs(integration): refresh stale permissions-scenario gate reason
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit b938f2ef5101841e130785ccc1c871424e4627ab
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 18:23:41 2026 +0200
docs(integration): refresh stale permissions-scenario gate reason
The cfg gate blamed missing permission enforcement, which server-ng
now has (covered by server::http_rbac). The scenario stays gated
because it drives get_stats/get_clients/snapshot across all four
transports and QUIC/WebSocket control-plane parity is not yet
verified on server-ng.
---
core/integration/tests/server/general.rs | 13 ++++++-------
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/core/integration/tests/server/general.rs b/core/integration/tests/server/general.rs
index 9f6a206de..1a6fe744e 100644
--- a/core/integration/tests/server/general.rs
+++ b/core/integration/tests/server/general.rs
@@ -79,13 +79,12 @@ async fn user(harness: &TestHarness) {
user_scenario::run(harness).await;
}
-// Blocked under vsr: password hashing and the metadata journal-append
-// race are now fixed, so the scenario runs deep into its permission
-// matrix. Remaining blocker is per-operation authorization: server-ng
-// does not enforce the caller's permissions on metadata ops, so a
-// `read_streams`-only user calling `create_stream` gets `InvalidFormat`
-// instead of `Unauthorized`. Needs the permission-enforcement subsystem
-// wired on the metadata plane.
+// Blocked under vsr, but no longer for RBAC: server-ng now enforces the
+// caller's permissions on the metadata apply path, and that matrix is
+// covered by `server::http_rbac`. This scenario stays gated because it
+// drives get_stats/get_clients/snapshot across all four transports
+// (Tcp/Http/Quic/WebSocket), and QUIC/WebSocket control-plane parity is
+// not yet verified on server-ng.
#[cfg(not(feature = "vsr"))]
#[iggy_harness(
test_client_transport = [Tcp, Http, Quic, WebSocket],
(iggy) 14/43: feat(server-ng): add HTTP produce route
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 2789a345ca73203971ddbe9fc08fb8d35a7f60ab
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 23:22:52 2026 +0200
feat(server-ng): add HTTP produce route
An HTTP caller holds no consensus socket, so it could not produce:
partition replies fire post-commit through send_to_client and the
control-plane submit path never touches the partition groups.
POST /streams/{stream_id}/topics/{topic_id}/messages accepts the
legacy SendMessages JSON and encodes it through the SDK's own wire
path (identifier/partitioning conversion + SendMessagesEncoder), so
HTTP and TCP produce byte-identical request bodies. The handler
installs a request-keyed in-process reply slot before dispatching
through the shared partition gates; one slot catches the committed
reply and every pre-dispatch gate failure. Partition replies carry
no error code - success and gate failure are both empty-bodied - so
the reply's op discriminates: a committed reply carries its
prepare's op (>= 1), a gate failure replies via build_empty_reply
with 0 and renders as the legacy 404.
The default answers 201 + X-Iggy-Durability: replicated-memory after
the quorum commit, bounded at 10s (504 = outcome unknown: the
partition plane is at-least-once, the prepare may still land).
?ack=none answers 202 right after dispatch; its reply sheds at the
bus. Data-plane request ids ride a separate gate-free per-session
counter because partition ops have no dedup and concurrent produces
are legal - they must not serialize behind the control-plane gate.
Session eviction tears the reply target down token-fenced,
cancelling parked waiters.
---
core/server-ng/src/http.rs | 491 ++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 481 insertions(+), 10 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 7d1799f74..ab7c6053a 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -22,10 +22,11 @@
mod extractor;
mod jwt;
-use std::cell::RefCell;
+use std::cell::{Cell, RefCell};
use std::collections::HashMap;
use std::net::{IpAddr, SocketAddr};
use std::rc::Rc;
+use std::time::Duration;
use axum::extract::{Path, Query, State};
use axum::http::header::LOCATION;
@@ -34,7 +35,7 @@ use axum::middleware::map_response;
use axum::response::{IntoResponse, Response};
use axum::routing::{delete, get, post, put};
use axum::{Json, Router};
-use bytes::Bytes;
+use bytes::{Bytes, BytesMut};
use configs::cluster::{ClusterConfig, ClusterNodeConfig};
use configs::http::HttpJwtConfig;
use consensus::{MetadataHandle, VsrConsensus};
@@ -49,6 +50,7 @@ use iggy_binary_protocol::requests::consumer_groups::{
CreateConsumerGroupRequest, DeleteConsumerGroupRequest, GetConsumerGroupRequest,
GetConsumerGroupsRequest,
};
+use iggy_binary_protocol::requests::messages::{RawMessage, SendMessagesEncoder};
use iggy_binary_protocol::requests::personal_access_tokens::{
CreatePersonalAccessTokenRequest, DeletePersonalAccessTokenRequest,
};
@@ -80,7 +82,7 @@ use iggy_binary_protocol::responses::users::get_user::UserDetailsResponse;
use iggy_binary_protocol::responses::users::get_users::GetUsersResponse;
use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
use iggy_binary_protocol::{
- GenericHeader, Operation, RequestHeader, WireDecode, WireEncode, WireName,
+ GenericHeader, Operation, ReplyHeader, RequestHeader, WireDecode, WireEncode, WireName,
};
use iggy_common::change_password::ChangePassword;
use iggy_common::create_consumer_group::CreateConsumerGroup;
@@ -96,16 +98,17 @@ use iggy_common::update_stream::UpdateStream;
use iggy_common::update_topic::UpdateTopic;
use iggy_common::update_user::UpdateUser;
use iggy_common::wire_conversions::{
- clients_from_wire, consumer_groups_from_wire, identifier_to_wire, permissions_to_wire,
- streams_from_wire, topics_from_wire, users_from_wire,
+ clients_from_wire, consumer_groups_from_wire, identifier_to_wire, partitioning_to_wire,
+ permissions_to_wire, streams_from_wire, topics_from_wire, users_from_wire,
};
use iggy_common::{
ClientInfo, ClientInfoDetails, ClusterMetadata, ClusterNode, ClusterNodeRole,
ClusterNodeStatus, ConsumerGroup, ConsumerGroupDetails, Identifier, IdentityInfo, IggyError,
- IggyTimestamp, RawPersonalAccessToken, Stats, Stream, StreamDetails, TokenInfo, Topic,
- TopicDetails, TransportEndpoints, UserInfo, UserInfoDetails, Validatable,
+ IggyMessageView, IggyTimestamp, RawPersonalAccessToken, SendMessages, Stats, Stream,
+ StreamDetails, TokenInfo, Topic, TopicDetails, TransportEndpoints, UserInfo, UserInfoDetails,
+ Validatable,
};
-use message_bus::client_listener;
+use message_bus::{BusMessage, InstanceToken, client_listener};
use metadata::impls::metadata::StreamsFrontend;
use secrecy::ExposeSecret;
use send_wrapper::SendWrapper;
@@ -117,7 +120,9 @@ use tracing::{error, info, warn};
use crate::auth::{verify_login_credentials, verify_pat_credentials};
use crate::bootstrap::ServerNgShard;
-use crate::dispatch::{submit_client_request_on_owner, submit_register_on_owner};
+use crate::dispatch::{
+ dispatch_partition_request, submit_client_request_on_owner, submit_register_on_owner,
+};
use crate::http::extractor::{Authenticated, Identity};
use crate::http::jwt::JwtManager;
use crate::login_register::LoginRegisterError;
@@ -143,6 +148,21 @@ const MAX_HTTP_SESSIONS: usize = 100_000;
/// are 1-based and strictly increasing within a session.
const FIRST_REQUEST_ID: u64 = 1;
+/// Bound on a produce's wait for its committed partition reply. Long enough to
+/// ride out a view change (plus the dispatch gates' own routable-wait budget),
+/// short enough not to pin HTTP connections behind a dead consensus group. On
+/// expiry the caller gets 504 and must treat the outcome as unknown: the
+/// partition plane is at-least-once and the prepare may still commit after the
+/// wait gave up, so the server never retries on the caller's behalf.
+const PRODUCE_REPLY_TIMEOUT: Duration = Duration::from_secs(10);
+
+/// Response header attesting what durability a produce response proves:
+/// [`DURABILITY_REPLICATED_MEMORY`] after an awaited quorum commit,
+/// [`DURABILITY_NONE`] for a `?ack=none` fire-and-forget.
+const DURABILITY_HEADER: HeaderName = HeaderName::from_static("x-iggy-durability");
+const DURABILITY_REPLICATED_MEMORY: &str = "replicated-memory";
+const DURABILITY_NONE: &str = "none";
+
/// Node name reported for the synthesized single self-node, matching the legacy
/// server's single-node synthesis.
const SELF_NODE_NAME: &str = "iggy-node";
@@ -177,6 +197,27 @@ struct HttpSession {
/// submit `.await` so each session's request numbers reach the primary in
/// order and stay gap-free for the depth-1 consensus dedup.
gate: Mutex<u64>,
+ /// Next data-plane request id. A separate, gate-free counter: partition ops
+ /// are at-least-once with no consensus dedup, so the id only correlates the
+ /// in-process reply slot and concurrent produces on one session are legal.
+ /// A plain `Cell` suffices on single-threaded shard 0; ids are minted
+ /// monotonically and never reused, which the slot-guard contract requires.
+ data_request: Cell<u64>,
+ /// Registry token of this session's lazily-installed in-process reply
+ /// target (`None` until the first awaited produce). Stored so session
+ /// eviction can tear the registry entry down fenced by the same token.
+ registry_token: Cell<Option<InstanceToken>>,
+}
+
+impl HttpSession {
+ /// Mint the next data-plane request id. Also consumed by the `?ack=none`
+ /// path, which installs no slot: sharing one counter keeps a shed reply's
+ /// id from ever colliding with a live awaited slot on this session.
+ fn next_data_request_id(&self) -> u64 {
+ let id = self.data_request.get();
+ self.data_request.set(id + 1);
+ id
+ }
}
/// Rejection for protected routes.
@@ -233,6 +274,58 @@ impl IntoResponse for WriteError {
}
}
+/// Rejection for a data-plane produce (`POST .../messages`).
+///
+/// Split differently from [`WriteError`] because the partition plane replies
+/// carry no committed error code: a pre-dispatch gate failure is an empty
+/// reply distinguishable only by header (see [`classify_partition_reply`]),
+/// and an unanswered produce is a distinct outcome the caller must treat as
+/// unknown rather than failed.
+enum ProduceError {
+ /// Caller-side rejection (bad identifier, oversized batch, the interim
+ /// non-root denial) or a malformed reply frame, rendered through the
+ /// legacy `IggyError -> status` map for SDK-identical bodies.
+ Rejected(IggyError),
+ /// The dispatch gates could not route the produce: the stream, topic, or
+ /// partition does not resolve (or never materialised within the routable
+ /// budget). Rendered as the legacy 404 body.
+ NotFound,
+ /// The in-process reply slot could not be installed. Transient server
+ /// condition -> the shared 503, retryable.
+ Unavailable,
+ /// No committed reply within [`PRODUCE_REPLY_TIMEOUT`], or the session's
+ /// reply target was torn down mid-wait. 504: the commit may still land
+ /// (at-least-once), so this is a hard "outcome unknown", not a failure
+ /// the server may transparently retry.
+ Timeout,
+}
+
+impl IntoResponse for ProduceError {
+ fn into_response(self) -> Response {
+ match self {
+ Self::Rejected(error) => CustomError::from(error).into_response(),
+ Self::NotFound => CustomError::ResourceNotFound.into_response(),
+ Self::Unavailable => service_unavailable(),
+ Self::Timeout => produce_timeout_response(),
+ }
+ }
+}
+
+/// 504 body for a produce whose commit outcome is unknown. Shaped like every
+/// other HTTP error (`ErrorResponse`) so clients parse one error schema.
+fn produce_timeout_response() -> Response {
+ (
+ StatusCode::GATEWAY_TIMEOUT,
+ Json(ErrorResponse {
+ id: StatusCode::GATEWAY_TIMEOUT.as_u16().into(),
+ code: "produce_timeout".to_owned(),
+ reason: "produce was not acknowledged in time; the write may still commit".to_owned(),
+ field: None,
+ }),
+ )
+ .into_response()
+}
+
/// The shared 503 body for a request that could not commit right now: no
/// caught-up primary, a full pipeline, or a view-change cancel. Retryable, and
/// rendered with the `CannotEstablishConnection` code the SDKs treat as a
@@ -269,6 +362,28 @@ struct ConsistencyQuery {
consistency: Consistency,
}
+/// Produce acknowledgement selected by the `?ack=` query param.
+///
+/// `replicated` (the default) answers 201 only after the partition group's
+/// quorum commit. `none` is fire-and-forget: the request is validated,
+/// dispatched, and answered 202 immediately; the commit still happens, but its
+/// reply is shed at the bus (no reply slot is installed).
+#[derive(Clone, Copy, Default, PartialEq, Eq, Deserialize)]
+#[serde(rename_all = "lowercase")]
+enum ProduceAck {
+ #[default]
+ Replicated,
+ None,
+}
+
+/// `?ack=` query wrapper. An absent param defaults to
+/// [`ProduceAck::Replicated`]; an unrecognized value is a 400 (axum `Query`).
+#[derive(Default, Deserialize)]
+struct ProduceQuery {
+ #[serde(default)]
+ ack: ProduceAck,
+}
+
/// Rejection for an authenticated read route (`GET /streams`,
/// `GET /streams/{id}`, and the reads that follow). Three classes, three
/// renderings, mirroring [`WriteError`]'s split so error bodies stay
@@ -456,7 +571,22 @@ impl HttpInner {
if let Some(session) = live_entry(&table, &key, now) {
return Ok(session);
}
- table.retain(|_, session| session.expiry > now);
+ table.retain(|_, session| {
+ if session.expiry > now {
+ return true;
+ }
+ // Evicting the session also tears down its in-process reply
+ // target (if a produce ever installed one), cancelling any
+ // still-parked reply waiters. Token-fenced so a stale sweep can
+ // never remove a later occupant of the key.
+ if let Some(token) = session.registry_token.get() {
+ self.shard
+ .bus
+ .clients()
+ .remove_if_token_matches(session.client_id, token);
+ }
+ false
+ });
if table.len() >= MAX_HTTP_SESSIONS {
// Still full after dropping expired entries: too many genuinely live
// sessions. Refuse rather than evict a live one; the client retries.
@@ -507,6 +637,8 @@ impl HttpInner {
user_id,
expiry,
gate: Mutex::new(FIRST_REQUEST_ID),
+ data_request: Cell::new(FIRST_REQUEST_ID),
+ registry_token: Cell::new(None),
}))
}
}
@@ -616,6 +748,10 @@ fn router(state: HttpState) -> Router {
"/streams/{stream_id}/topics/{topic_id}/purge",
delete(purge_topic),
)
+ .route(
+ "/streams/{stream_id}/topics/{topic_id}/messages",
+ post(send_messages),
+ )
.route(
"/streams/{stream_id}/topics/{topic_id}/consumer-groups",
get(get_cgs).post(create_cg),
@@ -1176,6 +1312,59 @@ async fn purge_topic(
Ok(StatusCode::NO_CONTENT)
}
+/// `POST /streams/{stream_id}/topics/{topic_id}/messages`: produce a batch of
+/// messages to a topic. The JSON body is the same `SendMessages` shape the
+/// legacy server accepts (partitioning + base64 messages); stream and topic
+/// come from the path.
+///
+/// Data plane, not control plane: the batch rides the partition group's own
+/// consensus (at-least-once, no dedup, no session gate - concurrent produces
+/// on one credential are legal), and the committed reply comes back through
+/// the session's in-process reply slot rather than a submit return value.
+/// The default answers 201 + `X-Iggy-Durability: replicated-memory` only
+/// after the quorum commit; `?ack=none` answers 202 + `X-Iggy-Durability:
+/// none` immediately after dispatch.
+async fn send_messages(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Query(query): Query<ProduceQuery>,
+ Json(command): Json<SendMessages>,
+) -> Result<Response, ProduceError> {
+ // Interim authorization: root-only until server-ng has an RBAC
+ // permissioner, mirroring the control-plane gate in `submit_committed`.
+ if identity.session.user_id != DEFAULT_ROOT_USER_ID {
+ return Err(ProduceError::Rejected(IggyError::Unauthorized));
+ }
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ProduceError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(ProduceError::Rejected)?;
+ // Rejects an oversized partitioning key and an empty or oversized batch.
+ command.validate().map_err(ProduceError::Rejected)?;
+ let body =
+ encode_send_messages(&stream_id, &topic_id, &command).map_err(ProduceError::Rejected)?;
+ match query.ack {
+ ProduceAck::Replicated => {
+ SendWrapper::new(produce_replicated(&state, &identity.session, &body)).await?;
+ Ok((
+ StatusCode::CREATED,
+ [(
+ DURABILITY_HEADER,
+ HeaderValue::from_static(DURABILITY_REPLICATED_MEMORY),
+ )],
+ )
+ .into_response())
+ }
+ ProduceAck::None => {
+ SendWrapper::new(produce_unacked(&state, &identity.session, &body)).await;
+ Ok((
+ StatusCode::ACCEPTED,
+ [(DURABILITY_HEADER, HeaderValue::from_static(DURABILITY_NONE))],
+ )
+ .into_response())
+ }
+ }
+}
+
/// `POST /streams/{stream_id}/topics/{topic_id}/consumer-groups`: create a
/// consumer group under a topic and render the committed reply as the same
/// `ConsumerGroupDetails` JSON the legacy server returns.
@@ -1526,6 +1715,165 @@ async fn submit_write(
Ok(Bytes::copy_from_slice(committed_payload(&reply)?))
}
+/// Encode a validated HTTP `SendMessages` into the `SendMessagesRequest` wire
+/// body, mirroring the SDK's TCP produce encode: identifier + partitioning
+/// wire conversion, then `RawMessage` borrows into [`SendMessagesEncoder`].
+/// Balanced / messages-key partitioning passes through untouched; the
+/// dispatch gates resolve it to a concrete partition server-side.
+fn encode_send_messages(
+ stream_id: &Identifier,
+ topic_id: &Identifier,
+ command: &SendMessages,
+) -> Result<Bytes, IggyError> {
+ let wire_stream_id = identifier_to_wire(stream_id)?;
+ let wire_topic_id = identifier_to_wire(topic_id)?;
+ let wire_partitioning = partitioning_to_wire(&command.partitioning)?;
+ // Two passes because the view accessors' return borrows are tied to the
+ // view value, not the batch buffer, so the views must outlive the borrows.
+ let views: Vec<IggyMessageView<'_>> = command.batch.iter().collect();
+ let raw_messages: Vec<RawMessage<'_>> = views
+ .iter()
+ .map(|view| RawMessage {
+ id: view.header().id(),
+ origin_timestamp: view.header().origin_timestamp(),
+ headers: view.user_headers(),
+ payload: view.payload(),
+ })
+ .collect();
+ let size = SendMessagesEncoder::encoded_size(
+ &wire_stream_id,
+ &wire_topic_id,
+ &wire_partitioning,
+ &raw_messages,
+ );
+ let mut buf = BytesMut::with_capacity(size);
+ SendMessagesEncoder::encode(
+ &mut buf,
+ &wire_stream_id,
+ &wire_topic_id,
+ &wire_partitioning,
+ &raw_messages,
+ );
+ Ok(buf.freeze())
+}
+
+/// Run one awaited produce end to end: install the reply slot, dispatch into
+/// the partition plane, and wait (bounded) for the committed reply.
+///
+/// Slot-before-dispatch is load-bearing: every pre-dispatch gate failure
+/// inside [`dispatch_partition_request`] replies through `send_to_client`,
+/// which fires an installed slot, so one slot catches every exit. The slot
+/// guard borrows the registry, which is why this whole future runs inside
+/// the caller's `SendWrapper` on shard 0.
+async fn produce_replicated(
+ state: &HttpInner,
+ session: &HttpSession,
+ body: &[u8],
+) -> Result<(), ProduceError> {
+ ensure_in_process_reply_target(state, session);
+ let request_id = session.next_data_request_id();
+ let message = build_request_message(
+ Operation::SendMessages,
+ session.client_id,
+ session.session,
+ request_id,
+ body,
+ );
+ let (guard, receiver) = state
+ .shard
+ .bus
+ .clients()
+ .install_reply_slot(session.client_id, request_id)
+ .map_err(|error| {
+ warn!(?error, "server-ng HTTP: produce reply slot install failed");
+ ProduceError::Unavailable
+ })?;
+ dispatch_partition_request(
+ &state.shard,
+ message,
+ session.client_id,
+ session.session,
+ session.client_id,
+ )
+ .await;
+ let outcome = compio::time::timeout(PRODUCE_REPLY_TIMEOUT, receiver).await;
+ // Removes the slot unless the reply already fired, so a late commit
+ // reply after a timeout sheds at the bus instead of leaking a waiter.
+ drop(guard);
+ match outcome {
+ Ok(Ok(reply)) => classify_partition_reply(&reply),
+ // Cancelled (reply target torn down by session eviction mid-wait) or
+ // elapsed: same caller contract either way - outcome unknown, 504.
+ Ok(Err(_)) | Err(_) => Err(ProduceError::Timeout),
+ }
+}
+
+/// Fire-and-forget produce (`?ack=none`): no reply slot, no wait. The commit
+/// still happens; its reply (and any gate-failure reply) targets a request id
+/// with no slot installed and is shed at the bus by design.
+async fn produce_unacked(state: &HttpInner, session: &HttpSession, body: &[u8]) {
+ let request_id = session.next_data_request_id();
+ let message = build_request_message(
+ Operation::SendMessages,
+ session.client_id,
+ session.session,
+ request_id,
+ body,
+ );
+ dispatch_partition_request(
+ &state.shard,
+ message,
+ session.client_id,
+ session.session,
+ session.client_id,
+ )
+ .await;
+}
+
+/// Install this session's in-process reply target on first data-plane use.
+///
+/// The registry key is the session's shard-0 client id - the same id stamped
+/// into `RequestHeader.client` - so a partition reply routed through
+/// `send_to_client` lands on this entry and resolves the request-keyed slot.
+/// `None` from the registry means the key is already occupied; treat it as
+/// installed but leave the token unset so this session never tears down an
+/// entry it does not own.
+fn ensure_in_process_reply_target(state: &HttpInner, session: &HttpSession) {
+ if session.registry_token.get().is_some() {
+ return;
+ }
+ if let Some(token) = state
+ .shard
+ .bus
+ .clients()
+ .insert_in_process(session.client_id)
+ {
+ session.registry_token.set(Some(token));
+ }
+}
+
+/// Discriminate a produce reply. Partition replies carry no result section
+/// and no error code - success and gate failure are both empty-bodied - so
+/// the only wire discriminator is the reply's `op`: a committed reply is
+/// built from its prepare header, whose op (the partition group's commit
+/// number) is always >= 1, while the pre-dispatch gate failures reply through
+/// `build_empty_reply` with 0 in that field. The gate reply cannot name which
+/// entity was missing, hence the generic legacy 404 body.
+fn classify_partition_reply(reply: &BusMessage) -> Result<(), ProduceError> {
+ let header = reply
+ .as_slice()
+ .get(..HEADER_SIZE)
+ .and_then(|bytes| bytemuck::checked::try_from_bytes::<ReplyHeader>(bytes).ok())
+ .ok_or(ProduceError::Rejected(IggyError::InvalidCommand))?;
+ if header.command != Command2::Reply {
+ return Err(ProduceError::Rejected(IggyError::InvalidCommand));
+ }
+ if header.op == 0 {
+ return Err(ProduceError::NotFound);
+ }
+ Ok(())
+}
+
/// The two cross-cutting gates every authenticated read enforces before it
/// touches state. Factored out of [`read_local`] so the cross-shard client
/// reads (`get_clients` / `get_client`) - which serve from the shard session
@@ -1819,8 +2167,11 @@ const fn login_error_to_iggy(error: &LoginRegisterError) -> IggyError {
#[cfg(test)]
mod tests {
use configs::cluster::TransportPorts;
+ use iggy_binary_protocol::PrepareHeader;
+ use iggy_common::{IggyMessage, IggyMessagesBatch, Partitioning, PartitioningKind};
use super::*;
+ use crate::responses::build_empty_reply;
const READ_PATH: &str = "/streams?consistency=linearizable";
@@ -1886,4 +2237,124 @@ mod tests {
Some("http://[::1]:8080/streams?consistency=linearizable".to_owned())
);
}
+
+ // -- encode_send_messages --
+
+ fn produce_command(partitioning: Partitioning) -> SendMessages {
+ let first = IggyMessage::builder()
+ .id(7)
+ .payload(Bytes::from_static(b"first"))
+ .build()
+ .expect("valid message");
+ // Raw pre-encoded user headers, mirroring the HTTP deserializer's
+ // base64 branch.
+ let mut second = IggyMessage::builder()
+ .id(8)
+ .payload(Bytes::from_static(b"second"))
+ .build()
+ .expect("valid message");
+ let raw_headers = Bytes::from_static(b"raw-header-bytes");
+ second.header.user_headers_length =
+ u32::try_from(raw_headers.len()).expect("test headers fit u32");
+ second.user_headers = Some(raw_headers);
+ let messages = vec![first, second];
+ SendMessages {
+ partitioning,
+ batch: IggyMessagesBatch::from(&messages),
+ ..Default::default()
+ }
+ }
+
+ #[test]
+ fn encode_send_messages_round_trips_through_wire_decoders() {
+ use iggy_binary_protocol::WireMessageIterator;
+ use iggy_binary_protocol::message_layout::WIRE_MESSAGE_INDEX_SIZE;
+ use iggy_binary_protocol::requests::messages::SendMessagesHeader;
+
+ let stream_id = Identifier::from_str_value("1").expect("valid stream id");
+ let topic_id = Identifier::from_str_value("orders").expect("valid topic id");
+ let command = produce_command(Partitioning::partition_id(3));
+ let origin_timestamps: Vec<u64> = command
+ .batch
+ .iter()
+ .map(|view| view.header().origin_timestamp())
+ .collect();
+
+ let bytes = encode_send_messages(&stream_id, &topic_id, &command).expect("encodes");
+
+ let metadata_length =
+ u32::from_le_bytes(bytes[..4].try_into().expect("length prefix")) as usize;
+ let (header, consumed) =
+ SendMessagesHeader::decode(&bytes[4..4 + metadata_length]).expect("valid metadata");
+ assert_eq!(consumed, metadata_length);
+ assert_eq!(header.stream_id, identifier_to_wire(&stream_id).unwrap());
+ assert_eq!(header.topic_id, identifier_to_wire(&topic_id).unwrap());
+ assert_eq!(
+ header.partitioning,
+ partitioning_to_wire(&command.partitioning).unwrap()
+ );
+ assert_eq!(header.messages_count, 2);
+
+ let data_offset = 4 + metadata_length + 2 * WIRE_MESSAGE_INDEX_SIZE;
+ let views: Vec<_> = WireMessageIterator::new(&bytes[data_offset..], 2)
+ .collect::<Result<Vec<_>, _>>()
+ .expect("valid message frames");
+ assert_eq!(views[0].id(), 7);
+ assert_eq!(views[0].payload(), b"first");
+ assert_eq!(views[0].user_headers(), b"");
+ assert_eq!(views[0].origin_timestamp(), origin_timestamps[0]);
+ assert_eq!(views[1].id(), 8);
+ assert_eq!(views[1].payload(), b"second");
+ assert_eq!(views[1].user_headers(), b"raw-header-bytes");
+ assert_eq!(views[1].origin_timestamp(), origin_timestamps[1]);
+ }
+
+ #[test]
+ fn send_messages_validate_rejects_oversized_partitioning_key() {
+ let command = produce_command(Partitioning {
+ kind: PartitioningKind::MessagesKey,
+ length: 0,
+ value: vec![0u8; 256],
+ });
+ assert!(command.validate().is_err());
+ }
+
+ // -- classify_partition_reply --
+
+ fn frozen(reply: Message<iggy_binary_protocol::ReplyHeader>) -> BusMessage {
+ reply.into_generic().into_frozen()
+ }
+
+ #[test]
+ fn committed_partition_reply_classifies_as_success() {
+ let prepare = PrepareHeader {
+ command: Command2::Prepare,
+ operation: Operation::SendMessages,
+ client: 42,
+ op: 1,
+ request: 1,
+ ..Default::default()
+ };
+ let reply = frozen(consensus::build_reply_message(&prepare, &Bytes::new()));
+ assert!(classify_partition_reply(&reply).is_ok());
+ }
+
+ #[test]
+ fn gate_failure_empty_reply_classifies_as_not_found() {
+ let request = build_request_message(Operation::SendMessages, 42, 7, 1, &[]);
+ let reply = frozen(build_empty_reply(request.header(), 42, 0, 9));
+ assert!(matches!(
+ classify_partition_reply(&reply),
+ Err(ProduceError::NotFound)
+ ));
+ }
+
+ #[test]
+ fn non_reply_frame_classifies_as_rejected() {
+ let request = build_request_message(Operation::SendMessages, 42, 7, 1, &[]);
+ assert!(matches!(
+ classify_partition_reply(&request.into_generic().into_frozen()),
+ Err(ProduceError::Rejected(IggyError::InvalidCommand))
+ ));
+ }
}
(iggy) 15/43: feat(server-ng): add HTTP poll and consumer-offset read routes
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit d5a479c421704a142b80f5766fb50bd3e39a4f1c
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 23:54:42 2026 +0200
feat(server-ng): add HTTP poll and consumer-offset read routes
Poll and get-consumer-offset are non-replicated reads served in band:
no VSR session, no reply slot, no consensus. Both routes run the same
resolution the TCP dispatch uses (its decode helpers split into shared
resolver cores), read the owning shard over the mesh, and decode the
re-encoded legacy wire body with the SDK's own
PolledMessages::from_bytes, so HTTP and TCP answers stay
field-identical.
HTTP reads carry VSR client id 0, which can never name a group member,
so a group-kind poll fences closed to the re-sync sentinel exactly like
a stale TCP member; the pinned query shape cannot express a group poll
anyway (Consumer.kind is serde-skipped), matching legacy HTTP. A
missing stored offset renders as the legacy 404, and an unanswered
mesh read as 504 through the shared gateway-timeout body.
---
core/server-ng/src/dispatch.rs | 28 ++-
core/server-ng/src/http.rs | 401 ++++++++++++++++++++++++++++++++++++++++-
2 files changed, 416 insertions(+), 13 deletions(-)
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index f942ebefb..50b3a1e65 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -1305,15 +1305,27 @@ fn empty_polled_messages_body(partition_id: u32) -> Bytes {
Bytes::from(body)
}
-type DecodedPollRequest = (IggyNamespace, u32, PollingConsumer, PollingArgs);
+pub(crate) type DecodedPollRequest = (IggyNamespace, u32, PollingConsumer, PollingArgs);
-#[allow(clippy::cast_possible_truncation)]
fn decode_poll_request(
shard: &Rc<ServerNgShard>,
request: &Message<RequestHeader>,
) -> Result<DecodedPollRequest, IggyError> {
let wire = PollMessagesRequest::decode_from(request_body(request))
.map_err(|_| IggyError::InvalidCommand)?;
+ resolve_poll_request(shard, &wire, request.header().client)
+}
+
+/// Resolve a decoded poll request into its owning-shard read: namespace,
+/// partition, polling consumer, and args. Shared by the TCP dispatch (client
+/// id = the connection's bound VSR client) and the HTTP route (client id 0,
+/// which fences group polls closed).
+#[allow(clippy::cast_possible_truncation)]
+pub(crate) fn resolve_poll_request(
+ shard: &Rc<ServerNgShard>,
+ wire: &PollMessagesRequest,
+ client_id: u128,
+) -> Result<DecodedPollRequest, IggyError> {
let strategy = polling_strategy_from_wire(&wire.strategy)?;
let args = PollingArgs::new(strategy, wire.count, wire.auto_commit);
@@ -1325,7 +1337,6 @@ fn decode_poll_request(
// both use, so `next()` reads back the offset it just committed.
if wire.consumer.kind == KIND_CONSUMER_GROUP {
let partition_id = wire.partition_id.ok_or(IggyError::InvalidIdentifier)?;
- let client_id = request.header().client;
let group_id = shard
.plane
.metadata()
@@ -1369,6 +1380,17 @@ fn decode_consumer_offset_request(
) -> Result<(IggyNamespace, u32, PollingConsumer), IggyError> {
let wire = GetConsumerOffsetRequest::decode_from(request_body(request))
.map_err(|_| IggyError::InvalidCommand)?;
+ resolve_consumer_offset_request(shard, &wire)
+}
+
+/// Resolve a decoded consumer-offset read into its owning-shard read:
+/// namespace, partition, and polling consumer. Shared by the TCP dispatch and
+/// the HTTP route; needs no client id because offset reads are not fenced
+/// (any client may read a group's offset, member or not).
+pub(crate) fn resolve_consumer_offset_request(
+ shard: &Rc<ServerNgShard>,
+ wire: &GetConsumerOffsetRequest,
+) -> Result<(IggyNamespace, u32, PollingConsumer), IggyError> {
let partition_id = wire.partition_id.ok_or(IggyError::InvalidIdentifier)?;
let namespace =
resolve_partition_namespace(shard, &wire.stream_id, &wire.topic_id, Some(partition_id))?;
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index ab7c6053a..c3a406d93 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -46,11 +46,16 @@ use iggy_binary_protocol::codes::{
use iggy_binary_protocol::consensus::{
Command2, EvictionHeader, EvictionReason, HEADER_SIZE, result_code, result_section_len,
};
+use iggy_binary_protocol::primitives::consumer::WireConsumer;
+use iggy_binary_protocol::primitives::polling_strategy::WirePollingStrategy;
use iggy_binary_protocol::requests::consumer_groups::{
CreateConsumerGroupRequest, DeleteConsumerGroupRequest, GetConsumerGroupRequest,
GetConsumerGroupsRequest,
};
-use iggy_binary_protocol::requests::messages::{RawMessage, SendMessagesEncoder};
+use iggy_binary_protocol::requests::consumer_offsets::GetConsumerOffsetRequest;
+use iggy_binary_protocol::requests::messages::{
+ PollMessagesRequest, RawMessage, SendMessagesEncoder,
+};
use iggy_binary_protocol::requests::personal_access_tokens::{
CreatePersonalAccessTokenRequest, DeletePersonalAccessTokenRequest,
};
@@ -91,8 +96,10 @@ use iggy_common::create_stream::CreateStream;
use iggy_common::create_topic::CreateTopic;
use iggy_common::create_user::CreateUser;
use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
+use iggy_common::get_consumer_offset::GetConsumerOffset;
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
+use iggy_common::poll_messages::DEFAULT_PARTITION_ID;
use iggy_common::update_permissions::UpdatePermissions;
use iggy_common::update_stream::UpdateStream;
use iggy_common::update_topic::UpdateTopic;
@@ -103,8 +110,9 @@ use iggy_common::wire_conversions::{
};
use iggy_common::{
ClientInfo, ClientInfoDetails, ClusterMetadata, ClusterNode, ClusterNodeRole,
- ClusterNodeStatus, ConsumerGroup, ConsumerGroupDetails, Identifier, IdentityInfo, IggyError,
- IggyMessageView, IggyTimestamp, RawPersonalAccessToken, SendMessages, Stats, Stream,
+ ClusterNodeStatus, ConsumerGroup, ConsumerGroupDetails, ConsumerOffsetInfo, Identifier,
+ IdentityInfo, IggyError, IggyMessageView, IggyTimestamp, PollMessages, PolledMessages,
+ RESYNC_REQUIRED_PARTITION_SENTINEL, RawPersonalAccessToken, SendMessages, Stats, Stream,
StreamDetails, TokenInfo, Topic, TopicDetails, TransportEndpoints, UserInfo, UserInfoDetails,
Validatable,
};
@@ -115,21 +123,23 @@ use send_wrapper::SendWrapper;
use serde::Deserialize;
use server::http::error::{CustomError, ErrorResponse};
use server_common::Message;
+use shard::{PartitionRead, PartitionReadReply};
use tokio::sync::Mutex;
use tracing::{error, info, warn};
use crate::auth::{verify_login_credentials, verify_pat_credentials};
use crate::bootstrap::ServerNgShard;
use crate::dispatch::{
- dispatch_partition_request, submit_client_request_on_owner, submit_register_on_owner,
+ dispatch_partition_request, resolve_consumer_offset_request, resolve_poll_request,
+ submit_client_request_on_owner, submit_register_on_owner,
};
use crate::http::extractor::{Authenticated, Identity};
use crate::http::jwt::JwtManager;
use crate::login_register::LoginRegisterError;
use crate::pat::rewrite_pat_request_for_user;
use crate::responses::{
- NonReplicatedResponse, build_non_replicated_response, build_raw_pat_reply,
- connected_client_to_response,
+ NonReplicatedResponse, build_non_replicated_response, build_polled_messages_body,
+ build_raw_pat_reply, connected_client_to_response,
};
use crate::server_error::ServerNgError;
use crate::users::maybe_rewrite_user_password_request;
@@ -156,6 +166,15 @@ const FIRST_REQUEST_ID: u64 = 1;
/// wait gave up, so the server never retries on the caller's behalf.
const PRODUCE_REPLY_TIMEOUT: Duration = Duration::from_secs(10);
+/// VSR client id stamped on HTTP data-plane reads (poll / consumer-offset).
+/// HTTP reads never Register a VSR client, and shard-0 client ids are minted
+/// from 1, so 0 can never name a live consumer-group member: a group-kind
+/// poll fences closed with `ConsumerGroupPartitionNotOwned` and answers the
+/// re-sync sentinel empty poll - the same failure a stale TCP member sees.
+/// Legacy HTTP polls with client id 0 for the same reason (no persistent
+/// sessions, so no group membership).
+const HTTP_READ_CLIENT_ID: u128 = 0;
+
/// Response header attesting what durability a produce response proves:
/// [`DURABILITY_REPLICATED_MEMORY`] after an awaited quorum commit,
/// [`DURABILITY_NONE`] for a `?ack=none` fire-and-forget.
@@ -314,12 +333,23 @@ impl IntoResponse for ProduceError {
/// 504 body for a produce whose commit outcome is unknown. Shaped like every
/// other HTTP error (`ErrorResponse`) so clients parse one error schema.
fn produce_timeout_response() -> Response {
+ gateway_timeout_response(
+ "produce_timeout",
+ "produce was not acknowledged in time; the write may still commit",
+ )
+}
+
+/// Shared 504 rendering for an in-band request the partition plane did not
+/// answer in time, shaped like every other HTTP error (`ErrorResponse`) so
+/// clients parse one error schema. Consumed by the produce reply wait and the
+/// partition reads ([`ReadError::Timeout`]).
+fn gateway_timeout_response(code: &str, reason: &str) -> Response {
(
StatusCode::GATEWAY_TIMEOUT,
Json(ErrorResponse {
id: StatusCode::GATEWAY_TIMEOUT.as_u16().into(),
- code: "produce_timeout".to_owned(),
- reason: "produce was not acknowledged in time; the write may still commit".to_owned(),
+ code: code.to_owned(),
+ reason: reason.to_owned(),
field: None,
}),
)
@@ -404,6 +434,10 @@ enum ReadError {
/// query, so the caller re-issues the read against the leader (see
/// [`primary_redirect_response`]).
RedirectToPrimary(String),
+ /// A partition read (poll / consumer-offset) got no reply from the owning
+ /// shard within the mesh budget. 504 like a produce timeout: the outcome is
+ /// unknown (the abandoned read may still be running), so the caller retries.
+ Timeout,
}
impl IntoResponse for ReadError {
@@ -415,6 +449,10 @@ impl IntoResponse for ReadError {
Self::NotFound => CustomError::ResourceNotFound.into_response(),
Self::NotPrimary => not_primary_response(),
Self::RedirectToPrimary(location) => primary_redirect_response(&location),
+ Self::Timeout => gateway_timeout_response(
+ "partition_read_timeout",
+ "the partition owner did not answer the read in time; retry",
+ ),
}
}
}
@@ -750,7 +788,11 @@ fn router(state: HttpState) -> Router {
)
.route(
"/streams/{stream_id}/topics/{topic_id}/messages",
- post(send_messages),
+ get(poll_messages).post(send_messages),
+ )
+ .route(
+ "/streams/{stream_id}/topics/{topic_id}/consumer-offsets",
+ get(get_consumer_offset),
)
.route(
"/streams/{stream_id}/topics/{topic_id}/consumer-groups",
@@ -1312,6 +1354,112 @@ async fn purge_topic(
Ok(StatusCode::NO_CONTENT)
}
+/// `GET /streams/{stream_id}/topics/{topic_id}/messages`: poll a batch of
+/// messages as the same `PolledMessages` JSON the legacy server returns. The
+/// query is the same flattened `PollMessages` shape the legacy server accepts
+/// (`consumer_id`, `partition_id`, strategy `kind`+`value`, `count`,
+/// `auto_commit`); stream and topic come from the path.
+///
+/// A non-replicated read served in band: the same resolution the TCP dispatch
+/// runs ([`resolve_poll_request`]), then a mesh read on the owning shard and a
+/// re-encode of the stored batches into the legacy wire body, decoded here by
+/// the SDK's own [`PolledMessages::from_bytes`] so the JSON is field-identical
+/// to a TCP poll. `auto_commit` rides [`resolve_poll_request`]'s args and is
+/// honored by the owning shard's poll plan; no extra HTTP work.
+async fn poll_messages(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Query(query): Query<PollMessages>,
+ Query(consistency): Query<ConsistencyQuery>,
+) -> Result<Json<PolledMessages>, ReadError> {
+ authorize_read(&state, &identity, consistency.consistency)?;
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let wire = poll_wire_request(&stream_id, &topic_id, &query).map_err(ReadError::Rejected)?;
+ let (namespace, partition_id, consumer, args) =
+ match resolve_poll_request(&state.shard, &wire, HTTP_READ_CLIENT_ID) {
+ Ok(decoded) => decoded,
+ // TCP parity: a fenced group poll answers 200 with the re-sync
+ // sentinel partition, not an error (see [`HTTP_READ_CLIENT_ID`]).
+ Err(IggyError::ConsumerGroupPartitionNotOwned(..)) => {
+ return Ok(Json(resync_required_polled_messages()));
+ }
+ // The remaining resolver failures are STM lookups that came up
+ // empty (unknown stream, topic, partition, or consumer group), so
+ // they render as the legacy 404 body.
+ Err(_) => return Err(ReadError::NotFound),
+ };
+ let reply = SendWrapper::new(
+ state
+ .shard
+ .partition_read(namespace, PartitionRead::Poll { consumer, args }),
+ )
+ .await;
+ match reply {
+ Some(PartitionReadReply::Poll {
+ fragments,
+ current_offset,
+ }) => {
+ let body = build_polled_messages_body(partition_id, current_offset, fragments)
+ .map_err(ReadError::Rejected)?;
+ Ok(Json(
+ PolledMessages::from_bytes(body).map_err(ReadError::Rejected)?,
+ ))
+ }
+ Some(PartitionReadReply::NotFound) => Err(ReadError::NotFound),
+ Some(_) => Err(ReadError::Rejected(IggyError::InvalidCommand)),
+ None => Err(ReadError::Timeout),
+ }
+}
+
+/// `GET /streams/{stream_id}/topics/{topic_id}/consumer-offsets`: fetch a
+/// consumer's stored offset as the same `ConsumerOffsetInfo` JSON the legacy
+/// server returns. The query is the same flattened `GetConsumerOffset` shape
+/// the legacy server accepts (`consumer_id`, optional `partition_id`).
+///
+/// A non-replicated read served in band, mirroring [`poll_messages`]. A
+/// missing offset (never stored, or the partition unknown to its owner) is
+/// the legacy 404: the TCP path replies an empty body the SDK decodes as
+/// `None`, and the legacy HTTP server renders that `None` as
+/// `CustomError::ResourceNotFound`.
+async fn get_consumer_offset(
+ State(state): State<HttpState>,
+ identity: Identity,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Query(query): Query<GetConsumerOffset>,
+ Query(consistency): Query<ConsistencyQuery>,
+) -> Result<Json<ConsumerOffsetInfo>, ReadError> {
+ authorize_read(&state, &identity, consistency.consistency)?;
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(ReadError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(ReadError::Rejected)?;
+ let wire =
+ consumer_offset_wire_request(&stream_id, &topic_id, &query).map_err(ReadError::Rejected)?;
+ let (namespace, partition_id, consumer) =
+ resolve_consumer_offset_request(&state.shard, &wire).map_err(|_| ReadError::NotFound)?;
+ let reply = SendWrapper::new(
+ state
+ .shard
+ .partition_read(namespace, PartitionRead::ConsumerOffset { consumer }),
+ )
+ .await;
+ match reply {
+ Some(PartitionReadReply::ConsumerOffset {
+ stored: Some(stored_offset),
+ current_offset,
+ }) => Ok(Json(ConsumerOffsetInfo {
+ partition_id,
+ current_offset,
+ stored_offset,
+ })),
+ Some(
+ PartitionReadReply::ConsumerOffset { stored: None, .. } | PartitionReadReply::NotFound,
+ ) => Err(ReadError::NotFound),
+ Some(_) => Err(ReadError::Rejected(IggyError::InvalidCommand)),
+ None => Err(ReadError::Timeout),
+ }
+}
+
/// `POST /streams/{stream_id}/topics/{topic_id}/messages`: produce a batch of
/// messages to a topic. The JSON body is the same `SendMessages` shape the
/// legacy server accepts (partitioning + base64 messages); stream and topic
@@ -1757,6 +1905,65 @@ fn encode_send_messages(
Ok(buf.freeze())
}
+/// Map a validated HTTP poll query onto the wire `PollMessagesRequest` the
+/// shared TCP resolver consumes, so both transports resolve one request shape.
+/// The query's consumer kind is structurally always `Consumer`
+/// (`Consumer::kind` is `#[serde(skip)]` - the flattened `kind` param names
+/// the polling strategy), matching the legacy HTTP server.
+fn poll_wire_request(
+ stream_id: &Identifier,
+ topic_id: &Identifier,
+ query: &PollMessages,
+) -> Result<PollMessagesRequest, IggyError> {
+ Ok(PollMessagesRequest {
+ consumer: WireConsumer {
+ kind: query.consumer.kind.as_code(),
+ id: identifier_to_wire(&query.consumer.id)?,
+ },
+ stream_id: identifier_to_wire(stream_id)?,
+ topic_id: identifier_to_wire(topic_id)?,
+ partition_id: query.partition_id,
+ strategy: WirePollingStrategy {
+ kind: query.strategy.kind.as_code(),
+ value: query.strategy.value,
+ },
+ count: query.count,
+ auto_commit: query.auto_commit,
+ })
+}
+
+/// Map a validated HTTP consumer-offset query onto the wire
+/// `GetConsumerOffsetRequest` the shared TCP resolver consumes. An omitted
+/// `partition_id` defaults to partition 0, matching the legacy server's
+/// `resolve_consumer_with_partition_id` (`partition_id.unwrap_or(0)`).
+fn consumer_offset_wire_request(
+ stream_id: &Identifier,
+ topic_id: &Identifier,
+ query: &GetConsumerOffset,
+) -> Result<GetConsumerOffsetRequest, IggyError> {
+ Ok(GetConsumerOffsetRequest {
+ consumer: WireConsumer {
+ kind: query.consumer.kind.as_code(),
+ id: identifier_to_wire(&query.consumer.id)?,
+ },
+ stream_id: identifier_to_wire(stream_id)?,
+ topic_id: identifier_to_wire(topic_id)?,
+ partition_id: Some(query.partition_id.unwrap_or(DEFAULT_PARTITION_ID)),
+ })
+}
+
+/// The empty `PolledMessages` a fenced consumer-group poll answers, carrying
+/// the re-sync sentinel in `partition_id` exactly as the TCP dispatch replies
+/// it, so an SDK re-syncs its assignment instead of reading end-of-partition.
+const fn resync_required_polled_messages() -> PolledMessages {
+ PolledMessages {
+ partition_id: RESYNC_REQUIRED_PARTITION_SENTINEL,
+ current_offset: 0,
+ count: 0,
+ messages: Vec::new(),
+ }
+}
+
/// Run one awaited produce end to end: install the reply slot, dispatch into
/// the partition plane, and wait (bounded) for the committed reply.
///
@@ -2166,9 +2373,20 @@ const fn login_error_to_iggy(error: &LoginRegisterError) -> IggyError {
#[cfg(test)]
mod tests {
+ use axum::http::Uri;
use configs::cluster::TransportPorts;
use iggy_binary_protocol::PrepareHeader;
- use iggy_common::{IggyMessage, IggyMessagesBatch, Partitioning, PartitioningKind};
+ use iggy_common::{
+ Consumer, ConsumerKind, IggyMessage, IggyMessagesBatch, Partitioning, PartitioningKind,
+ PollingKind, PollingStrategy,
+ };
+ use partitions::{Fragment, PollFragments};
+ use server_common::MESSAGE_ALIGN;
+ use server_common::iobuf::Owned;
+ use server_common::send_messages2::{
+ COMMAND_HEADER_SIZE, IggyMessage2, IggyMessage2Header, IggyMessages2, SendMessages2Header,
+ SendMessages2Owned,
+ };
use super::*;
use crate::responses::build_empty_reply;
@@ -2319,6 +2537,169 @@ mod tests {
assert!(command.validate().is_err());
}
+ // -- poll / consumer-offset query parsing and wire mapping --
+
+ #[test]
+ fn poll_query_parses_with_documented_defaults() {
+ let uri: Uri = "/streams/1/topics/1/messages?consumer_id=42"
+ .parse()
+ .expect("valid uri");
+ let Query(query) = Query::<PollMessages>::try_from_uri(&uri).expect("parses");
+ assert_eq!(query.consumer.kind, ConsumerKind::Consumer);
+ assert_eq!(
+ query.consumer.id,
+ Identifier::numeric(42).expect("valid id")
+ );
+ assert_eq!(query.partition_id, Some(0));
+ assert_eq!(query.strategy, PollingStrategy::offset(0));
+ assert_eq!(query.count, 10);
+ assert!(!query.auto_commit);
+ }
+
+ #[test]
+ fn poll_query_parses_explicit_strategy_count_and_tolerates_consistency_param() {
+ let uri: Uri = "/x?consumer_id=app&partition_id=3&kind=timestamp&value=42&count=5&auto_commit=true&consistency=linearizable"
+ .parse()
+ .expect("valid uri");
+ let Query(query) = Query::<PollMessages>::try_from_uri(&uri).expect("parses");
+ assert_eq!(
+ query.consumer.id,
+ Identifier::named("app").expect("valid id")
+ );
+ assert_eq!(query.partition_id, Some(3));
+ assert_eq!(
+ query.strategy,
+ PollingStrategy::timestamp(IggyTimestamp::from(42))
+ );
+ assert_eq!(query.count, 5);
+ assert!(query.auto_commit);
+ let Query(consistency) = Query::<ConsistencyQuery>::try_from_uri(&uri).expect("parses");
+ assert!(consistency.consistency == Consistency::Linearizable);
+ }
+
+ #[test]
+ fn poll_wire_request_maps_query_onto_tcp_request_shape() {
+ let stream_id = Identifier::from_str_value("orders").expect("valid stream id");
+ let topic_id = Identifier::from_str_value("1").expect("valid topic id");
+ let query = PollMessages {
+ consumer: Consumer {
+ kind: ConsumerKind::Consumer,
+ id: Identifier::numeric(9).expect("valid id"),
+ },
+ partition_id: Some(4),
+ strategy: PollingStrategy::next(),
+ count: 25,
+ auto_commit: true,
+ ..Default::default()
+ };
+ let wire = poll_wire_request(&stream_id, &topic_id, &query).expect("maps");
+ assert_eq!(wire.consumer.kind, 1);
+ assert_eq!(
+ wire.consumer.id,
+ identifier_to_wire(&query.consumer.id).unwrap()
+ );
+ assert_eq!(wire.stream_id, identifier_to_wire(&stream_id).unwrap());
+ assert_eq!(wire.topic_id, identifier_to_wire(&topic_id).unwrap());
+ assert_eq!(wire.partition_id, Some(4));
+ assert_eq!(wire.strategy.kind, PollingKind::Next.as_code());
+ assert_eq!(wire.strategy.value, 0);
+ assert_eq!(wire.count, 25);
+ assert!(wire.auto_commit);
+ }
+
+ #[test]
+ fn consumer_offset_wire_request_defaults_omitted_partition_to_zero() {
+ let stream_id = Identifier::numeric(1).expect("valid stream id");
+ let topic_id = Identifier::numeric(1).expect("valid topic id");
+ let query = GetConsumerOffset {
+ consumer: Consumer::new(Identifier::numeric(7).expect("valid id")),
+ partition_id: None,
+ };
+ let wire = consumer_offset_wire_request(&stream_id, &topic_id, &query).expect("maps");
+ assert_eq!(wire.partition_id, Some(DEFAULT_PARTITION_ID));
+ assert_eq!(wire.consumer.kind, 1);
+ }
+
+ // -- polled messages body decode --
+
+ /// Wrap one stored `SendMessages2` batch (`[256B header][blob]`) as the
+ /// poll fragment the owning shard replies, the shape
+ /// `build_polled_messages_body` consumes.
+ fn fragment_from_stored_batch(header: &SendMessages2Header, blob: &[u8]) -> PollFragments {
+ let mut buffer = Owned::<MESSAGE_ALIGN>::zeroed(COMMAND_HEADER_SIZE + blob.len());
+ header.encode_into(buffer.as_mut_slice());
+ buffer.as_mut_slice()[COMMAND_HEADER_SIZE..].copy_from_slice(blob);
+ let mut fragments = PollFragments::new();
+ fragments.push(Fragment::whole(buffer.into()));
+ fragments
+ }
+
+ /// Round-trip the poll route's encode/decode seam: the store's own batch
+ /// writer (`SendMessages2Owned::from_messages`) is the encoder oracle,
+ /// `build_polled_messages_body` re-encodes to the legacy wire body, and
+ /// the SDK's `PolledMessages::from_bytes` must read back every field.
+ #[test]
+ fn polled_messages_body_decodes_into_common_polled_messages() {
+ let mut messages = IggyMessages2::with_capacity(2);
+ messages.push(IggyMessage2 {
+ header: IggyMessage2Header {
+ id: 7,
+ origin_timestamp: 1_000,
+ ..Default::default()
+ },
+ payload: Bytes::from_static(b"first"),
+ user_headers: None,
+ });
+ messages.push(IggyMessage2 {
+ header: IggyMessage2Header {
+ id: 8,
+ origin_timestamp: 1_050,
+ ..Default::default()
+ },
+ payload: Bytes::from_static(b"second"),
+ user_headers: Some(Bytes::from_static(b"raw-header-bytes")),
+ });
+ let namespace = server_common::sharding::IggyNamespace::new(0, 0, 3);
+ let stored =
+ SendMessages2Owned::from_messages(namespace, &messages).expect("encodes stored batch");
+ // The store stamps these on append; `from_messages` leaves them zero.
+ let mut header = stored.header;
+ header.base_offset = 41;
+ header.base_timestamp = 999_999;
+
+ let body =
+ build_polled_messages_body(3, 42, fragment_from_stored_batch(&header, &stored.blob))
+ .expect("re-encodes wire body");
+ let polled = PolledMessages::from_bytes(body).expect("decodes as the SDK does");
+
+ assert_eq!(polled.partition_id, 3);
+ assert_eq!(polled.current_offset, 42);
+ assert_eq!(polled.count, 2);
+ assert_eq!(polled.messages.len(), 2);
+ assert_eq!(polled.messages[0].header.id, 7);
+ assert_eq!(polled.messages[0].header.offset, 41);
+ assert_eq!(polled.messages[0].header.timestamp, 999_999);
+ assert_eq!(polled.messages[0].header.origin_timestamp, 1_000);
+ assert_eq!(polled.messages[0].payload.as_ref(), b"first");
+ assert!(polled.messages[0].user_headers.is_none());
+ assert_eq!(polled.messages[1].header.id, 8);
+ assert_eq!(polled.messages[1].header.offset, 42);
+ assert_eq!(polled.messages[1].header.origin_timestamp, 1_050);
+ assert_eq!(polled.messages[1].payload.as_ref(), b"second");
+ assert_eq!(
+ polled.messages[1].user_headers.as_deref(),
+ Some(b"raw-header-bytes".as_ref())
+ );
+ }
+
+ #[test]
+ fn resync_required_polled_messages_carries_sentinel_partition() {
+ let polled = resync_required_polled_messages();
+ assert_eq!(polled.partition_id, RESYNC_REQUIRED_PARTITION_SENTINEL);
+ assert_eq!(polled.count, 0);
+ assert!(polled.messages.is_empty());
+ }
+
// -- classify_partition_reply --
fn frozen(reply: Message<iggy_binary_protocol::ReplyHeader>) -> BusMessage {
(iggy) 42/43: fix(partitions): reply typed deny on delete of missing consumer offset
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 80be4a9ad73fe0a28ac69f48b6147f4c17f3107b
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 23:09:17 2026 +0200
fix(partitions): reply typed deny on delete of missing consumer offset
Deleting a consumer offset that was never stored was silently
dropped on the partition primary, so an awaited HTTP write hung
for the full 10s reply timeout into a 504 and a TCP client waited
out its read timeout instead of learning the offset does not
exist.
The primary now answers a deny reply before the op enters the VSR
pipeline: an empty body with ConsumerOffsetNotFound carried in
ReplyHeader.status, the pre-commit error channel the SDK already
peeks before body decode. Nothing is prepared or replicated, so
backups never see the rejected delete and deterministic apply is
unaffected. The HTTP write classifier reads status ahead of the
op==0 gate-failure fallback and renders the legacy typed 404;
committed replies (op >= 1) and status-0 gate failures behave as
before.
---
core/binary_protocol/src/consensus/header.rs | 6 +-
core/consensus/src/plane_helpers.rs | 71 ++++++++++-
core/integration/tests/server/http_vsr.rs | 84 +++++++++++++
core/partitions/src/iggy_partition.rs | 179 ++++++++++++++++++++++++++-
core/server-ng/src/http.rs | 64 +++++++---
core/server-ng/src/responses.rs | 5 +-
6 files changed, 384 insertions(+), 25 deletions(-)
diff --git a/core/binary_protocol/src/consensus/header.rs b/core/binary_protocol/src/consensus/header.rs
index bc0d7b2fd..f374c86a6 100644
--- a/core/binary_protocol/src/consensus/header.rs
+++ b/core/binary_protocol/src/consensus/header.rs
@@ -280,10 +280,10 @@ pub struct ReplyHeader {
pub operation_padding: [u8; 7],
pub namespace: u64,
/// Request-level status: 0 = ok; nonzero = the `IggyError` code for a
- /// failure decided at dispatch time (authorization denial today; other
- /// pre-commit gate failures may adopt it later).
+ /// failure decided before commit (a dispatch-time authorization denial, or
+ /// the partition primary rejecting a delete of a missing consumer offset).
///
- /// Contract: this is nonzero ONLY on a dispatch-time denial, and a deny
+ /// Contract: this is nonzero ONLY on a pre-commit denial, and a deny
/// reply always carries an EMPTY body. So this header channel and the
/// committed per-sub-op results in the metadata result section are mutually
/// exclusive by construction: a reply either commits (status 0, result
diff --git a/core/consensus/src/plane_helpers.rs b/core/consensus/src/plane_helpers.rs
index f40562cb3..2a2585351 100644
--- a/core/consensus/src/plane_helpers.rs
+++ b/core/consensus/src/plane_helpers.rs
@@ -388,6 +388,37 @@ where
.expect("reply buffer must contain a valid reply message")
}
+/// Reply that denies a request on the primary before it enters the VSR
+/// pipeline.
+///
+/// The request's frame with an empty body and a nonzero `ReplyHeader.status`
+/// (the request-level error channel the SDK peeks before body decode).
+/// Nothing is prepared or replicated, so backups never see the denied
+/// request; `op` stays 0, the shape reply consumers already read as "nothing
+/// committed".
+///
+/// # Panics
+/// If the constructed message buffer is not valid.
+pub fn build_deny_reply_from_request<B, P>(
+ consensus: &VsrConsensus<B, P>,
+ request_header: &RequestHeader,
+ status: u32,
+) -> Message<ReplyHeader>
+where
+ B: MessageBus,
+ P: Pipeline<Entry = PipelineEntry>,
+{
+ let mut reply = build_reply_from_request(consensus, request_header, bytes::Bytes::new());
+ let header_size = std::mem::size_of::<ReplyHeader>();
+ let header = bytemuck::checked::try_from_bytes_mut::<ReplyHeader>(
+ &mut reply.as_mut_slice()[..header_size],
+ )
+ .expect("freshly built reply header is valid");
+ header.status = status;
+ header.op = 0;
+ reply
+}
+
/// Verify hash chain would not break if we add this header.
///
/// # Panics
@@ -475,7 +506,7 @@ pub async fn send_prepare_ok<B, P>(
mod tests {
use super::*;
use crate::{Consensus, LocalPipeline, VsrAction};
- use iggy_binary_protocol::StartViewChangeHeader;
+ use iggy_binary_protocol::{ConsensusHeader, Operation, StartViewChangeHeader};
use message_bus::SendError;
use server_common::{MESSAGE_ALIGN, iobuf::Frozen};
@@ -920,4 +951,42 @@ mod tests {
Some(7)
);
}
+
+ // A deny echoes the request frame with an empty body, carries the error
+ // code in `status`, and keeps `op` at 0 even when the group has committed
+ // ops -- reply consumers read `op == 0` as "nothing committed", and the
+ // deny must never be mistaken for a committed reply.
+ #[test]
+ fn deny_reply_from_request_stamps_status_and_commits_nothing() {
+ let consensus = VsrConsensus::new(1, 0, 3, 0, NoopBus, LocalPipeline::new());
+ consensus.init();
+ consensus.advance_commit_max(4);
+
+ let request = RequestHeader {
+ command: Command2::Request,
+ operation: Operation::DeleteConsumerOffset2,
+ client: 42,
+ request: 7,
+ namespace: 9,
+ ..Default::default()
+ };
+ let status = 3021;
+ let reply = build_deny_reply_from_request(&consensus, &request, status);
+
+ let header = reply.header();
+ assert_eq!(header.command, Command2::Reply);
+ assert_eq!(header.status, status);
+ assert_eq!(header.op, 0, "a deny commits nothing");
+ assert_eq!(header.commit, 4);
+ assert_eq!(header.client, 42);
+ assert_eq!(header.request, 7);
+ assert_eq!(header.namespace, 9);
+ assert_eq!(header.operation, Operation::DeleteConsumerOffset2);
+ assert_eq!(
+ header.size as usize,
+ std::mem::size_of::<ReplyHeader>(),
+ "deny reply body must be empty"
+ );
+ assert!(header.validate().is_ok());
+ }
}
diff --git a/core/integration/tests/server/http_vsr.rs b/core/integration/tests/server/http_vsr.rs
index cbd203b70..9876c2de0 100644
--- a/core/integration/tests/server/http_vsr.rs
+++ b/core/integration/tests/server/http_vsr.rs
@@ -520,6 +520,90 @@ async fn given_consumer_offset_when_stored_and_deleted_should_round_trip(harness
);
}
+/// A DELETE of a never-stored consumer offset is denied by the partition
+/// primary before consensus (`ReplyHeader.status`), so the route must answer
+/// a typed 404 immediately - not wait out the 10s partition-write reply
+/// timeout into a 504 - and the deny gate must not break a store-then-delete
+/// of an offset that does exist.
+#[iggy_harness]
+async fn given_missing_consumer_offset_when_deleting_should_reject_404_fast(harness: &TestHarness) {
+ /// Well under the partition-write reply timeout: a deny is a single
+ /// round trip, a silent drop only answers at the full 10s.
+ const DENY_LATENCY_BUDGET: Duration = Duration::from_secs(5);
+
+ let http = HttpSession::login_root(harness).await;
+ http.create_stream_and_topic("http-missing-offset", "offsets", 1)
+ .await;
+
+ let offsets_path = "/streams/http-missing-offset/topics/offsets/consumer-offsets";
+ let delete_path = format!("{offsets_path}/{CONSUMER_ID}?partition_id={PARTITION_ID}");
+ let started = Instant::now();
+ let response = http
+ .client
+ .delete(http.url(&delete_path))
+ .bearer_auth(&http.token)
+ .send()
+ .await
+ .expect("delete missing offset request");
+ let elapsed = started.elapsed();
+ assert_eq!(
+ response.status(),
+ StatusCode::NOT_FOUND,
+ "missing offset delete must reject 404"
+ );
+ assert!(
+ elapsed < DENY_LATENCY_BUDGET,
+ "deny must answer fast, not ride the reply timeout (took {elapsed:?})"
+ );
+ // The typed deny body, not the generic pre-dispatch not-found shape.
+ let body: serde_json::Value = response.json().await.expect("decode error body");
+ assert_eq!(
+ body["code"], "consumer_offset_not_found",
+ "deny must carry the typed error code, got {body}"
+ );
+
+ // The deny gate must not reject offsets that exist: store, then delete.
+ let messages = vec![text_message(1, "offset-fodder".to_string())];
+ let response = http
+ .produce("http-missing-offset", "offsets", PARTITION_ID, messages)
+ .await;
+ assert_eq!(
+ response.status(),
+ StatusCode::CREATED,
+ "produce must commit"
+ );
+ let store = StoreConsumerOffset {
+ consumer: Consumer::new(Identifier::numeric(CONSUMER_ID).expect("consumer id")),
+ partition_id: Some(PARTITION_ID),
+ offset: 0,
+ };
+ let response = http
+ .client
+ .put(http.url(offsets_path))
+ .bearer_auth(&http.token)
+ .json(&store)
+ .send()
+ .await
+ .expect("store offset request");
+ assert_eq!(
+ response.status(),
+ StatusCode::NO_CONTENT,
+ "store must commit"
+ );
+ let response = http
+ .client
+ .delete(http.url(&delete_path))
+ .bearer_auth(&http.token)
+ .send()
+ .await
+ .expect("delete stored offset request");
+ assert_eq!(
+ response.status(),
+ StatusCode::NO_CONTENT,
+ "existing offset delete must commit"
+ );
+}
+
#[iggy_harness]
async fn given_oversized_body_when_producing_should_reject_413(harness: &TestHarness) {
let http = HttpSession::login_root(harness).await;
diff --git a/core/partitions/src/iggy_partition.rs b/core/partitions/src/iggy_partition.rs
index dc7fbe113..226e89f71 100644
--- a/core/partitions/src/iggy_partition.rs
+++ b/core/partitions/src/iggy_partition.rs
@@ -33,10 +33,10 @@ use crate::{
use consensus::{
CommitLogEvent, Consensus, PartitionDiagEvent, Pipeline, PipelineEntry, PlaneKind, Project,
ReplicaLogContext, RequestLogEvent, Sequencer, SimEventKind, VsrConsensus, ack_preflight,
- ack_quorum_reached, build_reply_from_request, build_reply_message, drain_committable_prefix,
- emit_namespace_progress_event, emit_partition_diag, emit_sim_event,
- fence_old_prepare_by_commit, replicate_preflight, replicate_to_next_in_chain,
- send_prepare_ok as send_prepare_ok_common,
+ ack_quorum_reached, build_deny_reply_from_request, build_reply_from_request,
+ build_reply_message, drain_committable_prefix, emit_namespace_progress_event,
+ emit_partition_diag, emit_sim_event, fence_old_prepare_by_commit, replicate_preflight,
+ replicate_to_next_in_chain, send_prepare_ok as send_prepare_ok_common,
};
use iggy_binary_protocol::requests::consumer_offsets::{
DeleteConsumerOffset2Request, DeleteConsumerOffsetRequest, StoreConsumerOffset2Request,
@@ -1027,6 +1027,27 @@ where
.with_operation(message.header().operation)
.with_error(error.to_string()),
);
+ // Deny on the primary before the op enters the pipeline:
+ // nothing replicates, so backups never see the rejected
+ // delete, and the client gets a typed failure instead of
+ // waiting out its reply timeout.
+ let reply =
+ build_deny_reply_from_request(consensus, message.header(), error.as_code());
+ if let Err(send_error) = consensus
+ .message_bus()
+ .send_to_client(message.header().client, reply.into_generic().into_frozen())
+ .await
+ {
+ emit_partition_diag(
+ tracing::Level::WARN,
+ &PartitionDiagEvent::new(
+ ReplicaLogContext::from_consensus(consensus, PlaneKind::Partitions),
+ "delete_consumer_offset deny reply send failed",
+ )
+ .with_operation(message.header().operation)
+ .with_error(send_error.to_string()),
+ );
+ }
return;
}
@@ -2714,9 +2735,14 @@ mod tests {
use bytes::Bytes;
use compio::io::AsyncWriteAtExt;
use consensus::LocalPipeline;
+ use iggy_binary_protocol::{Command2, ReplyHeader, WireConsumer, WireEncode};
+ use message_bus::SendError;
+ use server_common::MESSAGE_ALIGN;
use server_common::send_messages2::{
COMMAND_HEADER_SIZE, IggyMessage2, IggyMessage2Header, IggyMessages2, SendMessages2Owned,
};
+ use std::cell::RefCell;
+ use std::rc::Rc;
const TEST_CLUSTER: u128 = 1;
@@ -2739,6 +2765,151 @@ mod tests {
)
}
+ /// Client-facing bus that records every `send_to_client` frame so tests
+ /// can assert on reply bytes without a connection registry (whose slot
+ /// guard would borrow the partition across `on_request(&mut self)`).
+ #[derive(Debug, Default)]
+ struct RecordingBus {
+ sent_to_clients: Rc<RefCell<Vec<(u128, Frozen<MESSAGE_ALIGN>)>>>,
+ }
+
+ impl MessageBus for RecordingBus {
+ fn track_background(&self, _handle: message_bus::JoinHandle<()>) {}
+
+ async fn send_to_client(
+ &self,
+ client_id: u128,
+ data: Frozen<MESSAGE_ALIGN>,
+ ) -> Result<(), SendError> {
+ self.sent_to_clients.borrow_mut().push((client_id, data));
+ Ok(())
+ }
+
+ async fn send_to_replica(
+ &self,
+ _replica: u8,
+ _data: Frozen<MESSAGE_ALIGN>,
+ ) -> Result<(), SendError> {
+ Ok(())
+ }
+
+ fn set_connection_lost_fn(&self, _f: message_bus::ConnectionLostFn) {}
+ fn set_replica_forward_fn(&self, _f: message_bus::ReplicaForwardFn) {}
+ fn set_client_forward_fn(&self, _f: message_bus::ClientForwardFn) {}
+ }
+
+ type SentFrames = Rc<RefCell<Vec<(u128, Frozen<MESSAGE_ALIGN>)>>>;
+
+ fn recording_partition() -> (IggyPartition<RecordingBus>, SentFrames) {
+ let namespace = IggyNamespace::new(1, 1, 0);
+ let bus = RecordingBus::default();
+ let sent_to_clients = bus.sent_to_clients.clone();
+ let consensus = VsrConsensus::new(
+ TEST_CLUSTER,
+ 0,
+ 1,
+ namespace.inner(),
+ bus,
+ LocalPipeline::new(),
+ );
+ consensus.init();
+ let partition = IggyPartition::with_in_memory_storage(
+ Arc::new(PartitionStats::default()),
+ consensus,
+ IggyByteSize::from(1024 * 1024),
+ false,
+ );
+ (partition, sent_to_clients)
+ }
+
+ fn delete_offset_request(
+ client_id: u128,
+ request_id: u64,
+ consumer_id: u32,
+ ) -> Message<RequestHeader> {
+ let body = DeleteConsumerOffset2Request {
+ consumer: WireConsumer::consumer(WireIdentifier::Numeric(consumer_id)),
+ stream_id: WireIdentifier::Numeric(1),
+ topic_id: WireIdentifier::Numeric(1),
+ partition_id: Some(0),
+ ack: AckLevel::Quorum,
+ }
+ .to_bytes();
+ let header_size = std::mem::size_of::<RequestHeader>();
+ let total = header_size + body.len();
+ let mut message = Message::<RequestHeader>::new(total);
+ message.as_mut_slice()[header_size..].copy_from_slice(&body);
+ message.transmute_header(|_, header: &mut RequestHeader| {
+ header.command = Command2::Request;
+ header.operation = Operation::DeleteConsumerOffset2;
+ header.client = client_id;
+ header.session = 1;
+ header.request = request_id;
+ header.namespace = IggyNamespace::new(1, 1, 0).inner();
+ header.size = u32::try_from(total).expect("request size fits u32");
+ })
+ }
+
+ /// Deleting a consumer offset that was never stored must answer with a
+ /// typed deny reply (empty body, `status` = `ConsumerOffsetNotFound`,
+ /// `op` 0) before consensus: nothing may enter the pipeline, and an
+ /// awaited client write must fail fast instead of waiting out its reply
+ /// timeout. Once the offset exists, the same request must pass the gate
+ /// into the pipeline without a deny.
+ #[compio::test]
+ async fn on_request_delete_of_missing_offset_replies_typed_deny() {
+ let (mut partition, sent_to_clients) = recording_partition();
+ let client_id: u128 = 42;
+ let consumer_id: u32 = 5;
+
+ partition
+ .on_request(delete_offset_request(client_id, 7, consumer_id))
+ .await;
+
+ {
+ let sent = sent_to_clients.borrow();
+ assert_eq!(sent.len(), 1, "exactly one deny reply");
+ let (reply_client, frame) = &sent[0];
+ assert_eq!(*reply_client, client_id);
+ let header = bytemuck::checked::try_from_bytes::<ReplyHeader>(
+ &frame.as_slice()[..std::mem::size_of::<ReplyHeader>()],
+ )
+ .expect("deny frame starts with a valid reply header");
+ assert_eq!(header.command, Command2::Reply);
+ assert_eq!(
+ header.status,
+ IggyError::ConsumerOffsetNotFound(0).as_code()
+ );
+ assert_eq!(header.op, 0, "a deny commits nothing");
+ assert_eq!(header.request, 7);
+ assert_eq!(
+ header.size as usize,
+ std::mem::size_of::<ReplyHeader>(),
+ "deny reply body must be empty"
+ );
+ }
+ assert_eq!(
+ partition.consensus().pipeline().borrow().len(),
+ 0,
+ "denied delete must not replicate"
+ );
+ assert!(partition.pending_consumer_offset_commits.is_empty());
+
+ // Existing offset: the gate passes and the delete enters the pipeline.
+ partition.consumer_offsets.pin().insert(
+ consumer_id as usize,
+ ConsumerOffset::new(ConsumerKind::Consumer, consumer_id, 3, String::new()),
+ );
+ partition
+ .on_request(delete_offset_request(client_id, 8, consumer_id))
+ .await;
+ assert_eq!(
+ partition.consensus().pipeline().borrow().len(),
+ 1,
+ "existing offset delete must replicate"
+ );
+ }
+
/// `reclaim_dead_group_offsets` must drop exactly the not-`is_live` groups
/// from the in-memory map and hand back their owned persisted-file paths,
/// leaving live groups untouched. The returned `Vec<String>` is what the
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 02ffc1f52..13f60fdd0 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -371,8 +371,9 @@ impl IntoResponse for WriteError {
#[derive(Debug)]
enum PartitionWriteError {
/// Caller-side rejection (bad identifier, oversized batch, an authorization
- /// denial) or a malformed reply frame, rendered through the legacy
- /// `IggyError -> status` map for SDK-identical bodies.
+ /// denial), a typed pre-commit deny from the partition plane
+ /// (`ReplyHeader.status`), or a malformed reply frame, rendered through the
+ /// legacy `IggyError -> status` map for SDK-identical bodies.
Rejected(IggyError),
/// The dispatch gates could not route the write: the stream, topic, or
/// partition does not resolve (or never materialised within the routable
@@ -1960,7 +1961,9 @@ async fn store_consumer_offset(
/// delete a consumer's stored offset. The consumer comes from the path and the
/// optional `partition_id` from the query, the same `DeleteConsumerOffset`
/// shape the legacy server accepts. Returns 204 on commit, matching the legacy
-/// server. Same replicated partition write as [`store_consumer_offset`].
+/// server; a delete of a never-stored offset is denied by the partition
+/// primary (`ReplyHeader.status`) and renders the legacy typed 404. Same
+/// replicated partition write as [`store_consumer_offset`].
async fn delete_consumer_offset(
State(state): State<HttpState>,
identity: Authenticated,
@@ -2699,12 +2702,16 @@ fn ensure_in_process_reply_target(state: &HttpInner, session: &HttpSession) {
}
/// Discriminate a partition write reply. Partition replies carry no result
-/// section and no error code (success and gate failure are both empty-bodied),
-/// so the only wire discriminator is the reply's `op`: a committed reply is
-/// built from its prepare header, whose op (the partition group's commit
-/// number) is always >= 1, while the pre-dispatch gate failures reply through
-/// `build_empty_reply` with 0 in that field. The gate reply cannot name which
-/// entity was missing, hence the generic legacy 404 body.
+/// section (success and denial are both empty-bodied), so the discriminators
+/// live in the header. `status` is read first: a nonzero value is the typed
+/// pre-commit denial (the partition primary's delete-of-missing-offset
+/// rejection, or a dispatch-time authorization denial) and renders through
+/// the legacy `IggyError -> status` map. With status 0, the reply's `op`
+/// splits the rest: a committed reply is built from its prepare header, whose
+/// op (the partition group's commit number) is always >= 1, while the
+/// pre-dispatch gate failures reply through `build_empty_reply` with 0 in
+/// that field. The gate reply cannot name which entity was missing, hence the
+/// generic legacy 404 body.
fn classify_partition_reply(reply: &BusMessage) -> Result<(), PartitionWriteError> {
let header = reply
.as_slice()
@@ -2714,6 +2721,11 @@ fn classify_partition_reply(reply: &BusMessage) -> Result<(), PartitionWriteErro
if header.command != Command2::Reply {
return Err(PartitionWriteError::Rejected(IggyError::InvalidCommand));
}
+ if header.status != 0 {
+ return Err(PartitionWriteError::Rejected(IggyError::from_code(
+ header.status,
+ )));
+ }
if header.op == 0 {
return Err(PartitionWriteError::NotFound);
}
@@ -3157,10 +3169,12 @@ mod tests {
assert_eq!(reply.header().operation, request.header().operation);
}
- // Enforces the status contract: every reply builder EXCEPT build_deny_reply
- // leaves status 0, so a future builder that leaks a nonzero reserved tail
- // (or sets status by mistake) trips here rather than silently shipping a
- // fake denial. build_deny_reply is the sole nonzero-status writer.
+ // Enforces the status contract: every ok-path reply builder leaves status
+ // 0, so a future builder that leaks a nonzero reserved tail (or sets
+ // status by mistake) trips here rather than silently shipping a fake
+ // denial. The deny builders (build_deny_reply here, the partition
+ // primary's consensus::build_deny_reply_from_request) are the only
+ // nonzero-status writers.
#[test]
fn only_the_deny_builder_emits_a_nonzero_status() {
let request = build_request_message(Operation::CreateStream, 42, 7, 3, &[]);
@@ -3205,13 +3219,13 @@ mod tests {
"committed replies carry status 0"
);
- // The one intentional writer stamps nonzero.
+ // The intentional deny writer stamps nonzero.
assert_ne!(
build_deny_reply(header, 42, 7, commit, IggyError::Unauthorized.as_code())
.header()
.status,
0,
- "the deny builder is the sole nonzero-status writer"
+ "the deny builder must stamp a nonzero status"
);
}
@@ -3655,6 +3669,26 @@ mod tests {
));
}
+ /// The partition primary's typed pre-commit deny (delete of a missing
+ /// consumer offset) rides `ReplyHeader.status` and must classify as the
+ /// mapped `IggyError`, not as the generic op-0 not-found.
+ #[test]
+ fn status_bearing_deny_reply_classifies_as_typed_rejection() {
+ let request = build_request_message(Operation::DeleteConsumerOffset2, 42, 7, 1, &[]);
+ let mut deny = build_empty_reply(request.header(), 42, 0, 9);
+ let header = bytemuck::checked::try_from_bytes_mut::<ReplyHeader>(
+ &mut deny.as_mut_slice()[..HEADER_SIZE],
+ )
+ .expect("empty reply header is valid");
+ header.status = IggyError::ConsumerOffsetNotFound(0).as_code();
+ assert!(matches!(
+ classify_partition_reply(&frozen(deny)),
+ Err(PartitionWriteError::Rejected(
+ IggyError::ConsumerOffsetNotFound(_)
+ ))
+ ));
+ }
+
#[test]
fn in_flight_write_guard_decrements_both_counters_on_drop() {
let session = Cell::new(0);
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index 27f014812..3dc41486c 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -870,8 +870,9 @@ pub(crate) fn build_empty_reply(
/// Build an empty reply that denies a dispatch-time authorization check: the
/// same request echo as [`build_empty_reply`] but with `ReplyHeader.status`
/// set to the rule's error code -- the request-level failure channel the SDK
-/// peeks before any body decode. The single writer of a nonzero status, so
-/// every deny path stamps it one way.
+/// peeks before any body decode. Every deny frame shares this shape (empty
+/// body, nonzero status, op 0); the partition primary's pre-pipeline deny
+/// stamps it through `consensus::build_deny_reply_from_request`.
pub(crate) fn build_deny_reply(
request_header: &RequestHeader,
client_id: u128,
(iggy) 43/43: fix(server-ng): reject flush with typed unavailable deny
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 7c78a34676b7bd459f865606391409cefa8334ae
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 23:59:40 2026 +0200
fix(server-ng): reject flush with typed unavailable deny
server-ng has no on-demand flush primitive: the journal
drains to disk on its own threshold. FLUSH_UNSAVED_BUFFER
had no handler, so it fell through the non-replicated
catch-all and fake-succeeded with an empty ok, attesting a
durability guarantee the server never made.
Return a typed FeatureUnavailable from the response builder
instead. The existing deny path surfaces it through the
reply status the SDK peeks, so clients get an honest error.
A real force-flush primitive remains future work.
---
core/integration/tests/server/flush_vsr.rs | 66 ++++++++++++++++++++++++++++++
core/integration/tests/server/mod.rs | 3 ++
core/server-ng/src/responses.rs | 11 +++--
3 files changed, 77 insertions(+), 3 deletions(-)
diff --git a/core/integration/tests/server/flush_vsr.rs b/core/integration/tests/server/flush_vsr.rs
new file mode 100644
index 000000000..68f41040e
--- /dev/null
+++ b/core/integration/tests/server/flush_vsr.rs
@@ -0,0 +1,66 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Flush contract against server-ng (vsr): the server has no on-demand flush
+//! primitive, so `FLUSH_UNSAVED_BUFFER` must surface a typed
+//! `FeatureUnavailable` over the SDK rather than the non-replicated catch-all's
+//! empty-ok, which would fake a durability guarantee.
+
+use iggy::prelude::*;
+use integration::iggy_harness;
+
+// server-ng partition ids are 0-based (CreateTopic assigns them from 0).
+const PARTITION_ID: u32 = 0;
+
+#[iggy_harness(
+ test_client_transport = [Tcp],
+ server(tcp.socket.override_defaults = true, tcp.socket.nodelay = true)
+)]
+async fn given_valid_partition_when_flushing_should_reject_feature_unavailable(
+ harness: &TestHarness,
+) {
+ let client = harness.tcp_root_client().await.expect("tcp root client");
+ client
+ .create_stream("flush-stream")
+ .await
+ .expect("create stream");
+ let stream_id = Identifier::from_str_value("flush-stream").expect("stream identifier");
+ client
+ .create_topic(
+ &stream_id,
+ "flush-topic",
+ 1,
+ CompressionAlgorithm::None,
+ None,
+ IggyExpiry::NeverExpire,
+ MaxTopicSize::ServerDefault,
+ )
+ .await
+ .expect("create topic");
+ let topic_id = Identifier::from_str_value("flush-topic").expect("topic identifier");
+
+ // A valid, existing partition proves the deny is feature-unavailable, not a
+ // target-not-found in disguise; fsync makes it the strongest durability ask.
+ let result = client
+ .flush_unsaved_buffer(&stream_id, &topic_id, PARTITION_ID, true)
+ .await;
+
+ assert!(
+ matches!(&result, Err(error) if error.as_code() == IggyError::FeatureUnavailable.as_code()),
+ "flush over TCP-ng must surface Err(FeatureUnavailable), got {result:?}"
+ );
+}
diff --git a/core/integration/tests/server/mod.rs b/core/integration/tests/server/mod.rs
index eeb536de9..619402c6f 100644
--- a/core/integration/tests/server/mod.rs
+++ b/core/integration/tests/server/mod.rs
@@ -24,6 +24,9 @@ mod cg;
// The ported round-robin membership join scenario runs against server-ng.
#[cfg(feature = "vsr")]
mod cg_vsr;
+// Flush (FLUSH_UNSAVED_BUFFER) has no server-ng primitive; it must deny typed.
+#[cfg(feature = "vsr")]
+mod flush_vsr;
// Raw-HTTP data-plane contract against server-ng's shard-0 listener.
#[cfg(feature = "vsr")]
mod http_vsr;
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index 3dc41486c..b5a8d26f3 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -29,9 +29,9 @@ use crate::wire::{transport_kind_to_wire, usize_to_u32};
use bytes::{BufMut, Bytes, BytesMut};
use consensus::{MetadataHandle, VsrConsensus};
use iggy_binary_protocol::codes::{
- GET_CLUSTER_METADATA_CODE, GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE,
- GET_PERSONAL_ACCESS_TOKENS_CODE, GET_STATS_CODE, GET_STREAM_CODE, GET_STREAMS_CODE,
- GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE, GET_USERS_CODE,
+ FLUSH_UNSAVED_BUFFER_CODE, GET_CLUSTER_METADATA_CODE, GET_CONSUMER_GROUP_CODE,
+ GET_CONSUMER_GROUPS_CODE, GET_PERSONAL_ACCESS_TOKENS_CODE, GET_STATS_CODE, GET_STREAM_CODE,
+ GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE, GET_USERS_CODE,
};
use iggy_binary_protocol::consensus::{RESULT_COUNT_LEN, result_code};
use iggy_binary_protocol::primitives::consumer::WireConsumer;
@@ -474,6 +474,11 @@ pub(crate) fn build_non_replicated_response(
NonReplicatedResponse::Bytes(GetConsumerGroupsResponse { groups }.to_bytes())
}))
}
+ // server-ng has no force-flush primitive: the journal drains to disk on
+ // its own threshold, not on demand. Answer an honest `FeatureUnavailable`
+ // rather than the non-replicated catch-all's empty-ok, which would attest
+ // a durability guarantee the server never made.
+ FLUSH_UNSAVED_BUFFER_CODE => Err(IggyError::FeatureUnavailable),
_ => match iggy_binary_protocol::dispatch::lookup_command(code) {
Some(meta) if !meta.is_replicated() => Ok(NonReplicatedResponse::Empty),
Some(_) => Err(IggyError::FeatureUnavailable),
(iggy) 20/43: feat(consensus): carry acting user id on Register through VSR
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 15962a5c33e68adcb0bb78155f366c6a28838d65
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 11:07:34 2026 +0200
feat(consensus): carry acting user id on Register through VSR
server-ng has no deterministic way to know which user issued a
committed op: the session->user binding lives only in the
transport-local SessionManager, so apply-time RBAC has no identity
source on replicas.
Stamp the credential-verified user_id into the server-originated
Register request (carved from reserved header bytes, 256B layout
and all existing field offsets preserved), project it into the
replicated PrepareHeader, and store it in ClientEntry at both
commit_register sites. ClientTable::get_user_id now resolves
client -> user identically on every replica.
Client-sent headers cannot reach commit_register: wire Register is
intercepted at dispatch and rebuilt in-process after credential
verification, so the field is never client-controlled.
---
core/binary_protocol/src/consensus/header.rs | 28 +++--
core/consensus/src/client_table.rs | 156 ++++++++++++++++++++++-----
core/consensus/src/impls.rs | 3 +
core/consensus/src/metadata_helpers.rs | 14 ++-
core/metadata/src/impls/metadata.rs | 16 ++-
core/sdk/src/vsr.rs | 2 +-
core/server-ng/src/auth.rs | 2 +-
core/server-ng/src/dispatch.rs | 7 +-
core/server-ng/src/http.rs | 2 +-
core/shard/src/lib.rs | 1 +
10 files changed, 186 insertions(+), 45 deletions(-)
diff --git a/core/binary_protocol/src/consensus/header.rs b/core/binary_protocol/src/consensus/header.rs
index ff87bf05f..230684211 100644
--- a/core/binary_protocol/src/consensus/header.rs
+++ b/core/binary_protocol/src/consensus/header.rs
@@ -151,7 +151,10 @@ pub struct RequestHeader {
pub operation_padding: [u8; 7],
pub namespace: u64,
pub session: u64,
- pub reserved: [u8; 56],
+ /// Acting user id for a `Register`; replicated so every replica resolves
+ /// session -> user deterministically. Zero for non-register ops.
+ pub user_id: u32,
+ pub reserved: [u8; 52],
}
const _: () = {
assert!(size_of::<RequestHeader>() == HEADER_SIZE);
@@ -159,7 +162,10 @@ const _: () = {
offset_of!(RequestHeader, client)
== offset_of!(RequestHeader, reserved_frame) + size_of::<[u8; 66]>()
);
- assert!(offset_of!(RequestHeader, reserved) + size_of::<[u8; 56]>() == HEADER_SIZE);
+ assert!(
+ offset_of!(RequestHeader, user_id) == offset_of!(RequestHeader, session) + size_of::<u64>()
+ );
+ assert!(offset_of!(RequestHeader, reserved) + size_of::<[u8; 52]>() == HEADER_SIZE);
};
impl Default for RequestHeader {
@@ -182,7 +188,8 @@ impl Default for RequestHeader {
operation_padding: [0; 7],
namespace: 0,
session: 0,
- reserved: [0; 56],
+ user_id: 0,
+ reserved: [0; 52],
}
}
}
@@ -591,7 +598,11 @@ pub struct PrepareHeader {
pub operation: Operation,
pub operation_padding: [u8; 7],
pub namespace: u64,
- pub reserved: [u8; 32],
+ /// Acting user id, copied from the originating `RequestHeader`. Lets every
+ /// replica resolve session -> user deterministically on `Register`. Zero
+ /// for non-register ops.
+ pub user_id: u32,
+ pub reserved: [u8; 28],
}
const _: () = {
assert!(size_of::<PrepareHeader>() == HEADER_SIZE);
@@ -599,7 +610,11 @@ const _: () = {
offset_of!(PrepareHeader, client)
== offset_of!(PrepareHeader, reserved_frame) + size_of::<[u8; 66]>()
);
- assert!(offset_of!(PrepareHeader, reserved) + size_of::<[u8; 32]>() == HEADER_SIZE);
+ assert!(
+ offset_of!(PrepareHeader, user_id)
+ == offset_of!(PrepareHeader, namespace) + size_of::<u64>()
+ );
+ assert!(offset_of!(PrepareHeader, reserved) + size_of::<[u8; 28]>() == HEADER_SIZE);
};
impl Default for PrepareHeader {
@@ -624,7 +639,8 @@ impl Default for PrepareHeader {
operation: Operation::Reserved,
operation_padding: [0; 7],
namespace: 0,
- reserved: [0; 32],
+ user_id: 0,
+ reserved: [0; 28],
}
}
}
diff --git a/core/consensus/src/client_table.rs b/core/consensus/src/client_table.rs
index cf582de65..ed517d0f2 100644
--- a/core/consensus/src/client_table.rs
+++ b/core/consensus/src/client_table.rs
@@ -79,6 +79,9 @@ pub struct ClientEntry {
/// Session number = commit op of the register. Monotonic across
/// registrations; new register always gets a higher session.
pub session: u64,
+ /// Acting user id captured at register. Fixed for the entry's lifetime;
+ /// lets every replica resolve session -> user without a metadata lookup.
+ pub user_id: u32,
/// Cached reply for client's latest committed request.
pub reply: CachedReply,
}
@@ -241,8 +244,13 @@ impl ClientTable {
/// # Panics
/// If `client_id == 0`, `session == 0`, or `client_id != reply.header().client`.
/// Session mismatch does NOT panic.
- pub fn commit_register<F>(&mut self, client_id: u128, reply: Message<ReplyHeader>, in_flight: F)
- where
+ pub fn commit_register<F>(
+ &mut self,
+ client_id: u128,
+ user_id: u32,
+ reply: Message<ReplyHeader>,
+ in_flight: F,
+ ) where
F: Fn(u128) -> bool,
{
assert!(client_id != 0, "client_id 0 is reserved for internal use");
@@ -291,6 +299,7 @@ impl ClientTable {
let slot_idx = self.first_free_slot().expect("eviction must free a slot");
self.slots[slot_idx] = Some(ClientEntry {
session,
+ user_id,
reply: cached,
});
self.index.insert(client_id, slot_idx);
@@ -462,6 +471,13 @@ impl ClientTable {
self.slots[slot_idx].as_ref().map(|entry| entry.session)
}
+ /// Acting user id captured when the client registered.
+ #[must_use]
+ pub fn get_user_id(&self, client_id: u128) -> Option<u32> {
+ let &slot_idx = self.index.get(&client_id)?;
+ self.slots[slot_idx].as_ref().map(|entry| entry.user_id)
+ }
+
/// Active committed entries.
#[must_use]
pub fn count(&self) -> usize {
@@ -474,6 +490,10 @@ mod tests {
use super::*;
use iggy_binary_protocol::{Command2, Operation};
+ /// Arbitrary non-zero user id for register fixtures; most tests don't
+ /// assert on it (see `register_stores_user_id` for the accessor check).
+ const TEST_USER_ID: u32 = 7;
+
fn make_register_reply(client: u128, commit: u64) -> Message<ReplyHeader> {
let header_size = std::mem::size_of::<ReplyHeader>();
let mut msg = Message::<ReplyHeader>::new(header_size);
@@ -519,7 +539,12 @@ mod tests {
fn table_with_client() -> (ClientTable, u64) {
let mut table = ClientTable::new(10);
let session = 10;
- table.commit_register(1, make_register_reply(1, session), no_in_flight());
+ table.commit_register(
+ 1,
+ TEST_USER_ID,
+ make_register_reply(1, session),
+ no_in_flight(),
+ );
(table, session)
}
@@ -528,11 +553,27 @@ mod tests {
#[test]
fn register_creates_session() {
let mut table = ClientTable::new(10);
- table.commit_register(1, make_register_reply(1, 42), no_in_flight());
+ table.commit_register(1, TEST_USER_ID, make_register_reply(1, 42), no_in_flight());
assert_eq!(table.get_session(1), Some(42));
+ assert_eq!(table.get_user_id(1), Some(TEST_USER_ID));
assert_eq!(table.count(), 1);
}
+ // Each entry keeps the user id it registered with; lookups are per-client.
+ #[test]
+ fn register_stores_user_id() {
+ let mut table = ClientTable::new(10);
+ table.commit_register(1, 11, make_register_reply(1, 10), no_in_flight());
+ table.commit_register(2, 22, make_register_reply(2, 20), no_in_flight());
+ assert_eq!(table.get_user_id(1), Some(11));
+ assert_eq!(table.get_user_id(2), Some(22));
+ assert_eq!(
+ table.get_user_id(3),
+ None,
+ "unregistered client has no user"
+ );
+ }
+
#[test]
fn check_register_new_client() {
let table = ClientTable::new(10);
@@ -696,9 +737,24 @@ mod tests {
#[test]
fn eviction_removes_oldest_commit() {
let mut table = ClientTable::new(2);
- table.commit_register(100, make_register_reply(100, 10), no_in_flight());
- table.commit_register(200, make_register_reply(200, 20), no_in_flight());
- table.commit_register(300, make_register_reply(300, 30), no_in_flight());
+ table.commit_register(
+ 100,
+ TEST_USER_ID,
+ make_register_reply(100, 10),
+ no_in_flight(),
+ );
+ table.commit_register(
+ 200,
+ TEST_USER_ID,
+ make_register_reply(200, 20),
+ no_in_flight(),
+ );
+ table.commit_register(
+ 300,
+ TEST_USER_ID,
+ make_register_reply(300, 30),
+ no_in_flight(),
+ );
assert!(table.get_reply(100).is_none());
assert!(table.get_reply(200).is_some());
assert!(table.get_reply(300).is_some());
@@ -708,9 +764,24 @@ mod tests {
#[test]
fn eviction_is_deterministic_by_slot_index() {
let mut table = ClientTable::new(2);
- table.commit_register(100, make_register_reply(100, 10), no_in_flight());
- table.commit_register(200, make_register_reply(200, 10), no_in_flight());
- table.commit_register(300, make_register_reply(300, 30), no_in_flight());
+ table.commit_register(
+ 100,
+ TEST_USER_ID,
+ make_register_reply(100, 10),
+ no_in_flight(),
+ );
+ table.commit_register(
+ 200,
+ TEST_USER_ID,
+ make_register_reply(200, 10),
+ no_in_flight(),
+ );
+ table.commit_register(
+ 300,
+ TEST_USER_ID,
+ make_register_reply(300, 30),
+ no_in_flight(),
+ );
assert!(table.get_reply(100).is_none());
assert!(table.get_reply(200).is_some());
assert!(table.get_reply(300).is_some());
@@ -719,8 +790,18 @@ mod tests {
#[test]
fn slot_reuse_after_eviction() {
let mut table = ClientTable::new(1);
- table.commit_register(100, make_register_reply(100, 10), no_in_flight());
- table.commit_register(200, make_register_reply(200, 20), no_in_flight());
+ table.commit_register(
+ 100,
+ TEST_USER_ID,
+ make_register_reply(100, 10),
+ no_in_flight(),
+ );
+ table.commit_register(
+ 200,
+ TEST_USER_ID,
+ make_register_reply(200, 20),
+ no_in_flight(),
+ );
assert!(table.get_reply(100).is_none());
assert!(table.get_reply(200).is_some());
assert_eq!(table.count(), 1);
@@ -732,11 +813,21 @@ mod tests {
#[test]
fn eviction_skips_in_flight_clients() {
let mut table = ClientTable::new(2);
- table.commit_register(100, make_register_reply(100, 10), no_in_flight());
- table.commit_register(200, make_register_reply(200, 20), no_in_flight());
+ table.commit_register(
+ 100,
+ TEST_USER_ID,
+ make_register_reply(100, 10),
+ no_in_flight(),
+ );
+ table.commit_register(
+ 200,
+ TEST_USER_ID,
+ make_register_reply(200, 20),
+ no_in_flight(),
+ );
// 100 in-flight; eviction must pick 200.
let in_flight = |c: u128| c == 100;
- table.commit_register(300, make_register_reply(300, 30), in_flight);
+ table.commit_register(300, TEST_USER_ID, make_register_reply(300, 30), in_flight);
assert!(
table.get_reply(100).is_some(),
"in-flight client must survive"
@@ -753,10 +844,25 @@ mod tests {
#[test]
fn eviction_falls_back_to_oldest_when_all_in_flight() {
let mut table = ClientTable::new(2);
- table.commit_register(100, make_register_reply(100, 10), no_in_flight());
- table.commit_register(200, make_register_reply(200, 20), no_in_flight());
+ table.commit_register(
+ 100,
+ TEST_USER_ID,
+ make_register_reply(100, 10),
+ no_in_flight(),
+ );
+ table.commit_register(
+ 200,
+ TEST_USER_ID,
+ make_register_reply(200, 20),
+ no_in_flight(),
+ );
let all_in_flight = |_| true;
- table.commit_register(300, make_register_reply(300, 30), all_in_flight);
+ table.commit_register(
+ 300,
+ TEST_USER_ID,
+ make_register_reply(300, 30),
+ all_in_flight,
+ );
assert!(
table.get_reply(100).is_none(),
"100 evicted (oldest fallback)"
@@ -770,9 +876,9 @@ mod tests {
#[test]
fn commit_register_idempotent_on_replay() {
let mut table = ClientTable::new(10);
- table.commit_register(1, make_register_reply(1, 10), no_in_flight());
+ table.commit_register(1, TEST_USER_ID, make_register_reply(1, 10), no_in_flight());
// Same client_id + session = idempotent (WAL replay).
- table.commit_register(1, make_register_reply(1, 10), no_in_flight());
+ table.commit_register(1, TEST_USER_ID, make_register_reply(1, 10), no_in_flight());
assert_eq!(table.get_session(1), Some(10));
assert_eq!(table.count(), 1);
}
@@ -783,12 +889,12 @@ mod tests {
#[test]
fn commit_register_different_session_logs_and_skips() {
let mut table = ClientTable::new(10);
- table.commit_register(1, make_register_reply(1, 10), no_in_flight());
+ table.commit_register(1, TEST_USER_ID, make_register_reply(1, 10), no_in_flight());
// existing=10, replay=20.
- table.commit_register(1, make_register_reply(1, 20), no_in_flight());
+ table.commit_register(1, TEST_USER_ID, make_register_reply(1, 20), no_in_flight());
assert_eq!(table.get_session(1), Some(10), "first session stays");
// Smaller replay session: same skip.
- table.commit_register(1, make_register_reply(1, 5), no_in_flight());
+ table.commit_register(1, TEST_USER_ID, make_register_reply(1, 5), no_in_flight());
assert_eq!(table.get_session(1), Some(10));
}
@@ -814,8 +920,8 @@ mod tests {
#[test]
fn different_clients_independent_sessions() {
let mut table = ClientTable::new(10);
- table.commit_register(1, make_register_reply(1, 10), no_in_flight());
- table.commit_register(2, make_register_reply(2, 20), no_in_flight());
+ table.commit_register(1, TEST_USER_ID, make_register_reply(1, 10), no_in_flight());
+ table.commit_register(2, TEST_USER_ID, make_register_reply(2, 20), no_in_flight());
assert_eq!(table.get_session(1), Some(10));
assert_eq!(table.get_session(2), Some(20));
assert!(matches!(table.check_request(1, 10, 1), RequestStatus::New));
diff --git a/core/consensus/src/impls.rs b/core/consensus/src/impls.rs
index 0edcb2fee..6521ecc3c 100644
--- a/core/consensus/src/impls.rs
+++ b/core/consensus/src/impls.rs
@@ -2087,6 +2087,9 @@ where
timestamp,
operation: old.operation,
namespace: old.namespace,
+ // Inert for non-register ops (they carry 0); on Register it
+ // replicates the acting user to every backup.
+ user_id: old.user_id,
..Default::default()
}
})
diff --git a/core/consensus/src/metadata_helpers.rs b/core/consensus/src/metadata_helpers.rs
index 2622d4bf3..8cda8533b 100644
--- a/core/consensus/src/metadata_helpers.rs
+++ b/core/consensus/src/metadata_helpers.rs
@@ -404,6 +404,10 @@ mod tests {
use iggy_binary_protocol::{Command2, Operation, ReplyHeader};
use message_bus::SendError;
+ /// Acting user for register fixtures; these tests exercise preflight /
+ /// replay, not user resolution, so the exact value is immaterial.
+ const ACTING_USER_ID: u32 = 1;
+
/// Production-sized `ClientTable`.
fn fresh_client_table() -> RefCell<ClientTable> {
RefCell::new(ClientTable::new(CLIENTS_TABLE_MAX))
@@ -465,7 +469,7 @@ mod tests {
let original_checksum = initial_reply.header().checksum;
client_table
.borrow_mut()
- .commit_register(client_id, initial_reply, |_| false);
+ .commit_register(client_id, ACTING_USER_ID, initial_reply, |_| false);
let result =
futures::executor::block_on(register_preflight(&consensus, &client_table, client_id));
@@ -501,7 +505,7 @@ mod tests {
let initial_reply = synthesize_register_reply(&consensus, client_id, session);
client_table
.borrow_mut()
- .commit_register(client_id, initial_reply, |_| false);
+ .commit_register(client_id, ACTING_USER_ID, initial_reply, |_| false);
// SendMessages commits -> cached is no longer the register reply.
let app_reply = synthesize_send_messages_reply(&consensus, client_id, session, 1, 18);
@@ -569,7 +573,7 @@ mod tests {
let initial_reply = synthesize_register_reply(&consensus, client_id, real_session);
client_table
.borrow_mut()
- .commit_register(client_id, initial_reply, |_| false);
+ .commit_register(client_id, ACTING_USER_ID, initial_reply, |_| false);
// Older retry (17 < 99): stale-session case.
let result = futures::executor::block_on(apply_preflight_consensus_plane(
@@ -605,7 +609,7 @@ mod tests {
let initial_reply = synthesize_register_reply(&consensus, client_id, real_session);
client_table
.borrow_mut()
- .commit_register(client_id, initial_reply, |_| false);
+ .commit_register(client_id, ACTING_USER_ID, initial_reply, |_| false);
// Client claims newer session (99 > 17), client bug.
let result = futures::executor::block_on(apply_preflight_consensus_plane(
@@ -667,7 +671,7 @@ mod tests {
let initial_reply = synthesize_register_reply(&consensus, client_id, session);
client_table
.borrow_mut()
- .commit_register(client_id, initial_reply, |_| false);
+ .commit_register(client_id, ACTING_USER_ID, initial_reply, |_| false);
// Cache at request 5 -> request 3 is stale.
let advanced = synthesize_send_messages_reply(&consensus, client_id, session, 5, 100);
client_table
diff --git a/core/metadata/src/impls/metadata.rs b/core/metadata/src/impls/metadata.rs
index 73511c701..b8c50f666 100644
--- a/core/metadata/src/impls/metadata.rs
+++ b/core/metadata/src/impls/metadata.rs
@@ -776,6 +776,7 @@ where
|c: u128| consensus.pipeline().borrow().has_message_from_client(c);
self.client_table.borrow_mut().commit_register(
prepare_header.client,
+ prepare_header.user_id,
reply.clone(),
in_flight,
);
@@ -936,6 +937,7 @@ where
pub async fn submit_register_in_process(
&self,
client_id: u128,
+ user_id: u32,
) -> Result<u64, MetadataSubmitError> {
assert!(client_id != 0, "client_id 0 is reserved for internal use");
let consensus = self
@@ -976,7 +978,7 @@ where
return Err(MetadataSubmitError::PipelineFull);
}
- let request = build_register_request_message(consensus, client_id);
+ let request = build_register_request_message(consensus, client_id, user_id);
// Wire path runs `RequestHeader::validate` at network boundary;
// in-process skips it. debug_assert pins drift.
debug_assert!(
@@ -1686,9 +1688,12 @@ where
// Register: commit_register creates session, no SM.
let reply = build_reply_message(&header, &bytes::Bytes::new());
let in_flight = |c: u128| consensus.pipeline().borrow().has_message_from_client(c);
- self.client_table
- .borrow_mut()
- .commit_register(header.client, reply, in_flight);
+ self.client_table.borrow_mut().commit_register(
+ header.client,
+ header.user_id,
+ reply,
+ in_flight,
+ );
} else if header.operation == Operation::Logout {
self.client_table.borrow_mut().remove_client(header.client);
// Mirror the on_ack path: drop the disconnected client from
@@ -1786,6 +1791,7 @@ where
fn build_register_request_message<B, P>(
consensus: &VsrConsensus<B, P>,
client_id: u128,
+ user_id: u32,
) -> Message<RequestHeader>
where
B: MessageBus,
@@ -1807,6 +1813,8 @@ where
client: client_id,
session: 0,
request: 0,
+ // Replicated on the prepare so every replica resolves session -> user.
+ user_id,
// Route through the metadata consensus group. The chain-forwarded
// prepare is re-routed on each peer by namespace; a `0` here would
// hash to a non-zero shard with no metadata consensus and be
diff --git a/core/sdk/src/vsr.rs b/core/sdk/src/vsr.rs
index 8f7f45624..f62b67b63 100644
--- a/core/sdk/src/vsr.rs
+++ b/core/sdk/src/vsr.rs
@@ -114,7 +114,7 @@ pub(crate) fn encode_request_header(
.checked_add(payload.len())
.ok_or(IggyError::InvalidConfiguration)?;
let size = u32::try_from(total_size).map_err(|_| IggyError::InvalidConfiguration)?;
- let mut reserved = [0; 56];
+ let mut reserved = [0; 52];
if operation == Operation::NonReplicated {
reserved[NON_REPLICATED_CODE_RANGE].copy_from_slice(&code.to_le_bytes());
}
diff --git a/core/server-ng/src/auth.rs b/core/server-ng/src/auth.rs
index a177b263d..b6e88e21f 100644
--- a/core/server-ng/src/auth.rs
+++ b/core/server-ng/src/auth.rs
@@ -160,7 +160,7 @@ pub(crate) async fn complete_login_register(
// optimistic Authenticated transition, so a transient submit failure
// needs no rollback -- the connection stays Connected and the SDK
// read-timeout replays.
- let session = match submit_register_on_owner(shard, vsr_client_id).await {
+ let session = match submit_register_on_owner(shard, vsr_client_id, user_id).await {
Ok(session) => session,
Err(error) => {
return Err(LoginRegisterError::Transient(error));
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index 50b3a1e65..4bf29a025 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -318,12 +318,13 @@ pub(crate) fn make_metadata_submit_handler(
match submit {
shard::MetadataSubmit::Register {
vsr_client_id,
+ user_id,
reply,
} => {
let session = shard
.plane
.metadata()
- .submit_register_in_process(vsr_client_id)
+ .submit_register_in_process(vsr_client_id, user_id)
.await
.ok();
let _ = reply.try_send(session);
@@ -1467,17 +1468,19 @@ fn polling_strategy_from_wire(
pub(crate) async fn submit_register_on_owner(
shard: &Rc<ServerNgShard>,
vsr_client_id: u128,
+ user_id: u32,
) -> Result<u64, MetadataSubmitError> {
if shard.id == 0 {
return shard
.plane
.metadata()
- .submit_register_in_process(vsr_client_id)
+ .submit_register_in_process(vsr_client_id, user_id)
.await;
}
let (reply, rx) = shard::channel::<Option<u64>>(1);
shard.forward_metadata_submit(shard::MetadataSubmit::Register {
vsr_client_id,
+ user_id,
reply,
});
match rx.recv().await {
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 121e37ce0..2e9f60e54 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -757,7 +757,7 @@ impl HttpInner {
}
// Shared Register entry point; on shard 0 (always, for HTTP) it runs
// `submit_register_in_process` directly on the metadata owner.
- let session = submit_register_on_owner(&self.shard, client_id)
+ let session = submit_register_on_owner(&self.shard, client_id, user_id)
.await
.map_err(|error| {
warn!(?error, "server-ng HTTP: VSR Register submit failed");
diff --git a/core/shard/src/lib.rs b/core/shard/src/lib.rs
index 424c71ac3..c0c3b972d 100644
--- a/core/shard/src/lib.rs
+++ b/core/shard/src/lib.rs
@@ -143,6 +143,7 @@ pub fn channel<T: Send + 'static>(capacity: usize) -> (Sender<T>, Receiver<T>) {
pub enum MetadataSubmit {
Register {
vsr_client_id: u128,
+ user_id: u32,
reply: Sender<Option<u64>>,
},
Logout {
(iggy) 35/43: fix(server-ng): guard raw PAT splice on committed result code
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit c791f09b0961220decb7dad049afaff33eaac6ae
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 22:03:00 2026 +0200
fix(server-ng): guard raw PAT splice on committed result code
A committed CreatePersonalAccessToken can still carry a business
rejection (duplicate name, invalid expiry), but build_raw_pat_reply
substituted the minted raw token into any committed Reply, so native
TCP returned a success-shaped token response for a failed create. HTTP
already guards via committed_payload; apply the same result-code gate
in the shared builder so a rejection passes through untouched.
---
core/server-ng/src/responses.rs | 103 ++++++++++++++++++++++++++++++++++++++--
1 file changed, 98 insertions(+), 5 deletions(-)
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index 16198ca5b..d5c9544d6 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -33,6 +33,7 @@ use iggy_binary_protocol::codes::{
GET_STREAM_CODE, GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE,
GET_USERS_CODE,
};
+use iggy_binary_protocol::consensus::result_code;
use iggy_binary_protocol::primitives::consumer::WireConsumer;
use iggy_binary_protocol::requests::consumer_groups::{
GetConsumerGroupRequest, GetConsumerGroupsRequest,
@@ -837,11 +838,12 @@ pub(crate) fn build_reply_from_bytes(
)
}
-/// If a raw PAT token was minted (`CreatePersonalAccessToken`), replace the
-/// committed reply -- whose body is empty because the raw token never entered
-/// consensus -- with a `RawPersonalAccessTokenResponse`, reusing the confirmed
-/// commit position from the committed reply. Otherwise the committed reply
-/// passes through unchanged.
+/// If a raw PAT token was minted (`CreatePersonalAccessToken`) and the commit
+/// succeeded, replace the committed reply -- whose body is empty because the
+/// raw token never entered consensus -- with a `RawPersonalAccessTokenResponse`,
+/// reusing the confirmed commit position from the committed reply. Otherwise
+/// (no token, a committed business rejection, or an eviction frame) the
+/// committed reply passes through unchanged.
pub(crate) fn build_raw_pat_reply(
request_header: &RequestHeader,
committed: Message<GenericHeader>,
@@ -865,6 +867,20 @@ pub(crate) fn build_raw_pat_reply(
bytemuck::checked::try_from_bytes::<ReplyHeader>(&committed.as_slice()[..header_len])
.map_err(|_| IggyError::InvalidFormat)?;
let commit = committed_header.commit;
+ let size = committed_header.size as usize;
+ // A committed create can still be a business rejection (duplicate name,
+ // invalid expiry) whose reply body carries a nonzero result code. Splice
+ // the secret only into a genuine success; pass everything else through
+ // untouched so the client decodes the committed error instead of a
+ // success-shaped token reply (the minted raw secret is simply dropped).
+ // Mirrors the HTTP handler's `committed_payload` gate.
+ let reply_body = committed
+ .as_slice()
+ .get(header_len..size)
+ .unwrap_or_default();
+ if result_code(reply_body) != Some(0) {
+ return Ok(committed);
+ }
let token = WireName::new(raw.as_str()).map_err(|_| IggyError::InvalidFormat)?;
let body = RawPersonalAccessTokenResponse { token }.to_bytes();
let reply = build_reply_from_bytes(
@@ -1044,3 +1060,80 @@ pub(crate) fn build_consumer_offset_body(
body.extend_from_slice(&stored_offset.to_le_bytes());
Bytes::from(body)
}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ fn pat_request_header() -> RequestHeader {
+ let zeroed = [0u8; std::mem::size_of::<RequestHeader>()];
+ let mut header = *bytemuck::checked::try_from_bytes::<RequestHeader>(&zeroed)
+ .expect("zeroed bytes form a valid RequestHeader");
+ header.command = Command2::Request;
+ header.operation = Operation::CreatePersonalAccessToken;
+ header.client = 42;
+ header.session = 7;
+ header.request = 3;
+ header
+ }
+
+ /// A committed metadata reply whose body is the given result section
+ /// (`[count][{index, result}]*`), as the commit path emits it.
+ fn committed_reply(result_body: &[u8]) -> Message<GenericHeader> {
+ let request_header = pat_request_header();
+ build_reply_from_bytes(
+ &request_header,
+ 42,
+ 7,
+ 9,
+ &Bytes::copy_from_slice(result_body),
+ )
+ .into_generic()
+ }
+
+ #[test]
+ fn raw_pat_reply_splices_token_into_a_committed_success() {
+ let success = committed_reply(&0u32.to_le_bytes());
+ let reply =
+ build_raw_pat_reply(&pat_request_header(), success, Some("raw-token".to_owned()))
+ .expect("splice succeeds");
+ let header_len = std::mem::size_of::<ReplyHeader>();
+ let body = &reply.as_slice()[header_len..reply.header().size as usize];
+ let response =
+ RawPersonalAccessTokenResponse::decode_from(body).expect("token body decodes");
+ assert_eq!(response.token.as_str(), "raw-token");
+ }
+
+ #[test]
+ fn raw_pat_reply_passes_a_committed_rejection_through_untouched() {
+ let rejection_code =
+ IggyError::PersonalAccessTokenAlreadyExists(String::new(), 0).as_code();
+ let mut result_body = Vec::new();
+ result_body.extend_from_slice(&1u32.to_le_bytes());
+ result_body.extend_from_slice(&0u32.to_le_bytes());
+ result_body.extend_from_slice(&rejection_code.to_le_bytes());
+ let rejection = committed_reply(&result_body);
+ let original = rejection.as_slice().to_vec();
+
+ let reply = build_raw_pat_reply(
+ &pat_request_header(),
+ rejection,
+ Some("raw-token".to_owned()),
+ )
+ .expect("pass-through succeeds");
+ assert_eq!(
+ reply.as_slice(),
+ original.as_slice(),
+ "a committed rejection must not be rewritten into a token reply"
+ );
+ }
+
+ #[test]
+ fn raw_pat_reply_without_a_token_passes_through() {
+ let success = committed_reply(&0u32.to_le_bytes());
+ let original = success.as_slice().to_vec();
+ let reply =
+ build_raw_pat_reply(&pat_request_header(), success, None).expect("pass-through");
+ assert_eq!(reply.as_slice(), original.as_slice());
+ }
+}
(iggy) 39/43: fix(server-ng): frame raw PAT reply with a result section
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 4cadd9e9b88719bde5a8ac4c4054294862579439
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 22:19:32 2026 +0200
fix(server-ng): frame raw PAT reply with a result section
The spliced CreatePersonalAccessToken reply carried a bare
RawPersonalAccessTokenResponse body, but the SDK strips a leading
result section from every metadata reply (split_metadata_result), so a
successful create over the binary transports decoded the token bytes
as a garbage result code and failed. Prefix the success section like
any committed metadata reply and strip it in the HTTP token decode.
Pre-existing; surfaced by the authentication scenario once the
duplicate-name path stopped masking replies.
---
core/server-ng/src/http.rs | 13 ++++++-------
core/server-ng/src/responses.rs | 22 ++++++++++++++++------
2 files changed, 22 insertions(+), 13 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 1f5579505..0fd2b846b 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -3000,14 +3000,13 @@ fn decode_consumer_group_details(payload: &[u8]) -> Result<ConsumerGroupDetails,
Ok(ConsumerGroupDetails::from(response))
}
-/// Extract the raw one-time token from a [`build_raw_pat_reply`] output. That
-/// reply's body is a bare `RawPersonalAccessTokenResponse` (no leading result
-/// section, unlike a committed metadata reply), so decode it straight past the
-/// header.
+/// Extract the raw one-time token from a [`build_raw_pat_reply`] output. The
+/// spliced reply is framed like any committed metadata reply (a success result
+/// section, then the `RawPersonalAccessTokenResponse`) so the SDK's
+/// `split_metadata_result` can decode it; strip the section the same way here.
fn decode_raw_pat_token(reply: &Message<GenericHeader>) -> Result<String, WriteError> {
- let size = reply.header().size as usize;
- let body = reply.as_slice().get(HEADER_SIZE..size).unwrap_or_default();
- let response = RawPersonalAccessTokenResponse::decode_from(body)
+ let payload = committed_payload(reply)?;
+ let response = RawPersonalAccessTokenResponse::decode_from(payload)
.map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
Ok(response.token.as_str().to_string())
}
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index e41659bd5..26946adb1 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -26,14 +26,14 @@
use crate::bootstrap::ServerNgShard;
use crate::session_manager::SessionManager;
use crate::wire::{transport_kind_to_wire, usize_to_u32};
-use bytes::Bytes;
+use bytes::{BufMut, Bytes, BytesMut};
use consensus::{MetadataHandle, VsrConsensus};
use iggy_binary_protocol::codes::{
GET_CLUSTER_METADATA_CODE, GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_STATS_CODE,
GET_STREAM_CODE, GET_STREAMS_CODE, GET_TOPIC_CODE, GET_TOPICS_CODE, GET_USER_CODE,
GET_USERS_CODE,
};
-use iggy_binary_protocol::consensus::result_code;
+use iggy_binary_protocol::consensus::{RESULT_COUNT_LEN, result_code};
use iggy_binary_protocol::primitives::consumer::WireConsumer;
use iggy_binary_protocol::requests::consumer_groups::{
GetConsumerGroupRequest, GetConsumerGroupsRequest,
@@ -917,13 +917,20 @@ pub(crate) fn build_raw_pat_reply(
return Ok(committed);
}
let token = WireName::new(raw.as_str()).map_err(|_| IggyError::InvalidFormat)?;
- let body = RawPersonalAccessTokenResponse { token }.to_bytes();
+ let response = RawPersonalAccessTokenResponse { token };
+ // The SDK strips a leading result section from every metadata reply
+ // (`split_metadata_result`), so the spliced body must carry the success
+ // section like any committed metadata reply; a bare token body would be
+ // read as a garbage result code.
+ let mut body = BytesMut::with_capacity(RESULT_COUNT_LEN + response.encoded_size());
+ body.put_u32_le(0);
+ response.encode(&mut body);
let reply = build_reply_from_bytes(
request_header,
request_header.client,
request_header.session,
commit,
- &body,
+ &body.freeze(),
);
Ok(reply.into_generic())
}
@@ -1134,8 +1141,11 @@ mod tests {
.expect("splice succeeds");
let header_len = std::mem::size_of::<ReplyHeader>();
let body = &reply.as_slice()[header_len..reply.header().size as usize];
- let response =
- RawPersonalAccessTokenResponse::decode_from(body).expect("token body decodes");
+ // Framed like every committed metadata reply: the SDK reads the result
+ // section first, then decodes the token payload past it.
+ assert_eq!(result_code(body), Some(0));
+ let response = RawPersonalAccessTokenResponse::decode_from(&body[RESULT_COUNT_LEN..])
+ .expect("token body decodes");
assert_eq!(response.token.as_str(), "raw-token");
}
(iggy) 03/43: feat(server-ng): route HTTP create-stream through VSR consensus
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 5d09f1ed50576954d8b1bd240e23996cf5442c44
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 12:47:55 2026 +0200
feat(server-ng): route HTTP create-stream through VSR consensus
The server-ng HTTP listener could authenticate and bind a session
but had no way to perform a control-plane write, leaving every
mutating route unserved.
POST /streams now builds a create-stream request header and submits
it on the owning shard via the in-process dispatch spine, the same
path TCP and QUIC use, so the write is replicated and linearized
identically. A per-session gate serializes writes and assigns the
gap-free request ids the depth-1 consensus dedup requires; the id
advances only once a Reply commits, so a transient failure or an
eviction leaves it free for the caller's retry.
The committed reply is decoded by frame command: a Reply's result
section renders the parity JSON or a committed business rejection
(duplicate name -> 400), while an Eviction maps to the same typed
401 a TCP client receives, so the caller re-authenticates rather
than retrying a dead session.
---
core/server-ng/src/dispatch.rs | 2 +-
core/server-ng/src/http.rs | 241 ++++++++++++++++++++++++++++++-----
core/server-ng/src/http/extractor.rs | 6 +-
3 files changed, 212 insertions(+), 37 deletions(-)
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index 74b81e92b..3bf5fa33b 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -1624,7 +1624,7 @@ fn submit_disconnect_logout(shard: Rc<ServerNgShard>, vsr_client_id: u128, sessi
/// `client` id (it's the VSR id, not the transport/home-shard-encoding id).
/// `None` = transient submit failure (SDK read-timeout replays).
#[allow(clippy::future_not_send)]
-async fn submit_client_request_on_owner(
+pub(crate) async fn submit_client_request_on_owner(
shard: &Rc<ServerNgShard>,
request: Message<RequestHeader>,
) -> Option<Message<GenericHeader>> {
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index e4f2d58f4..a2de4b11c 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -33,19 +33,30 @@ use axum::response::{IntoResponse, Response};
use axum::routing::{get, post};
use axum::{Json, Router};
use configs::http::HttpJwtConfig;
+use iggy_binary_protocol::consensus::{
+ Command2, EvictionHeader, EvictionReason, HEADER_SIZE, result_code, result_section_len,
+};
+use iggy_binary_protocol::requests::streams::CreateStreamRequest;
+use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
+use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
+use iggy_binary_protocol::{
+ GenericHeader, Operation, RequestHeader, WireDecode, WireEncode, WireName,
+};
+use iggy_common::create_stream::CreateStream;
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
-use iggy_common::{IdentityInfo, IggyError, IggyTimestamp, TokenInfo};
+use iggy_common::{IdentityInfo, IggyError, IggyTimestamp, StreamDetails, TokenInfo};
use message_bus::client_listener;
use secrecy::ExposeSecret;
use send_wrapper::SendWrapper;
use server::http::error::{CustomError, ErrorResponse};
+use server_common::Message;
use tokio::sync::Mutex;
use tracing::{error, info, warn};
use crate::auth::{verify_login_credentials, verify_pat_credentials};
use crate::bootstrap::ServerNgShard;
-use crate::dispatch::submit_register_on_owner;
+use crate::dispatch::{submit_client_request_on_owner, submit_register_on_owner};
use crate::http::extractor::Authenticated;
use crate::http::jwt::JwtManager;
use crate::login_register::LoginRegisterError;
@@ -61,8 +72,8 @@ const PONG: &str = "pong";
/// entries are dropped first, so the cap only bites on live oversubscription.
const MAX_HTTP_SESSIONS: usize = 100_000;
-/// First per-session request id the write path (2c) will hand out. VSR request
-/// numbers are 1-based and strictly increasing within a session.
+/// First per-session request id the write path hands out. VSR request numbers
+/// are 1-based and strictly increasing within a session.
const FIRST_REQUEST_ID: u64 = 1;
/// One VSR session established for a single login credential (a JWT `jti` or a
@@ -71,32 +82,22 @@ const FIRST_REQUEST_ID: u64 = 1;
struct HttpSession {
/// Shard-0 client id minted for this credential; its top 16 bits are 0, so
/// it shares the shard-0 id space with TCP virtual clients without
- /// colliding. Consumed by the write path (2c).
- #[expect(
- dead_code,
- reason = "read by the write path (2c) to fill RequestHeader.client"
- )]
+ /// colliding. Fills `RequestHeader.client` on every write.
client_id: u128,
- /// Cluster session number returned by the VSR `Register` commit. Consumed
- /// by the write path (2c).
- #[expect(
- dead_code,
- reason = "read by the write path (2c) to fill RequestHeader.session"
- )]
+ /// Cluster session number returned by the VSR `Register` commit. Fills
+ /// `RequestHeader.session` on every write.
session: u64,
- /// User the credential authenticated as. Consumed by the write path (2c).
- #[expect(dead_code, reason = "read by the write path (2c) for authorization")]
+ /// User the credential authenticated as. Consumed by the write path for
+ /// authorization.
+ #[expect(dead_code, reason = "read once authorization is wired for writes")]
user_id: u32,
/// Credential expiry in unix seconds (`u64::MAX` = never). Drives lazy
/// eviction of stale table entries.
expiry: u64,
- /// Serializes the write path (2c): the guarded value is the NEXT request
- /// id. A `tokio::sync::Mutex` because 2c holds it across the submit `.await`
- /// so each session's request numbers stay monotonic and gap-free.
- #[expect(
- dead_code,
- reason = "locked by the write path (2c) to serialize per-session writes"
- )]
+ /// Serializes this session's writes: the guarded value is the NEXT request
+ /// id. A `tokio::sync::Mutex` because the write path holds it across the
+ /// submit `.await` so each session's request numbers reach the primary in
+ /// order and stay gap-free for the depth-1 consensus dedup.
gate: Mutex<u64>,
}
@@ -126,17 +127,48 @@ impl IntoResponse for AuthError {
Self::Unauthenticated(error) => CustomError::from(error).into_response(),
// Register could not commit (no caught-up primary, pipeline full, or
// a view-change cancel). Transient server condition -> 503, retryable.
- Self::SessionUnavailable => (
- StatusCode::SERVICE_UNAVAILABLE,
- Json(ErrorResponse::from_error(
- IggyError::CannotEstablishConnection,
- )),
- )
- .into_response(),
+ Self::SessionUnavailable => service_unavailable(),
}
}
}
+/// Rejection for an authenticated control-plane write (`POST /streams` and the
+/// writes that follow it).
+///
+/// Same two-class split as [`AuthError`], for the same reasons: a caller-side
+/// validation failure or a committed business rejection (e.g. a duplicate
+/// stream name) renders through the legacy `IggyError -> CustomError` map so
+/// SDK error bodies stay byte-identical, while a write that cannot commit right
+/// now is a transient server condition (503) and must never surface as a
+/// business error or, worse, a 200 with a stale body.
+enum WriteError {
+ Rejected(IggyError),
+ Unavailable,
+}
+
+impl IntoResponse for WriteError {
+ fn into_response(self) -> Response {
+ match self {
+ Self::Rejected(error) => CustomError::from(error).into_response(),
+ Self::Unavailable => service_unavailable(),
+ }
+ }
+}
+
+/// The shared 503 body for a request that could not commit right now: no
+/// caught-up primary, a full pipeline, or a view-change cancel. Retryable, and
+/// rendered with the `CannotEstablishConnection` code the SDKs treat as a
+/// connection-level retry rather than a terminal error.
+fn service_unavailable() -> Response {
+ (
+ StatusCode::SERVICE_UNAVAILABLE,
+ Json(ErrorResponse::from_error(
+ IggyError::CannotEstablishConnection,
+ )),
+ )
+ .into_response()
+}
+
/// Shared shard-0 HTTP state.
///
/// Groups the shard handle, the JWT issuer/verifier, and the per-credential VSR
@@ -307,6 +339,7 @@ fn router(state: HttpState) -> Router {
post(login_with_personal_access_token),
)
.route("/users/me", get(get_me))
+ .route("/streams", post(create_stream))
.with_state(state)
}
@@ -350,6 +383,152 @@ async fn get_me(identity: Authenticated) -> Json<IdentityInfo> {
})
}
+/// `POST /streams`: create a stream through the per-session gate and the
+/// in-process VSR dispatch spine, then render the committed reply as the same
+/// `StreamDetails` JSON the legacy server returns.
+///
+/// The accepted body is name-only (`{"name": ...}`), matching the legacy
+/// request; server-ng's wire `CreateStreamRequest` is likewise name-only and
+/// auto-assigns the id, so there is no client-supplied stream id to honor.
+async fn create_stream(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Json(command): Json<CreateStream>,
+) -> Result<Json<StreamDetails>, WriteError> {
+ let request = CreateStreamRequest {
+ name: WireName::new(command.name)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidStreamName))?,
+ };
+
+ let client_id = identity.session.client_id;
+ let session_id = identity.session.session;
+
+ // Serialize this session's writes behind the gate and take the next VSR
+ // request id. The gate is held across the commit so ids reach the primary
+ // strictly in order, and it advances only once the request commits (a
+ // `Reply` frame): a transient failure or an eviction leaves the id free for
+ // the caller's retry, which the depth-1 dedup requires (the next accepted
+ // request must be `committed_request + 1`, so a consumed-but-uncommitted id
+ // would wedge the session on `RequestGap`).
+ let mut next_request_id = identity.session.gate.lock().await;
+ let message = build_create_stream_message(client_id, session_id, *next_request_id, &request);
+ // `submit_client_request_on_owner` is `!Send` (Rc-based); `SendWrapper`
+ // bridges it onto axum's `Send` handler future, sound because the whole
+ // handler runs pinned to shard 0's compio thread.
+ let reply = SendWrapper::new(submit_client_request_on_owner(&state.shard, message)).await;
+ let Some(reply) = reply else {
+ return Err(WriteError::Unavailable);
+ };
+
+ // Peek the reply frame before decoding: an eviction carries no result
+ // section and means the session is dead (re-authenticate), so unlike a
+ // committed `Reply` it must not consume this request id.
+ match reply.header().command {
+ Command2::Reply => {
+ // Committed: the cluster consumed this id, so advance the gate before
+ // rendering, whether or not the body decodes.
+ *next_request_id += 1;
+ drop(next_request_id);
+ Ok(Json(decode_committed_stream(&reply)?))
+ }
+ Command2::Eviction => Err(WriteError::Rejected(eviction_error(&reply))),
+ _ => Err(WriteError::Rejected(IggyError::InvalidCommand)),
+ }
+}
+
+/// Build the `Message<RequestHeader>` for a create-stream by filling a zeroed
+/// `#[repr(C)]` header, mirroring `wire::rewrite_request_body` and the
+/// partition reconciler's prepare builder.
+fn build_create_stream_message(
+ client_id: u128,
+ session_id: u64,
+ request_id: u64,
+ request: &CreateStreamRequest,
+) -> Message<RequestHeader> {
+ let body = request.to_bytes();
+ let total = HEADER_SIZE + body.len();
+ let mut message = Message::<RequestHeader>::new(total);
+ message.as_mut_slice()[HEADER_SIZE..].copy_from_slice(&body);
+ let header = bytemuck::checked::try_from_bytes_mut::<RequestHeader>(
+ &mut message.as_mut_slice()[..HEADER_SIZE],
+ )
+ .expect("zeroed bytes form a valid RequestHeader");
+ header.command = Command2::Request;
+ header.operation = Operation::CreateStream;
+ header.client = client_id;
+ header.session = session_id;
+ header.request = request_id;
+ header.size = u32::try_from(total).expect("create-stream message size fits u32");
+ message
+}
+
+/// Decode a committed `Reply` frame's body into `StreamDetails`, mirroring the
+/// SDK's `split_metadata_result`. A metadata reply body leads with a result
+/// section (`[count][{index, result}]*`): `count == 0` is success followed by
+/// the `GetStreamResponse` payload, and a nonzero first `result` is a committed
+/// business rejection carrying an `IggyError` code (e.g. duplicate name). The
+/// caller has already confirmed the frame is a `Reply`, so a missing result
+/// section is a malformed committed reply, mapped to an error rather than a 200
+/// with a stale body.
+fn decode_committed_stream(reply: &Message<GenericHeader>) -> Result<StreamDetails, WriteError> {
+ let size = reply.header().size as usize;
+ let body = reply.as_slice().get(HEADER_SIZE..size).unwrap_or_default();
+ match result_code(body) {
+ Some(0) => {
+ let payload_start =
+ result_section_len(body).ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
+ let payload = body
+ .get(payload_start..)
+ .ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
+ let response = GetStreamResponse::decode_from(payload)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
+ StreamDetails::try_from(response).map_err(WriteError::Rejected)
+ }
+ Some(code) => Err(WriteError::Rejected(IggyError::from_code(code))),
+ None => Err(WriteError::Rejected(IggyError::InvalidCommand)),
+ }
+}
+
+/// Map an eviction frame to the same typed [`IggyError`] the SDK's
+/// `decode_eviction` produces, so an HTTP caller sees the identical status a TCP
+/// caller would (session-terminal reasons render as 401 -> re-authenticate).
+/// Reuses the shared [`EvictionHeader`] primitive rather than hand-decoding
+/// offsets; an unreadable frame falls back to re-authentication.
+fn eviction_error(reply: &Message<GenericHeader>) -> IggyError {
+ let Some(eviction) = reply
+ .as_slice()
+ .get(..HEADER_SIZE)
+ .and_then(|bytes| bytemuck::checked::try_from_bytes::<EvictionHeader>(bytes).ok())
+ else {
+ return IggyError::Unauthenticated;
+ };
+ match eviction.reason {
+ EvictionReason::InvalidCredentials => IggyError::InvalidCredentials,
+ EvictionReason::InvalidToken => IggyError::InvalidPersonalAccessToken,
+ EvictionReason::UserInactive
+ | EvictionReason::SessionError
+ | EvictionReason::NoSession
+ | EvictionReason::SessionTooLow
+ | EvictionReason::SessionReleaseMismatch => IggyError::Unauthenticated,
+ EvictionReason::StaleClient => IggyError::StaleClient,
+ EvictionReason::IncompatibleProtocol => {
+ let server_max = eviction.server_protocol_version;
+ let server_min = eviction.server_protocol_version_min;
+ if server_min == 0 || server_max < server_min {
+ IggyError::Unauthenticated
+ } else {
+ IggyError::IncompatibleProtocolVersion(
+ IGGY_PROTOCOL_VERSION,
+ server_min,
+ server_max,
+ )
+ }
+ }
+ EvictionReason::MalformedLogin => IggyError::InvalidFormat,
+ _ => IggyError::InvalidCommand,
+ }
+}
+
/// Issue a fresh access token for `user_id` and wrap it in the exact
/// `IdentityInfo` shape the SDKs pin: numeric `user_id` plus an `access_token`
/// carrying the token string and its unix-seconds expiry.
diff --git a/core/server-ng/src/http/extractor.rs b/core/server-ng/src/http/extractor.rs
index fd2e90051..87d8921ff 100644
--- a/core/server-ng/src/http/extractor.rs
+++ b/core/server-ng/src/http/extractor.rs
@@ -40,7 +40,7 @@ const PAT_KEY_PREFIX: &str = "pat:";
/// shared VSR session established once per presenting credential.
pub struct Authenticated {
pub user_id: u32,
- /// Per-credential VSR session. The write path (2c) reads its client id and
+ /// Per-credential VSR session. The write path reads its client id and
/// session number and serializes writes through its gate; `get_me` and
/// other read-only probes ignore it.
///
@@ -48,10 +48,6 @@ pub struct Authenticated {
/// `Rc` is not; this mirrors the `HttpState` bridge. Sound for the same
/// reason: the extractor mints it on shard 0's compio thread and every
/// handler runs on that same thread, so the wrapper is never crossed.
- #[expect(
- dead_code,
- reason = "read by the write path (2c) via client id, session, and gate"
- )]
pub session: SendWrapper<Rc<HttpSession>>,
}
(iggy) 06/43: feat(server-ng): gate HTTP write routes to root user
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit bf38ea794de201b21aab989030eb0acfe49bc770
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 15:08:27 2026 +0200
feat(server-ng): gate HTTP write routes to root user
server-ng has no RBAC permissioner yet, so any authenticated token could
create, update, delete, or purge streams and topics. That is a privilege
escalation: a valid but non-root credential must not mutate management
state.
Add an interim authorization gate inside submit_write, the single
chokepoint every HTTP control-plane write funnels through, rejecting any
non-root user with Unauthorized (403) before any consensus work is spent.
Root (user_id 0) is the sole admin marker until per-op RBAC (F8) replaces
this blanket gate.
---
core/server-ng/src/http.rs | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index f42a129ce..be7dcd9b5 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -51,6 +51,7 @@ use iggy_binary_protocol::{
};
use iggy_common::create_stream::CreateStream;
use iggy_common::create_topic::CreateTopic;
+use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
use iggy_common::update_stream::UpdateStream;
@@ -103,7 +104,6 @@ struct HttpSession {
session: u64,
/// User the credential authenticated as. Consumed by the write path for
/// authorization.
- #[expect(dead_code, reason = "read once authorization is wired for writes")]
user_id: u32,
/// Credential expiry in unix seconds (`u64::MAX` = never). Drives lazy
/// eviction of stale table entries.
@@ -644,6 +644,12 @@ async fn submit_write(
operation: Operation,
body: &[u8],
) -> Result<Bytes, WriteError> {
+ // Interim authorization: until server-ng has an RBAC permissioner, every
+ // control-plane write is root-only. A non-root credential is authenticated
+ // but unprivileged, rejected before any consensus work is spent.
+ if session.user_id != DEFAULT_ROOT_USER_ID {
+ return Err(WriteError::Rejected(IggyError::Unauthorized));
+ }
let mut next_request_id = session.gate.lock().await;
let message = build_request_message(
operation,
(iggy) 13/43: feat(server-ng): add in-process reply slots for HTTP data plane
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit dd542f0b2fae4ac026ff3a9f50bef7003b25de0e
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 22:51:49 2026 +0200
feat(server-ng): add in-process reply slots for HTTP data plane
An HTTP caller holds no consensus socket: partition replies fire
post-commit through send_to_client, which until now could only
resolve to a per-peer wire queue, and the partition routing gates
lived inline in handle_client_request where a transport-less caller
could not reach them.
Teach the connection registry an in-process entry (no reader/writer
tasks) with a request-keyed reply slot API: install_reply_slot hands
back a oneshot receiver plus an RAII guard that removes the slot on
drop, so a timed-out caller never leaks its waiter. Duplicate request
ids are rejected instead of silently replacing a live waiter, and
entry teardown cancels every pending receiver. Extract the partition
tail of handle_client_request into dispatch_partition_request so an
already-authenticated in-process caller routes partition ops through
the same namespace and routability gates; TCP behavior is unchanged.
Drop the partition plane's speculative take_reply_sender call site:
in-process partition replies correlate by request id at the bus, not
by pipeline subscriber, so it would never gain a caller.
---
core/message_bus/src/lib.rs | 53 +++-
.../src/lifecycle/connection_registry.rs | 287 ++++++++++++++++++++-
core/message_bus/src/lifecycle/mod.rs | 2 +-
core/partitions/src/iggy_partition.rs | 10 +-
core/server-ng/src/dispatch.rs | 185 +++++++------
5 files changed, 444 insertions(+), 93 deletions(-)
diff --git a/core/message_bus/src/lib.rs b/core/message_bus/src/lib.rs
index ea50e4cd1..9679c6906 100644
--- a/core/message_bus/src/lib.rs
+++ b/core/message_bus/src/lib.rs
@@ -95,7 +95,8 @@ pub use installer::conn_info::{
};
pub use lifecycle::{
BusMessage, BusReceiver, BusSender, ConnectionRegistry, DrainOutcome, FusedShutdown,
- ReplicaRegistry, ReplyRoute, Shutdown, ShutdownToken,
+ InstanceToken, ReplicaRegistry, ReplyRoute, ReplySlotError, ReplySlotGuard, Shutdown,
+ ShutdownToken,
};
pub use transports::tls::TlsServerCredentials;
@@ -1395,6 +1396,56 @@ mod tests {
assert!(matches!(err, SendError::ClientNotFound(_)));
}
+ #[allow(clippy::cast_possible_truncation)]
+ fn reply_message(request: u64) -> Message<ReplyHeader> {
+ Message::<ReplyHeader>::new(HEADER_SIZE).transmute_header(|_, h: &mut ReplyHeader| {
+ h.command = Command2::Reply;
+ h.size = HEADER_SIZE as u32;
+ h.request = request;
+ })
+ }
+
+ /// Full in-process seam: entry + slot installed, `send_to_client`
+ /// resolves the request id from the reply header and fires the oneshot.
+ #[compio::test]
+ #[allow(clippy::future_not_send)]
+ async fn send_to_client_in_process_fires_matching_reply_slot() {
+ let bus = IggyMessageBus::new(0);
+ let client_id = 1u128;
+ bus.clients()
+ .insert_in_process(client_id)
+ .expect("fresh key");
+ let (_guard, reply_rx) = bus
+ .clients()
+ .install_reply_slot(client_id, 42)
+ .expect("slot installs");
+
+ bus.send_to_client(client_id, reply_message(42).into_generic().into_frozen())
+ .await
+ .expect("in-process delivery");
+
+ let received = reply_rx.await.expect("reply delivered");
+ assert_eq!(reply_request_id(&received), 42);
+ }
+
+ /// Reply whose request id has no waiter (timed-out caller already
+ /// removed its slot): shed as `ClientNotFound`, nothing panics.
+ #[compio::test]
+ #[allow(clippy::future_not_send)]
+ async fn send_to_client_in_process_sheds_reply_without_waiter() {
+ let bus = IggyMessageBus::new(0);
+ let client_id = 1u128;
+ bus.clients()
+ .insert_in_process(client_id)
+ .expect("fresh key");
+
+ let err = bus
+ .send_to_client(client_id, reply_message(42).into_generic().into_frozen())
+ .await
+ .unwrap_err();
+ assert!(matches!(err, SendError::ClientNotFound(_)));
+ }
+
#[compio::test]
#[allow(clippy::future_not_send)]
async fn send_to_client_returns_route_missing_when_remote_and_no_forward_fn() {
diff --git a/core/message_bus/src/lifecycle/connection_registry.rs b/core/message_bus/src/lifecycle/connection_registry.rs
index 6a01110b8..5858ab906 100644
--- a/core/message_bus/src/lifecycle/connection_registry.rs
+++ b/core/message_bus/src/lifecycle/connection_registry.rs
@@ -41,6 +41,7 @@ use std::collections::hash_map::Entry as HmEntry;
use std::fmt::Debug;
use std::hash::Hash;
use std::time::{Duration, Instant};
+use thiserror::Error;
use tracing::{debug, warn};
/// Opaque per-insert generation token.
@@ -139,7 +140,6 @@ pub struct DrainOutcome {
#[derive(Debug)]
enum ReplyTarget {
Socket(BusSender),
- #[allow(dead_code)] // wired in P2 (HTTP data plane)
InProcess {
by_request: HashMap<u64, oneshot::Sender<BusMessage>>,
},
@@ -170,6 +170,68 @@ pub enum ReplyRoute {
NoSlot(BusMessage),
}
+/// Rejection reasons for [`ConnectionRegistry::install_reply_slot`].
+#[derive(Debug, Error, PartialEq, Eq)]
+pub enum ReplySlotError {
+ /// No registry entry exists for the key: the in-process entry was
+ /// never installed or has already been torn down.
+ #[error("no registry entry for this key")]
+ EntryNotFound,
+ /// The entry is a socket target. Socket replies go through the
+ /// per-peer queue; a reply slot on such an entry could never fire.
+ #[error("registry entry is a socket target")]
+ NotInProcess,
+ /// A waiter is already registered under this request id. Replacing it
+ /// would drop the first waiter's sender and could deliver its reply
+ /// to the wrong caller, so the second install is rejected instead.
+ #[error("a reply slot for this request id is already installed")]
+ DuplicateRequest,
+}
+
+/// Cancel-safety handle for an installed in-process reply slot.
+///
+/// Dropping the guard removes the slot unless the reply already fired
+/// ([`ConnectionRegistry::fire_in_process`] consumes the slot on delivery,
+/// making the drop a no-op) or the whole entry was replaced (fenced by the
+/// entry's [`InstanceToken`], mirroring `remove_if_token_matches`). A
+/// caller that times out or is cancelled therefore never leaks its oneshot
+/// sender.
+///
+/// Contract: a request id must not be reused under the same key while a
+/// previous guard for that id is still live; the stale guard's drop would
+/// remove the new slot. Callers mint monotonically increasing request ids,
+/// which rules this out.
+#[must_use = "dropping the guard removes the reply slot"]
+#[derive(Debug)]
+pub struct ReplySlotGuard<'a, K>
+where
+ K: Eq + Hash + Copy + Debug + 'static,
+{
+ registry: &'a ConnectionRegistry<K>,
+ key: K,
+ request: u64,
+ token: InstanceToken,
+}
+
+impl<K> Drop for ReplySlotGuard<'_, K>
+where
+ K: Eq + Hash + Copy + Debug + 'static,
+{
+ fn drop(&mut self) {
+ let mut entries = self.registry.entries.borrow_mut();
+ let Some(entry) = entries.get_mut(&self.key) else {
+ return;
+ };
+ if entry.token != self.token {
+ return;
+ }
+ let ReplyTarget::InProcess { by_request } = &mut entry.reply else {
+ return;
+ };
+ by_request.remove(&self.request);
+ }
+}
+
#[derive(Debug)]
struct Entry {
reply: ReplyTarget,
@@ -313,6 +375,49 @@ where
}
}
+ /// Register a oneshot waiter for the reply to `request` on `key`'s
+ /// in-process entry.
+ ///
+ /// The returned receiver resolves when `send_to_client` routes a reply
+ /// carrying this request id (via [`Self::fire_in_process`]), or with
+ /// `Canceled` when the entry is torn down first. The guard removes the
+ /// slot on drop; see [`ReplySlotGuard`] for the exact semantics.
+ ///
+ /// # Errors
+ ///
+ /// [`ReplySlotError::EntryNotFound`] when `key` has no entry,
+ /// [`ReplySlotError::NotInProcess`] when the entry is a socket target,
+ /// [`ReplySlotError::DuplicateRequest`] when a waiter is already
+ /// registered under `request`.
+ pub fn install_reply_slot(
+ &self,
+ key: K,
+ request: u64,
+ ) -> Result<(ReplySlotGuard<'_, K>, oneshot::Receiver<BusMessage>), ReplySlotError> {
+ let mut entries = self.entries.borrow_mut();
+ let Some(entry) = entries.get_mut(&key) else {
+ return Err(ReplySlotError::EntryNotFound);
+ };
+ let token = entry.token;
+ let ReplyTarget::InProcess { by_request } = &mut entry.reply else {
+ return Err(ReplySlotError::NotInProcess);
+ };
+ let HmEntry::Vacant(slot) = by_request.entry(request) else {
+ return Err(ReplySlotError::DuplicateRequest);
+ };
+ let (reply_tx, reply_rx) = oneshot::channel();
+ slot.insert(reply_tx);
+ Ok((
+ ReplySlotGuard {
+ registry: self,
+ key,
+ request,
+ token,
+ },
+ reply_rx,
+ ))
+ }
+
/// Register a new connection. Stores the producer side of the per-peer
/// queue alongside the writer + reader task handles so graceful shutdown
/// can drain everything.
@@ -365,6 +470,38 @@ where
Ok(token)
}
+ /// Register an in-process reply target for `key`.
+ ///
+ /// In-process peers have no socket and no reader / writer tasks:
+ /// replies routed to `key` resolve request-keyed oneshots installed
+ /// via [`Self::install_reply_slot`]. Teardown goes through
+ /// [`Self::remove`] / [`Self::remove_if_token_matches`]; dropping the
+ /// entry drops every pending sender, so all outstanding receivers
+ /// resolve `Canceled`.
+ ///
+ /// Returns `None` when `key` is already registered (socket or
+ /// in-process); the caller must tear the existing entry down first.
+ pub fn insert_in_process(&self, key: K) -> Option<InstanceToken> {
+ let mut entries = self.entries.borrow_mut();
+ let HmEntry::Vacant(slot) = entries.entry(key) else {
+ return None;
+ };
+ let token = self.mint_token();
+ // Nothing ever waits on this shutdown (no reader to wake); it
+ // exists only because every entry owns one.
+ let (conn_shutdown, _) = super::Shutdown::new();
+ slot.insert(Entry {
+ reply: ReplyTarget::InProcess {
+ by_request: HashMap::new(),
+ },
+ writer_handle: None,
+ reader_handle: None,
+ token,
+ _conn_shutdown: conn_shutdown,
+ });
+ Some(token)
+ }
+
/// Unregister a connection without awaiting its tasks.
///
/// Returns `true` if an entry was removed. Dropping the entry drops the
@@ -1206,4 +1343,152 @@ mod tests {
let returned = outcome.expect_err("missing slot should return msg");
assert_eq!(returned.len(), original_len);
}
+
+ /// End-to-end registry seam: install entry + slot, route, fire, await.
+ #[compio::test]
+ async fn in_process_slot_receives_fired_reply() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ reg.insert_in_process(1u8).expect("fresh key");
+
+ let (guard, reply_rx) = reg.install_reply_slot(1u8, 42).expect("slot installs");
+ let msg = match reg.try_send_or_return(1u8, make_bus_msg()) {
+ ReplyRoute::InProcess(msg) => msg,
+ other => panic!("expected InProcess route, got {other:?}"),
+ };
+ reg.fire_in_process(1u8, 42, msg).expect("waiter present");
+
+ let received = reply_rx.await.expect("reply delivered");
+ assert_eq!(received.len(), HEADER_SIZE);
+ drop(guard);
+ assert!(reg.contains(1u8), "guard drop must not remove the entry");
+ }
+
+ #[compio::test]
+ async fn insert_in_process_duplicate_key_returns_none() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ assert!(reg.insert_in_process(1u8).is_some());
+ assert!(reg.insert_in_process(1u8).is_none());
+ assert_eq!(reg.len(), 1);
+ }
+
+ #[compio::test]
+ async fn install_reply_slot_rejects_missing_and_socket_entries() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ assert_eq!(
+ reg.install_reply_slot(1u8, 1).unwrap_err(),
+ ReplySlotError::EntryNotFound
+ );
+
+ let (_shutdown, token) = Shutdown::new();
+ let (tx, rx) = async_channel::bounded(8);
+ reg.insert(
+ 1u8,
+ tx,
+ spawn_dummy_writer(rx),
+ spawn_dummy_reader(token),
+ dummy_conn_shutdown(),
+ )
+ .expect("socket insert ok");
+ assert_eq!(
+ reg.install_reply_slot(1u8, 1).unwrap_err(),
+ ReplySlotError::NotInProcess
+ );
+ }
+
+ /// Second install under a live request id must be rejected, and the
+ /// first waiter must stay wired (the reply still reaches it).
+ #[compio::test]
+ async fn install_reply_slot_rejects_duplicate_request() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ reg.insert_in_process(1u8).expect("fresh key");
+
+ let (_guard, reply_rx) = reg.install_reply_slot(1u8, 42).expect("first install");
+ assert_eq!(
+ reg.install_reply_slot(1u8, 42).unwrap_err(),
+ ReplySlotError::DuplicateRequest
+ );
+
+ reg.fire_in_process(1u8, 42, make_bus_msg())
+ .expect("first waiter still registered");
+ assert_eq!(reply_rx.await.expect("reply delivered").len(), HEADER_SIZE);
+ }
+
+ /// Reply for a request id nobody waits on: handed back, no panic, and
+ /// the unrelated slot stays registered.
+ #[compio::test]
+ async fn fire_in_process_unknown_request_returns_msg() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ reg.insert_in_process(1u8).expect("fresh key");
+ let (_guard, reply_rx) = reg.install_reply_slot(1u8, 42).expect("slot installs");
+
+ let msg = make_bus_msg();
+ let original_len = msg.len();
+ let returned = reg
+ .fire_in_process(1u8, 7, msg)
+ .expect_err("no waiter for request 7");
+ assert_eq!(returned.len(), original_len);
+
+ reg.fire_in_process(1u8, 42, make_bus_msg())
+ .expect("slot 42 untouched");
+ assert_eq!(reply_rx.await.expect("reply delivered").len(), HEADER_SIZE);
+ }
+
+ /// Timeout / cancellation path: dropping the guard removes the slot,
+ /// cancels the receiver, and a later reply is shed as no-waiter.
+ #[compio::test]
+ async fn dropped_guard_removes_slot_and_cancels_receiver() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ reg.insert_in_process(1u8).expect("fresh key");
+
+ let (guard, reply_rx) = reg.install_reply_slot(1u8, 42).expect("slot installs");
+ drop(guard);
+
+ assert!(reply_rx.await.is_err(), "sender dropped with the slot");
+ assert!(
+ reg.fire_in_process(1u8, 42, make_bus_msg()).is_err(),
+ "late reply must be handed back, not delivered"
+ );
+ let (_guard, _reply_rx) = reg
+ .install_reply_slot(1u8, 42)
+ .expect("request id reusable after guard drop");
+ }
+
+ /// After the reply fired the slot is gone; the guard's drop is a no-op
+ /// and the request id becomes reusable.
+ #[compio::test]
+ async fn guard_drop_is_noop_after_reply_fired() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ reg.insert_in_process(1u8).expect("fresh key");
+
+ let (guard, reply_rx) = reg.install_reply_slot(1u8, 42).expect("slot installs");
+ reg.fire_in_process(1u8, 42, make_bus_msg())
+ .expect("waiter present");
+ drop(guard);
+
+ assert_eq!(reply_rx.await.expect("reply delivered").len(), HEADER_SIZE);
+ let (_guard, _reply_rx) = reg
+ .install_reply_slot(1u8, 42)
+ .expect("request id reusable after fire + drop");
+ }
+
+ /// Entry teardown with a pending waiter: the receiver resolves
+ /// `Canceled` instead of hanging, and the stale guard's later drop
+ /// must not disturb a reinstalled entry's slot (token fence).
+ #[compio::test]
+ async fn entry_teardown_cancels_pending_receiver_and_fences_stale_guard() {
+ let reg: ConnectionRegistry<u8> = ConnectionRegistry::new();
+ reg.insert_in_process(1u8).expect("fresh key");
+ let (stale_guard, reply_rx) = reg.install_reply_slot(1u8, 42).expect("slot installs");
+
+ assert!(reg.remove(1u8));
+ assert!(reply_rx.await.is_err(), "teardown cancels pending waiter");
+
+ reg.insert_in_process(1u8)
+ .expect("reinstall after teardown");
+ let (_guard, reply_rx) = reg.install_reply_slot(1u8, 42).expect("slot installs");
+ drop(stale_guard);
+ reg.fire_in_process(1u8, 42, make_bus_msg())
+ .expect("stale guard must not evict the new slot");
+ assert_eq!(reply_rx.await.expect("reply delivered").len(), HEADER_SIZE);
+ }
}
diff --git a/core/message_bus/src/lifecycle/mod.rs b/core/message_bus/src/lifecycle/mod.rs
index e80bec1f7..5081a516a 100644
--- a/core/message_bus/src/lifecycle/mod.rs
+++ b/core/message_bus/src/lifecycle/mod.rs
@@ -26,6 +26,6 @@ pub mod shutdown;
pub use connection_registry::{
BusMessage, BusReceiver, BusSender, ConnectionRegistry, DrainOutcome, InstanceToken,
- RejectedRegistration, ReplicaRegistry, ReplyRoute,
+ RejectedRegistration, ReplicaRegistry, ReplyRoute, ReplySlotError, ReplySlotGuard,
};
pub use shutdown::{FusedShutdown, Shutdown, ShutdownToken};
diff --git a/core/partitions/src/iggy_partition.rs b/core/partitions/src/iggy_partition.rs
index c396eb948..dc7fbe113 100644
--- a/core/partitions/src/iggy_partition.rs
+++ b/core/partitions/src/iggy_partition.rs
@@ -1762,7 +1762,7 @@ where
let committed_visible_offsets = self.resolve_committed_visible_offsets(&drained).await;
let mut messages_committed = false;
- for mut entry in drained {
+ for entry in drained {
let prepare_header = entry.header;
if !self
.commit_partition_entry(
@@ -1814,14 +1814,6 @@ where
// commit. Session lifecycle is metadata-only.
let reply = build_reply_message(&prepare_header, &bytes::Bytes::new());
- // TODO: no production caller yet. Partition has no in-process
- // subscriber (only metadata uses pipeline_message_with_subscriber);
- // wired for forward-compat. Fired AFTER local commit (slot-first
- // ordering analog). Dropped receiver ignored.
- if let Some(sender) = entry.take_reply_sender() {
- let _ = sender.send(reply.clone());
- }
-
if send_client_replies {
let reply_buffers = reply.into_generic().into_frozen();
emit_sim_event(SimEventKind::ClientReplyEmitted, &event);
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index 3bf5fa33b..f942ebefb 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -548,90 +548,17 @@ async fn handle_client_request(
return;
}
- // Partition data-plane op (SendMessages / consumer-offset writes): the
- // op belongs to the partition's own consensus group, not the metadata
- // group. Route it through the shard mesh by namespace; the owning
- // shard's partitions plane runs at-least-once consensus and replies
- // directly via `send_to_client`. `header.client` therefore stays the
- // TRANSPORT id (home-shard routing bits), not the VSR session id --
- // partition ops are sessionless ("session lifecycle is metadata-only").
if header.operation.is_partition() {
- // The consumer-group offset fence keys on the bound VSR client id (the
- // member id), not the transport id stamped into the partition-op
- // header. `bound` is Some here (unbound transports returned above).
- let vsr_client_id = bound.map_or(0, |(client_id, _)| client_id);
- let namespace = match resolve_partition_request_namespace(
+ // `bound` is Some here (unbound transports returned above).
+ let (vsr_client_id, bound_session) = bound.unwrap_or((0, 0));
+ dispatch_partition_request(
shard,
- header.operation,
- request_body(&request),
+ request,
vsr_client_id,
- ) {
- Ok(namespace) => namespace,
- Err(error) => {
- // A partition op against a stream/topic that no longer
- // resolves (e.g. a consumer's trailing auto-commit racing a
- // `delete_stream`). A silent drop wedges the SDK connection
- // forever; reply empty so the client fails fast instead.
- warn!(
- transport_client_id,
- error = %error,
- operation = ?header.operation,
- "partition request with unresolved namespace; replying empty"
- );
- send_empty_partition_reply(shard, transport_client_id, &header).await;
- return;
- }
- };
- // Convergence wait: a CreateTopic commit returns to the client
- // before the per-shard reconcilers seed routing rows and
- // materialise the partition (next wake/periodic tick). The SDK
- // does not replay sends, so an immediately-following partition op
- // would be dropped as unroutable. Absorb that window here with a
- // bounded wait; steady-state sends (row present, partition probed
- // once) skip it entirely.
- if !wait_for_partition_routable(shard, IggyNamespace::from_raw(namespace)).await {
- warn!(
- transport_client_id,
- namespace,
- operation = ?header.operation,
- "partition request not routable within budget; replying empty"
- );
- send_empty_partition_reply(shard, transport_client_id, &header).await;
- return;
- }
- // A group consumer-offset op carries the group NAME on the wire; the
- // partition plane keys the offset by the group's monotonic id (the same
- // key the poll path auto-commits under and the read path resolves), so
- // rewrite the consumer id before replication -- the apply layer has no
- // metadata access to resolve it.
- let request = match maybe_rewrite_consumer_offset_request(shard, request) {
- Ok(rewritten) => rewritten,
- Err(error) => {
- warn!(
- transport_client_id,
- error = %error,
- operation = ?header.operation,
- "failed to rewrite consumer-offset request; replying empty"
- );
- send_empty_partition_reply(shard, transport_client_id, &header).await;
- return;
- }
- };
- let request = request.transmute_header(|header, new_header: &mut RequestHeader| {
- *new_header = header;
- new_header.namespace = namespace;
- new_header.client = transport_client_id;
- // Header validation requires `session > 0 && request > 0` for
- // non-register ops. The partition plane itself is sessionless
- // (at-least-once, no `ClientTable` dedup), so the bound VSR
- // session merely satisfies validation, and a zero request id
- // (the SDK does not number data-plane ops) is normalized.
- if let Some((_, bound_session)) = bound {
- new_header.session = bound_session;
- }
- new_header.request = new_header.request.max(1);
- });
- shard.dispatch(request.into_generic());
+ bound_session,
+ transport_client_id,
+ )
+ .await;
return;
}
@@ -732,6 +659,102 @@ async fn handle_client_request(
}
}
+/// Route a partition data-plane op (`SendMessages` / consumer-offset writes)
+/// through the shard mesh by namespace: the op belongs to the partition's
+/// own consensus group, not the metadata group. The owning shard's
+/// partitions plane runs at-least-once consensus and replies directly via
+/// `send_to_client`. `header.client` therefore stays the TRANSPORT id
+/// (home-shard routing bits), not the VSR session id -- partition ops are
+/// sessionless ("session lifecycle is metadata-only").
+///
+/// Callers must have authenticated the transport already: `vsr_client_id` /
+/// `bound_session` come from its bound VSR session. Every failure before
+/// dispatch replies empty so the client fails fast instead of wedging on a
+/// silent drop.
+///
+/// `vsr_client_id` keys the consumer-group offset fence (the member id),
+/// not the transport id stamped into the partition-op header.
+#[allow(clippy::future_not_send)]
+pub(crate) async fn dispatch_partition_request(
+ shard: &Rc<ServerNgShard>,
+ request: Message<RequestHeader>,
+ vsr_client_id: u128,
+ bound_session: u64,
+ transport_client_id: u128,
+) {
+ let header = *request.header();
+ let namespace = match resolve_partition_request_namespace(
+ shard,
+ header.operation,
+ request_body(&request),
+ vsr_client_id,
+ ) {
+ Ok(namespace) => namespace,
+ Err(error) => {
+ // A partition op against a stream/topic that no longer
+ // resolves (e.g. a consumer's trailing auto-commit racing a
+ // `delete_stream`). A silent drop wedges the SDK connection
+ // forever; reply empty so the client fails fast instead.
+ warn!(
+ transport_client_id,
+ error = %error,
+ operation = ?header.operation,
+ "partition request with unresolved namespace; replying empty"
+ );
+ send_empty_partition_reply(shard, transport_client_id, &header).await;
+ return;
+ }
+ };
+ // Convergence wait: a CreateTopic commit returns to the client
+ // before the per-shard reconcilers seed routing rows and
+ // materialise the partition (next wake/periodic tick). The SDK
+ // does not replay sends, so an immediately-following partition op
+ // would be dropped as unroutable. Absorb that window here with a
+ // bounded wait; steady-state sends (row present, partition probed
+ // once) skip it entirely.
+ if !wait_for_partition_routable(shard, IggyNamespace::from_raw(namespace)).await {
+ warn!(
+ transport_client_id,
+ namespace,
+ operation = ?header.operation,
+ "partition request not routable within budget; replying empty"
+ );
+ send_empty_partition_reply(shard, transport_client_id, &header).await;
+ return;
+ }
+ // A group consumer-offset op carries the group NAME on the wire; the
+ // partition plane keys the offset by the group's monotonic id (the same
+ // key the poll path auto-commits under and the read path resolves), so
+ // rewrite the consumer id before replication -- the apply layer has no
+ // metadata access to resolve it.
+ let request = match maybe_rewrite_consumer_offset_request(shard, request) {
+ Ok(rewritten) => rewritten,
+ Err(error) => {
+ warn!(
+ transport_client_id,
+ error = %error,
+ operation = ?header.operation,
+ "failed to rewrite consumer-offset request; replying empty"
+ );
+ send_empty_partition_reply(shard, transport_client_id, &header).await;
+ return;
+ }
+ };
+ let request = request.transmute_header(|header, new_header: &mut RequestHeader| {
+ *new_header = header;
+ new_header.namespace = namespace;
+ new_header.client = transport_client_id;
+ // Header validation requires `session > 0 && request > 0` for
+ // non-register ops. The partition plane itself is sessionless
+ // (at-least-once, no `ClientTable` dedup), so the bound VSR
+ // session merely satisfies validation, and a zero request id
+ // (the SDK does not number data-plane ops) is normalized.
+ new_header.session = bound_session;
+ new_header.request = new_header.request.max(1);
+ });
+ shard.dispatch(request.into_generic());
+}
+
#[allow(clippy::future_not_send)]
async fn handle_non_replicated_request(
shard: &Rc<ServerNgShard>,
(iggy) 19/43: test(server-ng): add HTTP data-plane integration tests
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 3b1dddcdd54a564f1e4768400f43275a63a9bdcc
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 01:13:55 2026 +0200
test(server-ng): add HTTP data-plane integration tests
Gate the new shard-0 HTTP surface end to end over raw reqwest rather
than the SDK client, so the wire contract itself is what regresses:
201 + x-iggy-durability on committed produces, 202 + async commit for
ack=none, 413 on the 2 MB body cap without poisoning the listener,
consumer-offset store/get/delete round trip, and 16 concurrent
produces on one login session polled back with no loss and no
cross-partition leakage.
---
core/integration/tests/server/http_vsr.rs | 546 ++++++++++++++++++++++++++++++
core/integration/tests/server/mod.rs | 3 +
2 files changed, 549 insertions(+)
diff --git a/core/integration/tests/server/http_vsr.rs b/core/integration/tests/server/http_vsr.rs
new file mode 100644
index 000000000..eb68dd355
--- /dev/null
+++ b/core/integration/tests/server/http_vsr.rs
@@ -0,0 +1,546 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! HTTP data-plane gate for server-ng: produce, poll, and consumer-offset
+//! routes exercised over raw `reqwest` (not the SDK HTTP client) so the wire
+//! contract itself is under test - exact status codes, the
+//! `x-iggy-durability` header, the body-size cap, and cross-request isolation
+//! of concurrent produces on one login session.
+
+use futures::future::join_all;
+use iggy::prelude::*;
+use iggy_common::create_stream::CreateStream;
+use iggy_common::create_topic::CreateTopic;
+use iggy_common::store_consumer_offset::StoreConsumerOffset;
+use iggy_common::{ConsumerOffsetInfo, IggyMessagesBatch};
+use integration::harness::TestHarness;
+use integration::iggy_harness;
+use reqwest::{Response, StatusCode};
+use std::collections::{BTreeMap, HashMap, HashSet};
+use std::str::FromStr;
+use std::time::{Duration, Instant};
+use tokio::time::sleep;
+
+// server-ng partition ids are 0-based (CreateTopic assigns them from 0).
+const PARTITION_ID: u32 = 0;
+
+/// Explicit consumer id shared by the offset store body and the read/delete
+/// query, so both sides address the same offset-table key
+/// (`Consumer::default()` would carry numeric id 0, not 1).
+const CONSUMER_ID: u32 = 1;
+
+const DURABILITY_HEADER: &str = "x-iggy-durability";
+const DURABILITY_REPLICATED_MEMORY: &str = "replicated-memory";
+const DURABILITY_NONE: &str = "none";
+
+/// The harness readiness gate waits for the dumped runtime config, which the
+/// server writes before binding the HTTP listener; a bounded login retry
+/// bridges that gap plus single-node consensus warmup.
+const LOGIN_TIMEOUT: Duration = Duration::from_secs(15);
+const LOGIN_RETRY_INTERVAL: Duration = Duration::from_millis(50);
+
+/// `?ack=none` answers before the commit; poll until visible, never unbounded.
+const ASYNC_COMMIT_TIMEOUT: Duration = Duration::from_secs(15);
+const ASYNC_COMMIT_RETRY_INTERVAL: Duration = Duration::from_millis(100);
+
+const REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// One authenticated root session against the test server's HTTP listener.
+struct HttpSession {
+ client: reqwest::Client,
+ base_url: String,
+ token: String,
+}
+
+impl HttpSession {
+ async fn login_root(harness: &TestHarness) -> Self {
+ let addr = harness
+ .server()
+ .http_addr()
+ .expect("HTTP transport not configured on test server");
+ let base_url = format!("http://{addr}");
+ let client = reqwest::Client::builder()
+ .timeout(REQUEST_TIMEOUT)
+ .build()
+ .expect("build reqwest client");
+
+ let body = serde_json::json!({
+ "username": DEFAULT_ROOT_USERNAME,
+ "password": DEFAULT_ROOT_PASSWORD,
+ });
+ let deadline = Instant::now() + LOGIN_TIMEOUT;
+ loop {
+ let attempt = client
+ .post(format!("{base_url}/users/login"))
+ .json(&body)
+ .send()
+ .await;
+ match attempt {
+ Ok(response) if response.status().is_success() => {
+ let identity: IdentityInfo =
+ response.json().await.expect("decode IdentityInfo");
+ let token = identity
+ .access_token
+ .expect("login must return an access token")
+ .token;
+ return Self {
+ client,
+ base_url,
+ token,
+ };
+ }
+ // Listener not bound yet or consensus still warming up; any
+ // non-5xx rejection (e.g. 401) is a real failure.
+ Ok(response) if response.status().is_server_error() => {}
+ Ok(response) => panic!("login rejected: {}", response.status()),
+ Err(error) if error.is_connect() => {}
+ Err(error) => panic!("login request failed: {error}"),
+ }
+ assert!(
+ Instant::now() < deadline,
+ "HTTP login did not succeed within {LOGIN_TIMEOUT:?}"
+ );
+ sleep(LOGIN_RETRY_INTERVAL).await;
+ }
+ }
+
+ fn url(&self, path: &str) -> String {
+ format!("{}{path}", self.base_url)
+ }
+
+ async fn create_stream_and_topic(&self, stream: &str, topic: &str, partitions: u32) {
+ let response = self
+ .client
+ .post(self.url("/streams"))
+ .bearer_auth(&self.token)
+ .json(&CreateStream {
+ name: stream.to_string(),
+ })
+ .send()
+ .await
+ .expect("create stream request");
+ assert!(
+ response.status().is_success(),
+ "create stream failed: {}",
+ response.status()
+ );
+
+ let command = CreateTopic {
+ stream_id: Identifier::default(),
+ partitions_count: partitions,
+ compression_algorithm: CompressionAlgorithm::None,
+ message_expiry: IggyExpiry::NeverExpire,
+ max_topic_size: MaxTopicSize::ServerDefault,
+ replication_factor: None,
+ name: topic.to_string(),
+ };
+ let response = self
+ .client
+ .post(self.url(&format!("/streams/{stream}/topics")))
+ .bearer_auth(&self.token)
+ .json(&command)
+ .send()
+ .await
+ .expect("create topic request");
+ assert!(
+ response.status().is_success(),
+ "create topic failed: {}",
+ response.status()
+ );
+ }
+
+ async fn produce(
+ &self,
+ stream: &str,
+ topic: &str,
+ partition_id: u32,
+ messages: Vec<IggyMessage>,
+ ) -> Response {
+ self.produce_with_query(stream, topic, partition_id, messages, "")
+ .await
+ }
+
+ async fn produce_with_query(
+ &self,
+ stream: &str,
+ topic: &str,
+ partition_id: u32,
+ messages: Vec<IggyMessage>,
+ query: &str,
+ ) -> Response {
+ let body = SendMessages {
+ metadata_length: 0,
+ stream_id: Identifier::default(),
+ topic_id: Identifier::default(),
+ partitioning: Partitioning::partition_id(partition_id),
+ batch: IggyMessagesBatch::from(&messages),
+ };
+ self.client
+ .post(self.url(&format!("/streams/{stream}/topics/{topic}/messages{query}")))
+ .bearer_auth(&self.token)
+ .json(&body)
+ .send()
+ .await
+ .expect("produce request")
+ }
+
+ async fn poll(
+ &self,
+ stream: &str,
+ topic: &str,
+ partition_id: u32,
+ offset: u64,
+ count: u32,
+ ) -> PolledMessages {
+ let path = format!(
+ "/streams/{stream}/topics/{topic}/messages\
+ ?consumer_id={CONSUMER_ID}&partition_id={partition_id}\
+ &kind=offset&value={offset}&count={count}&auto_commit=false"
+ );
+ let response = self
+ .client
+ .get(self.url(&path))
+ .bearer_auth(&self.token)
+ .send()
+ .await
+ .expect("poll request");
+ assert_eq!(response.status(), StatusCode::OK, "poll must answer 200");
+ response.json().await.expect("decode PolledMessages")
+ }
+}
+
+fn durability(response: &Response) -> &str {
+ response
+ .headers()
+ .get(DURABILITY_HEADER)
+ .expect("produce response must carry the durability header")
+ .to_str()
+ .expect("durability header is ASCII")
+}
+
+fn text_message(id: u128, payload: String) -> IggyMessage {
+ IggyMessage::builder()
+ .id(id)
+ .payload(payload.into())
+ .build()
+ .expect("message build")
+}
+
+#[iggy_harness]
+async fn given_http_session_when_producing_and_polling_should_round_trip(harness: &TestHarness) {
+ let http = HttpSession::login_root(harness).await;
+ http.create_stream_and_topic("http-stream", "http-topic", 1)
+ .await;
+
+ let header_key = HeaderKey::try_from("trace-id").expect("header key");
+ let header_value = HeaderValue::from_str("trace-42").expect("header value");
+ let messages: Vec<IggyMessage> = (0..3u32)
+ .map(|i| {
+ let builder = IggyMessage::builder()
+ .id(u128::from(i + 1))
+ .payload(format!("round-trip-{i}").into());
+ if i == 0 {
+ let user_headers = BTreeMap::from([(header_key.clone(), header_value.clone())]);
+ builder
+ .user_headers(user_headers)
+ .build()
+ .expect("message build")
+ } else {
+ builder.build().expect("message build")
+ }
+ })
+ .collect();
+
+ let response = http
+ .produce("http-stream", "http-topic", PARTITION_ID, messages)
+ .await;
+ assert_eq!(
+ response.status(),
+ StatusCode::CREATED,
+ "produce must commit"
+ );
+ assert_eq!(durability(&response), DURABILITY_REPLICATED_MEMORY);
+
+ let polled = http
+ .poll("http-stream", "http-topic", PARTITION_ID, 0, 10)
+ .await;
+ assert_eq!(polled.messages.len(), 3, "all sent messages must come back");
+ for (i, message) in polled.messages.iter().enumerate() {
+ assert_eq!(message.header.offset, i as u64, "offsets must be dense");
+ assert_eq!(
+ message.payload,
+ bytes::Bytes::from(format!("round-trip-{i}")),
+ "payload round trip for message {i}"
+ );
+ }
+ let user_headers = polled.messages[0]
+ .user_headers_map()
+ .expect("headers decode")
+ .expect("first message carries user headers");
+ assert_eq!(
+ user_headers.get(&header_key),
+ Some(&header_value),
+ "user header must survive the round trip"
+ );
+}
+
+const CONCURRENT_PRODUCES: usize = 16;
+const CONCURRENT_PARTITIONS: u32 = 4;
+
+#[iggy_harness]
+async fn given_one_session_when_producing_concurrently_should_not_cross_talk(
+ harness: &TestHarness,
+) {
+ let http = HttpSession::login_root(harness).await;
+ http.create_stream_and_topic("http-concurrent", "hammer", CONCURRENT_PARTITIONS)
+ .await;
+
+ let produces = (0..CONCURRENT_PRODUCES).map(|i| {
+ let http = &http;
+ let partition_id = (i as u32) % CONCURRENT_PARTITIONS;
+ async move {
+ let message = text_message(i as u128 + 1, format!("concurrent-{i}"));
+ let response = http
+ .produce("http-concurrent", "hammer", partition_id, vec![message])
+ .await;
+ (i, response.status(), durability(&response).to_string())
+ }
+ });
+ let outcomes = join_all(produces).await;
+
+ for (i, status, durability) in &outcomes {
+ assert_eq!(
+ *status,
+ StatusCode::CREATED,
+ "concurrent produce {i} must commit"
+ );
+ assert_eq!(
+ durability, DURABILITY_REPLICATED_MEMORY,
+ "concurrent produce {i} must attest a replicated commit"
+ );
+ }
+
+ let mut produced: HashMap<u32, HashSet<String>> = HashMap::new();
+ for i in 0..CONCURRENT_PRODUCES {
+ produced
+ .entry((i as u32) % CONCURRENT_PARTITIONS)
+ .or_default()
+ .insert(format!("concurrent-{i}"));
+ }
+
+ // Every partition must return exactly its own produced payloads: nothing
+ // lost, nothing foreign. The produce path is at-least-once, so duplicates
+ // are tolerated but must be byte-identical to a produced payload.
+ let mut polled_union: HashSet<String> = HashSet::new();
+ for partition_id in 0..CONCURRENT_PARTITIONS {
+ let polled = http
+ .poll("http-concurrent", "hammer", partition_id, 0, 100)
+ .await;
+ let expected = &produced[&partition_id];
+ let mut seen: HashSet<String> = HashSet::new();
+ for message in &polled.messages {
+ let payload = String::from_utf8(message.payload.to_vec()).expect("payload is UTF-8");
+ assert!(
+ expected.contains(&payload),
+ "partition {partition_id} returned foreign payload {payload:?}"
+ );
+ seen.insert(payload);
+ }
+ assert_eq!(
+ &seen, expected,
+ "partition {partition_id} must return exactly its own produced payloads"
+ );
+ polled_union.extend(seen);
+ }
+ assert_eq!(
+ polled_union.len(),
+ CONCURRENT_PRODUCES,
+ "every produced payload must be polled back"
+ );
+}
+
+#[iggy_harness]
+async fn given_consumer_offset_when_stored_and_deleted_should_round_trip(harness: &TestHarness) {
+ let http = HttpSession::login_root(harness).await;
+ http.create_stream_and_topic("http-offsets", "offsets", 1)
+ .await;
+
+ // Two committed messages so offset 1 names a real position.
+ let messages = (0..2u32)
+ .map(|i| text_message(u128::from(i + 1), format!("offset-fodder-{i}")))
+ .collect();
+ let response = http
+ .produce("http-offsets", "offsets", PARTITION_ID, messages)
+ .await;
+ assert_eq!(
+ response.status(),
+ StatusCode::CREATED,
+ "produce must commit"
+ );
+
+ let offsets_path = "/streams/http-offsets/topics/offsets/consumer-offsets";
+ let store = StoreConsumerOffset {
+ consumer: Consumer::new(Identifier::numeric(CONSUMER_ID).expect("consumer id")),
+ partition_id: Some(PARTITION_ID),
+ offset: 1,
+ };
+ let response = http
+ .client
+ .put(http.url(offsets_path))
+ .bearer_auth(&http.token)
+ .json(&store)
+ .send()
+ .await
+ .expect("store offset request");
+ assert_eq!(
+ response.status(),
+ StatusCode::NO_CONTENT,
+ "store must commit"
+ );
+
+ let get_path = format!("{offsets_path}?consumer_id={CONSUMER_ID}&partition_id={PARTITION_ID}");
+ let response = http
+ .client
+ .get(http.url(&get_path))
+ .bearer_auth(&http.token)
+ .send()
+ .await
+ .expect("get offset request");
+ assert_eq!(
+ response.status(),
+ StatusCode::OK,
+ "stored offset must be readable"
+ );
+ let info: ConsumerOffsetInfo = response.json().await.expect("decode ConsumerOffsetInfo");
+ assert_eq!(info.stored_offset, 1, "stored offset must round trip");
+ assert_eq!(info.partition_id, PARTITION_ID);
+
+ let delete_path = format!("{offsets_path}/{CONSUMER_ID}?partition_id={PARTITION_ID}");
+ let response = http
+ .client
+ .delete(http.url(&delete_path))
+ .bearer_auth(&http.token)
+ .send()
+ .await
+ .expect("delete offset request");
+ assert_eq!(
+ response.status(),
+ StatusCode::NO_CONTENT,
+ "delete must commit"
+ );
+
+ let response = http
+ .client
+ .get(http.url(&get_path))
+ .bearer_auth(&http.token)
+ .send()
+ .await
+ .expect("get offset request after delete");
+ assert_eq!(
+ response.status(),
+ StatusCode::NOT_FOUND,
+ "deleted offset must be gone"
+ );
+}
+
+#[iggy_harness]
+async fn given_oversized_body_when_producing_should_reject_413(harness: &TestHarness) {
+ let http = HttpSession::login_root(harness).await;
+ http.create_stream_and_topic("http-oversized", "big", 1)
+ .await;
+
+ // 3 MiB payload -> ~4 MiB JSON after base64: over the 2 MB body cap under
+ // either decimal or binary megabyte semantics.
+ let oversized = IggyMessage::builder()
+ .payload(vec![0x42u8; 3 * 1024 * 1024].into())
+ .build()
+ .expect("message build");
+ let response = http
+ .produce("http-oversized", "big", PARTITION_ID, vec![oversized])
+ .await;
+ assert_eq!(
+ response.status(),
+ StatusCode::PAYLOAD_TOO_LARGE,
+ "oversized body must be rejected"
+ );
+
+ // The rejection must not poison the listener: a normal produce still commits.
+ let normal = text_message(1, "small-after-large".to_string());
+ let response = http
+ .produce("http-oversized", "big", PARTITION_ID, vec![normal])
+ .await;
+ assert_eq!(
+ response.status(),
+ StatusCode::CREATED,
+ "server must stay healthy after a 413"
+ );
+ let polled = http
+ .poll("http-oversized", "big", PARTITION_ID, 0, 10)
+ .await;
+ assert_eq!(polled.messages.len(), 1, "normal produce must be pollable");
+ assert_eq!(
+ polled.messages[0].payload,
+ bytes::Bytes::from("small-after-large")
+ );
+}
+
+#[iggy_harness]
+async fn given_ack_none_when_producing_should_return_202_and_commit(harness: &TestHarness) {
+ let http = HttpSession::login_root(harness).await;
+ http.create_stream_and_topic("http-ack-none", "fire", 1)
+ .await;
+
+ let message = text_message(1, "fire-and-forget".to_string());
+ let response = http
+ .produce_with_query(
+ "http-ack-none",
+ "fire",
+ PARTITION_ID,
+ vec![message],
+ "?ack=none",
+ )
+ .await;
+ assert_eq!(
+ response.status(),
+ StatusCode::ACCEPTED,
+ "ack=none must answer before the commit"
+ );
+ assert_eq!(durability(&response), DURABILITY_NONE);
+
+ // The commit still happens, just asynchronously: poll bounded until the
+ // message becomes visible.
+ let deadline = Instant::now() + ASYNC_COMMIT_TIMEOUT;
+ loop {
+ let polled = http
+ .poll("http-ack-none", "fire", PARTITION_ID, 0, 10)
+ .await;
+ if !polled.messages.is_empty() {
+ assert_eq!(polled.messages.len(), 1, "exactly one message was produced");
+ assert_eq!(
+ polled.messages[0].payload,
+ bytes::Bytes::from("fire-and-forget"),
+ "ack=none payload must round trip"
+ );
+ break;
+ }
+ assert!(
+ Instant::now() < deadline,
+ "ack=none produce did not become pollable within {ASYNC_COMMIT_TIMEOUT:?}"
+ );
+ sleep(ASYNC_COMMIT_RETRY_INTERVAL).await;
+ }
+}
diff --git a/core/integration/tests/server/mod.rs b/core/integration/tests/server/mod.rs
index 53e8c70a4..b2f1d8351 100644
--- a/core/integration/tests/server/mod.rs
+++ b/core/integration/tests/server/mod.rs
@@ -24,6 +24,9 @@ mod cg;
// The ported round-robin membership join scenario runs against server-ng.
#[cfg(feature = "vsr")]
mod cg_vsr;
+// Raw-HTTP data-plane contract against server-ng's shard-0 listener.
+#[cfg(feature = "vsr")]
+mod http_vsr;
// 80-case race matrix with hardcoded HTTP variants (test_matrix bypasses
// the harness transport filter); revisit under vsr once basics are green.
#[cfg(not(feature = "vsr"))]
(iggy) 38/43: test(integration): delete only the created user in cg dup-name scenario
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit ed2b83fb2c12c3ef90e671b20e98c9cc46a5dc4c
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 22:13:13 2026 +0200
test(integration): delete only the created user in cg dup-name scenario
The scenario creates USERNAME_1 only but ran the shared cleanup with
delete_users=true, which also deletes USERNAME_2/USERNAME_3 and panics
on the committed UserNotFound. Pre-existing deterministic failure,
unrelated to the parity fixes in this series.
---
.../scenarios/consumer_group_duplicate_name_create_scenario.rs | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/core/integration/tests/server/scenarios/consumer_group_duplicate_name_create_scenario.rs b/core/integration/tests/server/scenarios/consumer_group_duplicate_name_create_scenario.rs
index fa1b928cb..658179408 100644
--- a/core/integration/tests/server/scenarios/consumer_group_duplicate_name_create_scenario.rs
+++ b/core/integration/tests/server/scenarios/consumer_group_duplicate_name_create_scenario.rs
@@ -30,7 +30,9 @@ use iggy::prelude::Identifier;
use iggy::prelude::IggyExpiry;
use iggy::prelude::MaxTopicSize;
use iggy::prelude::{ConsumerGroupClient, StreamClient, TopicClient};
-use integration::harness::{TestHarness, assert_clean_system, create_user, login_user};
+use integration::harness::{
+ TestHarness, assert_clean_system, create_user, delete_user, login_user,
+};
pub async fn run(harness: &TestHarness) {
let system_client = harness
@@ -99,7 +101,10 @@ pub async fn run(harness: &TestHarness) {
"the surviving member must keep its partition assignment"
);
- cleanup(&system_client, true).await;
+ // Only USERNAME_1 exists here; the shared `cleanup(_, true)` deletes all
+ // three scenario users and dies on the absent ones.
+ delete_user(&system_client, USERNAME_1).await;
+ cleanup(&system_client, false).await;
assert_clean_system(&system_client).await;
}
(iggy) 08/43: feat(server-ng): add HTTP personal-access-token create/delete routes
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 354ce587ef4f06b70c70f19b1a1b4b954095f897
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 16:09:59 2026 +0200
feat(server-ng): add HTTP personal-access-token create/delete routes
The shard-0 HTTP listener had no way to mint or revoke personal
access tokens, leaving REST callers short of legacy parity.
A raw PAT is non-deterministic and secret, so it must never enter
consensus. rewrite_pat_request_for_user mints it on shard 0 and
replicates only its hash, leaving a successful committed reply body
empty; build_raw_pat_reply then splices the raw secret back in
locally after the commit. Reaching that committed reply Message
means submit_write's core (gate + lock + rewrites + submit) is
extracted into submit_committed, which returns the Message and
request header; submit_write now wraps it and decodes the body, so
every existing stream/topic/user caller is behavior-unchanged.
A committed create can still carry a business rejection (duplicate
name), so create_pat honors the reply's result code through the
shared committed_payload classifier before splicing the secret:
only a genuine success gets a token, a rejection renders the legacy
error. Both new routes inherit the root-only admin gate through the
shared core.
---
core/server-ng/src/http.rs | 193 ++++++++++++++++++++++++++++++++++++---------
core/server-ng/src/pat.rs | 21 ++++-
2 files changed, 172 insertions(+), 42 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 719219c2f..ad3f3968f 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -37,6 +37,9 @@ use configs::http::HttpJwtConfig;
use iggy_binary_protocol::consensus::{
Command2, EvictionHeader, EvictionReason, HEADER_SIZE, result_code, result_section_len,
};
+use iggy_binary_protocol::requests::personal_access_tokens::{
+ CreatePersonalAccessTokenRequest, DeletePersonalAccessTokenRequest,
+};
use iggy_binary_protocol::requests::streams::{
CreateStreamRequest, DeleteStreamRequest, PurgeStreamRequest, UpdateStreamRequest,
};
@@ -47,6 +50,7 @@ use iggy_binary_protocol::requests::users::{
ChangePasswordRequest, CreateUserRequest, DeleteUserRequest, UpdatePermissionsRequest,
UpdateUserRequest,
};
+use iggy_binary_protocol::responses::personal_access_tokens::RawPersonalAccessTokenResponse;
use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
use iggy_binary_protocol::responses::topics::get_topic::GetTopicResponse;
use iggy_binary_protocol::responses::users::get_user::UserDetailsResponse;
@@ -55,6 +59,7 @@ use iggy_binary_protocol::{
GenericHeader, Operation, RequestHeader, WireDecode, WireEncode, WireName,
};
use iggy_common::change_password::ChangePassword;
+use iggy_common::create_personal_access_token::CreatePersonalAccessToken;
use iggy_common::create_stream::CreateStream;
use iggy_common::create_topic::CreateTopic;
use iggy_common::create_user::CreateUser;
@@ -67,8 +72,8 @@ use iggy_common::update_topic::UpdateTopic;
use iggy_common::update_user::UpdateUser;
use iggy_common::wire_conversions::{identifier_to_wire, permissions_to_wire};
use iggy_common::{
- Identifier, IdentityInfo, IggyError, IggyTimestamp, StreamDetails, TokenInfo, TopicDetails,
- UserInfoDetails, Validatable,
+ Identifier, IdentityInfo, IggyError, IggyTimestamp, RawPersonalAccessToken, StreamDetails,
+ TokenInfo, TopicDetails, UserInfoDetails, Validatable,
};
use message_bus::client_listener;
use secrecy::ExposeSecret;
@@ -84,6 +89,8 @@ use crate::dispatch::{submit_client_request_on_owner, submit_register_on_owner};
use crate::http::extractor::Authenticated;
use crate::http::jwt::JwtManager;
use crate::login_register::LoginRegisterError;
+use crate::pat::rewrite_pat_request_for_user;
+use crate::responses::build_raw_pat_reply;
use crate::server_error::ServerNgError;
use crate::users::maybe_rewrite_user_password_request;
@@ -382,6 +389,8 @@ fn router(state: HttpState) -> Router {
"/streams/{stream_id}/topics/{topic_id}/purge",
delete(purge_topic),
)
+ .route("/personal-access-tokens", post(create_pat))
+ .route("/personal-access-tokens/{name}", delete(delete_pat))
.with_state(state)
}
@@ -770,30 +779,101 @@ async fn update_permissions(
Ok(StatusCode::NO_CONTENT)
}
-/// Run one authenticated control-plane write end to end and return the committed
-/// reply's typed payload. Shared by every write route: `create_stream` decodes
-/// the payload into an entity, the update/delete/purge routes ignore it (it is
-/// empty) and answer 204.
+/// `POST /personal-access-tokens`: mint a personal access token for the caller
+/// and return its one-time raw secret as the same `{"token": ...}` JSON the
+/// legacy server returns, with HTTP 200.
///
-/// Serializes this session's writes behind its gate and holds it across the
-/// submit so request ids reach the primary strictly in order. The gate advances
-/// only on a committed `Reply`: a transient failure or an eviction leaves the id
-/// free for the caller's retry, which the depth-1 dedup requires (the next
-/// accepted request must be `committed + 1`, so a consumed-but-uncommitted id
-/// would wedge the session on `RequestGap`).
+/// The raw token is non-deterministic and secret, so it must never enter
+/// consensus: [`rewrite_pat_request_for_user`] (invoked inside
+/// [`submit_committed`]) mints it on shard 0 and replicates only its hash, so a
+/// successful committed reply body is empty. [`build_raw_pat_reply`] then splices
+/// the raw secret back into that reply locally, using the confirmed commit
+/// position. The token is surfaced only after the write commits; a malformed
+/// splice fails closed rather than emitting a blank token.
///
-/// Reply discrimination mirrors the SDK's `split_metadata_result`: the body
-/// leads with a result section (`Some(0)` is success followed by the typed
-/// payload; a nonzero first result is a committed business rejection carrying an
-/// `IggyError` code). An eviction frame carries no result section and means the
-/// session is dead (re-authenticate); a missing or short result section is a
-/// malformed committed reply, mapped to an error rather than a false success.
-async fn submit_write(
+/// A committed create can still carry a business rejection (duplicate name,
+/// invalid expiry), so the result code is honored via [`committed_payload`] -
+/// exactly as the mechanical routes do - BEFORE the secret is spliced. Only a
+/// genuine success gets a token; a rejection renders the legacy error instead of
+/// a bogus 200 + token.
+async fn create_pat(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Json(command): Json<CreatePersonalAccessToken>,
+) -> Result<Json<RawPersonalAccessToken>, WriteError> {
+ // Rejects an empty/oversized token name before any consensus work.
+ command.validate().map_err(WriteError::Rejected)?;
+ let request = CreatePersonalAccessTokenRequest {
+ name: WireName::new(&command.name)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidPersonalAccessTokenName))?,
+ expiry: command.expiry.into(),
+ };
+ let body = request.to_bytes();
+ let (request_header, committed, raw_token) = SendWrapper::new(submit_committed(
+ &state,
+ &identity.session,
+ Operation::CreatePersonalAccessToken,
+ &body,
+ ))
+ .await?;
+ // Reject a committed business error before splicing the secret; the success
+ // payload is empty, so the returned slice is discarded.
+ committed_payload(&committed)?;
+ let reply =
+ build_raw_pat_reply(&request_header, committed, raw_token).map_err(WriteError::Rejected)?;
+ Ok(Json(RawPersonalAccessToken {
+ token: decode_raw_pat_token(&reply)?,
+ }))
+}
+
+/// `DELETE /personal-access-tokens/{name}`: delete one of the caller's tokens by
+/// name. Returns 204 on commit. Inherits the root-only gate via [`submit_write`].
+async fn delete_pat(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(name): Path<String>,
+) -> Result<StatusCode, WriteError> {
+ let request = DeletePersonalAccessTokenRequest {
+ name: WireName::new(&name)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidPersonalAccessTokenName))?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::DeletePersonalAccessToken,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// Run one authenticated control-plane write to commit and hand back the
+/// committed reply `Message`, the request header, and any raw PAT token minted
+/// along the way. Shared core of every HTTP write: [`submit_write`] decodes the
+/// reply body for the stream/topic/user routes, while [`create_pat`] needs the
+/// raw `Message` + request header to substitute the one-time token.
+///
+/// Enforces the root-only admin gate, then serializes this session's writes
+/// behind its gate and holds it across the submit so request ids reach the
+/// primary strictly in order. The gate advances only on a committed `Reply`: a
+/// transient failure or an eviction leaves the id free for the caller's retry,
+/// which the depth-1 dedup requires (the next accepted request must be
+/// `committed + 1`, so a consumed-but-uncommitted id would wedge the session on
+/// `RequestGap`). An eviction means the session is dead, mapped to 401.
+///
+/// Both shard-0 request rewrites run here before consensus, mirroring the TCP
+/// dispatch path: the PAT rewrite mints a raw token and replicates only its hash
+/// (`CreatePersonalAccessToken`), and the user-password rewrite hashes plaintext
+/// (`CreateUser` / `ChangePassword`). Both are no-ops for every other operation,
+/// so plaintext secrets never enter consensus on any write. On either rewrite's
+/// decode failure the gate is released without advancing, leaving the id free.
+async fn submit_committed(
state: &HttpInner,
session: &HttpSession,
operation: Operation,
body: &[u8],
-) -> Result<Bytes, WriteError> {
+) -> Result<(RequestHeader, Message<GenericHeader>, Option<String>), WriteError> {
// Interim authorization: until server-ng has an RBAC permissioner, every
// control-plane write is root-only. A non-root credential is authenticated
// but unprivileged, rejected before any consensus work is spent.
@@ -808,11 +888,10 @@ async fn submit_write(
*next_request_id,
body,
);
- // Hash raw passwords on shard 0 before replication (CreateUser /
- // ChangePassword); a no-op for every other operation. Mirrors the TCP
- // dispatch path so plaintext never enters consensus. On decode failure the
- // gate is released without advancing, leaving the request id free to retry.
+ let (message, raw_token) =
+ rewrite_pat_request_for_user(session.user_id, message).map_err(WriteError::Rejected)?;
let message = maybe_rewrite_user_password_request(message).map_err(WriteError::Rejected)?;
+ let request_header = *message.header();
let reply = submit_client_request_on_owner(&state.shard, message).await;
let Some(reply) = reply else {
return Err(WriteError::Unavailable);
@@ -822,26 +901,52 @@ async fn submit_write(
Command2::Reply => {
*next_request_id += 1;
drop(next_request_id);
- let size = reply.header().size as usize;
- let reply_body = reply.as_slice().get(HEADER_SIZE..size).unwrap_or_default();
- match result_code(reply_body) {
- Some(0) => {
- let payload_start = result_section_len(reply_body)
- .ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
- let payload = reply_body
- .get(payload_start..)
- .ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
- Ok(Bytes::copy_from_slice(payload))
- }
- Some(code) => Err(WriteError::Rejected(IggyError::from_code(code))),
- None => Err(WriteError::Rejected(IggyError::InvalidCommand)),
- }
+ Ok((request_header, reply, raw_token))
}
Command2::Eviction => Err(WriteError::Rejected(eviction_error(&reply))),
_ => Err(WriteError::Rejected(IggyError::InvalidCommand)),
}
}
+/// Run one authenticated control-plane write end to end and return the committed
+/// reply's typed payload. Wraps [`submit_committed`] and decodes the reply body
+/// via [`committed_payload`]: `create_stream` decodes the payload into an entity,
+/// the update/delete/purge routes ignore it (it is empty) and answer 204.
+async fn submit_write(
+ state: &HttpInner,
+ session: &HttpSession,
+ operation: Operation,
+ body: &[u8],
+) -> Result<Bytes, WriteError> {
+ let (_request_header, reply, _raw_token) =
+ submit_committed(state, session, operation, body).await?;
+ Ok(Bytes::copy_from_slice(committed_payload(&reply)?))
+}
+
+/// Classify a committed reply's leading result section and return the typed
+/// payload slice on success. Mirrors the SDK's `split_metadata_result`:
+/// `Some(0)` is success and the payload follows the result section; a nonzero
+/// first result is a committed business rejection carrying an `IggyError` code
+/// (e.g. a duplicate token name); a missing or short section is a malformed
+/// committed reply, mapped to an error rather than a false success. Shared by
+/// [`submit_write`] and [`create_pat`] so a committed rejection can never render
+/// as a 2xx.
+fn committed_payload(reply: &Message<GenericHeader>) -> Result<&[u8], WriteError> {
+ let size = reply.header().size as usize;
+ let reply_body = reply.as_slice().get(HEADER_SIZE..size).unwrap_or_default();
+ match result_code(reply_body) {
+ Some(0) => {
+ let payload_start = result_section_len(reply_body)
+ .ok_or(WriteError::Rejected(IggyError::InvalidCommand))?;
+ reply_body
+ .get(payload_start..)
+ .ok_or(WriteError::Rejected(IggyError::InvalidCommand))
+ }
+ Some(code) => Err(WriteError::Rejected(IggyError::from_code(code))),
+ None => Err(WriteError::Rejected(IggyError::InvalidCommand)),
+ }
+}
+
/// Build a `Message<RequestHeader>` for a control-plane write by filling a zeroed
/// `#[repr(C)]` header, mirroring `wire::rewrite_request_body` and the partition
/// reconciler's prepare builder. `body` is the already-encoded wire request,
@@ -896,6 +1001,18 @@ fn decode_user_details(payload: &[u8]) -> Result<UserInfoDetails, WriteError> {
UserInfoDetails::try_from(response).map_err(WriteError::Rejected)
}
+/// Extract the raw one-time token from a [`build_raw_pat_reply`] output. That
+/// reply's body is a bare `RawPersonalAccessTokenResponse` (no leading result
+/// section, unlike a committed metadata reply), so decode it straight past the
+/// header.
+fn decode_raw_pat_token(reply: &Message<GenericHeader>) -> Result<String, WriteError> {
+ let size = reply.header().size as usize;
+ let body = reply.as_slice().get(HEADER_SIZE..size).unwrap_or_default();
+ let response = RawPersonalAccessTokenResponse::decode_from(body)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
+ Ok(response.token.as_str().to_string())
+}
+
/// Map an eviction frame to the same typed [`IggyError`] the SDK's
/// `decode_eviction` produces, so an HTTP caller sees the identical status a TCP
/// caller would (session-terminal reasons render as 401 -> re-authenticate).
diff --git a/core/server-ng/src/pat.rs b/core/server-ng/src/pat.rs
index 0a8199fa9..69bd83566 100644
--- a/core/server-ng/src/pat.rs
+++ b/core/server-ng/src/pat.rs
@@ -42,18 +42,31 @@ pub(crate) fn maybe_rewrite_pat_request(
transport_client_id: u128,
request: Message<RequestHeader>,
) -> Result<(Message<RequestHeader>, Option<String>), IggyError> {
- let operation = request.header().operation;
- let user_id = match operation {
+ let user_id = match request.header().operation {
Operation::CreatePersonalAccessToken | Operation::DeletePersonalAccessToken => sessions
.borrow()
.get_user_id(transport_client_id)
.ok_or(IggyError::Unauthenticated)?,
_ => return Ok((request, None)),
};
+ rewrite_pat_request_for_user(user_id, request)
+}
+/// PAT rewrite for a caller that already holds the authenticated `user_id`.
+///
+/// The HTTP listener authenticates against its own session table rather than
+/// the transport `SessionManager`, so it resolves the id itself and calls this
+/// directly. Mints the raw token + hash for `CreatePersonalAccessToken` and
+/// hands the raw token back; every other operation passes through unchanged
+/// with `None` (the HTTP write core routes all of its ops here, so the non-PAT
+/// arm is a no-op, not `unreachable`).
+pub(crate) fn rewrite_pat_request_for_user(
+ user_id: u32,
+ request: Message<RequestHeader>,
+) -> Result<(Message<RequestHeader>, Option<String>), IggyError> {
let body = request_body(&request);
let mut raw_token = None;
- let rewritten = match operation {
+ let rewritten = match request.header().operation {
Operation::CreatePersonalAccessToken => {
let wire = WireCreatePersonalAccessTokenRequest::decode_from(body)
.map_err(|_| IggyError::InvalidCommand)?;
@@ -84,7 +97,7 @@ pub(crate) fn maybe_rewrite_pat_request(
}
.to_bytes()
}
- _ => unreachable!(),
+ _ => return Ok((request, None)),
};
Ok((rewrite_request_body(&request, &rewritten)?, raw_token))
(iggy) 22/43: feat(metadata): enforce RBAC in the metadata apply path
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit dc246adea799d8db614c5b2870439959da0c079e
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 13:17:26 2026 +0200
feat(metadata): enforce RBAC in the metadata apply path
server-ng authorized nothing on the control plane: any authenticated
user could run every metadata op. The permissioner was live but had no
caller.
Gate every committed metadata op through an authorize step before its
state handler runs, at all three apply sites (primary commit, journal
replay, recovery) so the decision is recomputed identically on every
replica from the acting user id carried in the prepare header. A denied
op commits as a deterministic no-op returning Unauthorized, keeping the
log identical everywhere rather than dropping the entry on one node.
The acting user is stamped from the client table at submit; it can only
be the credential-verified user, never a wire-supplied value. The gate
short-circuits the root user (slab id 0), so root deletion and root
permission changes are now rejected in-apply as legacy does, keeping
that slot permanently root and the short-circuit sound. The simulator
seeds root to match the always-rooted production world.
---
core/consensus/src/impls.rs | 5 +-
core/metadata/src/impls/metadata.rs | 45 +-
core/metadata/src/impls/recovery.rs | 6 +-
core/metadata/src/stm/authz.rs | 758 +++++++++++++++++++++
core/metadata/src/stm/mod.rs | 1 +
core/metadata/src/stm/result.rs | 50 +-
core/metadata/src/stm/stream.rs | 8 +-
core/metadata/src/stm/user.rs | 118 +++-
core/simulator/src/replica.rs | 4 +
core/simulator/src/workload/ops/delete_user.rs | 7 +-
.../src/workload/ops/update_permissions.rs | 2 +
11 files changed, 990 insertions(+), 14 deletions(-)
diff --git a/core/consensus/src/impls.rs b/core/consensus/src/impls.rs
index 6521ecc3c..6b9a9c457 100644
--- a/core/consensus/src/impls.rs
+++ b/core/consensus/src/impls.rs
@@ -2087,8 +2087,9 @@ where
timestamp,
operation: old.operation,
namespace: old.namespace,
- // Inert for non-register ops (they carry 0); on Register it
- // replicates the acting user to every backup.
+ // Copied verbatim: carries the stamped acting user for client
+ // ops (and the authenticated user on Register), so the in-apply
+ // RBAC gate resolves the same identity on every backup.
user_id: old.user_id,
..Default::default()
}
diff --git a/core/metadata/src/impls/metadata.rs b/core/metadata/src/impls/metadata.rs
index b8c50f666..8667818f8 100644
--- a/core/metadata/src/impls/metadata.rs
+++ b/core/metadata/src/impls/metadata.rs
@@ -16,6 +16,7 @@
// under the License.
use crate::MuxStateMachine;
+use crate::stm::authz::gated_apply;
use crate::stm::consumer_group::CompleteConsumerGroupRevocationRequest;
use crate::stm::snapshot::{FillSnapshot, MetadataSnapshot, Snapshot, SnapshotError};
use crate::stm::stream::{Streams, TruncatePartitionRequest};
@@ -341,8 +342,8 @@ fn require_shard_zero<'a, T>(
slot
}
-/// Late-bound callback invoked after every successful
-/// `mux_stm.update(prepare)` on shard 0's metadata commit path.
+/// Late-bound callback invoked after every committed op on shard 0's metadata
+/// commit path (via `gated_apply`, including a gated no-op).
///
/// Wired by server-ng bootstrap once the metadata bundle has broadcast;
/// receives the committed [`Operation`] so the recipient can filter (the
@@ -799,7 +800,7 @@ where
// Normal op: apply SM, commit_reply. `Err` is decode/corruption
// only; a business rejection commits as a deterministic no-op
// whose `code` rides the reply body, replayed on retry.
- let apply = self.mux_stm.update(prepare).unwrap_or_else(|err| {
+ let apply = gated_apply(&self.mux_stm, prepare).unwrap_or_else(|err| {
panic!(
"on_ack: committed metadata op={} failed to apply: {err}",
prepare_header.op
@@ -1293,7 +1294,7 @@ where
#[allow(clippy::future_not_send)]
pub async fn submit_request_in_process(
&self,
- message: Message<RequestHeader>,
+ mut message: Message<RequestHeader>,
) -> Result<Message<GenericHeader>, MetadataSubmitError> {
let request_header = *message.header();
let client_id = request_header.client;
@@ -1342,6 +1343,33 @@ where
return Err(MetadataSubmitError::PipelineFull);
}
+ // Stamp the acting user id into the request header for the in-apply RBAC
+ // gate, overwriting any wire-supplied value (never trusted). The cluster
+ // resolves session -> user at Register (f1) and the gate must run
+ // identically on every replica, so identity rides the replicated prepare
+ // rather than a per-node session table. `request_preflight` above proved
+ // the session is live, so `get_user_id` is `Some`; the `unwrap_or(0)` is
+ // an unreachable guard, not a root grant (0 is only root's genuine id).
+ //
+ // Kept here rather than hoisted into the shared `prepare_request`:
+ // `Register` and `Logout` funnel through it too, and a mid-`Register`
+ // client has no committed session yet (`get_user_id` is `None`), so a
+ // blanket stamp there would zero `Register`'s f1 user id. The wire
+ // ingresses (`on_request`, `drain_request_queue_into_prepares`) are not
+ // client-reachable today; closing that latent gap needs an op-guarded
+ // stamp, tracked separately.
+ let acting_user_id = self
+ .client_table
+ .borrow()
+ .get_user_id(client_id)
+ .unwrap_or(0);
+ {
+ let header = bytemuck::checked::from_bytes_mut::<RequestHeader>(
+ &mut message.as_mut_slice()[..size_of::<RequestHeader>()],
+ );
+ header.user_id = acting_user_id;
+ }
+
let Ok(prepare) = self.prepare_request(message) else {
// Structurally-invalid request. Return an eviction frame (relayed to
// the socket by the home shard) rather than `Canceled`, which leaves
@@ -1706,7 +1734,7 @@ where
// Normal op: apply SM, commit_reply. `Err` is decode/corruption
// only; a business rejection commits as a deterministic no-op
// whose `code` rides the reply body, replayed on retry.
- let apply = self.mux_stm.update(prepare).unwrap_or_else(|err| {
+ let apply = gated_apply(&self.mux_stm, prepare).unwrap_or_else(|err| {
panic!("commit_journal: committed metadata op={op} failed to apply: {err}");
});
// Post-commit notifier (e.g. partition reconciler
@@ -2001,6 +2029,13 @@ where
timestamp,
operation,
namespace: request.namespace,
+ // Carry the acting user id so the in-apply RBAC gate sees the same
+ // identity on every replica. The default projection copies it (see
+ // `Project::project`); this helper builds prepares for the ops it
+ // rewrites (CreateTopic/CreatePartitions assignment forms,
+ // TruncatePartition), which would otherwise reset it to 0 via
+ // `..Default::default()`.
+ user_id: request.user_id,
..Default::default()
};
diff --git a/core/metadata/src/impls/recovery.rs b/core/metadata/src/impls/recovery.rs
index a32e558f9..4268c1633 100644
--- a/core/metadata/src/impls/recovery.rs
+++ b/core/metadata/src/impls/recovery.rs
@@ -17,6 +17,7 @@
use crate::impls::metadata::IggySnapshot;
use crate::stm::StateMachine;
+use crate::stm::authz::GatedApply;
use crate::stm::snapshot::{MetadataSnapshot, RestoreSnapshot, Snapshot, SnapshotError};
use iggy_binary_protocol::consensus::PrepareHeader;
use iggy_common::IggyError;
@@ -107,6 +108,7 @@ pub struct RecoveredMetadata<M> {
pub async fn recover<M>(data_dir: &Path) -> Result<RecoveredMetadata<M>, RecoveryError>
where
M: StateMachine<Input = Message<PrepareHeader>, Error = IggyError>
+ + GatedApply
+ RestoreSnapshot<MetadataSnapshot>
+ Default,
{
@@ -148,7 +150,9 @@ where
format!("failed to read journal entry for op={}", header.op),
))
})?;
- mux_stm.update(entry)?;
+ // WAL replay must recompute authorization denials identically to the
+ // primary/backup commit paths, so it goes through the same gate.
+ mux_stm.gated_update(entry)?;
last_applied_op = Some(header.op);
}
diff --git a/core/metadata/src/stm/authz.rs b/core/metadata/src/stm/authz.rs
new file mode 100644
index 000000000..35bfdf122
--- /dev/null
+++ b/core/metadata/src/stm/authz.rs
@@ -0,0 +1,758 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! In-apply control-plane authorization for the metadata state machine.
+//!
+//! Every committed client op is gated here before its `StateHandler::apply`
+//! runs. A denial returns an `Unauthorized` [`ApplyReply`] WITHOUT touching
+//! state, so the op commits as a deterministic no-op whose error rides the
+//! cached reply and replays on retry, exactly like a business rejection.
+//!
+//! Replay-determinism invariant: [`authorize`] is a pure function of the
+//! prepare header (`operation` + the acting `user_id`, stamped into the
+//! replicated header at submit time) and the committed permission/stream
+//! state as of the op immediately before this one. That state is applied in
+//! the same order on every replica, so the primary (`on_ack`), each backup
+//! (`commit_journal`), and WAL replay (`recover`) all recompute the identical
+//! decision. This is why identity must ride the header rather than being
+//! resolved from a local session table: a backup has no session for the
+//! acting client.
+
+use crate::MuxStateMachine;
+use crate::impls::metadata::StreamsFrontend;
+use crate::permissioner::Permissioner;
+use crate::stm::StateMachine;
+use crate::stm::consumer_group::{JoinConsumerGroupRequest, LeaveConsumerGroupRequest};
+use crate::stm::result::ApplyReply;
+use crate::stm::stream::{Streams, TruncatePartitionRequest};
+use crate::stm::user::Users;
+use iggy_binary_protocol::requests::consumer_groups::{
+ CreateConsumerGroupRequest, DeleteConsumerGroupRequest,
+};
+use iggy_binary_protocol::requests::partitions::{
+ CreatePartitionsWithAssignmentsRequest, DeletePartitionsRequest,
+};
+use iggy_binary_protocol::requests::streams::{
+ DeleteStreamRequest, PurgeStreamRequest, UpdateStreamRequest,
+};
+use iggy_binary_protocol::requests::topics::{
+ CreateTopicWithAssignmentsRequest, DeleteTopicRequest, PurgeTopicRequest, UpdateTopicRequest,
+};
+use iggy_binary_protocol::requests::users::ChangePasswordRequest;
+use iggy_binary_protocol::{Operation, PrepareHeader, WireDecode, WireIdentifier};
+use iggy_common::{IggyError, variadic};
+use server_common::Message;
+use std::mem::size_of;
+
+/// Gate a committed prepare, then apply it. A denial commits as an
+/// `Unauthorized` no-op (the gate never mutates state); an allow proceeds to
+/// the normal state-machine dispatch. This is the single funnel every
+/// production apply path uses, so no committed op can bypass the RBAC check.
+///
+/// The primary (`on_ack`) and backup (`commit_journal`) paths hold the concrete
+/// mux and call this directly; WAL replay (`recover`) is generic over the state
+/// machine and reaches it through [`GatedApply`].
+///
+/// # Errors
+/// Propagates the underlying [`StateMachine::update`] error (decode / corruption
+/// of a committed body), which is fatal on the commit path.
+pub(crate) fn gated_apply<M>(
+ mux: &M,
+ prepare: Message<PrepareHeader>,
+) -> Result<ApplyReply, IggyError>
+where
+ M: StreamsFrontend
+ + StateMachine<Input = Message<PrepareHeader>, Output = ApplyReply, Error = IggyError>,
+{
+ if let Some(denied) = authorize(&prepare, mux.users(), mux.streams()) {
+ return Ok(denied);
+ }
+ mux.update(prepare)
+}
+
+/// Generic entry point to [`gated_apply`] for the WAL-replay path.
+///
+/// The replay path is generic over the state machine and cannot name the
+/// concrete accessors, so it dispatches through this trait. Implemented only
+/// for the metadata mux (and, in tests, the empty state machine used by the
+/// recovery harness).
+pub trait GatedApply:
+ StateMachine<Input = Message<PrepareHeader>, Output = ApplyReply, Error = IggyError>
+{
+ /// See [`gated_apply`].
+ ///
+ /// # Errors
+ /// Propagates the underlying [`StateMachine::update`] error.
+ fn gated_update(&self, prepare: Message<PrepareHeader>) -> Result<ApplyReply, IggyError>;
+}
+
+impl GatedApply for MuxStateMachine<variadic!(Users, Streams)> {
+ fn gated_update(&self, prepare: Message<PrepareHeader>) -> Result<ApplyReply, IggyError> {
+ gated_apply(self, prepare)
+ }
+}
+
+/// The root user is always the first user created (slab id 0) and holds every
+/// grant. Short-circuited below rather than checked against the permissioner so
+/// the gate is correct even in a cluster that provisions no users (the
+/// deterministic simulator boots none, so its permissioner is empty).
+const ROOT_USER_ID: u32 = 0;
+
+/// Decides whether the acting user may run this committed op.
+///
+/// `Some(reply)` denies with a committed `Unauthorized` no-op; `None` allows
+/// the op to reach its `StateHandler::apply`. Only control-plane client ops are
+/// checked; internal / server-originated ops (which carry `user_id` 0), the
+/// self-scoped personal-access-token ops (no legacy RBAC), the pre-projection
+/// wire forms, and partition-plane ops all pass through.
+///
+/// Object ids are resolved against committed stream state first; a resolution
+/// miss passes through so `apply` produces its own `NotFound`, preserving the
+/// legacy notfound-before-permission ordering.
+#[allow(clippy::too_many_lines)]
+pub(crate) fn authorize(
+ prepare: &Message<PrepareHeader>,
+ users: &Users,
+ streams: &Streams,
+) -> Option<ApplyReply> {
+ let header = prepare.header();
+ let user_id = header.user_id;
+
+ if user_id == ROOT_USER_ID {
+ return None;
+ }
+ let body = &prepare.as_slice()[size_of::<PrepareHeader>()..header.size as usize];
+
+ match header.operation {
+ // Streams. `create_stream` is unscoped; the rest resolve the stream id.
+ Operation::CreateStream => check(users, |perm| perm.create_stream(user_id)),
+ Operation::UpdateStream => {
+ let Ok(request) = UpdateStreamRequest::decode_from(body) else {
+ return None;
+ };
+ stream_scoped(users, streams, &request.stream_id, |perm, sid| {
+ perm.update_stream(user_id, sid)
+ })
+ }
+ Operation::DeleteStream => {
+ let Ok(request) = DeleteStreamRequest::decode_from(body) else {
+ return None;
+ };
+ stream_scoped(users, streams, &request.stream_id, |perm, sid| {
+ perm.delete_stream(user_id, sid)
+ })
+ }
+ Operation::PurgeStream => {
+ let Ok(request) = PurgeStreamRequest::decode_from(body) else {
+ return None;
+ };
+ stream_scoped(users, streams, &request.stream_id, |perm, sid| {
+ perm.purge_stream(user_id, sid)
+ })
+ }
+
+ // Topics. The wire `CreateTopic` is projected to
+ // `CreateTopicWithAssignments` before it reaches apply, so only the
+ // enriched form is gated. `create_topic` is stream-scoped.
+ Operation::CreateTopicWithAssignments => {
+ let Ok(request) = CreateTopicWithAssignmentsRequest::decode_from(body) else {
+ return None;
+ };
+ stream_scoped(users, streams, &request.request.stream_id, |perm, sid| {
+ perm.create_topic(user_id, sid)
+ })
+ }
+ Operation::UpdateTopic => {
+ let Ok(request) = UpdateTopicRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.update_topic(user_id, sid, tid),
+ )
+ }
+ Operation::DeleteTopic => {
+ let Ok(request) = DeleteTopicRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.delete_topic(user_id, sid, tid),
+ )
+ }
+ Operation::PurgeTopic => {
+ let Ok(request) = PurgeTopicRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.purge_topic(user_id, sid, tid),
+ )
+ }
+
+ // Partitions. Wire `CreatePartitions` is projected to the enriched
+ // form; `DeleteSegments` is resolved server-side to `TruncatePartition`
+ // carrying the original client's id.
+ Operation::CreatePartitionsWithAssignments => {
+ let Ok(request) = CreatePartitionsWithAssignmentsRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.request.stream_id,
+ &request.request.topic_id,
+ |perm, sid, tid| perm.create_partitions(user_id, sid, tid),
+ )
+ }
+ Operation::DeletePartitions => {
+ let Ok(request) = DeletePartitionsRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.delete_partitions(user_id, sid, tid),
+ )
+ }
+ Operation::TruncatePartition => {
+ let Ok(request) = TruncatePartitionRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.delete_segments(user_id, sid, tid),
+ )
+ }
+
+ // Consumer groups. Join/Leave carry the enriched request types that add
+ // the VSR client id; Create/Delete use the wire types.
+ Operation::CreateConsumerGroup => {
+ let Ok(request) = CreateConsumerGroupRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.create_consumer_group(user_id, sid, tid),
+ )
+ }
+ Operation::DeleteConsumerGroup => {
+ let Ok(request) = DeleteConsumerGroupRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.delete_consumer_group(user_id, sid, tid),
+ )
+ }
+ Operation::JoinConsumerGroup => {
+ let Ok(request) = JoinConsumerGroupRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.join_consumer_group(user_id, sid, tid),
+ )
+ }
+ Operation::LeaveConsumerGroup => {
+ let Ok(request) = LeaveConsumerGroupRequest::decode_from(body) else {
+ return None;
+ };
+ topic_scoped(
+ users,
+ streams,
+ &request.stream_id,
+ &request.topic_id,
+ |perm, sid, tid| perm.leave_consumer_group(user_id, sid, tid),
+ )
+ }
+
+ // Users. Unscoped `manage_users`, except `ChangePassword` which the
+ // legacy execution layer exempts when the target is the acting user.
+ Operation::CreateUser => check(users, |perm| perm.create_user(user_id)),
+ Operation::DeleteUser => check(users, |perm| perm.delete_user(user_id)),
+ Operation::UpdateUser => check(users, |perm| perm.update_user(user_id)),
+ Operation::UpdatePermissions => check(users, |perm| perm.update_permissions(user_id)),
+ Operation::ChangePassword => {
+ let Ok(request) = ChangePasswordRequest::decode_from(body) else {
+ return None;
+ };
+ match users.read(|inner| inner.resolve_user_id(&request.user_id)) {
+ // Self-service password change: allowed without `manage_users`.
+ Some(target_id) if target_id == user_id as usize => None,
+ Some(_) => check(users, |perm| perm.change_password(user_id)),
+ // Unknown target passes through to apply's `UserNotFound`.
+ None => None,
+ }
+ }
+
+ // Not gated (listed exhaustively so a new op cannot silently skip the
+ // gate): VSR-reserved ops (never reach apply as client ops),
+ // server-originated internal ops (user_id 0), self-scoped PAT ops (no
+ // legacy RBAC), the pre-projection wire forms, and every partition op.
+ Operation::Reserved
+ | Operation::Register
+ | Operation::NonReplicated
+ | Operation::Logout
+ | Operation::RemoveConsumerGroupMember
+ | Operation::CompleteConsumerGroupRevocation
+ | Operation::CreateTopic
+ | Operation::CreatePartitions
+ | Operation::DeleteSegments
+ | Operation::CreatePersonalAccessToken
+ | Operation::DeletePersonalAccessToken
+ | Operation::SendMessages
+ | Operation::StoreConsumerOffset
+ | Operation::DeleteConsumerOffset
+ | Operation::StoreConsumerOffset2
+ | Operation::DeleteConsumerOffset2 => None,
+ }
+}
+
+/// Maps a permissioner rule outcome to a gate decision: any `Err` (always
+/// `Unauthorized`) denies with a committed `Unauthorized` reply.
+fn check(
+ users: &Users,
+ rule: impl FnOnce(&Permissioner) -> Result<(), IggyError>,
+) -> Option<ApplyReply> {
+ match users.read(|inner| rule(&inner.permissioner)) {
+ Ok(()) => None,
+ Err(error) => Some(ApplyReply::err(error.as_code())),
+ }
+}
+
+/// Resolves a stream id, then runs a stream-scoped rule. A resolution miss
+/// passes through (returns `None`) so apply surfaces its own `NotFound`.
+fn stream_scoped(
+ users: &Users,
+ streams: &Streams,
+ stream_id: &WireIdentifier,
+ rule: impl FnOnce(&Permissioner, usize) -> Result<(), IggyError>,
+) -> Option<ApplyReply> {
+ let stream_id = streams.read(|inner| inner.resolve_stream_id(stream_id))?;
+ check(users, |perm| rule(perm, stream_id))
+}
+
+/// Resolves a (stream, topic) pair, then runs a topic-scoped rule. A miss on
+/// either passes through so apply surfaces its own `NotFound`.
+fn topic_scoped(
+ users: &Users,
+ streams: &Streams,
+ stream_id: &WireIdentifier,
+ topic_id: &WireIdentifier,
+ rule: impl FnOnce(&Permissioner, usize, usize) -> Result<(), IggyError>,
+) -> Option<ApplyReply> {
+ let (stream_id, topic_id) = streams.read(|inner| {
+ let stream_id = inner.resolve_stream_id(stream_id)?;
+ let topic_id = inner.resolve_topic_id(stream_id, topic_id)?;
+ Some((stream_id, topic_id))
+ })?;
+ check(users, |perm| rule(perm, stream_id, topic_id))
+}
+
+/// The recovery unit tests exercise WAL-replay mechanics with an empty state
+/// list (`MuxStateMachine<()>`), which has no permission state and so needs no
+/// gate. Production always uses the `variadic!(Users, Streams)` machine above.
+#[cfg(test)]
+impl GatedApply for MuxStateMachine<()> {
+ fn gated_update(&self, prepare: Message<PrepareHeader>) -> Result<ApplyReply, IggyError> {
+ self.update(prepare)
+ }
+}
+
+#[cfg(test)]
+#[allow(clippy::cast_possible_truncation)]
+mod tests {
+ use super::*;
+ use iggy_binary_protocol::primitives::partition_assignment::CreatedPartitionAssignment;
+ use iggy_binary_protocol::primitives::permissions::{
+ WireGlobalPermissions, WirePermissions, WireStreamPermissions,
+ };
+ use iggy_binary_protocol::requests::streams::CreateStreamRequest;
+ use iggy_binary_protocol::requests::topics::CreateTopicRequest;
+ use iggy_binary_protocol::requests::users::CreateUserRequest;
+ use iggy_binary_protocol::{Command2, WireEncode, WireName};
+ use iggy_common::UserStatus;
+ use server_common::iobuf::Owned;
+
+ const HEADER_SIZE: usize = size_of::<PrepareHeader>();
+ /// Root is the first user, so it takes slab id 0; the first user created
+ /// after it is id 1.
+ const ROOT: u32 = 0;
+ const ALICE: u32 = 1;
+
+ fn mux() -> MuxStateMachine<variadic!(Users, Streams)> {
+ let users = Users::default();
+ users.ensure_root_user("iggy", "hash");
+ let streams = Streams::default();
+ MuxStateMachine::new(variadic!(users, streams))
+ }
+
+ fn make_prepare(
+ op: u64,
+ operation: Operation,
+ user_id: u32,
+ body: &[u8],
+ ) -> Message<PrepareHeader> {
+ let total = HEADER_SIZE + body.len();
+ let mut buffer = Owned::<4096>::zeroed(total);
+ {
+ let header = bytemuck::checked::from_bytes_mut::<PrepareHeader>(
+ &mut buffer.as_mut_slice()[..HEADER_SIZE],
+ );
+ header.command = Command2::Prepare;
+ header.operation = operation;
+ header.op = op;
+ header.user_id = user_id;
+ header.size = u32::try_from(total).unwrap();
+ }
+ buffer.as_mut_slice()[HEADER_SIZE..].copy_from_slice(body);
+ Message::try_from(buffer).unwrap()
+ }
+
+ fn no_grants() -> WireGlobalPermissions {
+ WireGlobalPermissions {
+ manage_servers: false,
+ read_servers: false,
+ manage_users: false,
+ read_users: false,
+ manage_streams: false,
+ read_streams: false,
+ manage_topics: false,
+ read_topics: false,
+ poll_messages: false,
+ send_messages: false,
+ }
+ }
+
+ fn create_user_body(
+ name: &str,
+ global: WireGlobalPermissions,
+ streams: Vec<WireStreamPermissions>,
+ ) -> bytes::Bytes {
+ CreateUserRequest {
+ username: WireName::new(name).unwrap(),
+ password: "hash".to_string(),
+ status: UserStatus::Active.as_code(),
+ permissions: Some(WirePermissions { global, streams }),
+ }
+ .to_bytes()
+ }
+
+ fn create_stream_body(name: &str) -> bytes::Bytes {
+ CreateStreamRequest {
+ name: WireName::new(name).unwrap(),
+ }
+ .to_bytes()
+ }
+
+ fn create_topic_body(stream_id: u32, name: &str) -> bytes::Bytes {
+ CreateTopicWithAssignmentsRequest {
+ request: CreateTopicRequest {
+ stream_id: WireIdentifier::numeric(stream_id),
+ partitions_count: 1,
+ compression_algorithm: 0,
+ message_expiry: 0,
+ max_topic_size: 0,
+ replication_factor: 1,
+ name: WireName::new(name).unwrap(),
+ },
+ partitions: vec![CreatedPartitionAssignment {
+ partition_id: 0,
+ consensus_group_id: 1,
+ }],
+ }
+ .to_bytes()
+ }
+
+ fn change_password_body(target: &str) -> bytes::Bytes {
+ ChangePasswordRequest {
+ user_id: WireIdentifier::named(target).unwrap(),
+ current_password: String::new(),
+ new_password: "new-hash".to_string(),
+ }
+ .to_bytes()
+ }
+
+ fn stream_manage_topics(stream_id: u32) -> WireStreamPermissions {
+ WireStreamPermissions {
+ stream_id,
+ manage_stream: false,
+ read_stream: false,
+ manage_topics: true,
+ read_topics: false,
+ poll_messages: false,
+ send_messages: false,
+ topics: Vec::new(),
+ }
+ }
+
+ const UNAUTHORIZED: u32 = 41;
+
+ #[test]
+ fn given_ungranted_user_when_create_stream_should_deny_and_leave_state_unchanged() {
+ let mux = mux();
+ assert_eq!(
+ mux.gated_update(make_prepare(
+ 1,
+ Operation::CreateUser,
+ ROOT,
+ &create_user_body("alice", no_grants(), Vec::new()),
+ ))
+ .unwrap()
+ .code,
+ 0,
+ "root may create a user"
+ );
+
+ let reply = mux
+ .gated_update(make_prepare(
+ 2,
+ Operation::CreateStream,
+ ALICE,
+ &create_stream_body("s1"),
+ ))
+ .unwrap();
+
+ assert_eq!(reply.code, UNAUTHORIZED);
+ assert_eq!(
+ mux.streams().read(|inner| inner.items.len()),
+ 0,
+ "denied op must not mutate stream state"
+ );
+ }
+
+ #[test]
+ fn given_manage_streams_grant_when_create_stream_should_allow() {
+ let mux = mux();
+ let grants = WireGlobalPermissions {
+ manage_streams: true,
+ ..no_grants()
+ };
+ mux.gated_update(make_prepare(
+ 1,
+ Operation::CreateUser,
+ ROOT,
+ &create_user_body("alice", grants, Vec::new()),
+ ))
+ .unwrap();
+
+ let reply = mux
+ .gated_update(make_prepare(
+ 2,
+ Operation::CreateStream,
+ ALICE,
+ &create_stream_body("s1"),
+ ))
+ .unwrap();
+
+ assert_eq!(reply.code, 0);
+ assert_eq!(mux.streams().read(|inner| inner.items.len()), 1);
+ }
+
+ #[test]
+ fn given_stream_scoped_topics_grant_when_create_topic_should_allow_only_that_stream() {
+ let mux = mux();
+ // Streams 0 and 1 (StreamsInner slab is separate from the user slab).
+ mux.gated_update(make_prepare(
+ 1,
+ Operation::CreateStream,
+ ROOT,
+ &create_stream_body("s0"),
+ ))
+ .unwrap();
+ mux.gated_update(make_prepare(
+ 2,
+ Operation::CreateStream,
+ ROOT,
+ &create_stream_body("s1"),
+ ))
+ .unwrap();
+ // Alice may manage topics on stream 0 only.
+ mux.gated_update(make_prepare(
+ 3,
+ Operation::CreateUser,
+ ROOT,
+ &create_user_body("alice", no_grants(), vec![stream_manage_topics(0)]),
+ ))
+ .unwrap();
+
+ let allowed = mux
+ .gated_update(make_prepare(
+ 4,
+ Operation::CreateTopicWithAssignments,
+ ALICE,
+ &create_topic_body(0, "t"),
+ ))
+ .unwrap();
+ assert_eq!(allowed.code, 0, "grant covers stream 0");
+
+ let denied = mux
+ .gated_update(make_prepare(
+ 5,
+ Operation::CreateTopicWithAssignments,
+ ALICE,
+ &create_topic_body(1, "t"),
+ ))
+ .unwrap();
+ assert_eq!(denied.code, UNAUTHORIZED, "grant does not cover stream 1");
+ }
+
+ #[test]
+ fn given_change_password_when_target_is_self_should_allow_else_deny() {
+ let mux = mux();
+ mux.gated_update(make_prepare(
+ 1,
+ Operation::CreateUser,
+ ROOT,
+ &create_user_body("alice", no_grants(), Vec::new()),
+ ))
+ .unwrap();
+ mux.gated_update(make_prepare(
+ 2,
+ Operation::CreateUser,
+ ROOT,
+ &create_user_body("bob", no_grants(), Vec::new()),
+ ))
+ .unwrap();
+
+ let own = mux
+ .gated_update(make_prepare(
+ 3,
+ Operation::ChangePassword,
+ ALICE,
+ &change_password_body("alice"),
+ ))
+ .unwrap();
+ assert_eq!(
+ own.code, 0,
+ "self password change is exempt from manage_users"
+ );
+
+ let other = mux
+ .gated_update(make_prepare(
+ 4,
+ Operation::ChangePassword,
+ ALICE,
+ &change_password_body("bob"),
+ ))
+ .unwrap();
+ assert_eq!(
+ other.code, UNAUTHORIZED,
+ "changing another user needs manage_users"
+ );
+ }
+
+ #[test]
+ fn given_internal_revocation_op_when_authorized_should_skip_the_gate() {
+ let mux = mux();
+ let prepare = make_prepare(1, Operation::CompleteConsumerGroupRevocation, ROOT, &[]);
+ assert!(
+ authorize(&prepare, mux.users(), mux.streams()).is_none(),
+ "server-originated revocation must not be gated"
+ );
+ }
+
+ #[test]
+ fn given_non_root_user_when_create_pat_should_skip_the_gate() {
+ let mux = mux();
+ mux.gated_update(make_prepare(
+ 1,
+ Operation::CreateUser,
+ ROOT,
+ &create_user_body("alice", no_grants(), Vec::new()),
+ ))
+ .unwrap();
+
+ let prepare = make_prepare(2, Operation::CreatePersonalAccessToken, ALICE, &[]);
+ assert!(
+ authorize(&prepare, mux.users(), mux.streams()).is_none(),
+ "PAT ops are self-scoped (parity: no legacy RBAC)"
+ );
+ }
+
+ // The linchpin: a sequence containing a denied op must produce byte-identical
+ // codes and final state whether applied once (primary on_ack) or replayed
+ // fresh (backup commit_journal / recover). Identity rides the header, so no
+ // session table is consulted.
+ #[test]
+ fn given_sequence_with_denial_when_replayed_fresh_should_be_state_identical() {
+ fn sequence() -> Vec<Message<PrepareHeader>> {
+ vec![
+ make_prepare(
+ 1,
+ Operation::CreateUser,
+ ROOT,
+ &create_user_body("alice", no_grants(), Vec::new()),
+ ),
+ make_prepare(
+ 2,
+ Operation::CreateStream,
+ ROOT,
+ &create_stream_body("s-root"),
+ ),
+ // Alice lacks manage_streams: denied on both passes.
+ make_prepare(
+ 3,
+ Operation::CreateStream,
+ ALICE,
+ &create_stream_body("s-alice"),
+ ),
+ ]
+ }
+
+ let primary = mux();
+ let primary_codes: Vec<u32> = sequence()
+ .into_iter()
+ .map(|prepare| primary.gated_update(prepare).unwrap().code)
+ .collect();
+
+ let replay = mux();
+ let replay_codes: Vec<u32> = sequence()
+ .into_iter()
+ .map(|prepare| replay.gated_update(prepare).unwrap().code)
+ .collect();
+
+ assert_eq!(primary_codes, replay_codes, "replay must be deterministic");
+ assert_eq!(primary_codes[2], UNAUTHORIZED, "alice's create is denied");
+ assert_eq!(primary.streams().read(|inner| inner.items.len()), 1);
+ assert_eq!(replay.streams().read(|inner| inner.items.len()), 1);
+ }
+}
diff --git a/core/metadata/src/stm/mod.rs b/core/metadata/src/stm/mod.rs
index f86c2d2e3..5fc0c10d1 100644
--- a/core/metadata/src/stm/mod.rs
+++ b/core/metadata/src/stm/mod.rs
@@ -15,6 +15,7 @@
// specific language governing permissions and limitations
// under the License.
+pub mod authz;
pub mod consumer_group;
pub mod mux;
pub mod result;
diff --git a/core/metadata/src/stm/result.rs b/core/metadata/src/stm/result.rs
index 7874729d8..9a3eef484 100644
--- a/core/metadata/src/stm/result.rs
+++ b/core/metadata/src/stm/result.rs
@@ -192,9 +192,15 @@ result_enum!(UpdateUserResult {
UserNotFound = 20,
UsernameAlreadyExists = 46,
});
-result_enum!(DeleteUserResult { UserNotFound = 20 });
+result_enum!(DeleteUserResult {
+ UserNotFound = 20,
+ CannotDeleteUser = 48,
+});
result_enum!(ChangePasswordResult { UserNotFound = 20 });
-result_enum!(UpdatePermissionsResult { UserNotFound = 20 });
+result_enum!(UpdatePermissionsResult {
+ UserNotFound = 20,
+ CannotChangePermissions = 49,
+});
// Personal access tokens.
result_enum!(CreatePersonalAccessTokenResult {
@@ -207,6 +213,12 @@ result_enum!(DeletePersonalAccessTokenResult { NotFound = 20 });
result_enum!(CreateConsumerGroupResult { NameAlreadyExists = 5004 });
result_enum!(DeleteConsumerGroupResult { NotFound = 5000 });
+/// `IggyError::Unauthorized`. Any control-plane op can commit as an in-apply
+/// authorization no-op, so this code is valid for every op regardless of its
+/// own result enum. Pinned to the wire discriminant in
+/// [`tests::given_result_discriminants_then_match_iggy_error_codes`].
+const UNAUTHORIZED_CODE: u32 = 41;
+
/// True if `code` is one this op's result enum declares (`Ok = 0` included).
///
/// The state machine only commits codes from the op's own enum, so an
@@ -215,6 +227,11 @@ result_enum!(DeleteConsumerGroupResult { NotFound = 5000 });
/// section (partition plane) return `true`.
#[must_use]
pub const fn result_code_recognized(operation: Operation, code: u32) -> bool {
+ // The in-apply RBAC gate can deny any control-plane op, committing an
+ // `Unauthorized` no-op, so accept it globally rather than per-op enum.
+ if code == UNAUTHORIZED_CODE {
+ return true;
+ }
match operation {
Operation::CreateStream => CreateStreamResult::from_u32(code).is_some(),
Operation::UpdateStream => UpdateStreamResult::from_u32(code).is_some(),
@@ -325,6 +342,20 @@ mod tests {
));
// Partition-plane ops carry no result section, so any code is accepted.
assert!(result_code_recognized(Operation::SendMessages, 12345));
+ // `Unauthorized` commits for any control-plane op via the in-apply gate,
+ // even though no per-op result enum declares it.
+ assert!(result_code_recognized(
+ Operation::CreateStream,
+ UNAUTHORIZED_CODE
+ ));
+ assert!(result_code_recognized(
+ Operation::CreateUser,
+ UNAUTHORIZED_CODE
+ ));
+ assert!(result_code_recognized(
+ Operation::CreateConsumerGroup,
+ UNAUTHORIZED_CODE
+ ));
}
// Pins every result-enum discriminant to the `IggyError` variant its doc
@@ -451,6 +482,18 @@ mod tests {
resource_not_found
);
+ // CannotDeleteUser (48) - root deletion is rejected in-apply.
+ assert_eq!(
+ u32::from(DeleteUserResult::CannotDeleteUser),
+ IggyError::CannotDeleteUser(0).as_code(),
+ );
+
+ // CannotChangePermissions (49) - root permissions are immutable in-apply.
+ assert_eq!(
+ u32::from(UpdatePermissionsResult::CannotChangePermissions),
+ IggyError::CannotChangePermissions(0).as_code(),
+ );
+
// PersonalAccessTokenAlreadyExists (51).
assert_eq!(
u32::from(CreatePersonalAccessTokenResult::AlreadyExists),
@@ -472,5 +515,8 @@ mod tests {
u32::from(DeleteConsumerGroupResult::NotFound),
IggyError::ConsumerGroupIdNotFound(id(), id()).as_code(),
);
+
+ // Unauthorized (41) - the global in-apply RBAC denial code.
+ assert_eq!(UNAUTHORIZED_CODE, IggyError::Unauthorized.as_code());
}
}
diff --git a/core/metadata/src/stm/stream.rs b/core/metadata/src/stm/stream.rs
index 765e643df..41fb0cce6 100644
--- a/core/metadata/src/stm/stream.rs
+++ b/core/metadata/src/stm/stream.rs
@@ -498,7 +498,7 @@ impl StreamsInner {
self.pending_revocations_count = count;
}
- fn resolve_stream_id(&self, identifier: &WireIdentifier) -> Option<usize> {
+ pub(crate) fn resolve_stream_id(&self, identifier: &WireIdentifier) -> Option<usize> {
match identifier {
WireIdentifier::Numeric(id) => {
let id = *id as usize;
@@ -512,7 +512,11 @@ impl StreamsInner {
}
}
- fn resolve_topic_id(&self, stream_id: usize, identifier: &WireIdentifier) -> Option<usize> {
+ pub(crate) fn resolve_topic_id(
+ &self,
+ stream_id: usize,
+ identifier: &WireIdentifier,
+ ) -> Option<usize> {
let stream = self.items.get(stream_id)?;
match identifier {
WireIdentifier::Numeric(id) => {
diff --git a/core/metadata/src/stm/user.rs b/core/metadata/src/stm/user.rs
index 51e51e60c..a13a50c1b 100644
--- a/core/metadata/src/stm/user.rs
+++ b/core/metadata/src/stm/user.rs
@@ -35,6 +35,7 @@ use iggy_binary_protocol::requests::users::{
use iggy_binary_protocol::responses::users::get_user::UserDetailsResponse;
use iggy_binary_protocol::responses::users::user_response::UserResponse;
use iggy_binary_protocol::{WireIdentifier, WireName};
+use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
use iggy_common::{
GlobalPermissions, IggyExpiry, IggyTimestamp, Permissions, PersonalAccessToken,
StreamPermissions, UserId, UserStatus,
@@ -124,7 +125,7 @@ collect_handlers! {
}
impl UsersInner {
- fn resolve_user_id(&self, identifier: &WireIdentifier) -> Option<usize> {
+ pub(crate) fn resolve_user_id(&self, identifier: &WireIdentifier) -> Option<usize> {
match identifier {
WireIdentifier::Numeric(id) => {
let id = *id as usize;
@@ -429,6 +430,15 @@ impl StateHandler for DeleteUserRequest {
return ApplyReply::err(DeleteUserResult::UserNotFound);
};
+ // Root (slab id 0) is undeletable, matching legacy `CannotDeleteUser`.
+ // The rejection is deterministic, so every replica keeps slab id 0 as
+ // the root user and the in-apply RBAC gate's `user_id == 0` root
+ // short-circuit can never be inherited by a recreated user (the slab
+ // reuses freed keys, so a deleted root's slot would be reclaimed).
+ if user_id == DEFAULT_ROOT_USER_ID as usize {
+ return ApplyReply::err(DeleteUserResult::CannotDeleteUser);
+ }
+
if let Some(user) = state.items.get(user_id) {
let username = user.username.clone();
state.items.remove(user_id);
@@ -475,6 +485,14 @@ impl StateHandler for UpdatePermissionsRequest {
return ApplyReply::err(UpdatePermissionsResult::UserNotFound);
};
+ // Root (slab id 0) permissions are immutable, matching legacy
+ // `CannotChangePermissions`. Rejected before any mutation, so the op is a
+ // committed no-op: root keeps its full grants and the permissioner index
+ // is untouched.
+ if user_id == DEFAULT_ROOT_USER_ID as usize {
+ return ApplyReply::err(UpdatePermissionsResult::CannotChangePermissions);
+ }
+
if let Some(user) = state.items.get_mut(user_id) {
user.permissions = self
.permissions
@@ -964,6 +982,98 @@ mod tests {
assert_eq!(apply.code, u32::from(DeleteUserResult::UserNotFound));
}
+ #[test]
+ fn given_root_user_when_apply_delete_user_should_reject_and_keep_root() {
+ // Production seeds root at slab id 0 (`ensure_root_user`), so the first
+ // slab slot is the root user. Deleting it must be rejected in-apply, or
+ // a later `CreateUser` would reclaim slot 0 and inherit the RBAC gate's
+ // `user_id == 0` root short-circuit.
+ let mut users = UsersInner::new();
+ let root_id = create_user_with(&mut users, "iggy", None);
+ assert_eq!(root_id, DEFAULT_ROOT_USER_ID, "root takes slab id 0");
+
+ let delete_root = DeleteUserRequest {
+ user_id: WireIdentifier::numeric(root_id),
+ };
+ let reply = StateHandler::apply(&delete_root, &mut users, IggyTimestamp::now());
+ assert_eq!(reply.code, u32::from(DeleteUserResult::CannotDeleteUser));
+ assert!(
+ users.items.contains(root_id as usize),
+ "root must survive a delete attempt"
+ );
+
+ // A non-root user (slab id 1) stays deletable: the guard is scoped to
+ // root, so there is no over-block regression.
+ let alice_id = create_user_with(&mut users, "alice", None);
+ assert_ne!(alice_id, DEFAULT_ROOT_USER_ID);
+ let delete_alice = DeleteUserRequest {
+ user_id: WireIdentifier::numeric(alice_id),
+ };
+ let reply = StateHandler::apply(&delete_alice, &mut users, IggyTimestamp::now());
+ assert_eq!(reply.code, 0, "non-root deletion still succeeds");
+ assert!(!users.items.contains(alice_id as usize));
+ }
+
+ #[test]
+ fn given_root_user_when_apply_update_permissions_should_reject_and_keep_grants() {
+ // Root occupies slab id 0 (production `ensure_root_user`). Its
+ // permissions must be immutable in-apply, or a delegated manage_users
+ // admin could tamper with root's grants.
+ let mut users = UsersInner::new();
+ let mut root_global = empty_global();
+ root_global.manage_servers = true;
+ root_global.poll_messages = true;
+ let root_id = create_user_with(
+ &mut users,
+ "iggy",
+ Some(WirePermissions {
+ global: root_global,
+ streams: Vec::new(),
+ }),
+ );
+ assert_eq!(root_id, DEFAULT_ROOT_USER_ID, "root takes slab id 0");
+ assert!(users.permissioner.poll_messages(root_id, 1, 1).is_ok());
+
+ // Attempt to strip root's grants: rejected, nothing mutates.
+ let strip = UpdatePermissionsRequest {
+ user_id: WireIdentifier::numeric(root_id),
+ permissions: None,
+ };
+ let reply = StateHandler::apply(&strip, &mut users, IggyTimestamp::now());
+ assert_eq!(
+ reply.code,
+ u32::from(UpdatePermissionsResult::CannotChangePermissions)
+ );
+ assert!(
+ users
+ .items
+ .get(root_id as usize)
+ .and_then(|user| user.permissions.as_ref())
+ .is_some(),
+ "root permissions must survive"
+ );
+ assert!(
+ users.permissioner.poll_messages(root_id, 1, 1).is_ok(),
+ "root gate access unchanged"
+ );
+
+ // A non-root user's permissions can still be updated: no over-block.
+ let alice_id = create_user_with(&mut users, "alice", None);
+ assert_ne!(alice_id, DEFAULT_ROOT_USER_ID);
+ let mut alice_global = empty_global();
+ alice_global.poll_messages = true;
+ let grant = UpdatePermissionsRequest {
+ user_id: WireIdentifier::numeric(alice_id),
+ permissions: Some(WirePermissions {
+ global: alice_global,
+ streams: Vec::new(),
+ }),
+ };
+ let reply = StateHandler::apply(&grant, &mut users, IggyTimestamp::now());
+ assert_eq!(reply.code, 0, "non-root permission update still succeeds");
+ assert!(users.permissioner.poll_messages(alice_id, 1, 1).is_ok());
+ }
+
fn empty_global() -> WireGlobalPermissions {
WireGlobalPermissions {
manage_servers: false,
@@ -1023,6 +1133,9 @@ mod tests {
#[test]
fn given_poll_grant_when_apply_update_permissions_removes_it_should_return_unauthorized() {
let mut users = UsersInner::new();
+ // Occupy slab id 0 with the (immutable) root user so the poller under
+ // test is a non-root user; root permission changes are rejected in-apply.
+ create_user_with(&mut users, "iggy", None);
let mut global = empty_global();
global.poll_messages = true;
let uid = create_user_with(
@@ -1057,6 +1170,9 @@ mod tests {
#[test]
fn given_permissioned_user_when_apply_delete_user_should_clear_all_indexes() {
let mut users = UsersInner::new();
+ // Occupy slab id 0 with the (undeletable) root user so the victim under
+ // test is a non-root user; root deletion is rejected in-apply.
+ create_user_with(&mut users, "iggy", None);
let mut global = empty_global();
global.poll_messages = true;
global.send_messages = true;
diff --git a/core/simulator/src/replica.rs b/core/simulator/src/replica.rs
index 7e44fefee..aea179ac1 100644
--- a/core/simulator/src/replica.rs
+++ b/core/simulator/src/replica.rs
@@ -47,6 +47,10 @@ pub type Replica = shard::IggyShard<
pub fn new_replica(id: u8, name: String, bus: &Arc<SimOutbox>, replica_count: u8) -> Replica {
let users: Users = UsersInner::new().into();
+ // Seed the root user at slab id 0, matching production bootstrap
+ // (`ensure_root_user`). Root is undeletable in-apply, so keeping it in slot
+ // 0 leaves the workload's users at slab id 1+ and out of any delete attempt.
+ users.ensure_root_user("iggy", "hash");
let streams: Streams = StreamsInner::new().into();
let mux = SimMuxStateMachine::new(variadic!(users, streams));
diff --git a/core/simulator/src/workload/ops/delete_user.rs b/core/simulator/src/workload/ops/delete_user.rs
index 225e025d9..964b8eee8 100644
--- a/core/simulator/src/workload/ops/delete_user.rs
+++ b/core/simulator/src/workload/ops/delete_user.rs
@@ -47,6 +47,8 @@ pub fn sample(
Outcome::UserNotFound => Some(Input {
user: shadow.fabricate_absent_name("user"),
}),
+ // Not a targeted outcome (absent from `OUTCOMES`); never sampled.
+ Outcome::CannotDeleteUser => None,
}
}
@@ -70,6 +72,9 @@ pub fn predicted_effect(input: &Input, outcome: Outcome) -> Effect {
Outcome::Ok => Effect::RemoveUser {
name: input.user.clone(),
},
- Outcome::UserNotFound => Effect::None,
+ // `CannotDeleteUser` is never targeted (the workload's users live at
+ // slab id 1+, never root), but effects dispatch on the committed
+ // outcome, so the arm is required: a rejected delete removes nothing.
+ Outcome::UserNotFound | Outcome::CannotDeleteUser => Effect::None,
}
}
diff --git a/core/simulator/src/workload/ops/update_permissions.rs b/core/simulator/src/workload/ops/update_permissions.rs
index e68e43e53..75169a382 100644
--- a/core/simulator/src/workload/ops/update_permissions.rs
+++ b/core/simulator/src/workload/ops/update_permissions.rs
@@ -47,6 +47,8 @@ pub fn sample(
Outcome::UserNotFound => Some(Input {
user: shadow.fabricate_absent_name("user"),
}),
+ // Not a targeted outcome (absent from `OUTCOMES`); never sampled.
+ Outcome::CannotChangePermissions => None,
}
}
(iggy) 27/43: fix(server-ng): verify and strip current_password on change
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 4816ae73fbc2718ae02334e38aacc9c8fc43e546
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 18:19:59 2026 +0200
fix(server-ng): verify and strip current_password on change
ChangePassword neither verified current_password nor removed it
before replication: any session holding the change_password
permission could rotate a password blind, and both supplied
passwords rode the replicated prepare into the WAL in plaintext on
every replica. Legacy verifies against the stored hash and strips
the field before writing state.
Restore that shape on the shard-0 pre-submit rewrite shared by all
transports: the rewrite now always hashes new_password and empties
current_password, and verifies current_password against the target's
stored hash whenever the target resolves locally, rejecting with
InvalidCredentials on mismatch. An unresolved target still commits a
stripped no-op that apply answers with UserNotFound; the narrow
window where a target materialises between rewrite and apply skips
only the verify and stays permission-gated. Denials are typed on
both transports: HTTP maps to 400, TCP answers through the reply
status field, which also replaces the silent drop on malformed
bodies that previously wedged the connection until the client
timeout.
---
core/integration/tests/server/http_rbac.rs | 126 +++++++++++++++++++++++++
core/server-ng/src/dispatch.rs | 26 +++++-
core/server-ng/src/http.rs | 21 +++--
core/server-ng/src/users.rs | 142 +++++++++++++++++++++++++++--
4 files changed, 298 insertions(+), 17 deletions(-)
diff --git a/core/integration/tests/server/http_rbac.rs b/core/integration/tests/server/http_rbac.rs
index 7fcd89f89..d8e5df831 100644
--- a/core/integration/tests/server/http_rbac.rs
+++ b/core/integration/tests/server/http_rbac.rs
@@ -55,12 +55,17 @@ const CONSUMER_ID: u32 = 1;
// `IggyError` discriminants (== the numeric `id` in the JSON error body).
const UNAUTHORIZED_ID: u64 = 41;
+const INVALID_CREDENTIALS_ID: u64 = 42;
const CANNOT_DELETE_USER_ID: u64 = 48;
const CANNOT_CHANGE_PERMISSIONS_ID: u64 = 49;
+// The apply's `ChangePasswordResult::UserNotFound` maps to `ResourceNotFound`.
+const RESOURCE_NOT_FOUND_ID: u64 = 20;
// Snake-case `IggyError` names (== the `code` string in the JSON error body).
const UNAUTHORIZED_CODE: &str = "unauthorized";
+const INVALID_CREDENTIALS_CODE: &str = "invalid_credentials";
const CANNOT_DELETE_USER_CODE: &str = "cannot_delete_user";
const CANNOT_CHANGE_PERMISSIONS_CODE: &str = "cannot_change_permissions";
+const RESOURCE_NOT_FOUND_CODE: &str = "resource_not_found";
/// Root is the first user (slab id 0) and is undeletable / permission-locked.
const ROOT_USER_PATH: &str = "0";
@@ -270,6 +275,26 @@ impl Client {
.await
}
+ /// Change a user's password via `PUT /users/{user_id}/password`.
+ async fn change_password(&self, user_id: &str, current: &str, new: &str) -> Response {
+ self.put_json(
+ &format!("/users/{user_id}/password"),
+ &json!({ "current_password": current, "new_password": new }),
+ )
+ .await
+ }
+
+ /// Anonymous login attempt returning the raw response, for exercising a
+ /// rejected login (the asserting [`Client::login`] cannot express failure).
+ async fn login_attempt(&self, username: &str, password: &str) -> Response {
+ self.http
+ .post(self.url("/users/login"))
+ .json(&json!({ "username": username, "password": password }))
+ .send()
+ .await
+ .expect("login request")
+ }
+
async fn produce(
&self,
stream: &str,
@@ -914,3 +939,104 @@ async fn given_non_root_tcp_session_when_polling_denied_should_surface_typed_una
"non-root TCP poll must surface Err(Unauthorized) via the reply status peek, got {result:?}"
);
}
+
+/// The current-password check gates a self-service rotation: a wrong current
+/// password is rejected pre-consensus as 400 `InvalidCredentials` (never
+/// reaching the replicated apply), and only the correct one rotates the
+/// credential so the old password stops working and the new one starts.
+#[iggy_harness]
+async fn given_own_password_when_current_wrong_should_reject_and_correct_should_rotate(
+ harness: &TestHarness,
+) {
+ let root = Client::login_root(harness).await;
+ root.create_user("carol", "carol-pass-1", no_permissions())
+ .await;
+ let carol = root.login("carol", "carol-pass-1").await;
+
+ let wrong = carol
+ .change_password("carol", "not-my-password", "carol-pass-2")
+ .await;
+ assert_eq!(
+ wrong.status(),
+ StatusCode::BAD_REQUEST,
+ "wrong current password must be 400"
+ );
+ assert_error_body(
+ wrong,
+ INVALID_CREDENTIALS_ID,
+ INVALID_CREDENTIALS_CODE,
+ "carol change-password wrong current",
+ )
+ .await;
+
+ let correct = carol
+ .change_password("carol", "carol-pass-1", "carol-pass-2")
+ .await;
+ assert_eq!(
+ correct.status(),
+ StatusCode::NO_CONTENT,
+ "correct current password must be 204"
+ );
+
+ assert!(
+ !carol
+ .login_attempt("carol", "carol-pass-1")
+ .await
+ .status()
+ .is_success(),
+ "old password must stop working after a successful rotation"
+ );
+ assert!(
+ carol
+ .login_attempt("carol", "carol-pass-2")
+ .await
+ .status()
+ .is_success(),
+ "new password must work after a successful rotation"
+ );
+}
+
+/// Admin-initiated change (target != caller): the current password is checked
+/// against the TARGET's stored hash, so a wrong one is 400 `InvalidCredentials`;
+/// a target that does not resolve is left to the replicated apply, which
+/// answers 404 `UserNotFound` (the plaintext is still stripped + hashed first).
+#[iggy_harness]
+async fn given_admin_change_password_when_target_current_wrong_or_missing_should_reject(
+ harness: &TestHarness,
+) {
+ let root = Client::login_root(harness).await;
+ root.create_user("dave", "dave-pass", no_permissions())
+ .await;
+
+ let wrong = root
+ .change_password("dave", "not-daves-password", "new-dave-pass")
+ .await;
+ assert_eq!(
+ wrong.status(),
+ StatusCode::BAD_REQUEST,
+ "admin change with the wrong target current password must be 400"
+ );
+ assert_error_body(
+ wrong,
+ INVALID_CREDENTIALS_ID,
+ INVALID_CREDENTIALS_CODE,
+ "admin change dave wrong current",
+ )
+ .await;
+
+ let missing = root
+ .change_password("ghost-does-not-exist", "whatever", "whatever-else")
+ .await;
+ assert_eq!(
+ missing.status(),
+ StatusCode::NOT_FOUND,
+ "change for an unknown target must be 404 from the apply"
+ );
+ assert_error_body(
+ missing,
+ RESOURCE_NOT_FOUND_ID,
+ RESOURCE_NOT_FOUND_CODE,
+ "admin change missing target",
+ )
+ .await;
+}
diff --git a/core/server-ng/src/dispatch.rs b/core/server-ng/src/dispatch.rs
index e27f27ceb..9f628a710 100644
--- a/core/server-ng/src/dispatch.rs
+++ b/core/server-ng/src/dispatch.rs
@@ -598,17 +598,35 @@ async fn handle_client_request(
return;
}
};
- // Hash raw passwords on the primary before replication (CreateUser /
- // ChangePassword); see `crate::users`. Replicas store the hash directly.
- let request = match maybe_rewrite_user_password_request(request) {
+ // Hash raw passwords and verify ChangePassword's current password on the
+ // primary before replication (CreateUser / ChangePassword); see
+ // `crate::users`. Replicas store the hash directly.
+ let request = match maybe_rewrite_user_password_request(shard, request) {
Ok(rewritten) => rewritten,
Err(error) => {
+ // Deny fast with the typed code (InvalidCredentials on a failed
+ // current-password check, InvalidCommand on a malformed body): a
+ // silent drop would wedge every later request on the connection
+ // until the socket read timeout.
warn!(
transport_client_id,
error = %error,
operation = ?header.operation,
- "dropping user request with invalid password payload"
+ "denying user password request"
);
+ let commit = current_metadata_commit(shard);
+ let reply = build_deny_reply(&header, transport_client_id, 0, commit, error.as_code());
+ if let Err(send_error) = shard
+ .bus
+ .send_to_client(transport_client_id, reply.into_generic().into_frozen())
+ .await
+ {
+ warn!(
+ transport_client_id,
+ error = %send_error,
+ "failed to send password-deny reply"
+ );
+ }
return;
}
};
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 7c8c45f3f..53931ca46 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -2126,8 +2126,11 @@ async fn delete_user(
/// `PUT /users/{user_id}/password`: change a user's password. Returns 204.
///
-/// Like [`create_user`], the new password rides the JSON body in plaintext and
-/// is hashed by [`submit_write`] on shard 0 before replication.
+/// Both passwords ride the JSON body in plaintext. On shard 0, before the op
+/// enters consensus, [`maybe_rewrite_user_password_request`] hashes the new
+/// password and strips the current one (so neither plaintext is ever
+/// replicated), and verifies `current_password` against the target's stored
+/// hash -- a wrong one rejects as 400 `InvalidCredentials`.
async fn change_password(
State(state): State<HttpState>,
identity: Authenticated,
@@ -2311,10 +2314,13 @@ async fn submit_committed(
///
/// Both shard-0 request rewrites run here before consensus, mirroring the TCP
/// dispatch path: the PAT rewrite mints a raw token and replicates only its hash
-/// (`CreatePersonalAccessToken`), and the user-password rewrite hashes plaintext
-/// (`CreateUser` / `ChangePassword`). Both are no-ops for every other operation,
-/// so plaintext secrets never enter consensus on any write. On either rewrite's
-/// decode failure the gate is released without advancing, leaving the id free.
+/// (`CreatePersonalAccessToken`), and the user-password rewrite hashes the new
+/// password and, for `ChangePassword`, strips the current one and verifies it
+/// against the stored hash (`CreateUser` / `ChangePassword`). Both are no-ops
+/// for every other operation, so plaintext secrets never enter consensus on any
+/// write. On either rewrite's rejection (a malformed body, or a failed
+/// current-password check) the gate is released without advancing, leaving the
+/// id free.
async fn submit_gated(
shard: &Rc<ServerNgShard>,
session: &HttpSession,
@@ -2331,7 +2337,8 @@ async fn submit_gated(
);
let (message, raw_token) =
rewrite_pat_request_for_user(session.user_id, message).map_err(WriteError::Rejected)?;
- let message = maybe_rewrite_user_password_request(message).map_err(WriteError::Rejected)?;
+ let message =
+ maybe_rewrite_user_password_request(shard, message).map_err(WriteError::Rejected)?;
let request_header = *message.header();
let reply = submit_client_request_on_owner(shard, message).await;
let Some(reply) = reply else {
diff --git a/core/server-ng/src/users.rs b/core/server-ng/src/users.rs
index b65ed357a..b8d9af4ae 100644
--- a/core/server-ng/src/users.rs
+++ b/core/server-ng/src/users.rs
@@ -25,19 +25,36 @@
//! replica and diverge state (and a raw password would never parse,
//! panicking `verify_password`). So the primary hashes once here and
//! replicates the hash, mirroring the PAT mint in [`crate::pat`].
+//!
+//! `ChangePassword` also verifies the caller-supplied `current_password`
+//! against the target's stored hash before the op replicates (the legacy
+//! server does this for self- and admin-initiated changes alike), and it
+//! always empties that field and hashes the new password, so no plaintext
+//! credential ever rides the replicated prepare/WAL.
+use crate::bootstrap::ServerNgShard;
use crate::wire::{request_body, rewrite_request_body};
+use bytes::Bytes;
+use consensus::MetadataHandle;
use iggy_binary_protocol::codec::{WireDecode, WireEncode};
use iggy_binary_protocol::requests::users::{ChangePasswordRequest, CreateUserRequest};
-use iggy_binary_protocol::{Operation, RequestHeader};
+use iggy_binary_protocol::{Operation, RequestHeader, WireIdentifier};
use iggy_common::IggyError;
+use metadata::impls::metadata::StreamsFrontend;
use server_common::{Message, crypto};
+use std::rc::Rc;
+use std::sync::Arc;
-/// Replace a raw wire password with its Argon2 hash before replication.
+/// Rewrite a raw wire password request before replication.
///
-/// Only touches `CreateUser` and `ChangePassword`; every other operation
-/// passes through unchanged.
+/// `CreateUser` hashes the new password. `ChangePassword` hashes the new
+/// password and strips the current one unconditionally (so neither plaintext
+/// enters consensus), and additionally verifies the supplied `current_password`
+/// against the target's stored hash when the target resolves. Every other
+/// operation passes through unchanged. Returns [`IggyError::InvalidCredentials`]
+/// when the current password does not match, matching the legacy server.
pub(crate) fn maybe_rewrite_user_password_request(
+ shard: &Rc<ServerNgShard>,
request: Message<RequestHeader>,
) -> Result<Message<RequestHeader>, IggyError> {
let operation = request.header().operation;
@@ -52,11 +69,124 @@ pub(crate) fn maybe_rewrite_user_password_request(
Operation::ChangePassword => {
let mut wire =
ChangePasswordRequest::decode_from(body).map_err(|_| IggyError::InvalidCommand)?;
- wire.new_password = crypto::hash_password(&wire.new_password);
- wire.to_bytes()
+ let stored_hash = resolve_password_hash(shard, &wire.user_id);
+ verify_and_rewrite_change_password(&mut wire, stored_hash.as_deref())?
}
_ => return Ok(request),
};
rewrite_request_body(&request, &rewritten)
}
+
+/// Hash the new password and strip the current one, always, then return the
+/// re-encoded body. When the target resolves (`stored_hash` is `Some`) the
+/// supplied `current_password` is verified against it first, returning
+/// [`IggyError::InvalidCredentials`] on mismatch (matching the legacy server).
+///
+/// The hash + strip are unconditional so no plaintext credential ever enters
+/// consensus, even for an unresolved target (which the replicated apply rejects
+/// with `UserNotFound`). ACCEPTED RESIDUAL: if the target is created in the
+/// narrow window between this rewrite and the apply, the current-password check
+/// is skipped for that one op; RBAC still gates cross-user changes.
+fn verify_and_rewrite_change_password(
+ wire: &mut ChangePasswordRequest,
+ stored_hash: Option<&str>,
+) -> Result<Bytes, IggyError> {
+ if let Some(stored_hash) = stored_hash
+ && !crypto::verify_password(&wire.current_password, stored_hash)
+ {
+ return Err(IggyError::InvalidCredentials);
+ }
+ wire.current_password = String::new();
+ wire.new_password = crypto::hash_password(&wire.new_password);
+ Ok(wire.to_bytes())
+}
+
+/// Stored Argon2 hash of the user named by `user_id`, read from the local
+/// metadata STM (the same replica login verifies against). `None` when the
+/// user does not resolve.
+fn resolve_password_hash(shard: &Rc<ServerNgShard>, user_id: &WireIdentifier) -> Option<Arc<str>> {
+ shard.plane.metadata().mux_stm.users().read(|users| {
+ let resolved = match user_id {
+ WireIdentifier::Numeric(id) => {
+ let id = *id as usize;
+ users.items.contains(id).then_some(id)
+ }
+ WireIdentifier::String(name) => users.index.get(name.as_str()).map(|&id| id as usize),
+ };
+ resolved
+ .and_then(|id| users.items.get(id))
+ .map(|user| Arc::clone(&user.password_hash))
+ })
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ const CURRENT_PASSWORD: &str = "current-password";
+ const NEW_PASSWORD: &str = "brand-new-password";
+
+ fn change_password_request() -> ChangePasswordRequest {
+ ChangePasswordRequest {
+ user_id: WireIdentifier::numeric(7),
+ current_password: CURRENT_PASSWORD.to_owned(),
+ new_password: NEW_PASSWORD.to_owned(),
+ }
+ }
+
+ #[test]
+ fn given_wrong_current_password_when_rewrite_should_reject() {
+ let stored_hash = crypto::hash_password("some-other-password");
+ let mut wire = change_password_request();
+
+ let result = verify_and_rewrite_change_password(&mut wire, Some(&stored_hash));
+
+ assert!(matches!(result, Err(IggyError::InvalidCredentials)));
+ }
+
+ #[test]
+ fn given_correct_current_password_when_rewrite_should_strip_and_hash() {
+ let stored_hash = crypto::hash_password(CURRENT_PASSWORD);
+ let mut wire = change_password_request();
+
+ let body = verify_and_rewrite_change_password(&mut wire, Some(&stored_hash))
+ .expect("verification succeeds and re-encodes a body");
+
+ let decoded =
+ ChangePasswordRequest::decode_from(&body).expect("re-encoded body round-trips");
+ assert!(
+ decoded.current_password.is_empty(),
+ "current password must be stripped before replication"
+ );
+ assert_ne!(
+ decoded.new_password, NEW_PASSWORD,
+ "new password must be hashed, not replicated raw"
+ );
+ assert!(
+ crypto::verify_password(NEW_PASSWORD, &decoded.new_password),
+ "hashed new password must verify against the raw input"
+ );
+ }
+
+ #[test]
+ fn given_unresolved_target_when_rewrite_should_hash_and_strip_without_verify() {
+ let mut wire = change_password_request();
+
+ // No stored hash: nothing to verify against, but the plaintext must
+ // still never reach consensus, so the body is rewritten regardless.
+ let body = verify_and_rewrite_change_password(&mut wire, None)
+ .expect("unresolved target is not an error");
+
+ let decoded =
+ ChangePasswordRequest::decode_from(&body).expect("re-encoded body round-trips");
+ assert!(
+ decoded.current_password.is_empty(),
+ "current password must be stripped even for an unresolved target"
+ );
+ assert!(
+ crypto::verify_password(NEW_PASSWORD, &decoded.new_password),
+ "new password must still be hashed for an unresolved target"
+ );
+ }
+}
(iggy) 26/43: fix(server-ng): harden HTTP session and auth edges
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit d43eb394789a011e6762262bc347377c13ed5da2
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 17:34:10 2026 +0200
fix(server-ng): harden HTTP session and auth edges
Five session/auth weaknesses from the pre-PR review, one slice:
The HTTP session cap (100k) exceeded the shared consensus client
table (8192), so consensus could LRU-evict a live idle HTTP session;
the cap is now half the table bound, imported so it cannot drift, and
an evicted session is now forgotten on the Eviction reply (occupancy
fenced by Rc identity) so the next request re-registers instead of
looping on 401. Concurrent first requests for one credential each
minted a client id; a per-key registration barrier now lets one
leader register while waiters park and reuse its session, waking on
guard drop so a cancelled leader cannot wedge them. The expiry sweep
used a timestamp sampled before the register await; it resamples.
Two deliberate tightenings beyond legacy parity: JWT nbf is now
validated on decode (every issued token carries nbf == iat by
default, so only future-dated tokens are rejected), and the login
path dummy-verifies against a static hash when the username is
unknown, closing the username-enumeration timing oracle legacy also
has.
---
core/server-ng/src/auth.rs | 25 ++-
core/server-ng/src/http.rs | 380 ++++++++++++++++++++++++++++++++++++-----
core/server-ng/src/http/jwt.rs | 43 +++++
3 files changed, 399 insertions(+), 49 deletions(-)
diff --git a/core/server-ng/src/auth.rs b/core/server-ng/src/auth.rs
index b6e88e21f..cc1c25412 100644
--- a/core/server-ng/src/auth.rs
+++ b/core/server-ng/src/auth.rs
@@ -37,8 +37,19 @@ use metadata::impls::metadata::StreamsFrontend;
use server_common::crypto;
use std::cell::RefCell;
use std::rc::Rc;
+use std::sync::LazyLock;
use tracing::warn;
+/// A well-formed Argon2 hash to verify against on the unknown-user login
+/// branch, so a missing username costs the same single `verify_password` a real
+/// user's wrong-password branch costs. Closes the username-existence timing
+/// oracle without changing the returned error. The password is a throwaway no
+/// account can present (login rejects out-of-range usernames before any hash
+/// check); computed once via the same hasher real users use, so the dummy
+/// verify runs an identical Argon2 KDF.
+static DUMMY_PASSWORD_HASH: LazyLock<String> =
+ LazyLock::new(|| crypto::hash_password("http-login-timing-guard"));
+
pub(crate) fn verify_login_credentials(
shard: &Rc<ServerNgShard>,
username: &str,
@@ -54,10 +65,16 @@ pub(crate) fn verify_login_credentials(
return Err(LoginRegisterError::InvalidCredentials);
}
shard.plane.metadata().mux_stm.users().read(|users| {
- let Some(user_id) = users.index.get(username).copied() else {
- return Err(LoginRegisterError::InvalidCredentials);
- };
- let Some(user) = users.items.get(user_id as usize) else {
+ let user = users
+ .index
+ .get(username)
+ .copied()
+ .and_then(|user_id| users.items.get(user_id as usize));
+ let Some(user) = user else {
+ // Constant-cost path: verify against a dummy hash so a missing
+ // username is indistinguishable by response timing from a wrong
+ // password (both return InvalidCredentials).
+ let _ = crypto::verify_password(password, DUMMY_PASSWORD_HASH.as_str());
return Err(LoginRegisterError::InvalidCredentials);
};
if user.status != UserStatus::Active {
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index 3b8a940d6..7c8c45f3f 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -24,6 +24,7 @@ mod jwt;
use std::cell::{Cell, RefCell};
use std::collections::HashMap;
+use std::collections::hash_map::Entry;
use std::net::{IpAddr, SocketAddr};
use std::rc::Rc;
use std::time::Duration;
@@ -38,7 +39,7 @@ use axum::{Json, Router};
use bytes::{Bytes, BytesMut};
use configs::cluster::{ClusterConfig, ClusterNodeConfig};
use configs::http::HttpConfig;
-use consensus::{MetadataHandle, VsrConsensus};
+use consensus::{CLIENTS_TABLE_MAX, MetadataHandle, VsrConsensus};
use futures::channel::oneshot;
use iggy_binary_protocol::codes::{
GET_CONSUMER_GROUP_CODE, GET_CONSUMER_GROUPS_CODE, GET_STATS_CODE, GET_STREAM_CODE,
@@ -161,7 +162,16 @@ const PONG: &str = "pong";
/// (pathological at P1 volume). New sessions past the cap are refused with a
/// transient 503 rather than evicting a live one; the client retries. Expired
/// entries are dropped first, so the cap only bites on live oversubscription.
-const MAX_HTTP_SESSIONS: usize = 100_000;
+///
+/// Bounded by the shared VSR client table: HTTP sessions and the TCP/QUIC/WS
+/// virtual clients all Register into the one [`CLIENTS_TABLE_MAX`]-slot table,
+/// which LRU-evicts the oldest client when full. Capping HTTP at half that
+/// bound keeps this plane from crowding the others out and keeps the combined
+/// steady state under the shared bound, so a live idle HTTP session is not
+/// routinely evicted consensus-side. The residual eviction race (both planes
+/// busy) degrades gracefully: an evicted session's next control write is
+/// classified as an eviction and re-registers (see [`HttpInner::forget_session`]).
+const MAX_HTTP_SESSIONS: usize = CLIENTS_TABLE_MAX / 2;
/// First per-session request id the write path hands out. VSR request numbers
/// are 1-based and strictly increasing within a session.
@@ -227,6 +237,11 @@ const VIEW_HEADER: HeaderName = HeaderName::from_static("x-iggy-view");
/// PAT). Shared via `Rc` by every concurrent request bearing that credential,
/// so the session granularity is per-login.
struct HttpSession {
+ /// Session-table key this entry lives under (`jwt:{jti}` / `pat:{sha}`).
+ /// Held so an eviction observed on the write path can remove exactly this
+ /// entry (see [`HttpInner::forget_session`]) without threading the key
+ /// through the whole write chain.
+ key: String,
/// Shard-0 client id minted for this credential; its top 16 bits are 0, so
/// it shares the shard-0 id space with TCP virtual clients without
/// colliding. Fills `RequestHeader.client` on every write.
@@ -315,13 +330,21 @@ impl IntoResponse for AuthError {
/// business error or, worse, a 200 with a stale body.
enum WriteError {
Rejected(IggyError),
+ /// The VSR session was evicted (its client slot was reclaimed cluster-side,
+ /// e.g. LRU-evicted from the full client table). Renders identically to a
+ /// terminal `Rejected` (401 -> re-authenticate), but is a distinct variant
+ /// so the submit path can drop the dead session entry and let the caller's
+ /// next request re-register cleanly instead of 401-looping on it.
+ Evicted(IggyError),
Unavailable,
}
impl IntoResponse for WriteError {
fn into_response(self) -> Response {
match self {
- Self::Rejected(error) => CustomError::from(error).into_response(),
+ Self::Rejected(error) | Self::Evicted(error) => {
+ CustomError::from(error).into_response()
+ }
Self::Unavailable => service_unavailable(),
}
}
@@ -621,6 +644,67 @@ struct ClusterRoster {
http_addr: SocketAddr,
}
+/// Serializes first-use VSR registration per credential key so a herd of
+/// concurrent first-requests for one token runs exactly one `Register` instead
+/// of N that each mint a client id and orphan N-1 slots last-writer-wins.
+///
+/// Shard 0 is single-threaded, so the whole check-or-claim is a synchronous
+/// `RefCell` critical section (no cross-thread machinery): the first caller for
+/// a key claims it and gets a [`RegistrationGuard`]; every later caller gets a
+/// waiter to park on until the registrant finishes. The guard clears the marker
+/// and wakes all waiters on drop - including a cancellation drop, so a
+/// disconnected registrant can never wedge its waiters.
+#[derive(Default)]
+struct RegistrationBarrier {
+ /// Key -> waiters parked on the in-flight registration. Dropping a waiter's
+ /// sender wakes its receiver; the guard drops the whole vec at once.
+ inflight: RefCell<HashMap<String, Vec<oneshot::Sender<()>>>>,
+}
+
+/// Outcome of [`RegistrationBarrier::enter`]: lead the registration or wait for
+/// the caller that is already leading it.
+enum BarrierEntry<'a> {
+ Lead(RegistrationGuard<'a>),
+ Wait(oneshot::Receiver<()>),
+}
+
+/// Held by the sole registrant for a key. On drop it removes the in-flight
+/// marker, which drops every parked waiter's sender and so wakes them to
+/// re-check the session table.
+struct RegistrationGuard<'a> {
+ barrier: &'a RegistrationBarrier,
+ key: String,
+}
+
+impl RegistrationBarrier {
+ /// Claim `key` for registration, or return a waiter if another caller
+ /// already holds it. Synchronous and borrow-free of any `.await`.
+ fn enter(&self, key: &str) -> BarrierEntry<'_> {
+ match self.inflight.borrow_mut().entry(key.to_owned()) {
+ Entry::Occupied(mut occupied) => {
+ let (sender, receiver) = oneshot::channel();
+ occupied.get_mut().push(sender);
+ BarrierEntry::Wait(receiver)
+ }
+ Entry::Vacant(vacant) => {
+ vacant.insert(Vec::new());
+ BarrierEntry::Lead(RegistrationGuard {
+ barrier: self,
+ key: key.to_owned(),
+ })
+ }
+ }
+ }
+}
+
+impl Drop for RegistrationGuard<'_> {
+ fn drop(&mut self) {
+ // Dropping the vec of senders wakes every waiter (Canceled), which then
+ // re-checks the table for the session this registrant installed.
+ self.barrier.inflight.borrow_mut().remove(&self.key);
+ }
+}
+
/// Shared shard-0 HTTP state.
///
/// Groups the shard handle, the JWT issuer/verifier, and the per-credential VSR
@@ -634,6 +718,9 @@ struct HttpInner {
/// bridge tolerates the `!Sync` interior - but the guard must never be held
/// across an `.await` (see [`HttpInner::resolve_session`]).
sessions: RefCell<HashMap<String, Rc<HttpSession>>>,
+ /// Per-key registration barrier: prevents a thundering herd of first
+ /// requests for one credential from each running its own `Register`.
+ registrations: RegistrationBarrier,
roster: ClusterRoster,
/// Awaited partition writes currently in flight across all sessions, gated
/// by [`MAX_IN_FLIGHT_WRITES_GLOBAL`]. Only [`InFlightWriteGuard`] touches
@@ -689,45 +776,48 @@ impl HttpInner {
user_id: u32,
expiry: u64,
) -> Result<Rc<HttpSession>, AuthError> {
- let now = IggyTimestamp::now().to_secs();
- if let Some(session) = self.live_session(&key, now) {
- return Ok(session);
- }
-
- // Miss: mint + Register with no borrow held (an async VSR commit).
- let fresh = self.register_session(user_id, expiry).await?;
-
- let mut table = self.sessions.borrow_mut();
- // A concurrent first request for the same key may have Registered and
- // inserted while we were parked. Last-writer-wins: reuse the installed
- // entry and drop `fresh`, whose client id is then orphaned on the peers
- // until they evict it - an accepted rare cost at P1 volume.
- if let Some(session) = live_entry(&table, &key, now) {
- return Ok(session);
- }
- table.retain(|_, session| {
- if session.expiry > now {
- return true;
+ loop {
+ let now = IggyTimestamp::now().to_secs();
+ if let Some(session) = self.live_session(&key, now) {
+ return Ok(session);
}
- // Evicting the session also tears down its in-process reply
- // target (if a partition write ever installed one), cancelling any
- // still-parked reply waiters. Token-fenced so a stale sweep can
- // never remove a later occupant of the key.
- if let Some(token) = session.registry_token.get() {
- self.shard
- .bus
- .clients()
- .remove_if_token_matches(session.client_id, token);
+
+ // Miss. Serialize registration per credential so a herd of
+ // concurrent first-requests runs one `Register`, not N.
+ match self.registrations.enter(&key) {
+ BarrierEntry::Wait(waiter) => {
+ // Another first-request is registering this credential.
+ // Park until it finishes (its guard wakes us on drop),
+ // then loop to re-check the table for what it installed.
+ let _ = waiter.await;
+ }
+ BarrierEntry::Lead(_guard) => {
+ // Sole registrant for this key: mint + Register with no
+ // borrow held (an async VSR commit). The guard wakes any
+ // waiters when this scope ends, cancellation drop included.
+ let fresh = self.register_session(key.clone(), user_id, expiry).await?;
+ // Resample after the await: the pre-await stamp is stale for
+ // the expiry sweep and cap check below.
+ let now = IggyTimestamp::now().to_secs();
+ let (admitted, torn) = {
+ let mut table = self.sessions.borrow_mut();
+ let torn = sweep_expired(&mut table, now);
+ if table.len() >= MAX_HTTP_SESSIONS {
+ // Still full after dropping expired entries: too many
+ // genuinely live sessions. Refuse rather than evict a
+ // live one (its `fresh` client id is orphaned on the
+ // peers until they evict it - a rare at-cap cost).
+ (None, torn)
+ } else {
+ table.insert(key.clone(), Rc::clone(&fresh));
+ (Some(fresh), torn)
+ }
+ };
+ self.teardown_reply_targets(torn);
+ return admitted.ok_or(AuthError::SessionUnavailable);
+ }
}
- false
- });
- if table.len() >= MAX_HTTP_SESSIONS {
- // Still full after dropping expired entries: too many genuinely live
- // sessions. Refuse rather than evict a live one; the client retries.
- return Err(AuthError::SessionUnavailable);
}
- table.insert(key, Rc::clone(&fresh));
- Ok(fresh)
}
/// Clone the live (non-expired) entry for `key`, if present. Confines the
@@ -737,9 +827,10 @@ impl HttpInner {
}
/// Mint a shard-0 client id and run the VSR `Register` for a fresh session.
- /// Holds no table borrow; the caller inserts the result.
+ /// Holds no table borrow; the caller inserts the result under `key`.
async fn register_session(
&self,
+ key: String,
user_id: u32,
expiry: u64,
) -> Result<Rc<HttpSession>, AuthError> {
@@ -766,6 +857,7 @@ impl HttpInner {
AuthError::SessionUnavailable
})?;
Ok(Rc::new(HttpSession {
+ key,
client_id,
session,
user_id,
@@ -776,6 +868,73 @@ impl HttpInner {
in_flight_writes: Cell::new(0),
}))
}
+
+ /// Drop the session table entry for `session`, but only if it is still the
+ /// current occupant of its key (pointer-fenced, so a later re-registration
+ /// under the same key is never purged). Also tears down its in-process
+ /// reply target. Called when a control write comes back evicted: the VSR
+ /// slot is gone, so leaving the entry would 401-loop every retry on the
+ /// same credential until the token expires; removing it makes the next
+ /// request re-register cleanly through the barrier.
+ fn forget_session(&self, session: &Rc<HttpSession>) {
+ let torn = forget_if_same(&mut self.sessions.borrow_mut(), session);
+ self.teardown_reply_targets(torn.into_iter().collect());
+ }
+
+ /// Tear down the in-process reply targets of swept/forgotten sessions,
+ /// token-fenced so a stale teardown can never remove a later occupant's
+ /// registry entry. Runs outside the `sessions` borrow.
+ fn teardown_reply_targets(&self, torn: Vec<(u128, InstanceToken)>) {
+ for (client_id, token) in torn {
+ self.shard
+ .bus
+ .clients()
+ .remove_if_token_matches(client_id, token);
+ }
+ }
+}
+
+/// Drop every expired entry from the session table, returning the
+/// `(client_id, registry token)` of each dropped entry that had installed an
+/// in-process reply target so the caller can tear those down outside the
+/// borrow. A pure map operation (no shard), so the cap/expiry policy is
+/// unit-testable without a live consensus.
+fn sweep_expired(
+ table: &mut HashMap<String, Rc<HttpSession>>,
+ now_secs: u64,
+) -> Vec<(u128, InstanceToken)> {
+ let mut torn = Vec::new();
+ table.retain(|_, session| {
+ if session.expiry > now_secs {
+ return true;
+ }
+ if let Some(token) = session.registry_token.get() {
+ torn.push((session.client_id, token));
+ }
+ false
+ });
+ torn
+}
+
+/// Remove `session` from the table only if it is still the current occupant of
+/// its key (`Rc::ptr_eq`), returning its reply target to tear down. A stale
+/// handle whose key was re-registered to a newer session removes nothing. Pure
+/// (no shard) so the eviction-recovery fencing is unit-testable.
+fn forget_if_same(
+ table: &mut HashMap<String, Rc<HttpSession>>,
+ session: &Rc<HttpSession>,
+) -> Option<(u128, InstanceToken)> {
+ match table.get(&session.key) {
+ Some(current) if Rc::ptr_eq(current, session) => {
+ let torn = session
+ .registry_token
+ .get()
+ .map(|token| (session.client_id, token));
+ table.remove(&session.key);
+ torn
+ }
+ _ => None,
+ }
}
/// Borrow-and-clone a live table entry, or `None` if missing or expired. Shared
@@ -821,6 +980,7 @@ pub async fn start(
shard: Rc::clone(shard),
jwt,
sessions: RefCell::new(HashMap::new()),
+ registrations: RegistrationBarrier::default(),
roster: ClusterRoster {
enabled: cluster.enabled,
name: cluster.name.clone(),
@@ -2117,12 +2277,12 @@ async fn submit_committed(
// legacy parity.
let (result_slot, committed) = oneshot::channel();
let shard = Rc::clone(&state.shard);
- let session = Rc::clone(session);
+ let task_session = Rc::clone(session);
let body = body.to_vec();
// Detached so a client disconnect cannot split a commit from its id
// advance; both happen inside the task regardless of handler liveness.
compio::runtime::spawn(async move {
- let result = submit_gated(&shard, &session, operation, &body).await;
+ let result = submit_gated(&shard, &task_session, operation, &body).await;
// A failed send means the handler died mid-await. The id advance
// already happened above, which is the invariant that matters.
let _ = result_slot.send(result);
@@ -2130,7 +2290,13 @@ async fn submit_committed(
.detach();
// `Canceled` = the task was dropped before sending (runtime teardown), a
// transient server condition like an unanswered submit.
- committed.await.map_err(|_| WriteError::Unavailable)?
+ let outcome = committed.await.map_err(|_| WriteError::Unavailable)?;
+ // An eviction means this session's VSR slot is gone: forget the entry so
+ // the caller's next request re-registers instead of 401-looping on it.
+ if matches!(&outcome, Err(WriteError::Evicted(_))) {
+ state.forget_session(session);
+ }
+ outcome
}
/// Gate-locked core of [`submit_committed`], run on a task the HTTP client
@@ -2140,7 +2306,8 @@ async fn submit_committed(
/// eviction leaves the id free for the caller's retry, which the depth-1
/// dedup requires (the next accepted request must be `committed + 1`, so a
/// consumed-but-uncommitted id would wedge the session on `RequestGap`). An
-/// eviction means the session is dead, mapped to 401.
+/// eviction means the session is dead: mapped to 401, and [`submit_committed`]
+/// then forgets the entry so the caller's retry re-registers cleanly.
///
/// Both shard-0 request rewrites run here before consensus, mirroring the TCP
/// dispatch path: the PAT rewrite mints a raw token and replicates only its hash
@@ -2177,7 +2344,7 @@ async fn submit_gated(
drop(next_request_id);
Ok((request_header, reply, raw_token))
}
- Command2::Eviction => Err(WriteError::Rejected(eviction_error(&reply))),
+ Command2::Eviction => Err(WriteError::Evicted(eviction_error(&reply))),
_ => Err(WriteError::Rejected(IggyError::InvalidCommand)),
}
}
@@ -3537,6 +3704,7 @@ mod tests {
#[compio::test]
async fn detached_task_advances_gate_and_ignores_dead_receiver() {
let session = Rc::new(HttpSession {
+ key: "jwt:test".to_owned(),
client_id: 7,
session: 1,
user_id: DEFAULT_ROOT_USER_ID,
@@ -3562,4 +3730,126 @@ mod tests {
done.await.expect("detached task must run to completion");
assert_eq!(*session.gate.lock().await, FIRST_REQUEST_ID + 1);
}
+
+ // -- session table + registration barrier --
+
+ /// `InstanceToken` has no public constructor, so fixtures carry no reply
+ /// target; the token-teardown branch of the sweep/forget helpers is
+ /// exercised via their `Option` path, not fabricated here.
+ fn fake_session(key: &str, client_id: u128, expiry: u64) -> Rc<HttpSession> {
+ Rc::new(HttpSession {
+ key: key.to_owned(),
+ client_id,
+ session: 1,
+ user_id: DEFAULT_ROOT_USER_ID,
+ expiry,
+ gate: Mutex::new(FIRST_REQUEST_ID),
+ data_request: Cell::new(FIRST_REQUEST_ID),
+ registry_token: Cell::new(None),
+ in_flight_writes: Cell::new(0),
+ })
+ }
+
+ // The barrier is what makes a herd of concurrent first-requests for one
+ // credential run a single `Register`: only the leader reaches
+ // `register_session`; every other caller waits and reuses what it installs.
+ // A different credential leads independently, and the key frees on drop.
+ #[compio::test]
+ async fn registration_barrier_leads_one_caller_and_parks_the_rest() {
+ let barrier = RegistrationBarrier::default();
+
+ let BarrierEntry::Lead(leader) = barrier.enter("jwt:a") else {
+ panic!("first caller for a key must lead");
+ };
+ let mut waiters = Vec::new();
+ for _ in 0..4 {
+ match barrier.enter("jwt:a") {
+ BarrierEntry::Wait(waiter) => waiters.push(waiter),
+ BarrierEntry::Lead(_) => panic!("a concurrent caller must not lead the same key"),
+ }
+ }
+ assert!(
+ matches!(barrier.enter("jwt:b"), BarrierEntry::Lead(_)),
+ "a different credential leads independently"
+ );
+
+ // Leader finishing wakes every waiter and frees the key.
+ drop(leader);
+ for waiter in waiters {
+ let _ = waiter.await;
+ }
+ assert!(
+ matches!(barrier.enter("jwt:a"), BarrierEntry::Lead(_)),
+ "the freed key leads a fresh registration"
+ );
+ }
+
+ #[test]
+ fn sweep_expired_drops_only_expired_entries() {
+ let mut table = HashMap::new();
+ let live = fake_session("jwt:live", 1, 1_000);
+ let stale = fake_session("jwt:stale", 2, 10);
+ table.insert(live.key.clone(), Rc::clone(&live));
+ table.insert(stale.key.clone(), Rc::clone(&stale));
+
+ // now = 100: live.expiry(1000) > now stays, stale.expiry(10) <= now goes.
+ let torn = sweep_expired(&mut table, 100);
+
+ assert!(torn.is_empty(), "fixtures install no reply target");
+ assert!(table.contains_key("jwt:live"), "live session retained");
+ assert!(!table.contains_key("jwt:stale"), "expired session swept");
+ }
+
+ // At the cap the sweep only reclaims expired entries, never a live one, so
+ // an all-live table stays full and `resolve_session` refuses (503) rather
+ // than evicting a live session.
+ #[test]
+ fn sweep_never_evicts_live_sessions_so_a_full_table_stays_full() {
+ let mut table = HashMap::new();
+ for client_id in 0..8u128 {
+ let session = fake_session(&format!("jwt:{client_id}"), client_id, 1_000);
+ table.insert(session.key.clone(), session);
+ }
+ let torn = sweep_expired(&mut table, 100);
+ assert!(torn.is_empty());
+ assert_eq!(table.len(), 8, "no live session is evicted to make room");
+ }
+
+ #[test]
+ fn http_session_cap_leaves_headroom_below_the_shared_client_table_bound() {
+ const {
+ assert!(
+ MAX_HTTP_SESSIONS < CLIENTS_TABLE_MAX,
+ "HTTP must not claim the whole shared VSR client table"
+ );
+ }
+ assert_eq!(MAX_HTTP_SESSIONS, CLIENTS_TABLE_MAX / 2);
+ }
+
+ // Eviction recovery: forgetting the evicted session drops exactly its
+ // entry, and a stale handle never purges a session that re-registered under
+ // the same key in the meantime (the `Rc::ptr_eq` fence).
+ #[test]
+ fn forget_removes_the_evicted_session_but_spares_a_re_registration() {
+ let mut table = HashMap::new();
+ let evicted = fake_session("jwt:a", 1, u64::MAX);
+ table.insert(evicted.key.clone(), Rc::clone(&evicted));
+
+ assert!(forget_if_same(&mut table, &evicted).is_none());
+ assert!(!table.contains_key("jwt:a"), "evicted session removed");
+
+ let replacement = fake_session("jwt:a", 2, u64::MAX);
+ table.insert(replacement.key.clone(), Rc::clone(&replacement));
+ assert!(
+ forget_if_same(&mut table, &evicted).is_none(),
+ "a stale handle removes nothing"
+ );
+ assert!(
+ Rc::ptr_eq(
+ table.get("jwt:a").expect("replacement present"),
+ &replacement
+ ),
+ "the pointer fence spares the re-registered session"
+ );
+ }
}
diff --git a/core/server-ng/src/http/jwt.rs b/core/server-ng/src/http/jwt.rs
index 1304ef692..1bb92ff02 100644
--- a/core/server-ng/src/http/jwt.rs
+++ b/core/server-ng/src/http/jwt.rs
@@ -71,6 +71,11 @@ impl JwtManager {
validation.set_issuer(&config.valid_issuers);
validation.set_audience(&config.valid_audiences);
validation.leeway = u64::from(config.clock_skew.as_secs());
+ // The issuer always stamps `nbf`, so enforce it: a token presented
+ // before its not-before instant (beyond `leeway`) is rejected.
+ // `jsonwebtoken` leaves `validate_nbf` off by default, so this is
+ // explicit; the claim is always present, so it only ever tightens.
+ validation.validate_nbf = true;
Ok(Self {
algorithm,
issuer: config.issuer,
@@ -173,3 +178,41 @@ fn normalize_secrets(mut config: HttpJwtConfig) -> HttpJwtConfig {
}
config
}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ /// A default config (HS256, matching issuer/audience, empty secrets ->
+ /// one ephemeral secret shared by encode and decode) with the two
+ /// timing knobs overridden.
+ fn config(not_before: &str, clock_skew: &str) -> HttpJwtConfig {
+ HttpJwtConfig {
+ not_before: not_before.parse().expect("valid duration"),
+ clock_skew: clock_skew.parse().expect("valid duration"),
+ ..HttpJwtConfig::default()
+ }
+ }
+
+ #[test]
+ fn token_presented_before_its_nbf_is_rejected() {
+ // not_before well past the leeway window, so the freshly issued token
+ // is not yet valid and decode must reject it.
+ let manager = JwtManager::build(&config("3600 s", "5 s")).expect("builds");
+ let token = manager.generate(7).expect("issues");
+ assert!(
+ manager.decode(&token.access_token).is_err(),
+ "a token presented before its nbf must be rejected"
+ );
+ }
+
+ #[test]
+ fn token_at_its_nbf_is_accepted() {
+ // Default not_before (0s) => nbf == iat, so the token is valid now and
+ // enabling nbf validation does not reject a normally issued token.
+ let manager = JwtManager::build(&config("0 s", "5 s")).expect("builds");
+ let token = manager.generate(7).expect("issues");
+ let claims = manager.decode(&token.access_token).expect("valid now");
+ assert_eq!(claims.sub, "7");
+ }
+}
(iggy) 16/43: feat(server-ng): add HTTP consumer-offset write routes
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 2195a5c3bfd4808a2e6b3093d1ce813bb2de77fd
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 00:13:08 2026 +0200
feat(server-ng): add HTTP consumer-offset write routes
The HTTP listener could read consumer offsets but not store or
delete them, so an HTTP consumer could not persist its position.
Add legacy-parity PUT /consumer-offsets and DELETE
/consumer-offsets/{consumer_id}, both returning 204 on commit.
Offset writes are partition-plane ops, not metadata writes: encode
the v2 wire requests (StoreConsumerOffset2/DeleteConsumerOffset2)
with ack pinned to Quorum and ride the same reply-slot machinery
the produce route proved - slot before dispatch, bounded wait,
op==0 gate reply as the generic 404. The produce-named shared
pieces are renamed plane-neutral (PartitionWriteError,
partition_write_replicated) instead of duplicated; the 504 body
keeps a per-kind code (produce_timeout / offset_write_timeout) so
callers can tell which write surface timed out.
---
core/server-ng/src/http.rs | 376 +++++++++++++++++++++++++++++++++++++--------
1 file changed, 310 insertions(+), 66 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index c3a406d93..596f00a94 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -52,7 +52,9 @@ use iggy_binary_protocol::requests::consumer_groups::{
CreateConsumerGroupRequest, DeleteConsumerGroupRequest, GetConsumerGroupRequest,
GetConsumerGroupsRequest,
};
-use iggy_binary_protocol::requests::consumer_offsets::GetConsumerOffsetRequest;
+use iggy_binary_protocol::requests::consumer_offsets::{
+ DeleteConsumerOffset2Request, GetConsumerOffsetRequest, StoreConsumerOffset2Request,
+};
use iggy_binary_protocol::requests::messages::{
PollMessagesRequest, RawMessage, SendMessagesEncoder,
};
@@ -87,7 +89,8 @@ use iggy_binary_protocol::responses::users::get_user::UserDetailsResponse;
use iggy_binary_protocol::responses::users::get_users::GetUsersResponse;
use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
use iggy_binary_protocol::{
- GenericHeader, Operation, ReplyHeader, RequestHeader, WireDecode, WireEncode, WireName,
+ AckLevel, GenericHeader, Operation, ReplyHeader, RequestHeader, WireDecode, WireEncode,
+ WireName,
};
use iggy_common::change_password::ChangePassword;
use iggy_common::create_consumer_group::CreateConsumerGroup;
@@ -96,25 +99,28 @@ use iggy_common::create_stream::CreateStream;
use iggy_common::create_topic::CreateTopic;
use iggy_common::create_user::CreateUser;
use iggy_common::defaults::DEFAULT_ROOT_USER_ID;
+use iggy_common::delete_consumer_offset::DeleteConsumerOffset;
use iggy_common::get_consumer_offset::GetConsumerOffset;
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
use iggy_common::poll_messages::DEFAULT_PARTITION_ID;
+use iggy_common::store_consumer_offset::StoreConsumerOffset;
use iggy_common::update_permissions::UpdatePermissions;
use iggy_common::update_stream::UpdateStream;
use iggy_common::update_topic::UpdateTopic;
use iggy_common::update_user::UpdateUser;
use iggy_common::wire_conversions::{
- clients_from_wire, consumer_groups_from_wire, identifier_to_wire, partitioning_to_wire,
- permissions_to_wire, streams_from_wire, topics_from_wire, users_from_wire,
+ clients_from_wire, consumer_groups_from_wire, consumer_to_wire, identifier_to_wire,
+ partitioning_to_wire, permissions_to_wire, streams_from_wire, topics_from_wire,
+ users_from_wire,
};
use iggy_common::{
ClientInfo, ClientInfoDetails, ClusterMetadata, ClusterNode, ClusterNodeRole,
- ClusterNodeStatus, ConsumerGroup, ConsumerGroupDetails, ConsumerOffsetInfo, Identifier,
- IdentityInfo, IggyError, IggyMessageView, IggyTimestamp, PollMessages, PolledMessages,
- RESYNC_REQUIRED_PARTITION_SENTINEL, RawPersonalAccessToken, SendMessages, Stats, Stream,
- StreamDetails, TokenInfo, Topic, TopicDetails, TransportEndpoints, UserInfo, UserInfoDetails,
- Validatable,
+ ClusterNodeStatus, Consumer, ConsumerGroup, ConsumerGroupDetails, ConsumerOffsetInfo,
+ Identifier, IdentityInfo, IggyError, IggyMessageView, IggyTimestamp, PollMessages,
+ PolledMessages, RESYNC_REQUIRED_PARTITION_SENTINEL, RawPersonalAccessToken, SendMessages,
+ Stats, Stream, StreamDetails, TokenInfo, Topic, TopicDetails, TransportEndpoints, UserInfo,
+ UserInfoDetails, Validatable,
};
use message_bus::{BusMessage, InstanceToken, client_listener};
use metadata::impls::metadata::StreamsFrontend;
@@ -158,13 +164,14 @@ const MAX_HTTP_SESSIONS: usize = 100_000;
/// are 1-based and strictly increasing within a session.
const FIRST_REQUEST_ID: u64 = 1;
-/// Bound on a produce's wait for its committed partition reply. Long enough to
-/// ride out a view change (plus the dispatch gates' own routable-wait budget),
-/// short enough not to pin HTTP connections behind a dead consensus group. On
-/// expiry the caller gets 504 and must treat the outcome as unknown: the
-/// partition plane is at-least-once and the prepare may still commit after the
-/// wait gave up, so the server never retries on the caller's behalf.
-const PRODUCE_REPLY_TIMEOUT: Duration = Duration::from_secs(10);
+/// Bound on a partition write's (produce / consumer-offset write) wait for its
+/// committed reply. Long enough to ride out a view change (plus the dispatch
+/// gates' own routable-wait budget), short enough not to pin HTTP connections
+/// behind a dead consensus group. On expiry the caller gets 504 and must treat
+/// the outcome as unknown: the partition plane is at-least-once and the prepare
+/// may still commit after the wait gave up, so the server never retries on the
+/// caller's behalf.
+const PARTITION_WRITE_REPLY_TIMEOUT: Duration = Duration::from_secs(10);
/// VSR client id stamped on HTTP data-plane reads (poll / consumer-offset).
/// HTTP reads never Register a VSR client, and shard-0 client ids are minted
@@ -223,8 +230,9 @@ struct HttpSession {
/// monotonically and never reused, which the slot-guard contract requires.
data_request: Cell<u64>,
/// Registry token of this session's lazily-installed in-process reply
- /// target (`None` until the first awaited produce). Stored so session
- /// eviction can tear the registry entry down fenced by the same token.
+ /// target (`None` until the first awaited partition write). Stored so
+ /// session eviction can tear the registry entry down fenced by the same
+ /// token.
registry_token: Cell<Option<InstanceToken>>,
}
@@ -293,56 +301,67 @@ impl IntoResponse for WriteError {
}
}
-/// Rejection for a data-plane produce (`POST .../messages`).
+/// Rejection for a data-plane partition write (`POST .../messages` produce and
+/// the `PUT`/`DELETE .../consumer-offsets` writes).
///
/// Split differently from [`WriteError`] because the partition plane replies
/// carry no committed error code: a pre-dispatch gate failure is an empty
/// reply distinguishable only by header (see [`classify_partition_reply`]),
-/// and an unanswered produce is a distinct outcome the caller must treat as
+/// and an unanswered write is a distinct outcome the caller must treat as
/// unknown rather than failed.
-enum ProduceError {
+enum PartitionWriteError {
/// Caller-side rejection (bad identifier, oversized batch, the interim
/// non-root denial) or a malformed reply frame, rendered through the
/// legacy `IggyError -> status` map for SDK-identical bodies.
Rejected(IggyError),
- /// The dispatch gates could not route the produce: the stream, topic, or
+ /// The dispatch gates could not route the write: the stream, topic, or
/// partition does not resolve (or never materialised within the routable
/// budget). Rendered as the legacy 404 body.
NotFound,
/// The in-process reply slot could not be installed. Transient server
/// condition -> the shared 503, retryable.
Unavailable,
- /// No committed reply within [`PRODUCE_REPLY_TIMEOUT`], or the session's
- /// reply target was torn down mid-wait. 504: the commit may still land
- /// (at-least-once), so this is a hard "outcome unknown", not a failure
- /// the server may transparently retry.
- Timeout,
+ /// No committed reply within [`PARTITION_WRITE_REPLY_TIMEOUT`], or the
+ /// session's reply target was torn down mid-wait. 504: the commit may
+ /// still land (at-least-once), so this is a hard "outcome unknown", not a
+ /// failure the server may transparently retry. Carries the write's
+ /// operation so the 504 body names which write kind timed out.
+ Timeout(Operation),
}
-impl IntoResponse for ProduceError {
+impl IntoResponse for PartitionWriteError {
fn into_response(self) -> Response {
match self {
Self::Rejected(error) => CustomError::from(error).into_response(),
Self::NotFound => CustomError::ResourceNotFound.into_response(),
Self::Unavailable => service_unavailable(),
- Self::Timeout => produce_timeout_response(),
+ Self::Timeout(operation) => partition_write_timeout_response(operation),
}
}
}
-/// 504 body for a produce whose commit outcome is unknown. Shaped like every
-/// other HTTP error (`ErrorResponse`) so clients parse one error schema.
-fn produce_timeout_response() -> Response {
- gateway_timeout_response(
- "produce_timeout",
- "produce was not acknowledged in time; the write may still commit",
- )
+/// 504 body for a partition write whose commit outcome is unknown, coded per
+/// write kind so a caller can tell a produce timeout from an offset-write
+/// timeout. Shaped like every other HTTP error (`ErrorResponse`) so clients
+/// parse one error schema.
+fn partition_write_timeout_response(operation: Operation) -> Response {
+ let (code, reason) = match operation {
+ Operation::SendMessages => (
+ "produce_timeout",
+ "produce was not acknowledged in time; the write may still commit",
+ ),
+ _ => (
+ "offset_write_timeout",
+ "consumer-offset write was not acknowledged in time; the write may still commit",
+ ),
+ };
+ gateway_timeout_response(code, reason)
}
/// Shared 504 rendering for an in-band request the partition plane did not
/// answer in time, shaped like every other HTTP error (`ErrorResponse`) so
-/// clients parse one error schema. Consumed by the produce reply wait and the
-/// partition reads ([`ReadError::Timeout`]).
+/// clients parse one error schema. Consumed by the partition-write reply wait
+/// and the partition reads ([`ReadError::Timeout`]).
fn gateway_timeout_response(code: &str, reason: &str) -> Response {
(
StatusCode::GATEWAY_TIMEOUT,
@@ -614,7 +633,7 @@ impl HttpInner {
return true;
}
// Evicting the session also tears down its in-process reply
- // target (if a produce ever installed one), cancelling any
+ // target (if a partition write ever installed one), cancelling any
// still-parked reply waiters. Token-fenced so a stale sweep can
// never remove a later occupant of the key.
if let Some(token) = session.registry_token.get() {
@@ -792,7 +811,11 @@ fn router(state: HttpState) -> Router {
)
.route(
"/streams/{stream_id}/topics/{topic_id}/consumer-offsets",
- get(get_consumer_offset),
+ get(get_consumer_offset).put(store_consumer_offset),
+ )
+ .route(
+ "/streams/{stream_id}/topics/{topic_id}/consumer-offsets/{consumer_id}",
+ delete(delete_consumer_offset),
)
.route(
"/streams/{stream_id}/topics/{topic_id}/consumer-groups",
@@ -1478,21 +1501,28 @@ async fn send_messages(
Path((stream_id, topic_id)): Path<(String, String)>,
Query(query): Query<ProduceQuery>,
Json(command): Json<SendMessages>,
-) -> Result<Response, ProduceError> {
+) -> Result<Response, PartitionWriteError> {
// Interim authorization: root-only until server-ng has an RBAC
// permissioner, mirroring the control-plane gate in `submit_committed`.
if identity.session.user_id != DEFAULT_ROOT_USER_ID {
- return Err(ProduceError::Rejected(IggyError::Unauthorized));
+ return Err(PartitionWriteError::Rejected(IggyError::Unauthorized));
}
- let stream_id = Identifier::from_str_value(&stream_id).map_err(ProduceError::Rejected)?;
- let topic_id = Identifier::from_str_value(&topic_id).map_err(ProduceError::Rejected)?;
+ let stream_id =
+ Identifier::from_str_value(&stream_id).map_err(PartitionWriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(PartitionWriteError::Rejected)?;
// Rejects an oversized partitioning key and an empty or oversized batch.
- command.validate().map_err(ProduceError::Rejected)?;
- let body =
- encode_send_messages(&stream_id, &topic_id, &command).map_err(ProduceError::Rejected)?;
+ command.validate().map_err(PartitionWriteError::Rejected)?;
+ let body = encode_send_messages(&stream_id, &topic_id, &command)
+ .map_err(PartitionWriteError::Rejected)?;
match query.ack {
ProduceAck::Replicated => {
- SendWrapper::new(produce_replicated(&state, &identity.session, &body)).await?;
+ SendWrapper::new(partition_write_replicated(
+ &state,
+ &identity.session,
+ Operation::SendMessages,
+ &body,
+ ))
+ .await?;
Ok((
StatusCode::CREATED,
[(
@@ -1513,6 +1543,82 @@ async fn send_messages(
}
}
+/// `PUT /streams/{stream_id}/topics/{topic_id}/consumer-offsets`: store a
+/// consumer's offset. The JSON body is the same `StoreConsumerOffset` shape the
+/// legacy server accepts (flattened `consumer_id`, optional `partition_id`,
+/// `offset`); stream and topic come from the path. Returns 204 on commit,
+/// matching the legacy server.
+///
+/// Data plane like a produce: the offset write is a replicated op on the
+/// partition group's own consensus, awaited through the session's in-process
+/// reply slot ([`partition_write_replicated`]). The v2 wire op is pinned to
+/// `ack = Quorum` - `?ack=none` is a produce-only surface. The consumer
+/// identifier passes through on the wire; the dispatch resolvers hash named
+/// consumers and rewrite group ids server-side, identically to TCP.
+async fn store_consumer_offset(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Json(command): Json<StoreConsumerOffset>,
+) -> Result<StatusCode, PartitionWriteError> {
+ // Interim authorization: root-only until server-ng has an RBAC
+ // permissioner, mirroring the control-plane gate in `submit_committed`.
+ if identity.session.user_id != DEFAULT_ROOT_USER_ID {
+ return Err(PartitionWriteError::Rejected(IggyError::Unauthorized));
+ }
+ let stream_id =
+ Identifier::from_str_value(&stream_id).map_err(PartitionWriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(PartitionWriteError::Rejected)?;
+ let request = store_offset_wire_request(&stream_id, &topic_id, &command)
+ .map_err(PartitionWriteError::Rejected)?;
+ let body = request.to_bytes();
+ SendWrapper::new(partition_write_replicated(
+ &state,
+ &identity.session,
+ Operation::StoreConsumerOffset2,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
+/// `DELETE /streams/{stream_id}/topics/{topic_id}/consumer-offsets/{consumer_id}`:
+/// delete a consumer's stored offset. The consumer comes from the path and the
+/// optional `partition_id` from the query, the same `DeleteConsumerOffset`
+/// shape the legacy server accepts. Returns 204 on commit, matching the legacy
+/// server. Same replicated partition write as [`store_consumer_offset`].
+async fn delete_consumer_offset(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id, consumer_id)): Path<(String, String, String)>,
+ Query(query): Query<DeleteConsumerOffset>,
+) -> Result<StatusCode, PartitionWriteError> {
+ // Interim authorization: root-only until server-ng has an RBAC
+ // permissioner, mirroring the control-plane gate in `submit_committed`.
+ if identity.session.user_id != DEFAULT_ROOT_USER_ID {
+ return Err(PartitionWriteError::Rejected(IggyError::Unauthorized));
+ }
+ let stream_id =
+ Identifier::from_str_value(&stream_id).map_err(PartitionWriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(PartitionWriteError::Rejected)?;
+ // `Consumer::new` fixes the kind to `Consumer`, exactly as the legacy
+ // handler does; HTTP cannot express a group-kind offset op.
+ let consumer = Consumer::new(
+ Identifier::from_str_value(&consumer_id).map_err(PartitionWriteError::Rejected)?,
+ );
+ let request = delete_offset_wire_request(&stream_id, &topic_id, &consumer, query.partition_id)
+ .map_err(PartitionWriteError::Rejected)?;
+ let body = request.to_bytes();
+ SendWrapper::new(partition_write_replicated(
+ &state,
+ &identity.session,
+ Operation::DeleteConsumerOffset2,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
/// `POST /streams/{stream_id}/topics/{topic_id}/consumer-groups`: create a
/// consumer group under a topic and render the committed reply as the same
/// `ConsumerGroupDetails` JSON the legacy server returns.
@@ -1952,6 +2058,45 @@ fn consumer_offset_wire_request(
})
}
+/// Map a validated HTTP store-offset body onto the v2 wire request
+/// (`StoreConsumerOffset2Request`), `ack` pinned to `Quorum` so the route can
+/// await the committed reply. The body's consumer kind is structurally always
+/// `Consumer` (`Consumer::kind` is `#[serde(skip)]`), matching the legacy HTTP
+/// server; `partition_id` passes through as the wire `Option` (flag byte +
+/// u32) for the server-side resolvers to ground.
+fn store_offset_wire_request(
+ stream_id: &Identifier,
+ topic_id: &Identifier,
+ command: &StoreConsumerOffset,
+) -> Result<StoreConsumerOffset2Request, IggyError> {
+ Ok(StoreConsumerOffset2Request {
+ consumer: consumer_to_wire(&command.consumer)?,
+ stream_id: identifier_to_wire(stream_id)?,
+ topic_id: identifier_to_wire(topic_id)?,
+ partition_id: command.partition_id,
+ offset: command.offset,
+ ack: AckLevel::Quorum,
+ })
+}
+
+/// Map a validated HTTP delete-offset request onto the v2 wire request
+/// (`DeleteConsumerOffset2Request`), `ack` pinned to `Quorum` like
+/// [`store_offset_wire_request`].
+fn delete_offset_wire_request(
+ stream_id: &Identifier,
+ topic_id: &Identifier,
+ consumer: &Consumer,
+ partition_id: Option<u32>,
+) -> Result<DeleteConsumerOffset2Request, IggyError> {
+ Ok(DeleteConsumerOffset2Request {
+ consumer: consumer_to_wire(consumer)?,
+ stream_id: identifier_to_wire(stream_id)?,
+ topic_id: identifier_to_wire(topic_id)?,
+ partition_id,
+ ack: AckLevel::Quorum,
+ })
+}
+
/// The empty `PolledMessages` a fenced consumer-group poll answers, carrying
/// the re-sync sentinel in `partition_id` exactly as the TCP dispatch replies
/// it, so an SDK re-syncs its assignment instead of reading end-of-partition.
@@ -1964,23 +2109,25 @@ const fn resync_required_polled_messages() -> PolledMessages {
}
}
-/// Run one awaited produce end to end: install the reply slot, dispatch into
-/// the partition plane, and wait (bounded) for the committed reply.
+/// Run one awaited partition write (produce / consumer-offset write) end to
+/// end: install the reply slot, dispatch into the partition plane, and wait
+/// (bounded) for the committed reply.
///
/// Slot-before-dispatch is load-bearing: every pre-dispatch gate failure
/// inside [`dispatch_partition_request`] replies through `send_to_client`,
/// which fires an installed slot, so one slot catches every exit. The slot
/// guard borrows the registry, which is why this whole future runs inside
/// the caller's `SendWrapper` on shard 0.
-async fn produce_replicated(
+async fn partition_write_replicated(
state: &HttpInner,
session: &HttpSession,
+ operation: Operation,
body: &[u8],
-) -> Result<(), ProduceError> {
+) -> Result<(), PartitionWriteError> {
ensure_in_process_reply_target(state, session);
let request_id = session.next_data_request_id();
let message = build_request_message(
- Operation::SendMessages,
+ operation,
session.client_id,
session.session,
request_id,
@@ -1992,8 +2139,12 @@ async fn produce_replicated(
.clients()
.install_reply_slot(session.client_id, request_id)
.map_err(|error| {
- warn!(?error, "server-ng HTTP: produce reply slot install failed");
- ProduceError::Unavailable
+ warn!(
+ ?error,
+ ?operation,
+ "server-ng HTTP: partition write reply slot install failed"
+ );
+ PartitionWriteError::Unavailable
})?;
dispatch_partition_request(
&state.shard,
@@ -2003,7 +2154,7 @@ async fn produce_replicated(
session.client_id,
)
.await;
- let outcome = compio::time::timeout(PRODUCE_REPLY_TIMEOUT, receiver).await;
+ let outcome = compio::time::timeout(PARTITION_WRITE_REPLY_TIMEOUT, receiver).await;
// Removes the slot unless the reply already fired, so a late commit
// reply after a timeout sheds at the bus instead of leaking a waiter.
drop(guard);
@@ -2011,7 +2162,7 @@ async fn produce_replicated(
Ok(Ok(reply)) => classify_partition_reply(&reply),
// Cancelled (reply target torn down by session eviction mid-wait) or
// elapsed: same caller contract either way - outcome unknown, 504.
- Ok(Err(_)) | Err(_) => Err(ProduceError::Timeout),
+ Ok(Err(_)) | Err(_) => Err(PartitionWriteError::Timeout(operation)),
}
}
@@ -2059,24 +2210,24 @@ fn ensure_in_process_reply_target(state: &HttpInner, session: &HttpSession) {
}
}
-/// Discriminate a produce reply. Partition replies carry no result section
-/// and no error code - success and gate failure are both empty-bodied - so
-/// the only wire discriminator is the reply's `op`: a committed reply is
+/// Discriminate a partition write reply. Partition replies carry no result
+/// section and no error code (success and gate failure are both empty-bodied),
+/// so the only wire discriminator is the reply's `op`: a committed reply is
/// built from its prepare header, whose op (the partition group's commit
/// number) is always >= 1, while the pre-dispatch gate failures reply through
/// `build_empty_reply` with 0 in that field. The gate reply cannot name which
/// entity was missing, hence the generic legacy 404 body.
-fn classify_partition_reply(reply: &BusMessage) -> Result<(), ProduceError> {
+fn classify_partition_reply(reply: &BusMessage) -> Result<(), PartitionWriteError> {
let header = reply
.as_slice()
.get(..HEADER_SIZE)
.and_then(|bytes| bytemuck::checked::try_from_bytes::<ReplyHeader>(bytes).ok())
- .ok_or(ProduceError::Rejected(IggyError::InvalidCommand))?;
+ .ok_or(PartitionWriteError::Rejected(IggyError::InvalidCommand))?;
if header.command != Command2::Reply {
- return Err(ProduceError::Rejected(IggyError::InvalidCommand));
+ return Err(PartitionWriteError::Rejected(IggyError::InvalidCommand));
}
if header.op == 0 {
- return Err(ProduceError::NotFound);
+ return Err(PartitionWriteError::NotFound);
}
Ok(())
}
@@ -2620,6 +2771,83 @@ mod tests {
assert_eq!(wire.consumer.kind, 1);
}
+ // -- consumer-offset write wire mapping --
+
+ #[test]
+ fn store_offset_wire_request_round_trips_with_quorum_ack() {
+ let stream_id = Identifier::numeric(1).expect("valid stream id");
+ let topic_id = Identifier::named("orders").expect("valid topic id");
+ let command = StoreConsumerOffset {
+ consumer: Consumer::new(Identifier::named("c1").expect("valid id")),
+ partition_id: Some(1),
+ offset: 42,
+ };
+ let request = store_offset_wire_request(&stream_id, &topic_id, &command).expect("maps");
+ let bytes = request.to_bytes();
+ assert_eq!(*bytes.last().expect("non-empty"), AckLevel::Quorum.as_u8());
+ let (decoded, consumed) =
+ StoreConsumerOffset2Request::decode(&bytes).expect("decodes as the server does");
+ assert_eq!(consumed, bytes.len());
+ assert_eq!(decoded, request);
+ assert_eq!(decoded.consumer.kind, 1);
+ assert_eq!(decoded.partition_id, Some(1));
+ assert_eq!(decoded.offset, 42);
+ assert_eq!(decoded.ack, AckLevel::Quorum);
+ }
+
+ #[test]
+ fn store_offset_wire_request_passes_omitted_partition_through() {
+ let stream_id = Identifier::numeric(1).expect("valid stream id");
+ let topic_id = Identifier::numeric(2).expect("valid topic id");
+ let command = StoreConsumerOffset {
+ consumer: Consumer::new(Identifier::numeric(7).expect("valid id")),
+ partition_id: None,
+ offset: u64::MAX,
+ };
+ let request = store_offset_wire_request(&stream_id, &topic_id, &command).expect("maps");
+ let bytes = request.to_bytes();
+ let (decoded, consumed) =
+ StoreConsumerOffset2Request::decode(&bytes).expect("decodes as the server does");
+ assert_eq!(consumed, bytes.len());
+ assert_eq!(decoded.partition_id, None);
+ assert_eq!(decoded.offset, u64::MAX);
+ assert_eq!(decoded.ack, AckLevel::Quorum);
+ }
+
+ #[test]
+ fn delete_offset_wire_request_round_trips_partition_variants() {
+ let stream_id = Identifier::named("stream-1").expect("valid stream id");
+ let topic_id = Identifier::numeric(2).expect("valid topic id");
+ for partition_id in [Some(1), None] {
+ let request = delete_offset_wire_request(
+ &stream_id,
+ &topic_id,
+ &Consumer::new(Identifier::named("c1").expect("valid id")),
+ partition_id,
+ )
+ .expect("maps");
+ let bytes = request.to_bytes();
+ assert_eq!(*bytes.last().expect("non-empty"), AckLevel::Quorum.as_u8());
+ let (decoded, consumed) =
+ DeleteConsumerOffset2Request::decode(&bytes).expect("decodes as the server does");
+ assert_eq!(consumed, bytes.len());
+ assert_eq!(decoded, request);
+ assert_eq!(decoded.consumer.kind, 1);
+ assert_eq!(decoded.partition_id, partition_id);
+ assert_eq!(decoded.ack, AckLevel::Quorum);
+ }
+ }
+
+ #[test]
+ fn delete_offset_query_parses_optional_partition_id() {
+ let uri: Uri = "/x?partition_id=3".parse().expect("valid uri");
+ let Query(query) = Query::<DeleteConsumerOffset>::try_from_uri(&uri).expect("parses");
+ assert_eq!(query.partition_id, Some(3));
+ let bare: Uri = "/x".parse().expect("valid uri");
+ let Query(query) = Query::<DeleteConsumerOffset>::try_from_uri(&bare).expect("parses");
+ assert_eq!(query.partition_id, None);
+ }
+
// -- polled messages body decode --
/// Wrap one stored `SendMessages2` batch (`[256B header][blob]`) as the
@@ -2720,13 +2948,29 @@ mod tests {
assert!(classify_partition_reply(&reply).is_ok());
}
+ /// The 204 path of the offset write routes: a committed
+ /// `StoreConsumerOffset2` reply (op >= 1) classifies as success.
+ #[test]
+ fn committed_offset_write_reply_classifies_as_success() {
+ let prepare = PrepareHeader {
+ command: Command2::Prepare,
+ operation: Operation::StoreConsumerOffset2,
+ client: 42,
+ op: 3,
+ request: 2,
+ ..Default::default()
+ };
+ let reply = frozen(consensus::build_reply_message(&prepare, &Bytes::new()));
+ assert!(classify_partition_reply(&reply).is_ok());
+ }
+
#[test]
fn gate_failure_empty_reply_classifies_as_not_found() {
let request = build_request_message(Operation::SendMessages, 42, 7, 1, &[]);
let reply = frozen(build_empty_reply(request.header(), 42, 0, 9));
assert!(matches!(
classify_partition_reply(&reply),
- Err(ProduceError::NotFound)
+ Err(PartitionWriteError::NotFound)
));
}
@@ -2735,7 +2979,7 @@ mod tests {
let request = build_request_message(Operation::SendMessages, 42, 7, 1, &[]);
assert!(matches!(
classify_partition_reply(&request.into_generic().into_frozen()),
- Err(ProduceError::Rejected(IggyError::InvalidCommand))
+ Err(PartitionWriteError::Rejected(IggyError::InvalidCommand))
));
}
}
(iggy) 09/43: feat(server-ng): add HTTP consumer-group create/delete routes
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 4c5436e439c9485be6d18bd1605e88bcb8ec2be5
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 16:42:11 2026 +0200
feat(server-ng): add HTTP consumer-group create/delete routes
Add HTTP consumer-group create and delete under the topic path. Both
are mechanical clones of the create-topic / delete-topic handlers routed
through the shared submit_write helper, so they inherit the root-only
admin gate and the committed result-code business-error mapping for
free. Create decodes the committed ConsumerGroupDetailsResponse into
ConsumerGroupDetails (HTTP 200); delete returns 204.
---
core/server-ng/src/http.rs | 86 ++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 84 insertions(+), 2 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index ad3f3968f..7d3c7b467 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -37,6 +37,9 @@ use configs::http::HttpJwtConfig;
use iggy_binary_protocol::consensus::{
Command2, EvictionHeader, EvictionReason, HEADER_SIZE, result_code, result_section_len,
};
+use iggy_binary_protocol::requests::consumer_groups::{
+ CreateConsumerGroupRequest, DeleteConsumerGroupRequest,
+};
use iggy_binary_protocol::requests::personal_access_tokens::{
CreatePersonalAccessTokenRequest, DeletePersonalAccessTokenRequest,
};
@@ -50,6 +53,7 @@ use iggy_binary_protocol::requests::users::{
ChangePasswordRequest, CreateUserRequest, DeleteUserRequest, UpdatePermissionsRequest,
UpdateUserRequest,
};
+use iggy_binary_protocol::responses::consumer_groups::get_consumer_group::ConsumerGroupDetailsResponse;
use iggy_binary_protocol::responses::personal_access_tokens::RawPersonalAccessTokenResponse;
use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
use iggy_binary_protocol::responses::topics::get_topic::GetTopicResponse;
@@ -59,6 +63,7 @@ use iggy_binary_protocol::{
GenericHeader, Operation, RequestHeader, WireDecode, WireEncode, WireName,
};
use iggy_common::change_password::ChangePassword;
+use iggy_common::create_consumer_group::CreateConsumerGroup;
use iggy_common::create_personal_access_token::CreatePersonalAccessToken;
use iggy_common::create_stream::CreateStream;
use iggy_common::create_topic::CreateTopic;
@@ -72,8 +77,8 @@ use iggy_common::update_topic::UpdateTopic;
use iggy_common::update_user::UpdateUser;
use iggy_common::wire_conversions::{identifier_to_wire, permissions_to_wire};
use iggy_common::{
- Identifier, IdentityInfo, IggyError, IggyTimestamp, RawPersonalAccessToken, StreamDetails,
- TokenInfo, TopicDetails, UserInfoDetails, Validatable,
+ ConsumerGroupDetails, Identifier, IdentityInfo, IggyError, IggyTimestamp,
+ RawPersonalAccessToken, StreamDetails, TokenInfo, TopicDetails, UserInfoDetails, Validatable,
};
use message_bus::client_listener;
use secrecy::ExposeSecret;
@@ -389,6 +394,14 @@ fn router(state: HttpState) -> Router {
"/streams/{stream_id}/topics/{topic_id}/purge",
delete(purge_topic),
)
+ .route(
+ "/streams/{stream_id}/topics/{topic_id}/consumer-groups",
+ post(create_cg),
+ )
+ .route(
+ "/streams/{stream_id}/topics/{topic_id}/consumer-groups/{group_id}",
+ delete(delete_cg),
+ )
.route("/personal-access-tokens", post(create_pat))
.route("/personal-access-tokens/{name}", delete(delete_pat))
.with_state(state)
@@ -643,6 +656,65 @@ async fn purge_topic(
Ok(StatusCode::NO_CONTENT)
}
+/// `POST /streams/{stream_id}/topics/{topic_id}/consumer-groups`: create a
+/// consumer group under a topic and render the committed reply as the same
+/// `ConsumerGroupDetails` JSON the legacy server returns.
+///
+/// The stream and topic come from the path; the JSON body is name-only
+/// (`{"name": ...}`), matching the legacy request. The submitted op is a plain
+/// `CreateConsumerGroup`; the metadata owner assigns the group id, so this
+/// handler never allocates one itself.
+async fn create_cg(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id)): Path<(String, String)>,
+ Json(command): Json<CreateConsumerGroup>,
+) -> Result<Json<ConsumerGroupDetails>, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(WriteError::Rejected)?;
+ let request = CreateConsumerGroupRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(WriteError::Rejected)?,
+ name: WireName::new(command.name)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidConsumerGroupName))?,
+ };
+ let body = request.to_bytes();
+ let payload = SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::CreateConsumerGroup,
+ &body,
+ ))
+ .await?;
+ Ok(Json(decode_consumer_group_details(&payload)?))
+}
+
+/// `DELETE /streams/{stream_id}/topics/{topic_id}/consumer-groups/{group_id}`:
+/// delete a consumer group. Returns 204 on commit.
+async fn delete_cg(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path((stream_id, topic_id, group_id)): Path<(String, String, String)>,
+) -> Result<StatusCode, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ let topic_id = Identifier::from_str_value(&topic_id).map_err(WriteError::Rejected)?;
+ let group_id = Identifier::from_str_value(&group_id).map_err(WriteError::Rejected)?;
+ let request = DeleteConsumerGroupRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ topic_id: identifier_to_wire(&topic_id).map_err(WriteError::Rejected)?,
+ group_id: identifier_to_wire(&group_id).map_err(WriteError::Rejected)?,
+ };
+ let body = request.to_bytes();
+ SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::DeleteConsumerGroup,
+ &body,
+ ))
+ .await?;
+ Ok(StatusCode::NO_CONTENT)
+}
+
/// `POST /users`: create a user and render the committed reply as the same
/// `UserInfoDetails` JSON the legacy server returns.
///
@@ -1001,6 +1073,16 @@ fn decode_user_details(payload: &[u8]) -> Result<UserInfoDetails, WriteError> {
UserInfoDetails::try_from(response).map_err(WriteError::Rejected)
}
+/// Decode the `ConsumerGroupDetailsResponse` payload of a committed
+/// create-consumer-group reply into `ConsumerGroupDetails`. `payload` is the
+/// slice past the result section that [`submit_write`] already validated as a
+/// success. The wire-to-domain conversion is infallible.
+fn decode_consumer_group_details(payload: &[u8]) -> Result<ConsumerGroupDetails, WriteError> {
+ let response = ConsumerGroupDetailsResponse::decode_from(payload)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
+ Ok(ConsumerGroupDetails::from(response))
+}
+
/// Extract the raw one-time token from a [`build_raw_pat_reply`] output. That
/// reply's body is a bare `RawPersonalAccessTokenResponse` (no leading result
/// section, unlike a committed metadata reply), so decode it straight past the
(iggy) 05/43: feat(server-ng): add HTTP create-topic route
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit c93619e763524019bc4ddd2dd9edf2b980132f1c
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Wed Jul 1 14:21:43 2026 +0200
feat(server-ng): add HTTP create-topic route
server-ng exposed topic update/delete/purge over HTTP but
not creation, leaving a management-parity hole.
A client CreateTopic is already converted to
CreateTopicWithAssignments at prepare time, where the
metadata owner allocates the consensus group ids. Reuse
that path: the handler submits a plain CreateTopic and
decodes the committed GetTopicResponse into TopicDetails,
so no metadata or consensus code changes. Allocation stays
on the primary's serialized prepare step, keeping ids
collision-free without a new state-machine handler.
---
core/server-ng/src/http.rs | 54 ++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 52 insertions(+), 2 deletions(-)
diff --git a/core/server-ng/src/http.rs b/core/server-ng/src/http.rs
index f2f52868b..f42a129ce 100644
--- a/core/server-ng/src/http.rs
+++ b/core/server-ng/src/http.rs
@@ -41,21 +41,24 @@ use iggy_binary_protocol::requests::streams::{
CreateStreamRequest, DeleteStreamRequest, PurgeStreamRequest, UpdateStreamRequest,
};
use iggy_binary_protocol::requests::topics::{
- DeleteTopicRequest, PurgeTopicRequest, UpdateTopicRequest,
+ CreateTopicRequest, DeleteTopicRequest, PurgeTopicRequest, UpdateTopicRequest,
};
use iggy_binary_protocol::responses::streams::get_stream::GetStreamResponse;
+use iggy_binary_protocol::responses::topics::get_topic::GetTopicResponse;
use iggy_binary_protocol::version::IGGY_PROTOCOL_VERSION;
use iggy_binary_protocol::{
GenericHeader, Operation, RequestHeader, WireDecode, WireEncode, WireName,
};
use iggy_common::create_stream::CreateStream;
+use iggy_common::create_topic::CreateTopic;
use iggy_common::login_user::LoginUser;
use iggy_common::login_with_personal_access_token::LoginWithPersonalAccessToken;
use iggy_common::update_stream::UpdateStream;
use iggy_common::update_topic::UpdateTopic;
use iggy_common::wire_conversions::identifier_to_wire;
use iggy_common::{
- Identifier, IdentityInfo, IggyError, IggyTimestamp, StreamDetails, TokenInfo, Validatable,
+ Identifier, IdentityInfo, IggyError, IggyTimestamp, StreamDetails, TokenInfo, TopicDetails,
+ Validatable,
};
use message_bus::client_listener;
use secrecy::ExposeSecret;
@@ -356,6 +359,7 @@ fn router(state: HttpState) -> Router {
put(update_stream).delete(delete_stream),
)
.route("/streams/{stream_id}/purge", delete(purge_stream))
+ .route("/streams/{stream_id}/topics", post(create_topic))
.route(
"/streams/{stream_id}/topics/{topic_id}",
put(update_topic).delete(delete_topic),
@@ -500,6 +504,43 @@ async fn purge_stream(
Ok(StatusCode::NO_CONTENT)
}
+/// `POST /streams/{stream_id}/topics`: create a topic under a stream and render
+/// the committed reply as the same `TopicDetails` JSON the legacy server returns.
+///
+/// The stream comes from the path; the JSON body carries the remaining fields.
+/// The submitted op is a plain `CreateTopic`; the metadata owner allocates the
+/// consensus group ids and rewrites it to `CreateTopicWithAssignments` before
+/// replication, so this handler stays a pure submit-and-decode.
+async fn create_topic(
+ State(state): State<HttpState>,
+ identity: Authenticated,
+ Path(stream_id): Path<String>,
+ Json(command): Json<CreateTopic>,
+) -> Result<Json<TopicDetails>, WriteError> {
+ let stream_id = Identifier::from_str_value(&stream_id).map_err(WriteError::Rejected)?;
+ // Rejects empty/oversized name, partitions_count > MAX, replication_factor == Some(0).
+ command.validate().map_err(WriteError::Rejected)?;
+ let request = CreateTopicRequest {
+ stream_id: identifier_to_wire(&stream_id).map_err(WriteError::Rejected)?,
+ partitions_count: command.partitions_count,
+ compression_algorithm: command.compression_algorithm.as_code(),
+ message_expiry: command.message_expiry.into(),
+ max_topic_size: command.max_topic_size.into(),
+ replication_factor: command.replication_factor.unwrap_or(0),
+ name: WireName::new(command.name)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidTopicName))?,
+ };
+ let body = request.to_bytes();
+ let payload = SendWrapper::new(submit_write(
+ &state,
+ &identity.session,
+ Operation::CreateTopic,
+ &body,
+ ))
+ .await?;
+ Ok(Json(decode_topic_details(&payload)?))
+}
+
/// `PUT /streams/{stream_id}/topics/{topic_id}`: update a topic. Returns 204.
async fn update_topic(
State(state): State<HttpState>,
@@ -676,6 +717,15 @@ fn decode_stream_details(payload: &[u8]) -> Result<StreamDetails, WriteError> {
StreamDetails::try_from(response).map_err(WriteError::Rejected)
}
+/// Decode the `GetTopicResponse` payload of a committed create-topic reply into
+/// `TopicDetails`. `payload` is the slice past the result section that
+/// [`submit_write`] already validated as a success.
+fn decode_topic_details(payload: &[u8]) -> Result<TopicDetails, WriteError> {
+ let response = GetTopicResponse::decode_from(payload)
+ .map_err(|_| WriteError::Rejected(IggyError::InvalidCommand))?;
+ TopicDetails::try_from(response).map_err(WriteError::Rejected)
+}
+
/// Map an eviction frame to the same typed [`IggyError`] the SDK's
/// `decode_eviction` produces, so an HTTP caller sees the identical status a TCP
/// caller would (session-terminal reasons render as 401 -> re-authenticate).
(iggy) 40/43: fix(server-ng): order topic-not-found identifiers per display contract
Posted by hg...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
hubcio pushed a commit to branch http-listener-rebased
in repository https://gitbox.apache.org/repos/asf/iggy.git
commit 45e1675281dc00d42a54bac0536906929d54d69f
Author: Hubert Gruszecki <h....@gmail.com>
AuthorDate: Thu Jul 2 22:22:49 2026 +0200
fix(server-ng): order topic-not-found identifiers per display contract
IggyError::TopicIdNotFound renders "Topic with ID: {0} for stream with
ID: {1}", so the topic identifier goes first; the helper had them
swapped (copied from the legacy call sites, which carry the same
quirk).
---
core/server-ng/src/responses.rs | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/core/server-ng/src/responses.rs b/core/server-ng/src/responses.rs
index 26946adb1..ed7e3225a 100644
--- a/core/server-ng/src/responses.rs
+++ b/core/server-ng/src/responses.rs
@@ -708,11 +708,12 @@ fn stream_not_found(stream_id: &WireIdentifier) -> IggyError {
IggyError::StreamIdNotFound(wire_id_to_identifier(stream_id).unwrap_or_default())
}
-/// Typed miss for a read's parent topic; see [`stream_not_found`].
+/// Typed miss for a read's parent topic; see [`stream_not_found`]. The variant's
+/// display order is (topic, stream).
fn topic_not_found(stream_id: &WireIdentifier, topic_id: &WireIdentifier) -> IggyError {
IggyError::TopicIdNotFound(
- wire_id_to_identifier(stream_id).unwrap_or_default(),
wire_id_to_identifier(topic_id).unwrap_or_default(),
+ wire_id_to_identifier(stream_id).unwrap_or_default(),
)
}