[PR] Feat(gateways)/3534 3538 kafka metadata createtopics [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla opened a new pull request, #4258:
URL: https://github.com/apache/iggy/pull/4258

   ##  Summary
   
     Wires the Kafka gateway's CreateTopics (#3538) and Metadata (#3534) handlers to the real Iggy bridge, instead of stub responses. Combined into one PR/branch because both touch IggyBridge::get_kafka_topic —
     merging up front eliminates the duplicate-implementation risk a sequential rebase would have carried, rather than just deferring it.
   
     With IGGY_KAFKA_BRIDGE_ENABLED=true:
     - CreateTopics creates a real Iggy stream/topic, atomically reporting AlreadyExists instead of racing a check-then-create.
     - Metadata reports real topics and partition counts pulled from the bridge.
   
     With the bridge off (default), both remain stubs as before (CreateTopics → NOT_CONTROLLER, Metadata → every topic unknown).
   
     Closes #3534, closes #3538.
     Related to #3560 
   
   
     ## What's included
   
     - IggyBridge::get_kafka_topic — single canonical existence/detail lookup, shared by both handlers (previously would have been two separate, differently-typed implementations).
     - IggyBridge::create_kafka_topic → TopicCreationOutcome::{Created, AlreadyExists} — atomic create-or-report-exists
     - IggyBridge::list_kafka_topics — enumerates override targets and default-stream topics for "list all," deduplicated.
     - Kafka wire error codes tightened: ERROR_INVALID_REQUEST(42), ERROR_TOPIC_ALREADY_EXISTS(36), ERROR_INVALID_CONFIG(40), ERROR_REQUEST_TIMED_OUT(7). Removed the incorrect ERROR_INVALID_REPLICA_ASSIGNMENT(39).
     - Bridge fan-out hardening, applied identically to both handlers: dedup requested topic names before any bridge call, MAX_BRIDGE_BACKED_TOPICS = 100 cap (rejected wholesale as INVALID_REQUEST, no bridge call at
     all above the cap), one wall-clock deadline over the request's aggregate bridge work (CreateTopics uses the wire timeout_ms, clamped [1s, 30s]; Metadata uses a fixed 20s, no timeout_ms field in its supported
     versions).
     - Metadata: fixed v0 isAllTopics() semantics (topics == null || (topics.isEmpty() && version == 0)); added a post-bridge-call, pre-encode frame-size check (response_would_exceed_frame_size) since partition-count
     expansion (up to 1000/topic) isn't priced by the pre-decode element-count bound in bounds_guard.rs.
   
    
    ## Test plan
   
     - cargo build -p iggy-gateway-kafka --all-targets — clean.
     - cargo fmt --all, cargo sort --no-format --workspace — clean.
     - cargo clippy -p iggy-gateway-kafka --all-features --all-targets -- -D warnings — no issues.
     - cargo test -p iggy-gateway-kafka — 332 passed, 16 suites (includes new create_topics_real_bridge_tests.rs, metadata_real_bridge_tests.rs, and 4 new list_kafka_topics_* cases appended to
     bridge_iggy_integration_tests.rs).
     - ./scripts/ci/{trailing-whitespace,trailing-newline,license-headers,markdownlint}.sh — clean.
     - git diff --check — no stray whitespace.
   
   
   
   ## AI Usage
   
   If AI tools were used, please answer:
   1. Which tools?  Claude code for implementation , Cursor for review and validation
   2. Scope of usage?  Implementation , review and validation
   3. How did you verify the generated code works correctly? Test Suite and Code walkthrough
   4. Can you explain every line of the code if asked? Yes
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4077904722


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,203 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
+        }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => topics.into_iter().map(found_result).collect(),
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => {
+            let distinct_names: HashSet<&str> = names.iter().map(StrBytes::as_str).collect();
+            if distinct_names.len() > MAX_BRIDGE_BACKED_TOPICS {
+                tracing::warn!(
+                    distinct_topics = distinct_names.len(),
+                    max = MAX_BRIDGE_BACKED_TOPICS,
+                    "Metadata request addresses too many distinct topics; rejecting"
+                );
+                names
+                    .iter()
+                    .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+                    .collect()
+            } else {
+                match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &names))
+                    .await
+                {
+                    Ok(results) => results,
+                    Err(_elapsed) => {
+                        tracing::warn!(
+                            distinct_topics = distinct_names.len(),
+                            deadline_secs = REQUEST_DEADLINE.as_secs(),
+                            "Metadata request's aggregate bridge work exceeded its deadline; \
+                             answering retriable instead of blocking further"
+                        );
+                        names
+                            .iter()
+                            .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                            .collect()
+                    }
+                }
+            }
         }
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names, or one `list_kafka_topics()` call, can still expand into a
+    // response `bounds_guard` never had the information to price in before this bridge round
+    // trip returned. Checked here, before `encode_real_response` builds one
+    // `MetadataResponsePartition` per partition, not after - the expensive part is building that
+    // `Vec`, not encoding the bytes that follow it.
+    let total_partitions: usize = results
+        .iter()
+        .filter(|result| result.error_code == ERROR_NONE)
+        .map(|result| result.partitions_count as usize)
+        .sum();
+    if response_would_exceed_frame_size(total_partitions, state.max_frame_size) {
+        tracing::warn!(
+            total_partitions,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;

Review Comment:
   All-topics Metadata response now truncates to fit max_frame_size (keeping whole topics, listing order) instead of closing the connection — closing was a permanent,  cluster-size-triggered outage for every client's bootstrap call. Named-lookup path still closes (unchanged, client-bounded via the 100-topic cap).  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on PR #4258:
URL: https://github.com/apache/iggy/pull/4258#issuecomment-5824996395

   /ready


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4088474558


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,378 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39), the one condition that code exists for. This bridge doesn't
+/// model per-partition replica placement, so only the index set is checked, not each entry's
+/// replica list.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments) {

Review Comment:
   New assignment_replicas_are_valid() — this gateway has exactly one broker, so the only legal replica  list is [1]; that single check rules out empty, duplicate, and unregistered in one shot. Fixed the   test helper (broker_id 0 → 1) and added 3 negative unit tests matching the reviewer's exact repro     shapes. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093427380


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge

Review Comment:
    Fixed: added to README's stub-warning paragraph — "CreateTopics has no authentication gate yet: with the bridge on, any client that can reach this port can create topics (up to 1000 partitions each) as the
     bridge's own Iggy user, until SASL (#3549) lands."



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093524576


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());

Review Comment:
     Fixed: result is now built with .with_error_message(None) explicitly at the top of create_one_topic, so every branch that doesn't deliberately set text (duplicate-name rejection, ALREADY_EXISTS, the new
     existence-wins-over-rejection branch) carries None instead of an empty string. Also applied the same explicit None to the two early-reject blocks in handle() (topic-cap rejection, duplicate-name rejection in
     create_all_topics) that had the same gap.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4088476645


##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,140 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets - checked so an overridden topic is never listed twice, once under its Kafka-side
+    /// name and once under its raw Iggy name.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {
+                results.push(KafkaTopicMetadata {
+                    kafka_topic: kafka_topic.to_string(),
+                    partitions_count: details.partitions_count,
+                });
+            }
+        }
+
+        let default_stream_id = Identifier::named(default_stream).map_err(BridgeError::Iggy)?;
+        if with_request_timeout(self.client.get_stream(&default_stream_id))
+            .await?
+            .is_some()
+        {
+            let topics = with_request_timeout(self.client.get_topics(&default_stream_id)).await?;
+            for topic in topics {
+                if default_stream_override_targets.contains(topic.name.as_str()) {

Review Comment:
   Now also collects override keys and skips default-stream topics matching either set.    



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093507333


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result

Review Comment:
     Fixed in both places. bridge_error_result (create_topics.rs) logs client-caused variants at debug!, everything else at error!. get_kafka_topics doesn't have a per-name error arm anymore (it's one batched call), so the split has  at the aggregate level: a get_kafka_topics failure logs once at error! and answers every
     distinct name that error's code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["numinnex (via GitHub)" <gi...@apache.org>]

numinnex commented on PR #4258:
URL: https://github.com/apache/iggy/pull/4258#issuecomment-5832958664

   @ryerraguntla resolve the conflict and from my side it's good to go. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093373592


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39). Each entry's own replica list is checked too, not just the
+/// index: this gateway advertises exactly one broker (node id 1, `metadata.rs`'s `BrokerId(1)`),
+/// so the only legal replica list is `[1]` - never empty, never repeating it, never naming a
+/// broker Metadata never advertised. Real Kafka's `ReplicationControlManager` validates the whole
+/// binding this way, not just the index.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments)
+            || !assignment_replicas_are_valid(&topic.assignments)
+        {
+            return Err(ERROR_INVALID_REPLICA_ASSIGNMENT);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));
+    }
+
+    let broker_default_ok = version >= 4;
+
+    let partitions_ok = if broker_default_ok {
+        topic.num_partitions == -1 || topic.num_partitions > 0
+    } else {
+        topic.num_partitions > 0
+    };
+    if !partitions_ok {
+        return Err(ERROR_INVALID_PARTITIONS);
+    }
+
+    let replication_ok = if broker_default_ok {

Review Comment:
     Fixed as suggested: numeric-path replication_ok now requires exactly 1 (or the -1 broker-default sentinel from v4+), matching the manual-assignment branch's existing [1]-only rule. Also means success()'s
     hardcoded .with_replication_factor(1) is no longer a potential lie — it's now always the truthful value. 2 new tests.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093490548


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct.len(),
+                deadline_secs = REQUEST_DEADLINE.as_secs(),
+                "Metadata request's aggregate bridge work exceeded its deadline; answering \
+                 retriable instead of blocking further"
+            );
+            distinct
+                .iter()
+                .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                .collect()
+        }
+    }
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;

Review Comment:
     Fixed , template_partition_bytes(version) now calls MetadataResponsePartition::compute_size(version) on a template shaped like the real encode (1 replica, 1 isr, empty offline), replacing the
     flat constant. Kept the old value as RESPONSE_BYTES_PER_PARTITION_FALLBACK, used only if compute_size somehow errors. 2 new tests confirm every version's real value stays at/under the old flat charge .



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093338755


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {

Review Comment:
     Fixed as a side effect of #1: the named-lookup path is now one batched get_kafka_topics call under one timeout, so there's no longer a per-name loop whose partial results could be discarded. (I'd initially fixed
     this the way you described — per-name timeout_at — before landing on the batching redesign for #1, which made that scaffolding unnecessary; removed it.)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093357051


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39). Each entry's own replica list is checked too, not just the
+/// index: this gateway advertises exactly one broker (node id 1, `metadata.rs`'s `BrokerId(1)`),
+/// so the only legal replica list is `[1]` - never empty, never repeating it, never naming a
+/// broker Metadata never advertised. Real Kafka's `ReplicationControlManager` validates the whole
+/// binding this way, not just the index.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments)
+            || !assignment_replicas_are_valid(&topic.assignments)
+        {
+            return Err(ERROR_INVALID_REPLICA_ASSIGNMENT);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));
+    }
+
+    let broker_default_ok = version >= 4;
+
+    let partitions_ok = if broker_default_ok {

Review Comment:
     Fixed in the shared validate_create_topic_shape (used by both validate_only and the real path), so both now reject num_partitions > 1000 the same way, with the same code the real path's TooManyPartitions bridge
     error already maps to (ERROR_INVALID_REQUEST, 42) — not INVALID_PARTITIONS (37), which means the opposite condition ("below 1"). Added MAX_PARTITIONS_COUNT const and 2 tests
     (partition_count_at_the_cap_is_accepted, partition_count_above_the_cap_is_rejected_with_the_real_creates_own_code).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093468469


##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets and isn't itself named the same as an override key - checked so an overridden
+    /// topic is never listed twice, once under its Kafka-side name and once under its raw Iggy
+    /// name, and so a raw default-stream topic never masquerades under a Kafka-side name an
+    /// override has already claimed for different data. The second check matters for a chained
+    /// override (`foo -> (kafka, bar)`, `bar -> (other, x)`): without it, a raw Iggy topic
+    /// literally named `foo` sitting in the default stream would be reported a second time under
+    /// the same `foo` name the override loop already emitted (backed by `kafka/bar`'s data), and
+    /// that second `foo` would be unreachable by name anyway, since `get_kafka_topic("foo")`
+    /// always resolves through the override to `kafka/bar`, never to the raw `kafka/foo`.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut override_keys: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            override_keys.insert(kafka_topic);
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {
+                results.push(KafkaTopicMetadata {
+                    kafka_topic: kafka_topic.to_string(),
+                    partitions_count: details.partitions_count,
+                });
+            }
+        }
+
+        let default_stream_id = Identifier::named(default_stream).map_err(BridgeError::Iggy)?;
+        if with_request_timeout(self.client.get_stream(&default_stream_id))
+            .await?
+            .is_some()
+        {
+            let topics = with_request_timeout(self.client.get_topics(&default_stream_id)).await?;
+            for topic in topics {

Review Comment:
     Fixed: list_kafka_topics now runs validate_kafka_topic_name on every raw default-stream topic before listing it, skipping (and debug!-logging) anything that fails, same gate every named path already enforces.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4088468066


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,249 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
+        }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_partitions: usize = results
+        .iter()
+        .filter(|result| result.error_code == ERROR_NONE)
+        .map(|result| result.partitions_count as usize)
+        .sum();
+    if response_would_exceed_frame_size(total_partitions, state.max_frame_size) {
+        tracing::warn!(
+            total_partitions,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
+    }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct.len(),
+                deadline_secs = REQUEST_DEADLINE.as_secs(),
+                "Metadata request's aggregate bridge work exceeded its deadline; answering \
+                 retriable instead of blocking further"
+            );
+            distinct
+                .iter()
+                .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                .collect()
+        }
+    }
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;
+
+const fn response_would_exceed_frame_size(total_partitions: usize, max_frame_size: usize) -> bool {
+    total_partitions.saturating_mul(RESPONSE_BYTES_PER_PARTITION) > max_frame_size
+}
+
+/// Trims an all-topics [`IggyBridge::list_kafka_topics`] result to fit `max_frame_size`, keeping
+/// as many whole topics (in listing order) as the budget allows.
+///
+/// Unlike the named-lookup arm, the all-topics response's size is a server-side property (the
+/// cluster's total partition count) that the requesting client never chose and cannot shrink -
+/// closing the connection over it, as the shared frame-size guard below does for the
+/// client-controllable named-lookup case, would make every all-topics Metadata call fail
+/// identically and permanently once the catalog crosses the trip point. That call is the
+/// bootstrap and refresh shape both librdkafka and the Java client use, so a hard close reads as
+/// "broker down" and reconnect-loops rather than surfacing a usable, if partial, result. This
+/// wire protocol has no pagination cursor to ask for the rest with, so a truncated list - honest
+/// about being incomplete via the dropped entries, not via an error code - is what's available.
+fn truncate_all_topics_to_frame_budget(
+    mut results: Vec<TopicResult>,
+    max_frame_size: usize,
+) -> Vec<TopicResult> {
+    let mut cumulative_partitions = 0usize;
+    let mut keep = results.len();
+    for (index, result) in results.iter().enumerate() {
+        if result.error_code != ERROR_NONE {
+            continue;
+        }
+        let next = cumulative_partitions + result.partitions_count as usize;

Review Comment:
   New estimated_response_bytes() charges fixed per-topic overhead (13 bytes) + actual name length +  partition cost — and charges it for every result, not just ERROR_NONE ones (error results still  encode a full wrapper with an empty partitions array). Used by both the named-lookup guard and the    all-topics truncation.   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on PR #4258:
URL: https://github.com/apache/iggy/pull/4258#issuecomment-5805043545

   /ready
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["tdwells90 (via GitHub)" <gi...@apache.org>]

tdwells90 commented on PR #4258:
URL: https://github.com/apache/iggy/pull/4258#issuecomment-5836670243

   While Iggy is unreachable, a named Metadata lookup answers each topic `NOT_LEADER_OR_FOLLOWER` with no partitions. librdkafka 1.x treats any topic error with no partitions, other than an unknown or invalid topic, as permanent and fails every queued record, so producers lose records within seconds of an Iggy outage, whatever their delivery timeout.
   
   We patched our copy to remember each topic's partition count whenever Iggy describes it, answer from that while Iggy can't be asked, and send `LEADER_NOT_AVAILABLE` for a topic not seen yet. Sends then fail per partition, which clients retry. Happy to open a follow-up PR once this lands.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on PR #4258:
URL: https://github.com/apache/iggy/pull/4258#issuecomment-5824995371

   /ready
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093430432


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct.len(),
+                deadline_secs = REQUEST_DEADLINE.as_secs(),
+                "Metadata request's aggregate bridge work exceeded its deadline; answering \
+                 retriable instead of blocking further"
+            );
+            distinct
+                .iter()
+                .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                .collect()
+        }
+    }
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;
+
+/// Fixed per-`MetadataResponseTopic` cost, independent of name length or partition count:
+/// `error_code`(2) + `is_internal`(1) + the `partitions` array's own compact-length varint
+/// (charged at its 5-byte worst case) + `topic_authorized_operations`(4) + empty tagged fields(1).
+/// Every result pays this once - `encode_real_response` emits a full wrapper even for an error
+/// result, just with an empty `partitions` array, so this cost isn't conditional on `ERROR_NONE`.
+const RESPONSE_BYTES_TOPIC_OVERHEAD: usize = 13;
+
+/// Estimated encoded byte cost of one [`TopicResult`]: the fixed per-topic overhead, the name's
+/// own bytes (compact string: length plus a short varint prefix, charged at a flat +2), and -
+/// only for a successful result, since `encode_real_response` sends an empty array otherwise -
+/// [`RESPONSE_BYTES_PER_PARTITION`] per partition.
+///
+/// `bounds_guard` cannot see any of this: it charges the request by name count alone, with no way
+/// to know a name's length, a topic's per-entry overhead, or its partition count before this
+/// bridge round trip returns real data. Charging partitions alone (this function's earlier form)
+/// undercounted every result: a name costs bytes whether or not the lookup succeeded, and a
+/// zero-partition or errored result - free under a partitions-only charge - still costs a full
+/// wrapper to encode. 500 topics with 255-byte names and one partition each encode to ~146 KB
+/// against the ~32 KB a partitions-only charge would have priced in.
+fn estimated_response_bytes(result: &TopicResult) -> usize {
+    let name_bytes = result.name.as_str().len() + 2;
+    let partition_bytes = if result.error_code == ERROR_NONE {
+        (result.partitions_count as usize).saturating_mul(RESPONSE_BYTES_PER_PARTITION)
+    } else {
+        0
+    };
+    RESPONSE_BYTES_TOPIC_OVERHEAD + name_bytes + partition_bytes
+}
+
+const fn response_would_exceed_frame_size(total_bytes: usize, max_frame_size: usize) -> bool {
+    total_bytes > max_frame_size
+}
+
+/// Trims an all-topics [`IggyBridge::list_kafka_topics`] result to fit `max_frame_size`, keeping
+/// as many whole topics (in listing order) as the budget allows.
+///
+/// Unlike the named-lookup arm, the all-topics response's size is a server-side property (the
+/// cluster's total partition count) that the requesting client never chose and cannot shrink -
+/// closing the connection over it, as the shared frame-size guard below does for the
+/// client-controllable named-lookup case, would make every all-topics Metadata call fail
+/// identically and permanently once the catalog crosses the trip point. That call is the
+/// bootstrap and refresh shape both librdkafka and the Java client use, so a hard close reads as
+/// "broker down" and reconnect-loops rather than surfacing a usable, if partial, result. This
+/// wire protocol has no pagination cursor to ask for the rest with, so a truncated list - honest
+/// about being incomplete via the dropped entries, not via an error code - is what's available.
+fn truncate_all_topics_to_frame_budget(
+    mut results: Vec<TopicResult>,
+    max_frame_size: usize,
+) -> Vec<TopicResult> {
+    let mut cumulative_bytes = 0usize;
+    let mut keep = results.len();
+    for (index, result) in results.iter().enumerate() {
+        let next = cumulative_bytes + estimated_response_bytes(result);
+        if response_would_exceed_frame_size(next, max_frame_size) {
+            keep = index;
+            break;
+        }
+        cumulative_bytes = next;
+    }
+    if keep < results.len() {
+        tracing::warn!(
+            total_topics = results.len(),
+            kept_topics = keep,
+            max_frame_size,
+            "All-topics Metadata response would exceed max_frame_size; truncating rather than \
+             closing the connection"
+        );
+        results.truncate(keep);
+    }
+    results
+}
+
+/// One resolved topic result for the real (bridge-backed) path - `partitions_count` is
+/// meaningless when `error_code != ERROR_NONE`.
+struct TopicResult {
+    name: StrBytes,
+    error_code: i16,
+    partitions_count: u32,
+}
+
+fn found_result(metadata: crate::bridge::KafkaTopicMetadata) -> TopicResult {
+    TopicResult {
+        name: StrBytes::from_string(metadata.kafka_topic),
+        error_code: ERROR_NONE,
+        partitions_count: metadata.partitions_count,
+    }
+}
+
+async fn lookup_one_topic(bridge: &IggyBridge, name: StrBytes) -> TopicResult {
+    match bridge.get_kafka_topic(&name).await {

Review Comment:
     Fixed, but via #1's batching redesign rather than a cache. A short-TTL cache would have needed its own invalidation story (this file's own doc comment on ensure_stream_and_topic already argues against a naive
     cache there for exactly that reason), and batching by stream removes the serialization cost without needing one at all — this call site no longer does one get_topic per name.
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093464112


##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets and isn't itself named the same as an override key - checked so an overridden
+    /// topic is never listed twice, once under its Kafka-side name and once under its raw Iggy
+    /// name, and so a raw default-stream topic never masquerades under a Kafka-side name an
+    /// override has already claimed for different data. The second check matters for a chained
+    /// override (`foo -> (kafka, bar)`, `bar -> (other, x)`): without it, a raw Iggy topic
+    /// literally named `foo` sitting in the default stream would be reported a second time under
+    /// the same `foo` name the override loop already emitted (backed by `kafka/bar`'s data), and
+    /// that second `foo` would be unreachable by name anyway, since `get_kafka_topic("foo")`
+    /// always resolves through the override to `kafka/bar`, never to the raw `kafka/foo`.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut override_keys: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            override_keys.insert(kafka_topic);
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {

Review Comment:
     Fixed: the override loop now matches on the lookup result — Unauthorized is skipped (logged at debug!) and the loop continues; every other error kind still propagates via ? (those aren't per-topic facts).
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093512306


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),

Review Comment:
     Fixed: new bridge_error_result helper sends only fixed, known-correct text to the client (the two client-caused variants' own precise strings, or a generic "internal error provisioning this topic" for anything
     bridge/Iggy-side) — never err.to_string()'s embedded detail, which can be wrong for an error reconstructed from a bare wire status code.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093482057


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -111,6 +379,51 @@ pub fn encode_response(
     encode_message(&resp, response_version, 256)
 }
 
+/// Real (bridge-backed) response: unlike [`encode_response`], each topic carries its own
+/// resolved error code and, on success, one [`MetadataResponsePartition`] per partition with
+/// this gateway's single broker (node id 1) as leader/replica/ISR - there is only ever one
+/// broker behind this gateway, so that triple is never actually in question.
+fn encode_real_response(
+    response_version: i16,
+    results: &[TopicResult],
+    broker: &BrokerAdvertise,
+) -> Result<Bytes> {
+    let response_topics = results
+        .iter()
+        .map(|result| {
+            let partitions = if result.error_code == ERROR_NONE {

Review Comment:
     Fixed . above BLOCKING_ENCODE_PARTITION_THRESHOLD (20,000 partitions total across the response — picked conservatively off one measured data point, not profiled, flagged for a
     real benchmark), encode_real_response runs via tokio::task::spawn_blocking instead of inline. Below the threshold, unchanged (no thread-hop cost added to the common case). 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["krishvishal (via GitHub)" <gi...@apache.org>]

krishvishal commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4091545989


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(

Review Comment:
   One `timeout` covers the whole batch, so on expiry every topic answers `REQUEST_TIMED_OUT`, even the topics that this request created.
   Fix: bound each bridge call with `timeout_at` against one deadline, and keep the finished results, as `list_offsets.rs` does.



##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {

Review Comment:
   Past 100 names, every topic fails, but a Java producer resends its full topic set (`ProducerMetadata.java:62-63`), so it never recovers.
   Fix: resolve the names with one `get_topics` call per stream, and remove the 100-name cap.



##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {

Review Comment:
   On expiry, `timeout` drops every topic that already resolved, but the `dedup_topic_names` doc promises a deterministic prefix.
   Fix: apply the deadline per name with `timeout_at`, and keep the resolved answers.



##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {

Review Comment:
   The all-topics path has no deadline, so one request can take `(overrides + 2) * 15s` on a slow bridge.
   Fix: wrap `list_kafka_topics` in `tokio::time::timeout(REQUEST_DEADLINE, ..)`, and close on expiry as the error arm does.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {

Review Comment:
   Kafka answers `TOPIC_ALREADY_EXISTS` before it reads configs, but this rejects configs first, so Kafka Connect, which sends `cleanup.policy=compact`, fails even on existing topics.
   Fix: if configs or shape fail, look up the topic, and if it exists, answer `TOPIC_ALREADY_EXISTS`.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());

Review Comment:
   `CreatableTopicResult::default()` sets `error_message` to `Some("")`, so Java clients raise `TopicExistsException` and other errors with empty text.
   Fix: set `error_message(None)` on every result without a message, as the stub does.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39). Each entry's own replica list is checked too, not just the
+/// index: this gateway advertises exactly one broker (node id 1, `metadata.rs`'s `BrokerId(1)`),
+/// so the only legal replica list is `[1]` - never empty, never repeating it, never naming a
+/// broker Metadata never advertised. Real Kafka's `ReplicationControlManager` validates the whole
+/// binding this way, not just the index.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments)
+            || !assignment_replicas_are_valid(&topic.assignments)
+        {
+            return Err(ERROR_INVALID_REPLICA_ASSIGNMENT);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));
+    }
+
+    let broker_default_ok = version >= 4;
+
+    let partitions_ok = if broker_default_ok {

Review Comment:
   This has no upper bound, so `validate_only` answers NONE for more than 1000 partitions, but the real create rejects that count.
   Fix: reject counts above 1000 here with the code of the real create, through a local constant (`iggy::prelude` does not export `MAX_PARTITIONS_COUNT`).



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),

Review Comment:
   This sends the Iggy error text to the Kafka client, and `IggyError::from_code` fills its data fields with defaults, so the text can be wrong.
   Fix: send fixed text for bridge errors, and keep the details in the log.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result

Review Comment:
   This arm and the one in `lookup_one_topic` log nothing, but `bridge/error.rs:155` says that handlers log the real Iggy error.
   Fix: log client-caused errors (`InvalidKafkaTopicName`, `TooManyPartitions`) at `debug!`, and every other non-not-found error at `error!`.



##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -111,6 +379,51 @@ pub fn encode_response(
     encode_message(&resp, response_version, 256)
 }
 
+/// Real (bridge-backed) response: unlike [`encode_response`], each topic carries its own
+/// resolved error code and, on success, one [`MetadataResponsePartition`] per partition with
+/// this gateway's single broker (node id 1) as leader/replica/ISR - there is only ever one
+/// broker behind this gateway, so that triple is never actually in question.
+fn encode_real_response(
+    response_version: i16,
+    results: &[TopicResult],
+    broker: &BrokerAdvertise,
+) -> Result<Bytes> {
+    let response_topics = results
+        .iter()
+        .map(|result| {
+            let partitions = if result.error_code == ERROR_NONE {

Review Comment:
   At the 131,000-partition bound, the build and encode take a measured 20-54 ms without a yield, above the 10 ms stall that `bounds_guard.rs` prevents.
   Fix: cache the encoded all-topics response briefly, or build it in `spawn_blocking` above an element budget.



##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct.len(),
+                deadline_secs = REQUEST_DEADLINE.as_secs(),
+                "Metadata request's aggregate bridge work exceeded its deadline; answering \
+                 retriable instead of blocking further"
+            );
+            distinct
+                .iter()
+                .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                .collect()
+        }
+    }
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;

Review Comment:
   The encoder writes 26 to 34 bytes per partition, so this 64-byte charge starts the silent all-topics truncation at 41-53% of `max_frame_size`.
   Fix: charge `Encodable::compute_size` of a template partition at the request version, instead of 64.



##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets and isn't itself named the same as an override key - checked so an overridden
+    /// topic is never listed twice, once under its Kafka-side name and once under its raw Iggy
+    /// name, and so a raw default-stream topic never masquerades under a Kafka-side name an
+    /// override has already claimed for different data. The second check matters for a chained
+    /// override (`foo -> (kafka, bar)`, `bar -> (other, x)`): without it, a raw Iggy topic
+    /// literally named `foo` sitting in the default stream would be reported a second time under
+    /// the same `foo` name the override loop already emitted (backed by `kafka/bar`'s data), and
+    /// that second `foo` would be unreachable by name anyway, since `get_kafka_topic("foo")`
+    /// always resolves through the override to `kafka/bar`, never to the raw `kafka/foo`.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut override_keys: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            override_keys.insert(kafka_topic);
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {
+                results.push(KafkaTopicMetadata {
+                    kafka_topic: kafka_topic.to_string(),
+                    partitions_count: details.partitions_count,
+                });
+            }
+        }
+
+        let default_stream_id = Identifier::named(default_stream).map_err(BridgeError::Iggy)?;
+        if with_request_timeout(self.client.get_stream(&default_stream_id))
+            .await?
+            .is_some()
+        {
+            let topics = with_request_timeout(self.client.get_topics(&default_stream_id)).await?;
+            for topic in topics {

Review Comment:
   This lists raw default-stream names without `validate_kafka_topic_name`, so Metadata can list a name, such as `orders v2`, that a named lookup rejects.
   Fix: skip raw names that fail `validate_kafka_topic_name`, and log each one at `debug!`.



##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;

Review Comment:
   `ensure_stream` runs per topic, and its `get_stream` returns every topic header in the stream, up to 409,600 headers per 100-topic request.
   Fix: call `create_topic` first, and create the stream only on `StreamIdNotFound`.



##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets and isn't itself named the same as an override key - checked so an overridden
+    /// topic is never listed twice, once under its Kafka-side name and once under its raw Iggy
+    /// name, and so a raw default-stream topic never masquerades under a Kafka-side name an
+    /// override has already claimed for different data. The second check matters for a chained
+    /// override (`foo -> (kafka, bar)`, `bar -> (other, x)`): without it, a raw Iggy topic
+    /// literally named `foo` sitting in the default stream would be reported a second time under
+    /// the same `foo` name the override loop already emitted (backed by `kafka/bar`'s data), and
+    /// that second `foo` would be unreachable by name anyway, since `get_kafka_topic("foo")`
+    /// always resolves through the override to `kafka/bar`, never to the raw `kafka/foo`.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut override_keys: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            override_keys.insert(kafka_topic);
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {
+                results.push(KafkaTopicMetadata {
+                    kafka_topic: kafka_topic.to_string(),
+                    partitions_count: details.partitions_count,
+                });
+            }
+        }
+
+        let default_stream_id = Identifier::named(default_stream).map_err(BridgeError::Iggy)?;
+        if with_request_timeout(self.client.get_stream(&default_stream_id))

Review Comment:
   `get_stream` and `get_topics` both return every topic header, so this fetches the list twice, and the probe fails for a user with only `read_topics`.
   Fix: call `get_topics` alone, because the server lists an absent stream as empty.



##########
gateways/kafka/README.md:
##########
@@ -2,13 +2,15 @@
 
 Foundation layer for [apache/iggy#3421](https://github.com/apache/iggy/issues/3421): a TCP listener on the Kafka wire port that decodes requests, validates scoped API keys and versions, and returns stub responses.
 
-> **Stub warning:** most APIs still don't persist or read real data. Produce and Fetch return
-> retriable `NOT_LEADER_OR_FOLLOWER` (6) so clients keep data locally / retry elsewhere instead of
-> trusting a fake success. CreateTopics does **not** create topics; valid requests return
-> `NOT_CONTROLLER` (41). Metadata still reports requested topics as unknown. ListOffsets is wired
-> to the Iggy bridge: with `IGGY_KAFKA_BRIDGE_ENABLED=true` it answers `EARLIEST`/`LATEST` from
-> real partition state; with the bridge off (the default) it stays a stub and answers
-> `NOT_LEADER_OR_FOLLOWER` (6). See [docs/SCOPE.md](docs/SCOPE.md).
+> **Stub warning:** Produce, Fetch, and ListOffsets still don't persist or read real data - they

Review Comment:
   ListOffsets answers from the bridge since #4259, so this stub warning is wrong for ListOffsets.
   Fix: remove ListOffsets from this list, and restore the ListOffsets sentence that this diff removed.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge

Review Comment:
   With the bridge on, any unauthenticated client on the Kafka port can create topics of up to 1000 partitions as the bridge user.
   Fix: state in the README that CreateTopics is an open write path until SASL (#3549) lands.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)

Review Comment:
   A request over the 100-topic cap gets `INVALID_REQUEST`, whose default text blames the client, but Kafka answers server limits with `POLICY_VIOLATION`.
   Fix: answer `POLICY_VIOLATION` (44), with an `error_message` that names the 100-topic limit.



##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets and isn't itself named the same as an override key - checked so an overridden
+    /// topic is never listed twice, once under its Kafka-side name and once under its raw Iggy
+    /// name, and so a raw default-stream topic never masquerades under a Kafka-side name an
+    /// override has already claimed for different data. The second check matters for a chained
+    /// override (`foo -> (kafka, bar)`, `bar -> (other, x)`): without it, a raw Iggy topic
+    /// literally named `foo` sitting in the default stream would be reported a second time under
+    /// the same `foo` name the override loop already emitted (backed by `kafka/bar`'s data), and
+    /// that second `foo` would be unreachable by name anyway, since `get_kafka_topic("foo")`
+    /// always resolves through the override to `kafka/bar`, never to the raw `kafka/foo`.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut override_keys: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            override_keys.insert(kafka_topic);
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {

Review Comment:
   The `?` returns on the first override lookup error, so one unauthorized override closes every all-topics request, but Kafka skips unauthorized topics.
   Fix: skip and log an override whose lookup fails with `Unauthorized`, and keep the other topics.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39). Each entry's own replica list is checked too, not just the
+/// index: this gateway advertises exactly one broker (node id 1, `metadata.rs`'s `BrokerId(1)`),
+/// so the only legal replica list is `[1]` - never empty, never repeating it, never naming a
+/// broker Metadata never advertised. Real Kafka's `ReplicationControlManager` validates the whole
+/// binding this way, not just the index.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments)
+            || !assignment_replicas_are_valid(&topic.assignments)
+        {
+            return Err(ERROR_INVALID_REPLICA_ASSIGNMENT);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));
+    }
+
+    let broker_default_ok = version >= 4;
+
+    let partitions_ok = if broker_default_ok {
+        topic.num_partitions == -1 || topic.num_partitions > 0
+    } else {
+        topic.num_partitions > 0
+    };
+    if !partitions_ok {
+        return Err(ERROR_INVALID_PARTITIONS);
+    }
+
+    let replication_ok = if broker_default_ok {

Review Comment:
   This accepts any positive `replication_factor` and reports 1, but it rejects the assignment `[1,2,3]`, and one-broker Kafka rejects both.
   Fix: reject `replication_factor` above 1 with `INVALID_REPLICATION_FACTOR`, as Kafka does (`ReplicationControlManager.java:800`).



##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct.len(),
+                deadline_secs = REQUEST_DEADLINE.as_secs(),
+                "Metadata request's aggregate bridge work exceeded its deadline; answering \
+                 retriable instead of blocking further"
+            );
+            distinct
+                .iter()
+                .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                .collect()
+        }
+    }
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;
+
+/// Fixed per-`MetadataResponseTopic` cost, independent of name length or partition count:
+/// `error_code`(2) + `is_internal`(1) + the `partitions` array's own compact-length varint
+/// (charged at its 5-byte worst case) + `topic_authorized_operations`(4) + empty tagged fields(1).
+/// Every result pays this once - `encode_real_response` emits a full wrapper even for an error
+/// result, just with an empty `partitions` array, so this cost isn't conditional on `ERROR_NONE`.
+const RESPONSE_BYTES_TOPIC_OVERHEAD: usize = 13;
+
+/// Estimated encoded byte cost of one [`TopicResult`]: the fixed per-topic overhead, the name's
+/// own bytes (compact string: length plus a short varint prefix, charged at a flat +2), and -
+/// only for a successful result, since `encode_real_response` sends an empty array otherwise -
+/// [`RESPONSE_BYTES_PER_PARTITION`] per partition.
+///
+/// `bounds_guard` cannot see any of this: it charges the request by name count alone, with no way
+/// to know a name's length, a topic's per-entry overhead, or its partition count before this
+/// bridge round trip returns real data. Charging partitions alone (this function's earlier form)
+/// undercounted every result: a name costs bytes whether or not the lookup succeeded, and a
+/// zero-partition or errored result - free under a partitions-only charge - still costs a full
+/// wrapper to encode. 500 topics with 255-byte names and one partition each encode to ~146 KB
+/// against the ~32 KB a partitions-only charge would have priced in.
+fn estimated_response_bytes(result: &TopicResult) -> usize {
+    let name_bytes = result.name.as_str().len() + 2;
+    let partition_bytes = if result.error_code == ERROR_NONE {
+        (result.partitions_count as usize).saturating_mul(RESPONSE_BYTES_PER_PARTITION)
+    } else {
+        0
+    };
+    RESPONSE_BYTES_TOPIC_OVERHEAD + name_bytes + partition_bytes
+}
+
+const fn response_would_exceed_frame_size(total_bytes: usize, max_frame_size: usize) -> bool {
+    total_bytes > max_frame_size
+}
+
+/// Trims an all-topics [`IggyBridge::list_kafka_topics`] result to fit `max_frame_size`, keeping
+/// as many whole topics (in listing order) as the budget allows.
+///
+/// Unlike the named-lookup arm, the all-topics response's size is a server-side property (the
+/// cluster's total partition count) that the requesting client never chose and cannot shrink -
+/// closing the connection over it, as the shared frame-size guard below does for the
+/// client-controllable named-lookup case, would make every all-topics Metadata call fail
+/// identically and permanently once the catalog crosses the trip point. That call is the
+/// bootstrap and refresh shape both librdkafka and the Java client use, so a hard close reads as
+/// "broker down" and reconnect-loops rather than surfacing a usable, if partial, result. This
+/// wire protocol has no pagination cursor to ask for the rest with, so a truncated list - honest
+/// about being incomplete via the dropped entries, not via an error code - is what's available.
+fn truncate_all_topics_to_frame_budget(
+    mut results: Vec<TopicResult>,
+    max_frame_size: usize,
+) -> Vec<TopicResult> {
+    let mut cumulative_bytes = 0usize;
+    let mut keep = results.len();
+    for (index, result) in results.iter().enumerate() {
+        let next = cumulative_bytes + estimated_response_bytes(result);
+        if response_would_exceed_frame_size(next, max_frame_size) {
+            keep = index;
+            break;
+        }
+        cumulative_bytes = next;
+    }
+    if keep < results.len() {
+        tracing::warn!(
+            total_topics = results.len(),
+            kept_topics = keep,
+            max_frame_size,
+            "All-topics Metadata response would exceed max_frame_size; truncating rather than \
+             closing the connection"
+        );
+        results.truncate(keep);
+    }
+    results
+}
+
+/// One resolved topic result for the real (bridge-backed) path - `partitions_count` is
+/// meaningless when `error_code != ERROR_NONE`.
+struct TopicResult {
+    name: StrBytes,
+    error_code: i16,
+    partitions_count: u32,
+}
+
+fn found_result(metadata: crate::bridge::KafkaTopicMetadata) -> TopicResult {
+    TopicResult {
+        name: StrBytes::from_string(metadata.kafka_topic),
+        error_code: ERROR_NONE,
+        partitions_count: metadata.partitions_count,
+    }
+}
+
+async fn lookup_one_topic(bridge: &IggyBridge, name: StrBytes) -> TopicResult {
+    match bridge.get_kafka_topic(&name).await {

Review Comment:
   Each name costs one `get_topic` round trip on the shared `IggyClient`, which runs one request at a time, but Kafka answers Metadata from memory.
   Fix: serve Metadata from a short-TTL topic cache in `IggyBridge`, and drop an entry after each create.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["numinnex (via GitHub)" <gi...@apache.org>]

numinnex commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4072774772


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,203 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
+        }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => topics.into_iter().map(found_result).collect(),
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => {
+            let distinct_names: HashSet<&str> = names.iter().map(StrBytes::as_str).collect();
+            if distinct_names.len() > MAX_BRIDGE_BACKED_TOPICS {
+                tracing::warn!(
+                    distinct_topics = distinct_names.len(),
+                    max = MAX_BRIDGE_BACKED_TOPICS,
+                    "Metadata request addresses too many distinct topics; rejecting"
+                );
+                names
+                    .iter()
+                    .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+                    .collect()
+            } else {
+                match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &names))
+                    .await
+                {
+                    Ok(results) => results,
+                    Err(_elapsed) => {
+                        tracing::warn!(
+                            distinct_topics = distinct_names.len(),
+                            deadline_secs = REQUEST_DEADLINE.as_secs(),
+                            "Metadata request's aggregate bridge work exceeded its deadline; \
+                             answering retriable instead of blocking further"
+                        );
+                        names
+                            .iter()
+                            .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                            .collect()
+                    }
+                }
+            }
         }
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names, or one `list_kafka_topics()` call, can still expand into a
+    // response `bounds_guard` never had the information to price in before this bridge round
+    // trip returned. Checked here, before `encode_real_response` builds one
+    // `MetadataResponsePartition` per partition, not after - the expensive part is building that
+    // `Vec`, not encoding the bytes that follow it.
+    let total_partitions: usize = results
+        .iter()
+        .filter(|result| result.error_code == ERROR_NONE)
+        .map(|result| result.partitions_count as usize)
+        .sum();
+    if response_would_exceed_frame_size(total_partitions, state.max_frame_size) {
+        tracing::warn!(
+            total_partitions,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;

Review Comment:
   This guard also covers the all-topics arm, where `total_partitions` is the whole catalog rather than anything the client asked for. With `max_frame_size` defaulting to 8 MiB the trip point is 131,072 partitions cluster-wide, after which every all-topics Metadata is a bare TCP close with no error code, for every client, permanently. All-topics is the bootstrap and refresh shape for librdkafka and the Java client, which read a close as "broker down" and reconnect-loop. Truncating seems better than closing here.



##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,203 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
+        }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => topics.into_iter().map(found_result).collect(),
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => {
+            let distinct_names: HashSet<&str> = names.iter().map(StrBytes::as_str).collect();
+            if distinct_names.len() > MAX_BRIDGE_BACKED_TOPICS {
+                tracing::warn!(
+                    distinct_topics = distinct_names.len(),
+                    max = MAX_BRIDGE_BACKED_TOPICS,
+                    "Metadata request addresses too many distinct topics; rejecting"
+                );
+                names
+                    .iter()
+                    .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+                    .collect()
+            } else {
+                match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &names))
+                    .await
+                {
+                    Ok(results) => results,
+                    Err(_elapsed) => {
+                        tracing::warn!(
+                            distinct_topics = distinct_names.len(),
+                            deadline_secs = REQUEST_DEADLINE.as_secs(),
+                            "Metadata request's aggregate bridge work exceeded its deadline; \
+                             answering retriable instead of blocking further"
+                        );
+                        names
+                            .iter()
+                            .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                            .collect()
+                    }
+                }
+            }
         }
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names, or one `list_kafka_topics()` call, can still expand into a
+    // response `bounds_guard` never had the information to price in before this bridge round
+    // trip returned. Checked here, before `encode_real_response` builds one
+    // `MetadataResponsePartition` per partition, not after - the expensive part is building that
+    // `Vec`, not encoding the bytes that follow it.
+    let total_partitions: usize = results
+        .iter()
+        .filter(|result| result.error_code == ERROR_NONE)
+        .map(|result| result.partitions_count as usize)
+        .sum();
+    if response_would_exceed_frame_size(total_partitions, state.max_frame_size) {
+        tracing::warn!(
+            total_partitions,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;
+
+const fn response_would_exceed_frame_size(total_partitions: usize, max_frame_size: usize) -> bool {
+    total_partitions.saturating_mul(RESPONSE_BYTES_PER_PARTITION) > max_frame_size
+}
+
+/// One resolved topic result for the real (bridge-backed) path - `partitions_count` is
+/// meaningless when `error_code != ERROR_NONE`.
+struct TopicResult {
+    name: StrBytes,
+    error_code: i16,
+    partitions_count: u32,
+}
+
+fn found_result(metadata: crate::bridge::KafkaTopicMetadata) -> TopicResult {
+    TopicResult {
+        name: StrBytes::from_string(metadata.kafka_topic),
+        error_code: ERROR_NONE,
+        partitions_count: metadata.partitions_count,
+    }
+}
+
+async fn lookup_one_topic(bridge: &IggyBridge, name: StrBytes) -> TopicResult {
+    match bridge.get_kafka_topic(&name).await {
+        // `get_kafka_topic` (unlike `list_kafka_topics`) returns the SDK's own `TopicDetails`,
+        // which carries the topic's raw Iggy-side name, not the Kafka-side one under an override
+        // - so this echoes the caller's own `name`, not a field off the result.
+        Ok(Some(details)) => TopicResult {
+            name,
+            error_code: ERROR_NONE,
+            partitions_count: details.partitions_count,
+        },
+        Ok(None) => TopicResult {
+            name,
+            error_code: ERROR_UNKNOWN_TOPIC_OR_PARTITION,
+            partitions_count: 0,
+        },
+        Err(err) => TopicResult {
+            name,
+            error_code: err.to_kafka_error_code(),
+            partitions_count: 0,
+        },
+    }
+}
+
+const fn error_result(name: StrBytes, error_code: i16) -> TopicResult {
+    TopicResult {
+        name,
+        error_code,
+        partitions_count: 0,
+    }
+}
+
+/// Resolves every requested name, deduping first so a name repeated in the request (or asked
+/// about more than once, which the wire technically allows) costs one `get_kafka_topic` round
+/// trip, not one per occurrence.
+async fn resolve_named_topics(bridge: &IggyBridge, names: &[StrBytes]) -> Vec<TopicResult> {
+    let mut seen = HashSet::with_capacity(names.len());
+    let mut distinct = Vec::new();
+    for name in names {
+        if seen.insert(name.as_str()) {
+            distinct.push(name.clone());
+        }
+    }
+
+    let mut results_by_name: HashMap<&str, TopicResult> = HashMap::with_capacity(distinct.len());
+    for name in &distinct {
+        let result = lookup_one_topic(bridge, name.clone()).await;
+        results_by_name.insert(name.as_str(), result);
+    }
+
+    names
+        .iter()
+        .map(|name| {
+            // Always present: `distinct` (and so results_by_name) was built from exactly these
+            // same requested names, just above.
+            let cached = results_by_name
+                .get(name.as_str())
+                .expect("every requested name was resolved above");

Review Comment:
   The bridge calls are deduped, but this map runs over every *occurrence*, so the response carries one entry per repeat. Real Kafka does `metadataRequest.topics.asScala.toSet` and answers once per distinct name. At v9, 131 copies of one 1000-partition name passes `bounds_guard` (131 elements), passes the 100-distinct cap (1 distinct), costs one round trip, and clears the new frame guard at 8,384,000 bytes, while building 131,000 partition records from a ~1 KB request.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,324 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
 use kafka_protocol::messages::create_topics_request::CreatableTopic;
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST, ERROR_NONE, ERROR_NOT_CONTROLLER,
+    ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS, GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(err.to_string()))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        Ok(TopicCreationOutcome::Created) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(err.to_string()))),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));

Review Comment:
   `assignments.len()` is the only thing read here, so the partition indices themselves are never checked. Kafka's `ReplicationControlManager.createTopic` answers `INVALID_REPLICA_ASSIGNMENT` (39) both for a duplicate `partition_index` and for a sequence that isn't consecutive from 0. `new NewTopic(name, Map.of(5, List.of(1), 7, List.of(1)))` sends `-1/-1`, clears the gate above, and gets `NONE` with `num_partitions=2` for a topic whose partitions are 0 and 1. 39 is now absent from `src/` entirely, so there's no code left for the conditions Kafka does use it for.



##########
gateways/kafka/src/bridge/error.rs:
##########
@@ -163,7 +172,17 @@ const fn iggy_error_to_kafka_code(err: &IggyError) -> i16 {
         | IggyError::TcpError
         | IggyError::TransientNotAccepted => ERROR_NOT_LEADER_OR_FOLLOWER,
         IggyError::TransientNotCommitted => ERROR_REQUEST_TIMED_OUT,
-        IggyError::TooManyPartitions => ERROR_INVALID_PARTITIONS,
+        // Not `ERROR_INVALID_PARTITIONS` (37): that code's own text, per `kafka-protocol`'s
+        // table, is "Number of partitions is below 1" - the opposite condition from "too many"
+        // (Iggy's server-side cap, above 1000). Reusing 37 for both directions would return a
+        // client-visible error message that contradicts the actual request it sent.
+        IggyError::TooManyPartitions => ERROR_INVALID_REQUEST,
+        // The operation *did* commit - the SDK's own reconnect path replayed a write whose first
+        // attempt already applied, and the server's client-table dedup caught the replay. Falling
+        // into the catch-all below would report a permanent server fault for a request that
+        // actually succeeded; a Java client treats `UNKNOWN_SERVER_ERROR` as non-retriable and
+        // would surface a spurious failure for a `CreateTopics` that in fact created the topic.
+        IggyError::RequestAlreadyApplied => ERROR_NONE,

Review Comment:
   The reasoning above is entirely about CreateTopics, but this arm is global. Reached through `metadata.rs`, it produces `TopicResult { error_code: NONE, partitions_count: 0 }`, and `encode_real_response` then emits a topic with error 0 and an empty partitions array: the topic appears to exist with nowhere to produce, and no retriable error for the client to act on. Scoping this to the CreateTopics write path keeps the fix without that side effect.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,324 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
 use kafka_protocol::messages::create_topics_request::CreatableTopic;
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST, ERROR_NONE, ERROR_NOT_CONTROLLER,
+    ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS, GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(err.to_string()))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        Ok(TopicCreationOutcome::Created) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(err.to_string()))),

Review Comment:
   `err.to_string()` embeds the full wire name for `InvalidKafkaTopicName`, so each result echoes the name twice while `bounds_guard` charged it once: the same one-echo modelling error this PR fixes on the Metadata side. `create_kafka_topic` validates the name before any I/O, so 100 names of 64 KiB clear the guard, the cap and the frame check, cost zero bridge calls, and build a ~13 MB response that `send_response` admits because it still only checks `i32::MAX`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4077891515


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,324 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
 use kafka_protocol::messages::create_topics_request::CreatableTopic;
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST, ERROR_NONE, ERROR_NOT_CONTROLLER,
+    ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS, GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(err.to_string()))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        Ok(TopicCreationOutcome::Created) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(err.to_string()))),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));

Review Comment:
   Re-added ERROR_INVALID_REPLICA_ASSIGNMENT (39) for its real use: a manual assignments list whose partition indices aren't exactly 0..len(), no duplicates. Added assignment_indices_are_consecutive_from_zero helper (sorted+deduped+min/max check — exact and O(n log n)).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093334503


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(

Review Comment:
   CreateTopics/Metadata named-lookup now bound each bridge call individually (timeout_at against one shared deadline), keeping already-resolved results instead of discarding the whole
     batch. All-topics list_kafka_topics() now has a deadline where it had none.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093347709


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {

Review Comment:
   similar as earlier timeout



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["krishvishal (via GitHub)" <gi...@apache.org>]

krishvishal merged PR #4258:
URL: https://github.com/apache/iggy/pull/4258


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["codecov[bot] (via GitHub)" <gi...@apache.org>]

codecov[bot] commented on PR #4258:
URL: https://github.com/apache/iggy/pull/4258#issuecomment-5770467665

   ## [Codecov](https://app.codecov.io/gh/apache/iggy/pull/4258?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 26.92%. Comparing base ([`7bd6967`](https://app.codecov.io/gh/apache/iggy/commit/7bd69670f9ad9946f6490359443ec25bf7e2a089?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`dafcba8`](https://app.codecov.io/gh/apache/iggy/commit/dafcba827e0289dd127955d1fde0fd4a2ef88aee?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   :warning: Report is 2 commits behind head on master.
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master    #4258       +/-   ##
   =============================================
   - Coverage     87.54%   26.92%   -60.63%     
     Complexity     1575     1575               
   =============================================
     Files          1284     1281        -3     
     Lines        223942   181110    -42832     
     Branches     187305   144473    -42832     
   =============================================
   - Hits         196050    48762   -147288     
   - Misses        23168   130620   +107452     
   + Partials       4724     1728     -2996     
   ```
   
   | [Components](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [Rust Core](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `11.62% <33.79%> (-77.01%)` | :arrow_down: |
   | [Java SDK](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `68.68% <ø> (ø)` | |
   | [C# SDK](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.41% <ø> (ø)` | |
   | [Python SDK](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.97% <ø> (ø)` | |
   | [PHP SDK](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.67% <ø> (ø)` | |
   | [Node SDK](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `96.36% <ø> (ø)` | |
   | [Go SDK](https://app.codecov.io/gh/apache/iggy/pull/4258/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.05% <ø> (ø)` | |
   [see 688 files with indirect coverage changes](https://app.codecov.io/gh/apache/iggy/pull/4258/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093357051


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39). Each entry's own replica list is checked too, not just the
+/// index: this gateway advertises exactly one broker (node id 1, `metadata.rs`'s `BrokerId(1)`),
+/// so the only legal replica list is `[1]` - never empty, never repeating it, never naming a
+/// broker Metadata never advertised. Real Kafka's `ReplicationControlManager` validates the whole
+/// binding this way, not just the index.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments)
+            || !assignment_replicas_are_valid(&topic.assignments)
+        {
+            return Err(ERROR_INVALID_REPLICA_ASSIGNMENT);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));
+    }
+
+    let broker_default_ok = version >= 4;
+
+    let partitions_ok = if broker_default_ok {

Review Comment:
   changed the implementation.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093446918


##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;

Review Comment:
    rather than restructuring ensure_topic/create_kafka_topic to try create_topic first and only create the stream on StreamIdNotFound,  fixed it at the shared
     ensure_stream helper directly — it now attempts create_stream first and treats StreamNameAlreadyExists as success, never calling get_stream (which returns StreamDetails embedding the full topics: Vec<Topic>) at
     all. Confirmed via iggy_common::StreamDetails's own field list that your cost claim is accurate. This fixes the cost for both call sites (ensure_stream_and_topic and create_kafka_topic) in one place rather than
     two.  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093457921


##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,151 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets and isn't itself named the same as an override key - checked so an overridden
+    /// topic is never listed twice, once under its Kafka-side name and once under its raw Iggy
+    /// name, and so a raw default-stream topic never masquerades under a Kafka-side name an
+    /// override has already claimed for different data. The second check matters for a chained
+    /// override (`foo -> (kafka, bar)`, `bar -> (other, x)`): without it, a raw Iggy topic
+    /// literally named `foo` sitting in the default stream would be reported a second time under
+    /// the same `foo` name the override loop already emitted (backed by `kafka/bar`'s data), and
+    /// that second `foo` would be unreachable by name anyway, since `get_kafka_topic("foo")`
+    /// always resolves through the override to `kafka/bar`, never to the raw `kafka/foo`.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut override_keys: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            override_keys.insert(kafka_topic);
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {
+                results.push(KafkaTopicMetadata {
+                    kafka_topic: kafka_topic.to_string(),
+                    partitions_count: details.partitions_count,
+                });
+            }
+        }
+
+        let default_stream_id = Identifier::named(default_stream).map_err(BridgeError::Iggy)?;
+        if with_request_timeout(self.client.get_stream(&default_stream_id))

Review Comment:
   dropped the get_stream probe, call get_topics alone — confirmed against core/server/src/responses.rs::build_get_topics_response's own comment ("Legacy parity: a missing stream lists
     as empty, not StreamNotFound") that this is safe.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093375621


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)

Review Comment:
     Fixed: added ERROR_POLICY_VIOLATION (44) to api.rs, this cap now answers it with a message naming the limit. Updated create_topics_rejects_more_than_the_topic_cap's assertion.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["numinnex (via GitHub)" <gi...@apache.org>]

numinnex commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4082045181


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,249 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
+        }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_partitions: usize = results
+        .iter()
+        .filter(|result| result.error_code == ERROR_NONE)
+        .map(|result| result.partitions_count as usize)
+        .sum();
+    if response_would_exceed_frame_size(total_partitions, state.max_frame_size) {
+        tracing::warn!(
+            total_partitions,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
+    }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct.len(),
+                deadline_secs = REQUEST_DEADLINE.as_secs(),
+                "Metadata request's aggregate bridge work exceeded its deadline; answering \
+                 retriable instead of blocking further"
+            );
+            distinct
+                .iter()
+                .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                .collect()
+        }
+    }
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;
+
+const fn response_would_exceed_frame_size(total_partitions: usize, max_frame_size: usize) -> bool {
+    total_partitions.saturating_mul(RESPONSE_BYTES_PER_PARTITION) > max_frame_size
+}
+
+/// Trims an all-topics [`IggyBridge::list_kafka_topics`] result to fit `max_frame_size`, keeping
+/// as many whole topics (in listing order) as the budget allows.
+///
+/// Unlike the named-lookup arm, the all-topics response's size is a server-side property (the
+/// cluster's total partition count) that the requesting client never chose and cannot shrink -
+/// closing the connection over it, as the shared frame-size guard below does for the
+/// client-controllable named-lookup case, would make every all-topics Metadata call fail
+/// identically and permanently once the catalog crosses the trip point. That call is the
+/// bootstrap and refresh shape both librdkafka and the Java client use, so a hard close reads as
+/// "broker down" and reconnect-loops rather than surfacing a usable, if partial, result. This
+/// wire protocol has no pagination cursor to ask for the rest with, so a truncated list - honest
+/// about being incomplete via the dropped entries, not via an error code - is what's available.
+fn truncate_all_topics_to_frame_budget(
+    mut results: Vec<TopicResult>,
+    max_frame_size: usize,
+) -> Vec<TopicResult> {
+    let mut cumulative_partitions = 0usize;
+    let mut keep = results.len();
+    for (index, result) in results.iter().enumerate() {
+        if result.error_code != ERROR_NONE {
+            continue;
+        }
+        let next = cumulative_partitions + result.partitions_count as usize;

Review Comment:
   This charges only `partitions × 64`, so topic names, per-topic overhead and zero-partition topics are free. Measured: 500 one-partition topics with 255-byte names encode to 146,037 B against 32,000 charged, and 200,000 zero-partition topics give a ~53 MB response against the 8 MiB budget. `send_response` only checks `i32::MAX`, so nothing downstream catches it, and the one-partition case can be built through CreateTopics without authentication. Charging each topic's real size (name, fixed overhead, partitions), or encoding and then measuring, would close it.



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,378 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39), the one condition that code exists for. This bridge doesn't
+/// model per-partition replica placement, so only the index set is checked, not each entry's
+/// replica list.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments) {

Review Comment:
   The indices are checked now, but each assignment's `broker_ids` never is. Real Kafka's `ReplicationControlManager` answers `INVALID_REPLICA_ASSIGNMENT` (39) for an empty replica list, a broker named twice, and a broker that isn't registered. Here `[]`, `[1, 1]` and `[7]` all answer `NONE` and create the topic, although this gateway only advertises node 1. The test helper at `create_topics_real_bridge_tests.rs:91` writes `broker_id 0`, so the suite currently asserts `NONE` for assignments a real broker would reject.



##########
gateways/kafka/src/bridge/iggy_bridge/topics.rs:
##########
@@ -217,4 +255,140 @@ impl IggyBridge {
             Err(err) => Err(err),
         }
     }
+
+    /// Looks up `kafka_topic`, resolved through the configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping), without creating it.
+    ///
+    /// Returns `Ok(None)` when either the mapped stream or the mapped topic doesn't exist -
+    /// callers (`CreateTopics`' existence check, `Metadata`'s lookup) treat both the same way:
+    /// nothing answers to this Kafka-side name yet.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::InvalidKafkaTopicName`] if `kafka_topic` fails Kafka's own
+    /// topic-naming rules. Returns [`BridgeError::Timeout`] if a call takes longer than
+    /// `REQUEST_TIMEOUT`. Returns [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn get_kafka_topic(
+        &self,
+        kafka_topic: &str,
+    ) -> Result<Option<TopicDetails>, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = Identifier::named(stream_name).map_err(BridgeError::Iggy)?;
+        let topic_id = Identifier::named(topic_name).map_err(BridgeError::Iggy)?;
+        // No separate get_stream probe: get_topic already answers Ok(None) when the stream
+        // itself is missing (see high_watermarks' own doc on this same fact), so a probe first
+        // would just pay a second round trip to learn something this one call already tells us.
+        with_request_timeout(self.client.get_topic(&stream_id, &topic_id)).await
+    }
+
+    /// Creates the Iggy stream/topic backing `kafka_topic`, or reports that it already exists.
+    ///
+    /// Atomic from this call's perspective, unlike a separate existence check
+    /// ([`Self::get_kafka_topic`]) followed by [`Self::ensure_stream_and_topic`]: that sequence
+    /// has a TOCTOU window between the two calls, and `ensure_stream_and_topic`'s own idempotent
+    /// contract would then absorb a second concurrent caller's create into a silent `Ok`, so both
+    /// callers see success for a `CreateTopics` request Kafka promises exactly one `NONE` for.
+    /// Here, the create attempt itself is the existence check: no separate read precedes it, and
+    /// [`TopicCreationOutcome::AlreadyExists`] comes from the server's own rejection of the write,
+    /// not from an earlier read that could already be stale by the time this call's write lands.
+    ///
+    /// # Errors
+    ///
+    /// Same as [`Self::ensure_stream_and_topic`], except an already-existing topic is reported as
+    /// [`TopicCreationOutcome::AlreadyExists`] rather than [`BridgeError::PartitionCountMismatch`].
+    /// `CreateTopics` is not an upsert, so a pre-existing topic is never itself an error here,
+    /// regardless of whether its partition count matches `partition_count`.
+    pub async fn create_kafka_topic(
+        &self,
+        kafka_topic: &str,
+        partition_count: u32,
+    ) -> Result<TopicCreationOutcome, BridgeError> {
+        validate_kafka_topic_name("kafka_topic", kafka_topic)?;
+        if partition_count == 0 {
+            return Err(BridgeError::InvalidPartitionCount {
+                kafka_topic: kafka_topic.to_string(),
+            });
+        }
+        let (stream_name, topic_name) = self.config.topic_mapping.resolve(kafka_topic);
+        let stream_id = self.ensure_stream(stream_name).await?;
+
+        let options = TopicCreateOptions {
+            partitions_count: Some(partition_count),
+            ..TopicCreateOptions::default()
+        };
+        match with_request_timeout(self.client.create_topic(&stream_id, topic_name, &options)).await
+        {
+            Ok(created) => {
+                info!("created Iggy topic '{topic_name}' with {partition_count} partitions");
+                // Same postcondition check ensure_topic's own create branch makes: partitions_count
+                // is a hard argument (Some(partition_count), never None), so a mismatch here means
+                // a future server-side clamp/cap, not a client input problem - fails loudly instead
+                // of silently reporting Created under a broken contract.
+                if created.partitions_count != partition_count {
+                    return Err(BridgeError::PartitionCountMismatch {
+                        topic: kafka_topic.to_string(),
+                        existing: created.partitions_count,
+                        requested: partition_count,
+                    });
+                }
+                Ok(TopicCreationOutcome::Created)
+            }
+            Err(BridgeError::Iggy(IggyError::TopicNameAlreadyExists(_, _))) => {
+                Ok(TopicCreationOutcome::AlreadyExists)
+            }
+            Err(err) => Err(err),
+        }
+    }
+
+    /// Every Kafka-visible topic: the target of every configured
+    /// [`TopicMapping`](crate::bridge::topic_map::TopicMapping) override that actually exists in
+    /// Iggy, plus every topic in the default stream that isn't itself one of those override
+    /// targets - checked so an overridden topic is never listed twice, once under its Kafka-side
+    /// name and once under its raw Iggy name.
+    ///
+    /// An Iggy stream this bridge has no mapping rule pointing at (neither the default stream nor
+    /// any override's target) holds data no Kafka client ever named - deliberately excluded, the
+    /// same way a real Kafka broker never reports storage it doesn't own.
+    ///
+    /// # Errors
+    ///
+    /// Returns [`BridgeError::Timeout`] if a call takes longer than `REQUEST_TIMEOUT`. Returns
+    /// [`BridgeError::Iggy`] for connectivity/auth failures.
+    pub async fn list_kafka_topics(&self) -> Result<Vec<KafkaTopicMetadata>, BridgeError> {
+        let default_stream = self.config.topic_mapping.default_stream();
+        let mut default_stream_override_targets: HashSet<&str> = HashSet::new();
+        let mut results = Vec::new();
+
+        for (kafka_topic, over) in self.config.topic_mapping.overrides() {
+            if over.stream == default_stream {
+                default_stream_override_targets.insert(over.topic.as_str());
+            }
+            if let Some(details) = self.get_kafka_topic(kafka_topic).await? {
+                results.push(KafkaTopicMetadata {
+                    kafka_topic: kafka_topic.to_string(),
+                    partitions_count: details.partitions_count,
+                });
+            }
+        }
+
+        let default_stream_id = Identifier::named(default_stream).map_err(BridgeError::Iggy)?;
+        if with_request_timeout(self.client.get_stream(&default_stream_id))
+            .await?
+            .is_some()
+        {
+            let topics = with_request_timeout(self.client.get_topics(&default_stream_id)).await?;
+            for topic in topics {
+                if default_stream_override_targets.contains(topic.name.as_str()) {

Review Comment:
   This skips default-stream topics that are an override's *target*, but not ones whose name is an override *key*. With overrides `foo → (kafka, bar)` and `bar → (other, x)` (both accepted by `TopicMapping::new`) and Iggy topics `kafka/foo` and `kafka/bar`, the override loop emits `foo` backed by `kafka/bar`, and this loop emits `kafka/foo` as a second `foo` with a different partition count. Real Kafka never repeats a name, and `kafka/foo` can't be reached by name anyway since `foo` routes to `kafka/bar`. Also skipping names that are override keys would fix it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093373592


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        // The second arm: the write committed on its first attempt, the SDK's own reconnect path
+        // replayed it, and the server's client-table dedup caught the replay - not a fault.
+        // `to_kafka_error_code`'s shared mapping deliberately doesn't special-case this - it's a
+        // write-only fact, checked here, at the one write this bridge makes, rather than assumed
+        // true for the reads that share that mapping too.
+        Ok(TopicCreationOutcome::Created)
+        | Err(BridgeError::Iggy(IggyError::RequestAlreadyApplied)) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(error_message_for(&err)))),
+    }
+}
+
+/// Error text for `CreatableTopicResult.error_message`, without re-embedding the topic name:
+/// `result.name` (`CreatableTopicResult::with_name`) already carries it, so `err.to_string()`'s
+/// own embedded copy for these two variants would double the per-topic response cost for a name
+/// the client already sent and already has back. Validation runs before any bridge I/O, so this
+/// is a purely local, zero-round-trip amplification if left in - 100 topics named with a maximal
+/// legal length build a response roughly twice the size the name alone would justify.
+fn error_message_for(err: &BridgeError) -> String {
+    match err {
+        BridgeError::InvalidKafkaTopicName { reason, .. } => reason.clone(),
+        BridgeError::InvalidPartitionCount { .. } => {
+            "partition count must be at least 1".to_string()
+        }
+        other => other.to_string(),
+    }
+}
+
+/// Validates one requested topic's KIP-464 shape and resolves its partition count.
+///
+/// A manual partition `assignments` list and an explicit `num_partitions`/`replication_factor`
+/// are mutually exclusive inputs, not two independently-checked values that happen to agree: real
+/// Kafka's own `ReplicationControlManager` rejects a manual assignment unless both are exactly
+/// `-1`, regardless of whether an explicit `num_partitions` matches `assignments.len()`. A count
+/// that only *disagrees* with the assignment length is not a distinct, more lenient case - the
+/// combination itself is what's invalid, so both are `INVALID_REQUEST` (42), never `NONE`.
+///
+/// An assignment's own partition indices are checked too, not just its length: real Kafka
+/// requires the key set to be exactly `0..assignments.len()`, each index appearing once, and
+/// rejects anything else - a duplicate or non-consecutive index (`{5: [...], 7: [...]}`) - with
+/// `INVALID_REPLICA_ASSIGNMENT` (39). Each entry's own replica list is checked too, not just the
+/// index: this gateway advertises exactly one broker (node id 1, `metadata.rs`'s `BrokerId(1)`),
+/// so the only legal replica list is `[1]` - never empty, never repeating it, never naming a
+/// broker Metadata never advertised. Real Kafka's `ReplicationControlManager` validates the whole
+/// binding this way, not just the index.
+///
+/// With no assignments, `num_partitions = -1` / `replication_factor = -1` mean "use the broker
+/// default" from v4+ (pre-v4 requires an explicit positive value for both, since v2/v3 have no
+/// broker-default sentinel absent a manual assignment).
+fn validate_create_topic_shape(
+    version: i16,
+    topic: &CreatableTopic,
+) -> core::result::Result<u32, i16> {
+    if !topic.assignments.is_empty() {
+        if topic.num_partitions != -1 || topic.replication_factor != -1 {
+            return Err(ERROR_INVALID_REQUEST);
+        }
+        if !assignment_indices_are_consecutive_from_zero(&topic.assignments)
+            || !assignment_replicas_are_valid(&topic.assignments)
+        {
+            return Err(ERROR_INVALID_REPLICA_ASSIGNMENT);
+        }
+        return Ok(u32::try_from(topic.assignments.len()).unwrap_or(DEFAULT_PARTITION_COUNT));
+    }
+
+    let broker_default_ok = version >= 4;
+
+    let partitions_ok = if broker_default_ok {
+        topic.num_partitions == -1 || topic.num_partitions > 0
+    } else {
+        topic.num_partitions > 0
+    };
+    if !partitions_ok {
+        return Err(ERROR_INVALID_PARTITIONS);
+    }
+
+    let replication_ok = if broker_default_ok {

Review Comment:
   done



##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)

Review Comment:
   fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093421019


##########
gateways/kafka/README.md:
##########
@@ -2,13 +2,15 @@
 
 Foundation layer for [apache/iggy#3421](https://github.com/apache/iggy/issues/3421): a TCP listener on the Kafka wire port that decodes requests, validates scoped API keys and versions, and returns stub responses.
 
-> **Stub warning:** most APIs still don't persist or read real data. Produce and Fetch return
-> retriable `NOT_LEADER_OR_FOLLOWER` (6) so clients keep data locally / retry elsewhere instead of
-> trusting a fake success. CreateTopics does **not** create topics; valid requests return
-> `NOT_CONTROLLER` (41). Metadata still reports requested topics as unknown. ListOffsets is wired
-> to the Iggy bridge: with `IGGY_KAFKA_BRIDGE_ENABLED=true` it answers `EARLIEST`/`LATEST` from
-> real partition state; with the bridge off (the default) it stays a stub and answers
-> `NOT_LEADER_OR_FOLLOWER` (6). See [docs/SCOPE.md](docs/SCOPE.md).
+> **Stub warning:** Produce, Fetch, and ListOffsets still don't persist or read real data - they

Review Comment:
    Fixed, and root-caused: 20d323793 (#4259) added the correct ListOffsets sentence to this paragraph, but a later doc-only commit (dafcba827) overwrote the whole paragraph back to older wording and a merge
     conflict resolution kept that older version, silently dropping #4259's addition. Rewrote the paragraph to include all three pieces of accurate status (Produce/Fetch stub, CreateTopics/Metadata bridge-wired,
     ListOffsets bridge-wired) in one coherent statement instead of just re-inserting the dropped sentence.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093334503


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(

Review Comment:
     Fixed. create_all_topics now wraps each topic's own create_one_topic call in timeout_at(shared_deadline, ...) instead of wrapping the whole loop in one timeout. A topic that already resolved before the deadline
     keeps its real result; only topics reached after the deadline (or the one that was mid-flight when it passed) answer REQUEST_TIMED_OUT.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4077895327


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,203 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
+        }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => topics.into_iter().map(found_result).collect(),
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => {
+            let distinct_names: HashSet<&str> = names.iter().map(StrBytes::as_str).collect();
+            if distinct_names.len() > MAX_BRIDGE_BACKED_TOPICS {
+                tracing::warn!(
+                    distinct_topics = distinct_names.len(),
+                    max = MAX_BRIDGE_BACKED_TOPICS,
+                    "Metadata request addresses too many distinct topics; rejecting"
+                );
+                names
+                    .iter()
+                    .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+                    .collect()
+            } else {
+                match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &names))
+                    .await
+                {
+                    Ok(results) => results,
+                    Err(_elapsed) => {
+                        tracing::warn!(
+                            distinct_topics = distinct_names.len(),
+                            deadline_secs = REQUEST_DEADLINE.as_secs(),
+                            "Metadata request's aggregate bridge work exceeded its deadline; \
+                             answering retriable instead of blocking further"
+                        );
+                        names
+                            .iter()
+                            .map(|name| error_result(name.clone(), ERROR_REQUEST_TIMED_OUT))
+                            .collect()
+                    }
+                }
+            }
         }
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names, or one `list_kafka_topics()` call, can still expand into a
+    // response `bounds_guard` never had the information to price in before this bridge round
+    // trip returned. Checked here, before `encode_real_response` builds one
+    // `MetadataResponsePartition` per partition, not after - the expensive part is building that
+    // `Vec`, not encoding the bytes that follow it.
+    let total_partitions: usize = results
+        .iter()
+        .filter(|result| result.error_code == ERROR_NONE)
+        .map(|result| result.partitions_count as usize)
+        .sum();
+    if response_would_exceed_frame_size(total_partitions, state.max_frame_size) {
+        tracing::warn!(
+            total_partitions,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Conservative per-partition byte cost of one encoded `MetadataResponsePartition` at v9 (the
+/// densest wire shape this handler emits): measured ~26 bytes (`error_code` + `partition_index` +
+/// `leader_id` + `leader_epoch` + a 1-entry `replica_nodes` + a 1-entry `isr_nodes` + an empty
+/// `offline_replicas` + tagged fields). 64 matches the margin `bounds_guard`'s own
+/// `RESPONSE_BYTES_PER_ELEMENT` uses for the same kind of estimate, rather than shaving this to
+/// the measured minimum.
+const RESPONSE_BYTES_PER_PARTITION: usize = 64;
+
+const fn response_would_exceed_frame_size(total_partitions: usize, max_frame_size: usize) -> bool {
+    total_partitions.saturating_mul(RESPONSE_BYTES_PER_PARTITION) > max_frame_size
+}
+
+/// One resolved topic result for the real (bridge-backed) path - `partitions_count` is
+/// meaningless when `error_code != ERROR_NONE`.
+struct TopicResult {
+    name: StrBytes,
+    error_code: i16,
+    partitions_count: u32,
+}
+
+fn found_result(metadata: crate::bridge::KafkaTopicMetadata) -> TopicResult {
+    TopicResult {
+        name: StrBytes::from_string(metadata.kafka_topic),
+        error_code: ERROR_NONE,
+        partitions_count: metadata.partitions_count,
+    }
+}
+
+async fn lookup_one_topic(bridge: &IggyBridge, name: StrBytes) -> TopicResult {
+    match bridge.get_kafka_topic(&name).await {
+        // `get_kafka_topic` (unlike `list_kafka_topics`) returns the SDK's own `TopicDetails`,
+        // which carries the topic's raw Iggy-side name, not the Kafka-side one under an override
+        // - so this echoes the caller's own `name`, not a field off the result.
+        Ok(Some(details)) => TopicResult {
+            name,
+            error_code: ERROR_NONE,
+            partitions_count: details.partitions_count,
+        },
+        Ok(None) => TopicResult {
+            name,
+            error_code: ERROR_UNKNOWN_TOPIC_OR_PARTITION,
+            partitions_count: 0,
+        },
+        Err(err) => TopicResult {
+            name,
+            error_code: err.to_kafka_error_code(),
+            partitions_count: 0,
+        },
+    }
+}
+
+const fn error_result(name: StrBytes, error_code: i16) -> TopicResult {
+    TopicResult {
+        name,
+        error_code,
+        partitions_count: 0,
+    }
+}
+
+/// Resolves every requested name, deduping first so a name repeated in the request (or asked
+/// about more than once, which the wire technically allows) costs one `get_kafka_topic` round
+/// trip, not one per occurrence.
+async fn resolve_named_topics(bridge: &IggyBridge, names: &[StrBytes]) -> Vec<TopicResult> {
+    let mut seen = HashSet::with_capacity(names.len());
+    let mut distinct = Vec::new();
+    for name in names {
+        if seen.insert(name.as_str()) {
+            distinct.push(name.clone());
+        }
+    }
+
+    let mut results_by_name: HashMap<&str, TopicResult> = HashMap::with_capacity(distinct.len());
+    for name in &distinct {
+        let result = lookup_one_topic(bridge, name.clone()).await;
+        results_by_name.insert(name.as_str(), result);
+    }
+
+    names
+        .iter()
+        .map(|name| {
+            // Always present: `distinct` (and so results_by_name) was built from exactly these
+            // same requested names, just above.
+            let cached = results_by_name
+                .get(name.as_str())
+                .expect("every requested name was resolved above");

Review Comment:
    Metadata named-lookup response now dedupes to one entry per distinct name (matching real Kafka's .toSet() behavior), not one per request occurrence. resolve_named_topics  simplified — no longer re-expands. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on PR #4258:
URL: https://github.com/apache/iggy/pull/4258#issuecomment-5787050466

   /ready


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4077908338


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,324 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
 use kafka_protocol::messages::create_topics_request::CreatableTopic;
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST, ERROR_NONE, ERROR_NOT_CONTROLLER,
+    ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS, GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {
+        return result
+            .with_error_code(ERROR_INVALID_CONFIG)
+            .with_error_message(Some(StrBytes::from(
+                "per-topic configs are not supported by this bridge".to_string(),
+            )));
+    }
+
+    let partition_count = match validate_create_topic_shape(version, topic) {
+        Ok(count) => count,
+        Err(code) => return result.with_error_code(code),
+    };
+
+    let kafka_topic = topic.name.as_str();
+    let success = || {
+        result
+            .clone()
+            .with_error_code(ERROR_NONE)
+            .with_num_partitions(i32::try_from(partition_count).unwrap_or(i32::MAX))
+            .with_replication_factor(1)
+    };
+
+    if validate_only {
+        return match bridge.get_kafka_topic(kafka_topic).await {
+            Ok(Some(_existing)) => result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS),
+            Ok(None) => success(),
+            Err(err) => result
+                .with_error_code(err.to_kafka_error_code())
+                .with_error_message(Some(StrBytes::from(err.to_string()))),
+        };
+    }
+
+    match bridge
+        .create_kafka_topic(kafka_topic, partition_count)
+        .await
+    {
+        Ok(TopicCreationOutcome::Created) => success(),
+        Ok(TopicCreationOutcome::AlreadyExists) => {
+            result.with_error_code(ERROR_TOPIC_ALREADY_EXISTS)
+        }
+        Err(err) => result
+            .with_error_code(err.to_kafka_error_code())
+            .with_error_message(Some(StrBytes::from(err.to_string()))),

Review Comment:
   error_message no longer re-embeds the topic name via err.to_string() (which InvalidKafkaTopicName/InvalidPartitionCount's Display does) — new error_message_for extracts  just the reason text. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4077898958


##########
gateways/kafka/src/bridge/error.rs:
##########
@@ -163,7 +172,17 @@ const fn iggy_error_to_kafka_code(err: &IggyError) -> i16 {
         | IggyError::TcpError
         | IggyError::TransientNotAccepted => ERROR_NOT_LEADER_OR_FOLLOWER,
         IggyError::TransientNotCommitted => ERROR_REQUEST_TIMED_OUT,
-        IggyError::TooManyPartitions => ERROR_INVALID_PARTITIONS,
+        // Not `ERROR_INVALID_PARTITIONS` (37): that code's own text, per `kafka-protocol`'s
+        // table, is "Number of partitions is below 1" - the opposite condition from "too many"
+        // (Iggy's server-side cap, above 1000). Reusing 37 for both directions would return a
+        // client-visible error message that contradicts the actual request it sent.
+        IggyError::TooManyPartitions => ERROR_INVALID_REQUEST,
+        // The operation *did* commit - the SDK's own reconnect path replayed a write whose first
+        // attempt already applied, and the server's client-table dedup caught the replay. Falling
+        // into the catch-all below would report a permanent server fault for a request that
+        // actually succeeded; a Java client treats `UNKNOWN_SERVER_ERROR` as non-retriable and
+        // would surface a spurious failure for a `CreateTopics` that in fact created the topic.
+        IggyError::RequestAlreadyApplied => ERROR_NONE,

Review Comment:
   RequestAlreadyApplied => ERROR_NONE moved out of the shared iggy_error_to_kafka_code (every handler's error path) into a local match arm in CreateTopics' own write call — the only place "the write already applied" is a meaningful fact.   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093329062


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {

Review Comment:
   removed. Added IggyBridge::get_kafka_topics, which batches by the stream each name resolves to rather than by name count — a long-lived Java producer whose ProducerMetadata cumulative topic set crosses 100 no longer gets permanently bricked.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093349976


##########
gateways/kafka/src/protocol/handlers/create_topics.rs:
##########
@@ -17,44 +17,394 @@
 
 //! `CreateTopics` (API key 19).
 
+use std::collections::HashSet;
+use std::time::Duration;
+
 use bytes::Bytes;
-use kafka_protocol::messages::create_topics_request::CreatableTopic;
+use iggy::prelude::IggyError;
+use kafka_protocol::messages::create_topics_request::{CreatableReplicaAssignment, CreatableTopic};
 use kafka_protocol::messages::create_topics_response::CreatableTopicResult;
-use kafka_protocol::messages::{CreateTopicsRequest, CreateTopicsResponse};
+use kafka_protocol::messages::{BrokerId, CreateTopicsRequest, CreateTopicsResponse, TopicName};
+use kafka_protocol::protocol::StrBytes;
 
+use crate::bridge::{BridgeError, IggyBridge, TopicCreationOutcome};
 use crate::error::Result;
 use crate::protocol::api::{
-    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_PARTITIONS,
-    ERROR_INVALID_REPLICATION_FACTOR, ERROR_NONE, ERROR_NOT_CONTROLLER, GatewayState,
-    HandleOutcome,
+    API_KEY_CREATE_TOPICS, ApiVersionRange, ERROR_INVALID_CONFIG, ERROR_INVALID_PARTITIONS,
+    ERROR_INVALID_REPLICA_ASSIGNMENT, ERROR_INVALID_REPLICATION_FACTOR, ERROR_INVALID_REQUEST,
+    ERROR_NONE, ERROR_NOT_CONTROLLER, ERROR_REQUEST_TIMED_OUT, ERROR_TOPIC_ALREADY_EXISTS,
+    GatewayState, HandleOutcome,
 };
 use crate::protocol::bounds_guard::validate_create_topics_shape;
-use crate::protocol::handlers::{decode_guarded, encode_message, handle_versioned_request};
+use crate::protocol::handlers::{
+    decode_guarded, encode_message, handle_versioned_request, is_supported_version,
+    respond_or_close, unsupported_version_response,
+};
 
 pub const RANGE: ApiVersionRange = ApiVersionRange {
     api_key: API_KEY_CREATE_TOPICS,
     min_version: 2,
     max_version: 5,
 };
 
-#[expect(
-    clippy::unused_async,
-    reason = "the shared handler signature, kept until a handler awaits the bridge"
-)]
+/// KIP-464 `num_partitions = -1` with no manual assignment: the count this bridge creates.
+///
+/// Matches real Kafka's own out-of-box `num.partitions=1` broker default. Not read from any
+/// bridge config - there is no such config surface today.
+const DEFAULT_PARTITION_COUNT: u32 = 1;
+
+/// Cap on distinct topic names one `CreateTopics` request may address through the bridge.
+///
+/// `bounds_guard`'s `MAX_REQUEST_ELEMENTS` (4,096) is a pre-decode `DoS` ceiling, not a usability
+/// recommendation: each non-duplicate requested name here costs up to ~4 Iggy round trips
+/// (`ensure_stream` + `create_topic`, plus a possible race-retry read on either) against the
+/// single lockstep `IggyClient` every Kafka connection on this gateway shares
+/// (`bridge/iggy_bridge/mod.rs`'s "Concurrency ceiling"). 100 keeps a worst-case batch's
+/// aggregate bridge cost small relative to that shared resource while remaining generous for any
+/// real admin batch. Duplicate names never count against this cap - they're rejected by
+/// [`find_duplicate_names`] before ever reaching the bridge.
+const MAX_BRIDGE_BACKED_TOPICS: usize = 100;
+
+/// Bounds imposed on the request's own `timeout_ms` before it becomes the aggregate bridge-work
+/// deadline. That value is client-supplied and otherwise unchecked: `0` or negative would abort
+/// every topic on arrival, and an oversized one would tie up the shared `IggyClient` past any
+/// reasonable request.
+const MIN_REQUEST_TIMEOUT: Duration = Duration::from_millis(1_000);
+const MAX_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
+
+/// Clamps the wire's own `timeout_ms` (KIP-4's field for exactly this) into
+/// `[MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT]` - unlike `ListOffsets`/`Metadata`, `CreateTopics`
+/// carries a real client-supplied deadline to honor, not just a fixed internal ceiling.
+fn clamp_request_timeout(timeout_ms: i32) -> Duration {
+    let requested = Duration::from_millis(u64::try_from(timeout_ms).unwrap_or(0));
+    requested.clamp(MIN_REQUEST_TIMEOUT, MAX_REQUEST_TIMEOUT)
+}
+
 pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> HandleOutcome {
-    handle_versioned_request(
-        API_KEY_CREATE_TOPICS,
-        api_version,
-        body,
-        |v, b| {
-            decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
-                validate_create_topics_shape(v, b, state.max_frame_size)
+    let Some(bridge) = &state.bridge else {
+        return handle_versioned_request(
+            API_KEY_CREATE_TOPICS,
+            api_version,
+            body,
+            |v, b| {
+                decode_guarded::<CreateTopicsRequest>(v, b, |v, b| {
+                    validate_create_topics_shape(v, b, state.max_frame_size)
+                })
+            },
+            encode_response,
+            encode_error_response,
+            "CreateTopics",
+        );
+    };
+
+    if !is_supported_version(API_KEY_CREATE_TOPICS, api_version) {
+        return unsupported_version_response(API_KEY_CREATE_TOPICS, api_version, |version| {
+            encode_error_response(version, ERROR_INVALID_REQUEST)
+        });
+    }
+
+    let req = match decode_guarded::<CreateTopicsRequest>(api_version, body, |v, b| {
+        validate_create_topics_shape(v, b, state.max_frame_size)
+    }) {
+        Ok(req) => req,
+        Err(error) => {
+            // debug!, not warn!: attacker-controlled, not operator-actionable.
+            tracing::debug!(%error, "Failed to decode CreateTopics request");
+            return respond_or_close(
+                encode_error_response(api_version, ERROR_INVALID_REQUEST),
+                "CreateTopics",
+            );
+        }
+    };
+
+    let duplicate_names = find_duplicate_names(&req.topics);
+
+    let distinct_bridge_backed: HashSet<&TopicName> = req
+        .topics
+        .iter()
+        .map(|topic| &topic.name)
+        .filter(|name| !duplicate_names.contains(*name))
+        .collect();
+    if distinct_bridge_backed.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct_bridge_backed.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "CreateTopics request addresses too many distinct topics; rejecting"
+        );
+        let results = req
+            .topics
+            .iter()
+            .map(|topic| {
+                CreatableTopicResult::default()
+                    .with_name(topic.name.clone())
+                    .with_error_code(ERROR_INVALID_REQUEST)
             })
-        },
-        encode_response,
-        encode_error_response,
-        "CreateTopics",
+            .collect();
+        let resp = CreateTopicsResponse::default().with_topics(results);
+        return respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics");
+    }
+
+    let deadline = clamp_request_timeout(req.timeout_ms);
+    let results = match tokio::time::timeout(
+        deadline,
+        create_all_topics(
+            bridge,
+            api_version,
+            &req.topics,
+            &duplicate_names,
+            req.validate_only,
+        ),
     )
+    .await
+    {
+        Ok(results) => results,
+        Err(_elapsed) => {
+            tracing::warn!(
+                distinct_topics = distinct_bridge_backed.len(),
+                deadline_ms = deadline.as_millis(),
+                "CreateTopics request's aggregate bridge work exceeded its deadline; \
+                 answering retriable instead of blocking further"
+            );
+            req.topics
+                .iter()
+                .map(|topic| {
+                    CreatableTopicResult::default()
+                        .with_name(topic.name.clone())
+                        .with_error_code(ERROR_REQUEST_TIMED_OUT)
+                })
+                .collect()
+        }
+    };
+    let resp = CreateTopicsResponse::default().with_topics(results);
+    respond_or_close(encode_message(&resp, api_version, 256), "CreateTopics")
+}
+
+/// Creates (or reports on) every requested topic, skipping the bridge entirely for a duplicate
+/// name - real Kafka refuses the whole name, not a first-wins/last-wins split: creating one
+/// occurrence and reporting `TOPIC_ALREADY_EXISTS` for the other would let a client observe a
+/// create it never got a `NONE` for (`AdminClient` keys its futures by name, so a second
+/// per-topic result for the same name is silently discarded client-side regardless of which one
+/// this bridge picked).
+async fn create_all_topics(
+    bridge: &IggyBridge,
+    api_version: i16,
+    topics: &[CreatableTopic],
+    duplicate_names: &HashSet<TopicName>,
+    validate_only: bool,
+) -> Vec<CreatableTopicResult> {
+    let mut results = Vec::with_capacity(topics.len());
+    for topic in topics {
+        let result = if duplicate_names.contains(&topic.name) {
+            CreatableTopicResult::default()
+                .with_name(topic.name.clone())
+                .with_error_code(ERROR_INVALID_REQUEST)
+        } else {
+            create_one_topic(bridge, api_version, topic, validate_only).await
+        };
+        results.push(result);
+    }
+    results
+}
+
+/// Every topic name that appears more than once in `topics` - real Kafka
+/// (`ControllerApis.createTopics`) refuses every occurrence of a duplicate name with
+/// `INVALID_REQUEST` (42) and creates nothing for it, rather than creating the first occurrence
+/// and reporting the rest as already existing.
+fn find_duplicate_names(topics: &[CreatableTopic]) -> HashSet<TopicName> {
+    let mut seen = HashSet::with_capacity(topics.len());
+    let mut duplicates = HashSet::new();
+    for topic in topics {
+        if !seen.insert(topic.name.clone()) {
+            duplicates.insert(topic.name.clone());
+        }
+    }
+    duplicates
+}
+
+/// Validates and, when the topic is not rejected outright, provisions one requested topic.
+///
+/// `configs` is rejected before the shape check - a config-bearing request is rejected the same
+/// way regardless of how its partitions/replication are shaped.
+///
+/// `validate_only` and the real create path diverge deliberately below that point, not just in
+/// whether they call the bridge: `validate_only` never mutates anything, so a plain existence
+/// read ([`IggyBridge::get_kafka_topic`]) is fine - there's no race to protect against when
+/// nothing gets created either way. The real path instead calls
+/// [`IggyBridge::create_kafka_topic`], which folds the existence check and the create into one
+/// atomic call - a separate read-then-write here would let two concurrent `CreateTopics` for the
+/// same new name both observe `Ok(None)` and both receive `NONE`, when Kafka guarantees exactly
+/// one caller does.
+async fn create_one_topic(
+    bridge: &IggyBridge,
+    version: i16,
+    topic: &CreatableTopic,
+    validate_only: bool,
+) -> CreatableTopicResult {
+    let result = CreatableTopicResult::default().with_name(topic.name.clone());
+
+    if !topic.configs.is_empty() {

Review Comment:
    existence now beats config/shape rejection (probes only on the rejection path, no extra round trip on the happy path — fixes Kafka Connect's idempotent bootstrap); validate_only now
     enforces the same 1000-partition cap the real path does; replication_factor > 1 now rejected (was silently accepted then reported back as 1); error_message defaults to None instead of Some(""); 100-topic cap now
     answers POLICY_VIOLATION (44) with a descriptive message instead of INVALID_REQUEST.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093338755


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {
+            Ok(topics) => {
+                let results: Vec<TopicResult> = topics.into_iter().map(found_result).collect();
+                truncate_all_topics_to_frame_budget(results, state.max_frame_size)
+            }
+            Err(error) => {
+                // Same "no top-level error field" constraint as a decode failure: there is no
+                // way to answer "the bridge itself is unreachable" for an all-topics request
+                // that doesn't also falsely claim zero topics exist.
+                tracing::warn!(%error, "Failed to list Kafka topics from the Iggy bridge; closing connection");
+                return HandleOutcome::Close;
+            }
+        },
+        Some(names) => resolve_requested_named_topics(bridge, &names).await,
+    };
+
+    // `bounds_guard` cannot see this: it charges the projected response by *requested* element
+    // (one topic name), but one real topic can carry up to Iggy's own per-topic partition cap
+    // (1000) - a handful of names can still expand into a response `bounds_guard` never had the
+    // information to price in before this bridge round trip returned. The all-topics arm is
+    // pre-truncated to this same budget above (`truncate_all_topics_to_frame_budget`) since its
+    // size is server-side, not client-controllable - closing over it would take down every
+    // client's bootstrap Metadata call, permanently, the moment the catalog grows past the
+    // trip point. This check is what still enforces the budget for the named-lookup arm, where
+    // the cap (100 distinct topics) bounds the request but not what each one costs to answer.
+    let total_bytes: usize = results.iter().map(estimated_response_bytes).sum();
+    if response_would_exceed_frame_size(total_bytes, state.max_frame_size) {
+        tracing::warn!(
+            total_bytes,
+            max_frame_size = state.max_frame_size,
+            "Metadata response would exceed max_frame_size; closing connection"
+        );
+        return HandleOutcome::Close;
     }
+
+    respond_or_close(
+        encode_real_response(api_version, &results, &state.broker),
+        "Metadata",
+    )
+}
+
+/// Resolves a named-lookup Metadata request's topics: dedupes first so every path below (cap,
+/// deadline, success) builds its response from the distinct set rather than one entry per
+/// request occurrence - real Kafka answers a topic named more than once with one response entry,
+/// not one per repeat, and re-expanding to match the request let a handful of repeats of one
+/// large topic name amplify a response sized off the repeat count instead of the distinct count.
+async fn resolve_requested_named_topics(
+    bridge: &IggyBridge,
+    names: &[StrBytes],
+) -> Vec<TopicResult> {
+    let distinct = dedup_topic_names(names);
+    if distinct.len() > MAX_BRIDGE_BACKED_TOPICS {
+        tracing::warn!(
+            distinct_topics = distinct.len(),
+            max = MAX_BRIDGE_BACKED_TOPICS,
+            "Metadata request addresses too many distinct topics; rejecting"
+        );
+        return distinct
+            .iter()
+            .map(|name| error_result(name.clone(), ERROR_INVALID_REQUEST))
+            .collect();
+    }
+
+    match tokio::time::timeout(REQUEST_DEADLINE, resolve_named_topics(bridge, &distinct)).await {

Review Comment:
   similar



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(gateways): Implement CreateTopics and Metadata Kafka Keys backed by the Iggy bridge [iggy]

["ryerraguntla (via GitHub)" <gi...@apache.org>]

ryerraguntla commented on code in PR #4258:
URL: https://github.com/apache/iggy/pull/4258#discussion_r4093347709


##########
gateways/kafka/src/protocol/handlers/metadata.rs:
##########
@@ -53,23 +73,271 @@ pub async fn handle(state: &GatewayState, api_version: i16, body: Bytes) -> Hand
         );
         return HandleOutcome::Close;
     }
-    match decode_topics(api_version, body, state.max_frame_size) {
-        Ok(topics) => respond_or_close(
-            encode_response(api_version, &topics, &state.broker, ERROR_NONE),
-            "Metadata",
-        ),
+
+    let Some(bridge) = &state.bridge else {
+        return match decode_topics(api_version, body, state.max_frame_size) {
+            Ok(topics) => respond_or_close(
+                encode_response(api_version, &topics, &state.broker, ERROR_NONE),
+                "Metadata",
+            ),
+            Err(error) => {
+                // Metadata has no top-level error field; a malformed body cannot carry
+                // INVALID_REQUEST in a version-correct way for every client. Close.
+                // debug!, not warn!: attacker-controlled, not operator-actionable.
+                tracing::debug!(
+                    %error,
+                    api_version,
+                    "Failed to decode Metadata request; closing connection"
+                );
+                HandleOutcome::Close
+            }
+        };
+    };
+
+    let requested = match decode_requested_topics(api_version, body, state.max_frame_size) {
+        Ok(requested) => requested,
         Err(error) => {
-            // Metadata has no top-level error field; a malformed body cannot carry
-            // INVALID_REQUEST in a version-correct way for every client. Close.
-            // debug!, not warn!: attacker-controlled, not operator-actionable.
             tracing::debug!(
                 %error,
                 api_version,
                 "Failed to decode Metadata request; closing connection"
             );
-            HandleOutcome::Close
+            return HandleOutcome::Close;
         }
+    };
+
+    let results = match requested {
+        None => match bridge.list_kafka_topics().await {

Review Comment:
     Fixed, bridge.list_kafka_topics() now runs under tokio::time::timeout(REQUEST_DEADLINE, ...), closing the connection on expiry (matches the existing error-arm behavior, since there's no per-topic retriable list to send for this server-driven path).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org