You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by "sujitn (via GitHub)" <gi...@apache.org> on 2026/04/12 12:35:28 UTC

[PR] feat(server): return base_offset and partition_id from send_messages [iggy]

sujitn opened a new pull request, #3101:
URL: https://github.com/apache/iggy/pull/3101

   The offset assigned during append was computed but discarded. Propagate it through ShardResponse::SendMessages and include it in the binary protocol response.
   
   Wire format change:
     Before: [status:4 LE][length:4 LE (0)]
     After:  [status:4 LE][length:4 LE (12)][base_offset:8 LE][partition_id:4 LE]
   
   existing SDK clients read the length field and skip unknown body bytes. The send_messages wrapper discards the response body and returns Ok(()).
   
   Ref: https://github.com/apache/iggy/discussions/3044
   
   ## Local Execution
   - Passed
   - Pre-commit hooks ran 
   
   ## AI Usage
   If AI tools were used, please answer:
   1. Which tools? 
   Claude
   
   2. Scope of usage? 
   SDK validation 
   
   3. How did you verify the generated code works correctly? 
     - 64/64 server unit tests pass (cargo test -p server --lib)
     - 36/36 offset retrieval integration tests pass, these send messages via the SDK
     - 87/87 concurrent produce/consume integration tests pass 
     - 6/6 authentication integration tests pass 
     - cargo clippy -p server --all-targets -- -D warnings - 0 warnings
   
   4. Can you explain every line of the code if asked? 
   Yes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4344449463

   We decided to postpone this until cluster v0 is merged.
   I'll resume discussion at some point in future one of these threads:
   https://github.com/apache/iggy/discussions/3081
   https://github.com/apache/iggy/discussions/6
   
   Until then, no code should be written besides spikes/PoCs. Of course, design studies can be continued.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "lukaszzborek (via GitHub)" <gi...@apache.org>.
lukaszzborek commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4246883968

   I think we should at least create issues for every sdk to implement that change in api (just after merge)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "sujitn (via GitHub)" <gi...@apache.org>.
sujitn commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3074940070


##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
         topic_id: topic.topic_id,
         partition_id,
     };
-    shard.append_messages(partition, batch).await?;
+    let base_offset = shard.append_messages(partition, batch).await?;
 
-    sender.send_empty_ok_response().await?;
+    const OFFSET_SIZE: usize = size_of::<u64>();
+    const PARTITION_SIZE: usize = size_of::<u32>();
+    const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+    let mut response = [0u8; RESPONSE_SIZE];
+    response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+    response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+    sender.send_ok_response(&response).await?;

Review Comment:
   thanks for the review, added a new SendMessageResponse struct



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4231561099

   ## [Codecov](https://app.codecov.io/gh/apache/iggy/pull/3101?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 71.58%. Comparing base ([`af92730`](https://app.codecov.io/gh/apache/iggy/commit/af9273081b6f0771a48e4b58ec08f1d513bb271c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`51865b8`](https://app.codecov.io/gh/apache/iggy/commit/51865b81bcf9fd0982b4306def75ac1152f58ade?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #3101      +/-   ##
   ============================================
   - Coverage     72.73%   71.58%   -1.15%     
     Complexity      943      943              
   ============================================
     Files          1117     1054      -63     
     Lines         96342    91527    -4815     
     Branches      73541    73446      -95     
   ============================================
   - Hits          70070    65517    -4553     
   + Misses        23722    23460     -262     
     Partials       2550     2550              
   ```
   
   | [Components](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [Rust Core](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.49% <ø> (-0.01%)` | :arrow_down: |
   | [Java SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.30% <ø> (ø)` | |
   | [C# SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `69.42% <ø> (ø)` | |
   | [Python SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.43% <ø> (ø)` | |
   | [Node SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `87.29% <ø> (-4.03%)` | :arrow_down: |
   | [Go SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.97% <ø> (ø)` | |
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/iggy/pull/3101?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [.../binary/handlers/messages/send\_messages\_handler.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fbinary%2Fhandlers%2Fmessages%2Fsend_messages_handler.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL2JpbmFyeS9oYW5kbGVycy9tZXNzYWdlcy9zZW5kX21lc3NhZ2VzX2hhbmRsZXIucnM=) | `100.00% <ø> (ø)` | |
   | [core/server/src/http/http\_shard\_wrapper.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fhttp%2Fhttp_shard_wrapper.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL2h0dHAvaHR0cF9zaGFyZF93cmFwcGVyLnJz) | `74.12% <ø> (-0.18%)` | :arrow_down: |
   | [core/server/src/shard/handlers.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fshard%2Fhandlers.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL3NoYXJkL2hhbmRsZXJzLnJz) | `77.83% <ø> (-0.12%)` | :arrow_down: |
   | [core/server/src/shard/system/messages.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fshard%2Fsystem%2Fmessages.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL3NoYXJkL3N5c3RlbS9tZXNzYWdlcy5ycw==) | `88.05% <ø> (+0.08%)` | :arrow_up: |
   | [core/server/src/shard/transmission/frame.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fshard%2Ftransmission%2Fframe.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL3NoYXJkL3RyYW5zbWlzc2lvbi9mcmFtZS5ycw==) | `100.00% <ø> (ø)` | |
   
   ... and [81 files with indirect coverage changes](https://app.codecov.io/gh/apache/iggy/pull/3101/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3072366859


##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
         topic_id: topic.topic_id,
         partition_id,
     };
-    shard.append_messages(partition, batch).await?;
+    let base_offset = shard.append_messages(partition, batch).await?;
 
-    sender.send_empty_ok_response().await?;
+    const OFFSET_SIZE: usize = size_of::<u64>();
+    const PARTITION_SIZE: usize = size_of::<u32>();
+    const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+    let mut response = [0u8; RESPONSE_SIZE];
+    response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+    response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+    sender.send_ok_response(&response).await?;

Review Comment:
   this is done incorrectly. check crate `binary_protocol` and `core/binary_protocol/src/requests/messages/send_messages.rs`. IMHO you need to add SendMessagesResponse struct.
   
   in other words, contract for binary protocol schema is in `binary_protocol` crate, not hardcoded in server sources.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4239705224

   I'm gonna review this properly once we finalize current release 0.8.0.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "sujitn (via GitHub)" <gi...@apache.org>.
sujitn commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3074940070


##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
         topic_id: topic.topic_id,
         partition_id,
     };
-    shard.append_messages(partition, batch).await?;
+    let base_offset = shard.append_messages(partition, batch).await?;
 
-    sender.send_empty_ok_response().await?;
+    const OFFSET_SIZE: usize = size_of::<u64>();
+    const PARTITION_SIZE: usize = size_of::<u32>();
+    const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+    let mut response = [0u8; RESPONSE_SIZE];
+    response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+    response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+    sender.send_ok_response(&response).await?;

Review Comment:
   thanks for the review, added a new SendMessagesResponse struct



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3072366859


##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
         topic_id: topic.topic_id,
         partition_id,
     };
-    shard.append_messages(partition, batch).await?;
+    let base_offset = shard.append_messages(partition, batch).await?;
 
-    sender.send_empty_ok_response().await?;
+    const OFFSET_SIZE: usize = size_of::<u64>();
+    const PARTITION_SIZE: usize = size_of::<u32>();
+    const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+    let mut response = [0u8; RESPONSE_SIZE];
+    response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+    response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+    sender.send_ok_response(&response).await?;

Review Comment:
   this is done incorrectly. check crate `binary_protocol` and `core/binary_protocol/src/requests/messages/send_messages.rs`. IMHO you need to add SendMessagesResponse struct.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio closed pull request #3101: feat(server): return base_offset and partition_id from send_messages
URL: https://github.com/apache/iggy/pull/3101


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "tungtose (via GitHub)" <gi...@apache.org>.
tungtose commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3070914679


##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,11 @@ pub async fn handle_send_messages(
         topic_id: topic.topic_id,
         partition_id,
     };
-    shard.append_messages(partition, batch).await?;
+    let base_offset = shard.append_messages(partition, batch).await?;
 
-    sender.send_empty_ok_response().await?;
+    let mut response = [0u8; 12];

Review Comment:
   Can we use explicit types or constants instead of magic numbers?
   eg:
   ```rust
   const OFFSET_SIZE: usize = 8;
   const PARTITION_SIZE: usize = 4;
   const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
   
   let mut response = [0u8; RESPONSE_SIZE];
   response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
   response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "sujitn (via GitHub)" <gi...@apache.org>.
sujitn commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3071087422


##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,11 @@ pub async fn handle_send_messages(
         topic_id: topic.topic_id,
         partition_id,
     };
-    shard.append_messages(partition, batch).await?;
+    let base_offset = shard.append_messages(partition, batch).await?;
 
-    sender.send_empty_ok_response().await?;
+    let mut response = [0u8; 12];

Review Comment:
   done. thanks



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3072366859


##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
         topic_id: topic.topic_id,
         partition_id,
     };
-    shard.append_messages(partition, batch).await?;
+    let base_offset = shard.append_messages(partition, batch).await?;
 
-    sender.send_empty_ok_response().await?;
+    const OFFSET_SIZE: usize = size_of::<u64>();
+    const PARTITION_SIZE: usize = size_of::<u32>();
+    const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+    let mut response = [0u8; RESPONSE_SIZE];
+    response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+    response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+    sender.send_ok_response(&response).await?;

Review Comment:
   this is done incorrectly. check crate `binary_protocol` and `core/binary_protocol/src/requests/messages/send_messages.rs`. IMHO you need to add SendMessagesResponse struct to `core/binary_protocol/src/responses/messages/send_messages.rs` (it doesnt yet exist)
   
   in other words, contract for binary protocol schema is in `binary_protocol` crate, not hardcoded in server sources.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4292791020

   This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.
   
   If you need a review, please ensure CI is green and the PR is rebased on the latest master. Don't hesitate to ping the maintainers - either @core on Discord or by mentioning them directly here on the PR.
   
   Thank you for your contribution!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org