You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by "sujitn (via GitHub)" <gi...@apache.org> on 2026/04/12 12:35:28 UTC
[PR] feat(server): return base_offset and partition_id from send_messages [iggy]
sujitn opened a new pull request, #3101:
URL: https://github.com/apache/iggy/pull/3101
The offset assigned during append was computed but discarded. Propagate it through ShardResponse::SendMessages and include it in the binary protocol response.
Wire format change:
Before: [status:4 LE][length:4 LE (0)]
After: [status:4 LE][length:4 LE (12)][base_offset:8 LE][partition_id:4 LE]
existing SDK clients read the length field and skip unknown body bytes. The send_messages wrapper discards the response body and returns Ok(()).
Ref: https://github.com/apache/iggy/discussions/3044
## Local Execution
- Passed
- Pre-commit hooks ran
## AI Usage
If AI tools were used, please answer:
1. Which tools?
Claude
2. Scope of usage?
SDK validation
3. How did you verify the generated code works correctly?
- 64/64 server unit tests pass (cargo test -p server --lib)
- 36/36 offset retrieval integration tests pass, these send messages via the SDK
- 87/87 concurrent produce/consume integration tests pass
- 6/6 authentication integration tests pass
- cargo clippy -p server --all-targets -- -D warnings - 0 warnings
4. Can you explain every line of the code if asked?
Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4344449463
We decided to postpone this until cluster v0 is merged.
I'll resume discussion at some point in future one of these threads:
https://github.com/apache/iggy/discussions/3081
https://github.com/apache/iggy/discussions/6
Until then, no code should be written besides spikes/PoCs. Of course, design studies can be continued.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "lukaszzborek (via GitHub)" <gi...@apache.org>.
lukaszzborek commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4246883968
I think we should at least create issues for every sdk to implement that change in api (just after merge)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "sujitn (via GitHub)" <gi...@apache.org>.
sujitn commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3074940070
##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
topic_id: topic.topic_id,
partition_id,
};
- shard.append_messages(partition, batch).await?;
+ let base_offset = shard.append_messages(partition, batch).await?;
- sender.send_empty_ok_response().await?;
+ const OFFSET_SIZE: usize = size_of::<u64>();
+ const PARTITION_SIZE: usize = size_of::<u32>();
+ const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+ let mut response = [0u8; RESPONSE_SIZE];
+ response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+ response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+ sender.send_ok_response(&response).await?;
Review Comment:
thanks for the review, added a new SendMessageResponse struct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4231561099
## [Codecov](https://app.codecov.io/gh/apache/iggy/pull/3101?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 71.58%. Comparing base ([`af92730`](https://app.codecov.io/gh/apache/iggy/commit/af9273081b6f0771a48e4b58ec08f1d513bb271c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`51865b8`](https://app.codecov.io/gh/apache/iggy/commit/51865b81bcf9fd0982b4306def75ac1152f58ade?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #3101 +/- ##
============================================
- Coverage 72.73% 71.58% -1.15%
Complexity 943 943
============================================
Files 1117 1054 -63
Lines 96342 91527 -4815
Branches 73541 73446 -95
============================================
- Hits 70070 65517 -4553
+ Misses 23722 23460 -262
Partials 2550 2550
```
| [Components](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [Rust Core](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.49% <ø> (-0.01%)` | :arrow_down: |
| [Java SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `62.30% <ø> (ø)` | |
| [C# SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `69.42% <ø> (ø)` | |
| [Python SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.43% <ø> (ø)` | |
| [Node SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `87.29% <ø> (-4.03%)` | :arrow_down: |
| [Go SDK](https://app.codecov.io/gh/apache/iggy/pull/3101/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `38.97% <ø> (ø)` | |
| [Files with missing lines](https://app.codecov.io/gh/apache/iggy/pull/3101?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../binary/handlers/messages/send\_messages\_handler.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fbinary%2Fhandlers%2Fmessages%2Fsend_messages_handler.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL2JpbmFyeS9oYW5kbGVycy9tZXNzYWdlcy9zZW5kX21lc3NhZ2VzX2hhbmRsZXIucnM=) | `100.00% <ø> (ø)` | |
| [core/server/src/http/http\_shard\_wrapper.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fhttp%2Fhttp_shard_wrapper.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL2h0dHAvaHR0cF9zaGFyZF93cmFwcGVyLnJz) | `74.12% <ø> (-0.18%)` | :arrow_down: |
| [core/server/src/shard/handlers.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fshard%2Fhandlers.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL3NoYXJkL2hhbmRsZXJzLnJz) | `77.83% <ø> (-0.12%)` | :arrow_down: |
| [core/server/src/shard/system/messages.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fshard%2Fsystem%2Fmessages.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL3NoYXJkL3N5c3RlbS9tZXNzYWdlcy5ycw==) | `88.05% <ø> (+0.08%)` | :arrow_up: |
| [core/server/src/shard/transmission/frame.rs](https://app.codecov.io/gh/apache/iggy/pull/3101?src=pr&el=tree&filepath=core%2Fserver%2Fsrc%2Fshard%2Ftransmission%2Fframe.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9zZXJ2ZXIvc3JjL3NoYXJkL3RyYW5zbWlzc2lvbi9mcmFtZS5ycw==) | `100.00% <ø> (ø)` | |
... and [81 files with indirect coverage changes](https://app.codecov.io/gh/apache/iggy/pull/3101/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3072366859
##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
topic_id: topic.topic_id,
partition_id,
};
- shard.append_messages(partition, batch).await?;
+ let base_offset = shard.append_messages(partition, batch).await?;
- sender.send_empty_ok_response().await?;
+ const OFFSET_SIZE: usize = size_of::<u64>();
+ const PARTITION_SIZE: usize = size_of::<u32>();
+ const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+ let mut response = [0u8; RESPONSE_SIZE];
+ response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+ response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+ sender.send_ok_response(&response).await?;
Review Comment:
this is done incorrectly. check crate `binary_protocol` and `core/binary_protocol/src/requests/messages/send_messages.rs`. IMHO you need to add SendMessagesResponse struct.
in other words, contract for binary protocol schema is in `binary_protocol` crate, not hardcoded in server sources.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4239705224
I'm gonna review this properly once we finalize current release 0.8.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "sujitn (via GitHub)" <gi...@apache.org>.
sujitn commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3074940070
##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
topic_id: topic.topic_id,
partition_id,
};
- shard.append_messages(partition, batch).await?;
+ let base_offset = shard.append_messages(partition, batch).await?;
- sender.send_empty_ok_response().await?;
+ const OFFSET_SIZE: usize = size_of::<u64>();
+ const PARTITION_SIZE: usize = size_of::<u32>();
+ const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+ let mut response = [0u8; RESPONSE_SIZE];
+ response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+ response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+ sender.send_ok_response(&response).await?;
Review Comment:
thanks for the review, added a new SendMessagesResponse struct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3072366859
##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
topic_id: topic.topic_id,
partition_id,
};
- shard.append_messages(partition, batch).await?;
+ let base_offset = shard.append_messages(partition, batch).await?;
- sender.send_empty_ok_response().await?;
+ const OFFSET_SIZE: usize = size_of::<u64>();
+ const PARTITION_SIZE: usize = size_of::<u32>();
+ const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+ let mut response = [0u8; RESPONSE_SIZE];
+ response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+ response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+ sender.send_ok_response(&response).await?;
Review Comment:
this is done incorrectly. check crate `binary_protocol` and `core/binary_protocol/src/requests/messages/send_messages.rs`. IMHO you need to add SendMessagesResponse struct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio closed pull request #3101: feat(server): return base_offset and partition_id from send_messages
URL: https://github.com/apache/iggy/pull/3101
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "tungtose (via GitHub)" <gi...@apache.org>.
tungtose commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3070914679
##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,11 @@ pub async fn handle_send_messages(
topic_id: topic.topic_id,
partition_id,
};
- shard.append_messages(partition, batch).await?;
+ let base_offset = shard.append_messages(partition, batch).await?;
- sender.send_empty_ok_response().await?;
+ let mut response = [0u8; 12];
Review Comment:
Can we use explicit types or constants instead of magic numbers?
eg:
```rust
const OFFSET_SIZE: usize = 8;
const PARTITION_SIZE: usize = 4;
const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
let mut response = [0u8; RESPONSE_SIZE];
response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "sujitn (via GitHub)" <gi...@apache.org>.
sujitn commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3071087422
##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,11 @@ pub async fn handle_send_messages(
topic_id: topic.topic_id,
partition_id,
};
- shard.append_messages(partition, batch).await?;
+ let base_offset = shard.append_messages(partition, batch).await?;
- sender.send_empty_ok_response().await?;
+ let mut response = [0u8; 12];
Review Comment:
done. thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on code in PR #3101:
URL: https://github.com/apache/iggy/pull/3101#discussion_r3072366859
##########
core/server/src/binary/handlers/messages/send_messages_handler.rs:
##########
@@ -197,8 +197,15 @@ pub async fn handle_send_messages(
topic_id: topic.topic_id,
partition_id,
};
- shard.append_messages(partition, batch).await?;
+ let base_offset = shard.append_messages(partition, batch).await?;
- sender.send_empty_ok_response().await?;
+ const OFFSET_SIZE: usize = size_of::<u64>();
+ const PARTITION_SIZE: usize = size_of::<u32>();
+ const RESPONSE_SIZE: usize = OFFSET_SIZE + PARTITION_SIZE;
+
+ let mut response = [0u8; RESPONSE_SIZE];
+ response[..OFFSET_SIZE].copy_from_slice(&base_offset.to_le_bytes());
+ response[OFFSET_SIZE..].copy_from_slice(&(partition_id as u32).to_le_bytes());
+ sender.send_ok_response(&response).await?;
Review Comment:
this is done incorrectly. check crate `binary_protocol` and `core/binary_protocol/src/requests/messages/send_messages.rs`. IMHO you need to add SendMessagesResponse struct to `core/binary_protocol/src/responses/messages/send_messages.rs` (it doesnt yet exist)
in other words, contract for binary protocol schema is in `binary_protocol` crate, not hardcoded in server sources.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(server): return base_offset and partition_id from send_messages [iggy]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #3101:
URL: https://github.com/apache/iggy/pull/3101#issuecomment-4292791020
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.
If you need a review, please ensure CI is green and the PR is rebased on the latest master. Don't hesitate to ping the maintainers - either @core on Discord or by mentioning them directly here on the PR.
Thank you for your contribution!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org