You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by "atharvalade (via GitHub)" <gi...@apache.org> on 2026/04/25 23:06:15 UTC
[I] Dynamic router extract_route_field produces JSON-escaped strings [iggy]
atharvalade opened a new issue, #3177:
URL: https://github.com/apache/iggy/issues/3177
In `iceberg_sink/src/router/dynamic_router.rs`, the `extract_route_field` method used `val.to_string()` on a `simd_json::OwnedValue`:
```rust
.map(|val| val.to_string())
```
For a string value like `tpch.lineitem`, `to_string()` produces the JSON representation `"tpch.lineitem"` (with literal quote characters). This means the route field value passed to `is_valid_namespaced_table` and used as a table lookup key contains surrounding quotes, so it either fails validation or looks up a nonexistent table name.
This means dynamic routing with string-valued route fields has never worked correctly, or has never been tested with a real Iceberg catalog end-to-end.
**Fix** : Use `val.as_str().map(|s| s.to_string())` instead of `val.to_string()`.
This should be covered by an integration test that writes a message with a string route field and verifies it lands in the correct Iceberg table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Dynamic router extract_route_field produces JSON-escaped strings [iggy]
Posted by "EdgarModesto23 (via GitHub)" <gi...@apache.org>.
EdgarModesto23 commented on issue #3177:
URL: https://github.com/apache/iggy/issues/3177#issuecomment-4464195142
Hi @atharvalade . Could you share more details as to why do you think this is an issue? Did you find this issue while running this on your own? You mentioned that it's using .to_string() on a simd_json::OwnedValue, and this causes it to turn the value from the message into a string with literal quotes. But simd_json uses the write! macro on [its implementation of the Display trait ](https://github.com/simd-lite/simd-json/blob/main/src/value/owned.rs#L254) and only gets the inner string. This is different to what [serde_json does](https://github.com/serde-rs/json/blob/v1.0.149/src/value/mod.rs#L197), which will include quotes.
I prepared a [repository ](https://github.com/EdgarModesto23/iggy/tree/test)with a docker-compose to setup a local iceberg environment for testing end-to-end if you'd like to test this yourself. In our end-to-end testing we didn't find any issues on dynamic routing. If in your testing you find that this is indeed an issue happening please share the results with us so we can address it appropiately.
Thank you!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Dynamic router extract_route_field produces JSON-escaped strings [iggy]
Posted by "atharvalade (via GitHub)" <gi...@apache.org>.
atharvalade commented on issue #3177:
URL: https://github.com/apache/iggy/issues/3177#issuecomment-4515654104
You're right, and I apologize for the inaccuracy. I went back and checked the `simd_json` source and confirmed that Display for` Value::String` uses `write!(f, "{s}")` which outputs the raw string without quotes. I conflated this with `serde_json` behavior.
What actually happened during my testing was that I had multiple issues at once (the source was emitting a nested `DatabaseRecord` envelope that the sink couldn't parse, plus type handling failures on NUMERIC/DATE columns). I incorrectly attributed part of the routing failure to `to_string()` adding quotes when the real problem was the message format incompatibility. The change to` as_str()` is marginally more type-safe (returns `None` if the value isn't a string rather than giving you the `Display` output of a non-string type) but it's not fixing an actual quoting bug. I'll close this issue. Thanks for taking the time to verify it. @EdgarModesto23
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Dynamic router extract_route_field produces JSON-escaped strings [iggy]
Posted by "atharvalade (via GitHub)" <gi...@apache.org>.
atharvalade closed issue #3177: Dynamic router extract_route_field produces JSON-escaped strings
URL: https://github.com/apache/iggy/issues/3177
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Dynamic router extract_route_field produces JSON-escaped strings [iggy]
Posted by "hubcio (via GitHub)" <gi...@apache.org>.
hubcio commented on issue #3177:
URL: https://github.com/apache/iggy/issues/3177#issuecomment-4505418119
@Standing-Man have you managed to reproduce this issue? what's the status for #3256 ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Dynamic router extract_route_field produces JSON-escaped strings [iggy]
Posted by "atharvalade (via GitHub)" <gi...@apache.org>.
atharvalade commented on issue #3177:
URL: https://github.com/apache/iggy/issues/3177#issuecomment-4331263397
@Standing-Man Sure, I don't mind. Someone from @core will assign this to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Dynamic router extract_route_field produces JSON-escaped strings [iggy]
Posted by "Standing-Man (via GitHub)" <gi...@apache.org>.
Standing-Man commented on issue #3177:
URL: https://github.com/apache/iggy/issues/3177#issuecomment-4321449634
Hi @atharvalade and @hubcio, I’d like to take a stab at this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org