You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Kriskras99 (via GitHub)" <gi...@apache.org> on 2026/01/27 14:45:31 UTC
[PR] feat!: Rework `SpecificSingleObjectWriter` [avro-rs]
Kriskras99 opened a new pull request, #445:
URL: https://github.com/apache/avro-rs/pull/445
The first commit changes `ResolvedSchema` and `ResolvedOwnedSchema` to use the same logic for resolving.
`ResolvedOwnedSchema` now also uses a self-referential struct when resolving the schema, so it does not need to clone all named schemas contained in it.
The second commit contains the rework of `SpecificSingleObjectWriter`
It now resolves the schema and caches it. It also no longer uses `GenericSingleObjectWriter` for the header and does it directly. This removes the need for a buffer.
This is a breaking change:
- `SpecificSingleObjectWriter::with_capcity` has been removed.
- `write_avro_datum_ref` now also takes a `names: &NamesRef` argument
This also fixes a bug in `reader::Block::read_writer_schema` where it unnecessarily parses the schemata twice.
Closes #439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat!: Rework `SpecificSingleObjectWriter` [avro-rs]
Posted by "Kriskras99 (via GitHub)" <gi...@apache.org>.
Kriskras99 commented on code in PR #445:
URL: https://github.com/apache/avro-rs/pull/445#discussion_r2733418409
##########
avro/src/writer.rs:
##########
@@ -617,44 +630,54 @@ impl<T> SpecificSingleObjectWriter<T>
where
T: AvroSchema + Into<Value>,
{
- /// Write the `Into<Value>` to the provided Write object. Returns a result with the number
- /// of bytes written including the header
- pub fn write_value<W: Write>(&mut self, data: T, writer: &mut W) -> AvroResult<usize> {
- let v: Value = data.into();
- self.inner.write_value_ref(&v, writer)
+ /// Write the value to the writer
+ ///
+ /// Returns the number of bytes written.
+ ///
+ /// Each call writes a complete single-object encoded message (header + data),
+ /// making each message independently decodable.
+ pub fn write_value<W: Write>(&self, data: T, writer: &mut W) -> AvroResult<usize> {
+ writer
+ .write_all(&self.header)
+ .map_err(Details::WriteBytes)?;
+ let value: Value = data.into();
+ let bytes = write_value_ref_owned_resolved(&self.resolved, &value, writer)?;
Review Comment:
`writer.write_all(&buffer)` can also fail partway through.
I don't think this is worth the performance penalty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat!: Rework `SpecificSingleObjectWriter` [avro-rs]
Posted by "martin-g (via GitHub)" <gi...@apache.org>.
martin-g commented on code in PR #445:
URL: https://github.com/apache/avro-rs/pull/445#discussion_r2732737171
##########
avro/tests/serde_human_readable_true.rs:
##########
@@ -127,7 +126,7 @@ fn avro_rs_440_uuid_fixed() -> TestResult {
let mut buffer = Vec::new();
assert!(apache_avro::util::set_serde_human_readable(true));
- let mut writer = SpecificSingleObjectWriter::with_capacity(64)?;
Review Comment:
nit: We can keep `with_capacity()` as a delegate to `new()` and deprecated for a version or two.
##########
avro/src/writer.rs:
##########
@@ -617,44 +630,54 @@ impl<T> SpecificSingleObjectWriter<T>
where
T: AvroSchema + Into<Value>,
{
- /// Write the `Into<Value>` to the provided Write object. Returns a result with the number
- /// of bytes written including the header
- pub fn write_value<W: Write>(&mut self, data: T, writer: &mut W) -> AvroResult<usize> {
- let v: Value = data.into();
- self.inner.write_value_ref(&v, writer)
+ /// Write the value to the writer
+ ///
+ /// Returns the number of bytes written.
+ ///
+ /// Each call writes a complete single-object encoded message (header + data),
+ /// making each message independently decodable.
+ pub fn write_value<W: Write>(&self, data: T, writer: &mut W) -> AvroResult<usize> {
+ writer
+ .write_all(&self.header)
+ .map_err(Details::WriteBytes)?;
+ let value: Value = data.into();
+ let bytes = write_value_ref_owned_resolved(&self.resolved, &value, writer)?;
Review Comment:
If this fails then the writer will contain the already written header.
How about something like:
```rust
let mut buffer = Vec::new();
let bytes = write_value_ref_owned_resolved(&self.resolved, &value, &mut buffer)?;
writer.write_all(&self.header).map_err(Details::WriteBytes)?;
writer.write_all(&buffer).map_err(Details::WriteBytes)?;
Ok(bytes + self.header.len())
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat!: Rework `SpecificSingleObjectWriter` [avro-rs]
Posted by "Kriskras99 (via GitHub)" <gi...@apache.org>.
Kriskras99 merged PR #445:
URL: https://github.com/apache/avro-rs/pull/445
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@avro.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org