You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ehoner (via GitHub)" <gi...@apache.org> on 2023/04/03 17:50:03 UTC
[GitHub] [samza] ehoner opened a new pull request, #1661: SAMZA-2777: Add support for GenericFixed to AzureBlobAvroWriter
ehoner opened a new pull request, #1661:
URL: https://github.com/apache/samza/pull/1661
The existing implementation requires that all messages are either of type `IndexedRecord` (a deserialized object with `Schema`) or an Avro-encoded `byte[]` that the writer aggregates in an Avro file format. All messages are required to use the same `Schema` and the `Schema` is set from the first `IndexedRecord` received. After the `Schema` is set, subsequent messages can be sent as `byte[]`s. `IndexedRecord`s are deserialized and must be encoded before aggregating. The `GenericFixed` is a `Schema` **and** Avro-encoded `byte[]` - [javadoc](https://avro.apache.org/docs/1.7.7/api/java/org/apache/avro/generic/GenericFixed.html). Adding support for `GenericFixed`, allows the caller to set the `Schema` and send **only** encoded messages.
There are no changes to the original design in [SEP-26](https://cwiki.apache.org/confluence/display/SAMZA/SEP-26:+Azure+Blob+Storage+Producer). These changes extend item 3 under "[Implementation](https://cwiki.apache.org/confluence/display/SAMZA/SEP-26:+Azure+Blob+Storage+Producer#SEP26:AzureBlobStorageProducer-Implementation/TestPlan)", see _AvroAzureBlobWriter is an AzureBlobWriter implementation to write messages which are Avro record..._.
#### Changes
1. No public API change.
2. New test: `testWriteGenericFixed`.
3. Updated javadocs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [samza] atoomula merged pull request #1661: SAMZA-2777: Add support for GenericFixed to AzureBlobAvroWriter
Posted by "atoomula (via GitHub)" <gi...@apache.org>.
atoomula merged PR #1661:
URL: https://github.com/apache/samza/pull/1661
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@samza.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org