You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@avro.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/03/13 07:11:00 UTC
[jira] [Commented] (AVRO-3683) Rust Writer, Reader can't use Schemas with dependencies in other Schemas. i.e. The output of Schema::parse_list
[ https://issues.apache.org/jira/browse/AVRO-3683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17699485#comment-17699485 ]
ASF subversion and git services commented on AVRO-3683:
-------------------------------------------------------
Commit 537025c0398313363e811e3e0258a433e8ee0d81 in avro's branch refs/heads/branch-1.11 from Martin Grigorov
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=537025c03 ]
AVRO-3683: [Rust] Read/Write with multiple schemas (#2014)
* AVRO-3683: Add support for using multiple schemata for resolve/validate/write
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: WIP
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: WIP compiles
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: WIP compiles and all tests pass
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: WIP Add support for reading
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: WIP
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: WIP
Use a main schema and pass all other schemata for resolution.
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: Formatting
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: WIP
Add support for multiple schemata in Reader/Writer APIs
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3646: Formatting
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: Use Vec instead of slice reference for schemata
It is much easier to deal with.
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: Fix the resolving of the writer schema when reading an Avro file
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
* AVRO-3683: Cleaup
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
---------
Signed-off-by: Martin Tzvetanov Grigorov <mg...@apache.org>
(cherry picked from commit b8b83b72f7184cf1b388fc20331d11eabbd93e06)
> Rust Writer, Reader can't use Schemas with dependencies in other Schemas. i.e. The output of Schema::parse_list
> -------------------------------------------------------------------------------------------------------------------
>
> Key: AVRO-3683
> URL: https://issues.apache.org/jira/browse/AVRO-3683
> Project: Apache Avro
> Issue Type: Bug
> Components: rust
> Affects Versions: 1.11.1
> Reporter: Mark Farnan
> Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Avro permits Schema definitions to reference Types defined in other Schema files.
> Currently these can be parsed with Schema:parse_list, but the output of this can't be used in Writer, Reader, (or to_avro_datum, from_avro_datum)
> * These functions only take a single 'schema' and can't resolve the References
> * Schema:parse_list puts in 'refs' for the external dependencies, but doesn't seem to populate the UnionSchema's, so they can be resolved.
>
> +Example of dependant schemas:+
> letschema_TypeA=r#"
> {
> "type": "record",
> "namespace": "base.name.space",
> "name": "MultiSchemaTestTypeA",
> "fields":
> [
> { "name": "b", "type": ["null", "some.name.space.MultiSchemaTestTypeB"] }
> ,
> { "name": "c", "type": \\{ "type": "map", "values": "some.name.space.MultiSchemaTestTypeB" }
> , "default": {}
> }
> ]
> }
> "#;
> letschema_TypeB=r#"
> {
> "type": "record",
> "namespace": "some.name.space",
> "name": "MultiSchemaTestTypeB",
> "fields":
> [
> { "name": "d", "type": "string" }
> ,
> { "name": "e", "type": "int" }
> ]
> }
> "#;
>
> PR Created with a failing test: https://github.com/apache/avro/pull/1997
--
This message was sent by Atlassian Jira
(v8.20.10#820010)