[jira] [Commented] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas

["Nagaraj Janardhana (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ATLAS-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16512248#comment-16512248 ] 

Nagaraj Janardhana commented on ATLAS-2694:
-------------------------------------------

Thanks [~barbara] for the AVRO typedef, very useful.

Would you also have a parser/converter to build Atlas Avro instance (based on the typedef) from the AVRO schema. 

Just trying to avoid some duplicate work :)

> Avro schema typedef and support for Avro schema evolution  in Atlas
> -------------------------------------------------------------------
>
>                 Key: ATLAS-2694
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2694
>             Project: Atlas
>          Issue Type: New Feature
>          Components:  atlas-core
>            Reporter: Srikanth Venkat
>            Assignee: Barbara Eckman
>            Priority: Critical
>             Fix For: 1.0.0
>
>         Attachments: ATLAS-2694-2.patch, ATLAS-2694.patch, avro_atlas_types_08.json
>
>
> Currently the base types in Atlas do not include Avro schemas. It would be nice to add typedef for Avro schema and any associated metadata to support schema evolution.
>  * For example, Avro_schema type supports:
>  ** All avro types, both primitive and complex, including union types, as fields of schema
>  ** All types have doc strings and defaults
>  ** A field of a schema can be another schema
>  ** Indefinite nesting of records, arrays.
>  ** Associated entities array attribute contains pointers to all datasets that reflect the avro schema
>  ** Fully expanded avroNotation for use in serDe
>  ** Schema evolution features such as isLatest (Boolean) and version number
>  * Schema evolution Process
>  ** Input: avro schema
>  ** Output: new version of avro schema
>  ** Compatibility: FULL, BACKWARD, FORWARD, NONE
>  ** IsBreakingChange (Boolean): does the change produce an incompatible schema? (ie its compatibility is not “FULL”)
>  *



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)