[jira] [Created] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas

["Srikanth Venkat (JIRA)" <ji...@apache.org>]

Srikanth Venkat created ATLAS-2694:
--------------------------------------

             Summary: Avro schema typedef and support for Avro schema evolution  in Atlas
                 Key: ATLAS-2694
                 URL: https://issues.apache.org/jira/browse/ATLAS-2694
             Project: Atlas
          Issue Type: Improvement
          Components:  atlas-core
            Reporter: Srikanth Venkat


Currently the base types in Atlas do not include Avro schemas. It would be nice to add typedef for Avro schema and any associated metadata to support schema evolution.
 * For example, Avro_schema type supports:
 ** All avro types, both primitive and complex, including union types, as fields of schema
 ** All types have doc strings and defaults
 ** A field of a schema can be another schema
 ** Indefinite nesting of records, arrays.
 ** Associated entities array attribute contains pointers to all datasets that reflect the avro schema
 ** Fully expanded avroNotation for use in serDe
 ** Schema evolution features such as isLatest (Boolean) and version number
 * Schema evolution Process
 ** Input: avro schema
 ** Output: new version of avro schema
 ** Compatibility: FULL, BACKWARD, FORWARD, NONE
 ** IsBreakingChange (Boolean): does the change produce an incompatible schema? (ie its compatibility is not “FULL”)
 * Atlas UI enhancement for JSON-valued attributes to support avro schema and avro schema evolution
 ** Currently JSON-valued attributes are fully displayed in-line with other attributes, not pretty-printed, cluttering the display.  To support a better display, we can display JSON-valued attributes in a one-line box that can be scrolled down, or fully expanded with a mouse click that pretty-prints the JSON. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)