Review Request 51936: Framework to apply updates to types in the type-system

[Sarath Kumar Subramanian <sa...@gmail.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51936/
-----------------------------------------------------------

Review request for atlas and Madhan Neethiraj.


Bugs: ATLAS-1174
    https://issues.apache.org/jira/browse/ATLAS-1174


Repository: atlas


Description
-------

1. Introduce "version" attribute to all types in the type-system, this helps to track changes made to the default types (hive, sqoop, falcon and storm types) and user created types. If version is not mentioned during creation of a type, default version "1.0" is assigned (optional attribute).
2. Using the version attributed for types, introduce a patch framework for type system. This framework applies patches to a type using its version number and can be used during upgrade - add new attributes to an existing types and it will be run during atlas startup.
The sequence of steps:
a. During atlas startup, check $ATLAS_HOME/models/patches directory for any available patch files (json files). If there any patch files handle them.
b. Sample patch json file looks like:
{
"patches": [
{ 
"action": "ADD_ATTRIBUTE",
"typeName": "hive_column",
"applyToVersion": "1.0",
"updateToVersion": "2.0",
"actionParams": [
{ "name": "position", "dataTypeName": "int", "multiplicity": "optional", "isComposite": false, "isUnique": false, "isIndexable": false, "reverseAttributeName": null }
]
} ]
}
c. The framework updates the type in "typeName" for the matching version number and after applying the patch, update the version to the one mentioned in "updateToVersion"
d. The json file can have more than one action (array of actions).
e. There can be multiple patch json files in the directory and are applied in the sort order of the filename. eg:
001-hive_column_add_position.json
002-hive_column_add_anotherattribute.json


Diffs
-----

  repository/src/main/java/org/apache/atlas/services/AtlasTypeUpdate.java PRE-CREATION 

Diff: https://reviews.apache.org/r/51936/diff/


Testing
-------


Thanks,

Sarath Kumar Subramanian