[jira] [Updated] (ATLAS-1095) Define Requirements: Data and metadata association via open connector framework

["Stephanie Hazlewood (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ATLAS-1095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stephanie Hazlewood updated ATLAS-1095:
---------------------------------------
             Assignee: Stephanie Hazlewood
    Affects Version/s: 0.8-incubating
          Description: 
Atlas is intended to provide a common approach to data governance and lineage across all systems and data within an organization.  Today Atlas provides access to metadata.   A connector provides access to a data source.  As connectors are the proxy of all data, they can also be explicit providers of metadata.   An open connector framework is being proposed to provide access to both data and the metadata Atlas provides together – the framework being the point at which data and metadata can be associated. This will help to better the exchange of information between platforms. It also offers new opportunities for the consistent enforcement of the governance policies and rules (e.g., rules of visibility).  Source connector/connection metadata provides the nucleus around which all other metadata describing the data builds.  

Introducing this framework:

Provides the necessary broker capabilities for normalized access to (finding and instantiating) a connector instance. 

Provides extension API supporting both partial and incremental adoption of the framework by existing connector provider runtimes allowing connector providers/contributors to adapt what they have vs re-write.

Introducing the notion of metadata-enabled open connectors:

Provides an API for the normalized access to the metadata describing a connector (asset type, connection type)

Provides base APIs for creating new connectors of different types and interaction styles.

The key personas using the framework are the tool developers and developers.  

As a connector developer, I want to create a new connector (or register an existing connector) to plug into the framework so that I can retrieve and manage metadata about what the source provides and ensure data used from that source has governance policies (e.g., rules of visibility) consistently configured and enforced. 

The tool developer uses a connector available through the framework to implement an application. Leveraging the APIs of the connector framework applications will be able to find relevant sources systems and connect to them. 



  was:
 Atlas is intended to provide a common approach to data governance and lineage across all systems and data within an organization.  Today Atlas provides access to metadata.   A connector provides access to a data source.  As connectors are the proxy of all data, they can also be explicit providers of metadata.   An open connector framework is being proposed to provide access to both data and metadata Atlas provides together – the framework being the point at which data and metadata can be associated. This will help to better the exchange of information between platforms (e.g., via connectors to Netezza) and in doing so, new opportunities for consistent enforcement of the governance rules (e.g., rules of visibility).  Source connector/connection metadata is the nucleus around which all other metadata describing the data builds.  
The open connector framework will to provide a simple, cohesive set of interfaces that make it easy to 

-	Find and maintain the metadata about the connectors available to a consuming application/developer
-	Create a connection to a system through a consistently defined API
-	Standardize the development of new metadata service-enabled connectors that have consistently implemented call outs to Atlas’ security and policy engine components.
-	Remain open so connectors from various connector providers may be registered and plugged into the framework and leveraged.

The key personas using the framework are the tool developers and developers.  

As a connector developer, I want to create a new connector (or register an existing connector) to plug into the framework so that I can retrieve and manage metadata about what the source provides and ensure data used from that source has governance policies (e.g., rules of visibility) consistently configured and enforced. 

The tool developer uses a connector available through the framework to implement an application. Leveraging the APIs of the connector framework applications will be able to find relevant sources systems and connect to them. 



              Summary: Define Requirements: Data and metadata association via open connector framework  (was: Data and metadata association via open connector framework)

> Define Requirements: Data and metadata association via open connector framework
> -------------------------------------------------------------------------------
>
>                 Key: ATLAS-1095
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1095
>             Project: Atlas
>          Issue Type: New Feature
>    Affects Versions: 0.8-incubating
>            Reporter: Stephanie Hazlewood
>            Assignee: Stephanie Hazlewood
>
> Atlas is intended to provide a common approach to data governance and lineage across all systems and data within an organization.  Today Atlas provides access to metadata.   A connector provides access to a data source.  As connectors are the proxy of all data, they can also be explicit providers of metadata.   An open connector framework is being proposed to provide access to both data and the metadata Atlas provides together – the framework being the point at which data and metadata can be associated. This will help to better the exchange of information between platforms. It also offers new opportunities for the consistent enforcement of the governance policies and rules (e.g., rules of visibility).  Source connector/connection metadata provides the nucleus around which all other metadata describing the data builds.  
> Introducing this framework:
> Provides the necessary broker capabilities for normalized access to (finding and instantiating) a connector instance. 
> Provides extension API supporting both partial and incremental adoption of the framework by existing connector provider runtimes allowing connector providers/contributors to adapt what they have vs re-write.
> Introducing the notion of metadata-enabled open connectors:
> Provides an API for the normalized access to the metadata describing a connector (asset type, connection type)
> Provides base APIs for creating new connectors of different types and interaction styles.
> The key personas using the framework are the tool developers and developers.  
> As a connector developer, I want to create a new connector (or register an existing connector) to plug into the framework so that I can retrieve and manage metadata about what the source provides and ensure data used from that source has governance policies (e.g., rules of visibility) consistently configured and enforced. 
> The tool developer uses a connector available through the framework to implement an application. Leveraging the APIs of the connector framework applications will be able to find relevant sources systems and connect to them. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)