[jira] [Created] (ATLAS-1869) Atlas "plugin" for Ranger (metadata capture)

["Nigel Jones (JIRA)" <ji...@apache.org>]

Nigel Jones created ATLAS-1869:
----------------------------------

             Summary: Atlas "plugin" for Ranger (metadata capture)
                 Key: ATLAS-1869
                 URL: https://issues.apache.org/jira/browse/ATLAS-1869
             Project: Atlas
          Issue Type: Bug
            Reporter: Nigel Jones


With a variety of data processing engines in Hadoop such as Hive, we have an Atlas plugin & hook that captures new & updated metadata from those engines and pushes it to Atlas. This can then be used to support governance including lineage.

We already have a "ranger plugin" for Atlas which allows ranger to control access to metadata in atlas - this is NOT the subject of this Jira, but rather "the other way around"

Examples might include
 * Capture information about the policies that are deployed in a ranger server - the types of assets they refer to, the classifications that are used. 
 * Capture information about the topology of ranger - by this I mean the plugins that are deployed and active, the nodes they run on, and feed this back into an operational model in Atlas

In each case the information could be published by Ranger, consumed by Atlas, and stewardship activities around the atlas metadata could help in tying things together

The benefit would be
 - better end to end view (since we know the endpoints, identifiers in audit logs)
 - optimizing the interfaces (rest & kafka) by being able to better targer useful information - ie if only a hive plugin is being used & configured for tags, let's just worry the tags it needs.

I see this of use around our open metadata work & specifically VDC, though not essential for an initial MVP

Placeholder for now... will elaborate further



At the same time the coupling would be loose, and shouldn't hinder any existing integrations, or decisions as to what is done in Atlas vs Ranger



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)