[jira] [Created] (KUDU-2895) Native Apache Atlas Support

["Grant Henke (JIRA)" <ji...@apache.org>]

Grant Henke created KUDU-2895:
---------------------------------

             Summary: Native Apache Atlas Support
                 Key: KUDU-2895
                 URL: https://issues.apache.org/jira/browse/KUDU-2895
             Project: Kudu
          Issue Type: New Feature
            Reporter: Grant Henke


This tracks adding lineage support to Kudu and Apache Atlas. 

A few notes based on some initial research:
 * It probably makes sense to generate a generic lineage file which can be consumed by Apache Atlas for lineage.
 ** This avoids the need for Java interaction in the server
 ** This is the approach Impala uses
 ** See ATLAS-3183 and [https://impala.apache.org/docs/build3x/html/topics/impala_lineage.html#lineage]
 * Creating lineage entries for table "DDL" initially makes sense
 ** CREATE, ALTER, DELETE
 ** This is what Hbase seems to do: [https://atlas.apache.org/Hook-HBase.html]
 ** "Only the namespace, table and column-family create/update/ delete operations are captured by Atlas HBase hook"
 * The need for lineage information by scans in unclear
 ** It would be super fine grained and difficult to interpret.
 ** Instead lineage from other tools doing the scanning would be more interpretable (Impala, Spark, etc).



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)