You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Sarath Subramanian (JIRA)" <ji...@apache.org> on 2019/07/09 21:35:00 UTC
[jira] [Updated] (ATLAS-3321) Introduce atlas metadata namespace
[ https://issues.apache.org/jira/browse/ATLAS-3321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sarath Subramanian updated ATLAS-3321:
--------------------------------------
Description:
Atlas has the notion of *clusterName* which is used as a global namespace in which you can define the context in which entity instances are defined and grouped. A cluster name defines a scope of operation in atlas and is specified in parent entity types - _hive_db (Hive), hbase_namespace (HBase), fs_path (File System), storm_topology (Storm), spark_db (Spark)_
cluster name is also used to construct _qualifiedName_ attribute in atlas -
* hive_db - db_name@cluster_name
* hive_table - db_name.table_name@cluster_name
* hive_column - db_name.table_name.column_name@cluster_name
Cluster name works for simple environments wherein we have a single cluster and single service instances running (single Hive or HBase service). In complex deployments - a single cluster may host multiple instances of the same service (two Hive services - Hive-1 and Hive-2), for such cases cluster name doesn't help to provide context to individual service instances and metadata groupings.
So we introduce - *"metadata namespace"* where a single cluster can contain multiple metadata namespaces. The qualified name for atlas will be generated using metadata namespace name instead of cluster name.
For each hook enabled we specify the property - *"atlas.metadata.namespace"* which defines the scope/context of the metadata.
> Introduce atlas metadata namespace
> -----------------------------------
>
> Key: ATLAS-3321
> URL: https://issues.apache.org/jira/browse/ATLAS-3321
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 2.0.0
> Reporter: Sarath Subramanian
> Assignee: Sarath Subramanian
> Priority: Major
> Fix For: 2.1.0
>
>
> Atlas has the notion of *clusterName* which is used as a global namespace in which you can define the context in which entity instances are defined and grouped. A cluster name defines a scope of operation in atlas and is specified in parent entity types - _hive_db (Hive), hbase_namespace (HBase), fs_path (File System), storm_topology (Storm), spark_db (Spark)_
> cluster name is also used to construct _qualifiedName_ attribute in atlas -
> * hive_db - db_name@cluster_name
> * hive_table - db_name.table_name@cluster_name
> * hive_column - db_name.table_name.column_name@cluster_name
> Cluster name works for simple environments wherein we have a single cluster and single service instances running (single Hive or HBase service). In complex deployments - a single cluster may host multiple instances of the same service (two Hive services - Hive-1 and Hive-2), for such cases cluster name doesn't help to provide context to individual service instances and metadata groupings.
> So we introduce - *"metadata namespace"* where a single cluster can contain multiple metadata namespaces. The qualified name for atlas will be generated using metadata namespace name instead of cluster name.
> For each hook enabled we specify the property - *"atlas.metadata.namespace"* which defines the scope/context of the metadata.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)