You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Madhan Neethiraj <ma...@apache.org> on 2020/02/03 01:30:15 UTC
Re: Review Request 72046: ATLAS-3534 EntityREST changes to allow
namespace
attributes to be added, updated and deleted to an entity instance
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72046/#review219459
-----------------------------------------------------------
intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java
Lines 133 (patched)
<https://reviews.apache.org/r/72046/#comment307625>
AtlasNamespaceType doesn't have any instance (compared to AtlasEntityType/AtlasStructType/AtlasEnumType/AtlasRelationshipType/AtlasClassificationType), hence following abstract methods should be empty/no-op:
- createDefaultValue()
- isValidValue()
- areEqualValues()
- getNormalizedValue()
- validateValue()
- isValidValueForUpdate()
- getNormalizedValueForUpdate()
- validateValueForUpdate()
Above methods in AtlasEntityType should be updated to handle namespace attributes. Please review and update.
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
Lines 824 (patched)
<https://reviews.apache.org/r/72046/#comment307626>
- checkIfNamespaceAttributesExist() is called only within AtlasEntityStoreV2. Consider marking this method as private
- given checkIfNamespaceAttributesExist() returns true if the vertex has at least one namespace attribute, consider renaming to containsAnyNamespaceAttribute()
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
Lines 915 (patched)
<https://reviews.apache.org/r/72046/#comment307628>
checkIfNamespaceAttributesExist() returns true on presence of first namespace-attribute, which may not be good here. Consider adding method that checks for presence of all given namespace-attributes - containsAllNamespaceAttributes(entityVertex, namespaceAttributes)
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
Lines 1449 (patched)
<https://reviews.apache.org/r/72046/#comment307629>
Consider following simper validation:
for (String nsName : entityNamespaces.keySet()) {
List<AtlasNamespaceAttribute> nsAttributes = entityType.getNamespaceAttributes(nsName);
if (nsAttributes == null) {
messages.add(nsName + ": invalid namespace for entity type " + entityType.getTypeName());
continue;
}
Map<String, Object> entityNsAttributes = entityNamespaces.get(nsName);
for (AtlasNamespaceAttribute nsAttribute : nsAttributes) {
AtlasType attrType = nsAttribute.getAttributeType();
String attrName = nsAttribute.getName();
Object attrValue = entityNsAttributes.get(attrName);
String fieldName = entityType.getTypeName() + "." + nsName + "." + attrName;
if (attrValue != null) {
attrType.validateValue(attrValue, fieldName, messages);
} else if (!nsAttribute.getAttributeDef().getIsOptional()) {
messages.add(fieldName + ": mandatory namespace attribute value missing in type " + entityType.getTypeName());
}
}
}
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
Line 617 (original), 620 (patched)
<https://reviews.apache.org/r/72046/#comment307623>
When 'attributes' contains namespace-attributes (named as "namespaceName:attributeName"), returned entity should have these namespace-attributes populated as well.
This update can be taken up in a subsequent patch, once basic-search is updated to support namespace attributes.
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
Line 694 (original), 697 (patched)
<https://reviews.apache.org/r/72046/#comment307624>
When 'attributes' contains namespace-attributes (named as "namespaceName:attributeName"?), returned entity should have these namespace-attributes populated as well.
This update can be taken up in a subsequent patch, once basic-search is updated to support namespace attributes.
- Madhan Neethiraj
On Jan. 31, 2020, 12:40 p.m., Mandar Ambawane wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72046/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2020, 12:40 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3534
> https://issues.apache.org/jira/browse/ATLAS-3534
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Created separate endpoints to associate Namespace attributes to entity
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/atlas/repository/Constants.java 0b28243
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 2054513
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 1b033b9
> intg/src/main/java/org/apache/atlas/type/AtlasNamespaceType.java a141d4a
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 97e27d0
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java 928c70d
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java 25284e9
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java e427a59
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java dc4c399
> webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 6845121
>
>
> Diff: https://reviews.apache.org/r/72046/diff/8/
>
>
> Testing
> -------
>
> Basic testing is done.
>
> Create NamespaceDef sports:
>
> curl --location --request POST 'http://localhost:21000/api/atlas/v2/types/typedefs' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
> --data-raw '{
> "namespaceDefs": [
> {
> "name": "sports",
> "description": "Sample sports",
> "serviceType": "NAMESPACES",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "sports_name",
> "typeName": "string",
> "options": {
> "applicableEntityTypes": "[\"hdfs_path\", \"hive_table\"]",
> "maxStrLength": "50"
> },
> "cardinality": "SINGLE",
> "isIndexable": true,
> "isOptional": true,
> "includeInNotification": true,
> "isUnique": false
> },
> {
> "name": "no_of_players",
> "typeName": "int",
> "options": {
> "applicableEntityTypes": "[\"hdfs_path\", \"hive_column\"]"
> },
> "cardinality": "SINGLE",
> "isIndexable": true,
> "isOptional": true,
> "includeInNotification": true,
> "isUnique": false
> }
> ]
> }
> ]
> }'
>
>
> Create NamespaceDef vehicle:
>
> curl --location --request POST 'http://localhost:21000/api/atlas/v2/types/typedefs' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
> --data-raw '{
> "namespaceDefs": [
> {
> "name": "vehicle",
> "description": "Sample vehicle",
> "serviceType": "NAMESPACES",
> "typeVersion": "1.0",
> "attributeDefs": [
> {
> "name": "vehicle_name",
> "typeName": "string",
> "options": {
> "applicableEntityTypes": "[\"hdfs_path\", \"hive_table\"]",
> "maxStrLength": "50"
> },
> "cardinality": "SINGLE",
> "isIndexable": true,
> "isOptional": true,
> "includeInNotification": true,
> "isUnique": false
> },
> {
> "name": "no_of_wheels",
> "typeName": "int",
> "options": {
> "applicableEntityTypes": "[\"hdfs_path\", \"hive_column\"]"
> },
> "cardinality": "SINGLE",
> "isIndexable": true,
> "isOptional": true,
> "includeInNotification": true,
> "isUnique": false
> }
> ]
> }
> ]
> }'
>
>
> Create hdfs_path entity:
>
> curl --location --request POST 'http://localhost:21000/api/atlas/v2/entity' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
> --data-raw '{
> "entity": {
> "guid": "-1",
> "typeName": "hdfs_path",
> "attributes": {
> "clusterName": "cm",
> "description": "desc",
> "location": "localhost",
> "name": "hdfs_path_entity1",
> "owner": "admin",
> "ownerType": "USER",
> "qualifiedName": "hdfs_path_entity1@cm",
> "path": "local_path"
> }
> }
> }'
>
>
> Associate namespace attributes with the created entity
>
>
> curl --location --request POST 'http://localhost:21000/api/atlas/v2/entity/guid/ea633224-ccfe-4573-9060-43190cd9531a/namespaces' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
> --data-raw '{
> "sports":{
> "sports_name":"cricket",
> "no_of_players":11
> },
> "vehicle":{
> "vehicle_name": "Car"
> }
> }'
>
>
> Add one more namespace attribute in the entity
>
> curl --location --request POST 'http://localhost:21000/api/atlas/v2/entity/guid/ea633224-ccfe-4573-9060-43190cd9531a/namespaces' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
> --data-raw '{
> "vehicle":{
> "no_of_wheels": 4
> }
> }'
>
>
> Update sports_name attribute with new value as Hockey
>
> curl --location --request PUT 'http://localhost:21000/api/atlas/v2/entity/guid/ea633224-ccfe-4573-9060-43190cd9531a/namespaces' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
> --data-raw '{
> "sports":{
> "sports_name": "Hockey"
> }
> }'
>
>
> Delete vehicle.vehicle_name namespace attribute
>
> curl --location --request DELETE 'http://localhost:21000/api/atlas/v2/entity/guid/ea633224-ccfe-4573-9060-43190cd9531a/namespaces' \
> --header 'Content-Type: application/json' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4=' \
> --data-raw '{
> "vehicle":{
> "vehicle_name": "Car"
> }
> }'
>
>
> Thanks,
>
> Mandar Ambawane
>
>