You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "UmeshPatil-1 (via GitHub)" <gi...@apache.org> on 2025/12/12 14:37:35 UTC
[PR] ATLAS-5159: Resolve Exporting Parent type doesn't export child type i… [atlas]
UmeshPatil-1 opened a new pull request, #489:
URL: https://github.com/apache/atlas/pull/489
**ATLAS-5159**
**What changes were proposed in this pull request?**
**Background:**
Exporting entities of a parent type (e.g., `Referenceable`) does not export entities of its child type (e.g., `hive_table`).
When attempting to export using the parent type with unique attributes, no child entities were returned, leading to incomplete exports.
**Root Cause:**
The export logic in StartEntityFetchByExportRequest did not properly handle abstract or parent types.
Methods like getEntitiesForMatchTypeType() and getEntitiesForMatchTypeUsingUniqueAttributes() only fetched concrete types when specified, ignoring child entities.
**Changes Proposed:**
1. Refactored StartEntityFetchByExportRequest to properly handle unique attribute lookups.
2. Updated getEntitiesForMatchTypeType() to include all subtypes of a parent type during export.
3. Simplified getEntitiesForMatchTypeUsingUniqueAttributes() for better performance and maintainability.
4. Updated getBindingsForTypeName() to ensure all subtypes are included in queries.
5. Improved logging and error handling for clarity.
**Impact:**
- Exporting parent types like Referenceable now correctly includes all child entities.
- Maintains consistent behavior for all fetch types (FULL, INCREMENTAL, CONNECTED).
- Minimal impact on performance; no additional schema changes are required.
**How was this patch tested?**
- Maven Build:- Build Successfull.
- Manual testing of export requests with all three fetch types: FULL, INCREMENTAL, CONNECTED.
- Verified entity GUIDs are correctly returned for parent types and child entities.
- Confirmed behavior for concrete types, abstract types, and invalid/misspelled types.
- Logs verified query execution, fetch type handling, and error reporting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@atlas.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ATLAS-5159: Export APIs: Exporting parent type doesn't export child type [atlas]
Posted by "UmeshPatil-1 (via GitHub)" <gi...@apache.org>.
UmeshPatil-1 closed pull request #489: ATLAS-5159: Export APIs: Exporting parent type doesn't export child type
URL: https://github.com/apache/atlas/pull/489
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@atlas.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ATLAS-5159: Export APIs: Exporting parent type doesn't export child type [atlas]
Posted by "chaitalicod (via GitHub)" <gi...@apache.org>.
chaitalicod commented on code in PR #489:
URL: https://github.com/apache/atlas/pull/489#discussion_r2719694896
##########
repository/src/main/java/org/apache/atlas/repository/impexp/StartEntityFetchByExportRequest.java:
##########
@@ -177,15 +189,61 @@ private List<String> getEntitiesForMatchTypeUsingUniqueAttributes(AtlasObjectId
return new ArrayList<>(ret);
}
- private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) {
- return executeGremlinQuery(getQueryTemplateForMatchType(matchType), getBindingsForTypeName(item.getTypeName()));
+ private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) throws AtlasBaseException {
+ String typeName = item.getTypeName();
+
+ AtlasEntityType entityType = typeRegistry.getEntityTypeByName(typeName);
+
+ if (StringUtils.isNotEmpty(typeName) && entityType == null) {
+ throw new AtlasBaseException(AtlasErrorCode.UNKNOWN_TYPENAME, typeName);
+ }
+
+ boolean isRootOrGenericSearch = StringUtils.isEmpty(typeName) || (CollectionUtils.isNotEmpty(entityType.getAllSubTypes()) && entityType.getSuperTypes().isEmpty());
Review Comment:
we are already checking not empty here
if (StringUtils.isNotEmpty(typeName) && entityType == null) {
throw new AtlasBaseException(AtlasErrorCode.UNKNOWN_TYPENAME, typeName);
}
then is it required to check empty or not here
boolean isRootOrGenericSearch = StringUtils.isEmpty(typeName) || (CollectionUtils.isNotEmpty(entityType.getAllSubTypes()) && entityType.getSuperTypes().isEmpty());
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@atlas.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] ATLAS-5159: Export APIs: Exporting parent type doesn't export child type [atlas]
Posted by "UmeshPatil-1 (via GitHub)" <gi...@apache.org>.
UmeshPatil-1 commented on code in PR #489:
URL: https://github.com/apache/atlas/pull/489#discussion_r2719950936
##########
repository/src/main/java/org/apache/atlas/repository/impexp/StartEntityFetchByExportRequest.java:
##########
@@ -177,15 +189,61 @@ private List<String> getEntitiesForMatchTypeUsingUniqueAttributes(AtlasObjectId
return new ArrayList<>(ret);
}
- private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) {
- return executeGremlinQuery(getQueryTemplateForMatchType(matchType), getBindingsForTypeName(item.getTypeName()));
+ private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) throws AtlasBaseException {
+ String typeName = item.getTypeName();
+
+ AtlasEntityType entityType = typeRegistry.getEntityTypeByName(typeName);
+
+ if (StringUtils.isNotEmpty(typeName) && entityType == null) {
+ throw new AtlasBaseException(AtlasErrorCode.UNKNOWN_TYPENAME, typeName);
+ }
+
+ boolean isRootOrGenericSearch = StringUtils.isEmpty(typeName) || (CollectionUtils.isNotEmpty(entityType.getAllSubTypes()) && entityType.getSuperTypes().isEmpty());
Review Comment:
Yes, both checks are required, the first ensures type validity and the second decides search scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@atlas.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Export APIs: Exporting parent type doesn't export child type [atlas]
Posted by "UmeshPatil-1 (via GitHub)" <gi...@apache.org>.
UmeshPatil-1 commented on code in PR #489:
URL: https://github.com/apache/atlas/pull/489#discussion_r2652218327
##########
repository/src/main/java/org/apache/atlas/repository/impexp/StartEntityFetchByExportRequest.java:
##########
@@ -177,15 +189,61 @@ private List<String> getEntitiesForMatchTypeUsingUniqueAttributes(AtlasObjectId
return new ArrayList<>(ret);
}
- private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) {
- return executeGremlinQuery(getQueryTemplateForMatchType(matchType), getBindingsForTypeName(item.getTypeName()));
+ private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) throws AtlasBaseException {
Review Comment:
Added UTS.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@atlas.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Export APIs: Exporting parent type doesn't export child type [atlas]
Posted by "chaitalicod (via GitHub)" <gi...@apache.org>.
chaitalicod commented on code in PR #489:
URL: https://github.com/apache/atlas/pull/489#discussion_r2633763733
##########
repository/src/main/java/org/apache/atlas/repository/impexp/StartEntityFetchByExportRequest.java:
##########
@@ -177,15 +189,61 @@ private List<String> getEntitiesForMatchTypeUsingUniqueAttributes(AtlasObjectId
return new ArrayList<>(ret);
}
- private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) {
- return executeGremlinQuery(getQueryTemplateForMatchType(matchType), getBindingsForTypeName(item.getTypeName()));
+ private List<String> getEntitiesForMatchTypeType(AtlasObjectId item, String matchType) throws AtlasBaseException {
Review Comment:
Please add UTS for each usecase to cover scenarios and regression behaviour
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@atlas.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org