You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@atlas.apache.org by Apoorv Naik <na...@gmail.com> on 2017/06/16 18:05:33 UTC

Review Request 60159: ATLAS-1880: Search using entity/trait attributes

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60159/
-----------------------------------------------------------

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath Subramanian, and Suma Shivaprasad.


Bugs: ATLAS-1880
    https://issues.apache.org/jira/browse/ATLAS-1880


Repository: atlas


Description
-------

See JIRA


Sample Request structure

```json
{
	"typeName": "hive_table",
	"excludeDeletedEntities": false,
	"limit": 25,
	"offset": 0,
	"entityFilters": {
		"condition": "AND",
		"criterion": [{
			"attributeName": "retention",
			"operator": "=",
			"attributeValue": "10"
		}, {
			"condition": "OR",
			"criterion": [{
				"attributeName": "createTime",
				"operator": ">",
				"attributeValue": "1111111111"
			}, {
				"attributeName": "lastAccessTime",
				"operator": "<=",
				"attributeValue": "2222222222"
			}]
		}]
	}
}
```


Diffs
-----

  client/src/main/java/org/apache/atlas/AtlasClientV2.java 61413428 
  common/src/main/java/org/apache/atlas/repository/Constants.java ac022528 
  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphQuery.java bd7b35e4 
  graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/query/NativeTitan0GraphQuery.java 5ad176b5 
  graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/query/NativeTitan1GraphQuery.java 9dc175b6 
  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 923a198b 
  repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 128cdbf9 
  repository/src/main/java/org/apache/atlas/discovery/GremlinSearchStep.java PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/SearchPipeline.java PRE-CREATION 
  repository/src/main/java/org/apache/atlas/discovery/SolrSearchStep.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/TestModules.java 095af417 
  repository/src/test/java/org/apache/atlas/discovery/GremlinSearchStepTest.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java 5d5b043e 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java ea550211 


Diff: https://reviews.apache.org/r/60159/diff/1/


Testing
-------

In progress

1. Tested Json marshalling and unmarshalling via REST


Thanks,

Apoorv Naik

Re: Review Request 60159: ATLAS-1880: Search using entity/trait attributes

Posted by Madhan Neethiraj <ma...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60159/#review179118
-----------------------------------------------------------




graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/TitanGraphQuery.java
Lines 163 (patched)
<https://reviews.apache.org/r/60159/#comment253613>

    if (LOG.isDebugEnabled()) { .. }



graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/TitanGraphQuery.java
Lines 171 (patched)
<https://reviews.apache.org/r/60159/#comment253615>

    Instead of multiple 'return's (line #171, #175), consider breaking from the loops to keep a single return at line #175.



graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/TitanGraphQuery.java
Lines 182 (patched)
<https://reviews.apache.org/r/60159/#comment253625>

    Instead of adding "offset + limit" number of entries to the list and then returning a subset, consider skipping "offset" number of entries in the first place. This might require duplicating the code in vertices(limit).



repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java
Lines 328 (patched)
<https://reviews.apache.org/r/60159/#comment253780>

    Given toGremlinFilterQuery() always return the passed in 'query', this condition "if" would always be false. Is this necessary? Same in line #336 as well.



repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java
Lines 335 (patched)
<https://reviews.apache.org/r/60159/#comment253782>

    Passing 'query' seems incorrect here - as it would endup adding .has() to it. For 'OR' case, a new query object needs to be instantiated here. Please review.



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
Lines 373 (patched)
<https://reviews.apache.org/r/60159/#comment253761>

    vertexIndexKeys should be recomputed after index updates for type-registry changes (like add/update types)


- Madhan Neethiraj


On June 28, 2017, 9:18 p.m., Apoorv Naik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60159/
> -----------------------------------------------------------
> 
> (Updated June 28, 2017, 9:18 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath Subramanian, and Suma Shivaprasad.
> 
> 
> Bugs: ATLAS-1880
>     https://issues.apache.org/jira/browse/ATLAS-1880
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> See JIRA
> 
> 
> Sample Request structure
> 
> ```json
> {
> 	"typeName": "hive_table",
> 	"excludeDeletedEntities": false,
> 	"limit": 25,
> 	"offset": 0,
> 	"entityFilters": {
> 		"condition": "AND",
> 		"criterion": [{
> 			"attributeName": "retention",
> 			"operator": "=",
> 			"attributeValue": "10"
> 		}, {
> 			"condition": "OR",
> 			"criterion": [{
> 				"attributeName": "createTime",
> 				"operator": ">",
> 				"attributeValue": "1111111111"
> 			}, {
> 				"attributeName": "lastAccessTime",
> 				"operator": "<=",
> 				"attributeValue": "2222222222"
> 			}]
> 		}]
> 	}
> }
> ```
> 
> 
> Diffs
> -----
> 
>   client/src/main/java/org/apache/atlas/AtlasClientV2.java 61413428 
>   common/src/main/java/org/apache/atlas/repository/Constants.java ac022528 
>   distro/src/conf/atlas-log4j.xml e6c0d9f6 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphQuery.java 841edf71 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasIndexQuery.java 1ff9d5ed 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertexQuery.java 53f490f5 
>   graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/NativeTitanGraphQuery.java 0211ff05 
>   graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/TitanGraphQuery.java 0077a21f 
>   graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/expr/AndCondition.java 68f0eb28 
>   graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/expr/HasPredicate.java 24e4f5bc 
>   graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0IndexQuery.java 1ed1734f 
>   graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0VertexQuery.java bd8b897f 
>   graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/query/NativeTitan0GraphQuery.java 7ec6ffeb 
>   graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1IndexQuery.java 4073dd2a 
>   graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1VertexQuery.java 4452bcdd 
>   graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/query/NativeTitan1GraphQuery.java 1ca900d8 
>   intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java a402c628 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 21b64273 
>   repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 923a198b 
>   repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 128cdbf9 
>   repository/src/main/java/org/apache/atlas/discovery/GremlinStep.java PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/SearchPipeline.java PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/discovery/SolrStep.java PRE-CREATION 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 3411f8de 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 42bd58fb 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java 9a8695af 
>   repository/src/main/java/org/apache/atlas/util/SearchContextContainer.java PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/TestModules.java fa2ac0d8 
>   repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java 5d5b043e 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 8c5623fa 
>   webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java ea550211 
>   webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 1fe31198 
> 
> 
> Diff: https://reviews.apache.org/r/60159/diff/9/
> 
> 
> Testing
> -------
> 
> 1. Tested Json marshalling and unmarshalling via REST
> 2. Tested with the attached request JSONs 
> 
> In progress
> 
> UTs (coding)
> 
> 
> File Attachments
> ----------------
> 
> Asset contains
>   https://reviews.apache.org/media/uploaded/files/2017/06/22/a8fdf120-5eb1-4a84-af4a-1236a08a765b__asset-contains.json
> Asset IN
>   https://reviews.apache.org/media/uploaded/files/2017/06/22/e2096d10-ea4d-4306-b68d-4f5c531fab3c__asset-in.json
> Asset like
>   https://reviews.apache.org/media/uploaded/files/2017/06/22/6774768f-26ba-4386-b0b6-8ea682813dc7__asset-like.json
> Hive table (date comparison)
>   https://reviews.apache.org/media/uploaded/files/2017/06/22/d6e54af0-f27f-4c6f-b9d3-4e3a0eb258ac__hive_table-date_2.json
> Hive table date
>   https://reviews.apache.org/media/uploaded/files/2017/06/22/f3558280-60bd-49dd-92c6-c1176f86bc70__hive_table-date.json
> Hive Table Like
>   https://reviews.apache.org/media/uploaded/files/2017/06/22/f07517dd-1c6c-4d2d-8bfd-3d494506b23c__hive_table-like.json
> Type and tag
>   https://reviews.apache.org/media/uploaded/files/2017/06/22/83768696-aa5c-4c08-a776-143753eb4548__type-tag.json
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>