Re: Review Request 45558: ATLAS-540 API to retrieve entity version events

[Shwetha GS <ss...@hortonworks.com>]

> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java, line 110
> > <https://reviews.apache.org/r/45558/diff/2/?file=1323995#file1323995line110>
> >
> >     Can we create an AtlasClient here with the endpoint? The advantage is that we would not need to store the endpoint as state. Also, we would keep chaining constructors, so that there would be only one initialization point in the end.

As part of AtlasClient creation, it writes config file and createss objects. Want to avoid all this in hive hook as hive hook doesn't require atlas client. Have cleaned up HiveMetaStoreBridge constructor calls though


> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java, line 161
> > <https://reviews.apache.org/r/45558/diff/2/?file=1324007#file1324007line161>
> >
> >     If the last byte has value 0x7F, this overflows to a negative number. In that case, the byte comparison would be different from expected, no?

The byte array here is for the entity id string which is UUID. So, don't expect the overflow


> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java, line 164
> > <https://reviews.apache.org/r/45558/diff/2/?file=1324007#file1324007line164>
> >
> >     Start rows seem to be inclusive in scans: http://stackoverflow.com/questions/17558547/hbase-easy-how-to-perform-range-prefix-scan-in-hbase-shell. Also, the javadoc for setStartRow says this: Note: In order to make startRow exclusive add a trailing 0 byte.
> >     
> >     I feel it will be easier for clients to just pass the last key in the result set and we add this trailing byte 0 to get results from the next row. Thoughts?

Start row by intuition, should be included in the results.

The thing is, the keys are ordered in the increasing order of timestamp in hbase. But while retrieving entries, we do reverse scan and return the results. So, if the clients want pagination with 20 results, they should retrieve 21 results and use the last one's key as startKey for next call. Yes, this is easier than clients having to do manual key modification to figure out the startkey. Will update the documentation


> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java, line 176
> > <https://reviews.apache.org/r/45558/diff/2/?file=1324007#file1324007line176>
> >
> >     Shouldn't we break here? Once an entityId is crossed, we would not get any results that would apply for this entityId anymore, right?

With reverse scan, this should be continue


- Shwetha


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45558/#review126818
-----------------------------------------------------------


On April 4, 2016, 8:36 a.m., Shwetha GS wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45558/
> -----------------------------------------------------------
> 
> (Updated April 4, 2016, 8:36 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-540
>     https://issues.apache.org/jira/browse/ATLAS-540
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Added REST API to get audit events. 
> 
> Pending: User info is no coming correctly. Each REST API call makes two calls - OPTIONS call and then actual call. OPTIONS call contains the actual user in httprequest, but the actual call contains the user in the queryparam doAs. Will debug this as part of ATLAS-585
> 
> 
> Diffs
> -----
> 
>   addons/falcon-bridge/pom.xml 03cb11e 
>   addons/falcon-bridge/src/test/java/org/apache/atlas/falcon/hook/FalconHookIT.java 4249a8f 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java 50a5311 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 68e32ff 
>   addons/sqoop-bridge/src/test/java/org/apache/atlas/sqoop/hook/SqoopHookIT.java 0e4658a 
>   client/src/main/java/org/apache/atlas/AtlasClient.java 0bb5264 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java PRE-CREATION 
>   common/src/main/java/org/apache/atlas/utils/ParamChecker.java 91adfaf 
>   notification/src/main/java/org/apache/atlas/notification/NotificationModule.java e8ae177 
>   pom.xml 226529d 
>   repository/pom.xml eca087a 
>   repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java ce1bdfb 
>   repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java 0c5c551 
>   repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java d41c4da 
>   repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java c4329a5 
>   repository/src/main/java/org/apache/atlas/repository/audit/InMemoryEntityAuditRepository.java df75290 
>   repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java cd1161a 
>   repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java 9c193f7 
>   repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java aa00417 
>   server-api/pom.xml d3e84c4 
>   server-api/src/main/java/org/apache/atlas/services/MetadataService.java ab402d7 
>   webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 6bfd780 
>   webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 53c503b 
>   webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java 8b0d0e9 
>   webapp/src/main/java/org/apache/atlas/web/service/ActiveInstanceElectorModule.java 065666d 
>   webapp/src/main/java/org/apache/atlas/web/service/ServiceModule.java PRE-CREATION 
>   webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java a8cd1ef 
> 
> Diff: https://reviews.apache.org/r/45558/diff/
> 
> 
> Testing
> -------
> 
> UTs, ITs and end to end test
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>