You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry via Review Board <no...@reviews.apache.org> on 2021/07/12 05:15:28 UTC
Re: Review Request 73423: ATLAS-4338: Import Service: Support change
notifiers for migration import
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73423/#review223211
-----------------------------------------------------------
Ship it!
Ship It!
- Ashutosh Mestry
On June 30, 2021, 11:07 a.m., Nikhil Bonte wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73423/
> -----------------------------------------------------------
>
> (Updated June 30, 2021, 11:07 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, madhan, Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
>
>
> Bugs: ATLAS-4338
> https://issues.apache.org/jira/browse/ATLAS-4338
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Currently Migration import feature (MigrationImport.java) does not support enabling notifiers (audits writer, notification to Atlas entities topic).
>
> With this improvement, Atlas now will enable notifiers for Migration import, based on a flag.
>
>
>
> Approach:
>
> * New option in AtlasImportRequest - sendNotifications (default : false)
> * MigrationImport will use the flag to select appropriate implementation of IAtlasEntityChangeNotifier to create EntityCreationManager.
> * EntityCreationManager will then notify listeners.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 2c1870444
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 1d29bf833
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java 99be169f2
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/BulkImporterImpl.java 8e17fd410
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/GuidMutationResponsePair.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/MigrationImport.java d6f23d6e2
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/pc/EntityConsumer.java b73988fd7
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/bulkimport/pc/EntityCreationManager.java 734add6d7
>
>
> Diff: https://reviews.apache.org/r/73423/diff/3/
>
>
> Testing
> -------
>
> * Tested Manually
> * Sample request
> {
> "options": {
> "migration": "true",
> "size": "100",
> "sendNotifications": "true",
> "batchSize": 500,
> "numWorkers": 40
> }
> }
> * took ~270 secs to import 12987 entities with notifiers enabled.
> * PC build : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/679/console
>
>
> Thanks,
>
> Nikhil Bonte
>
>