You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry via Review Board <no...@reviews.apache.org> on 2020/02/13 04:30:14 UTC
Re: Review Request 72039: ATLAS-3595 Migration import: support zip
file for data migration
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72039/#review219567
-----------------------------------------------------------
Integrate auto-start of migration import with this patch.
common/src/main/java/org/apache/atlas/service/Services.java
Lines 105 (patched)
<https://reviews.apache.org/r/72039/#comment307755>
Use StringUtils.endsWithIgnoreCase.
repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java
Line 62 (original), 65 (patched)
<https://reviews.apache.org/r/72039/#comment307756>
Refactor: Leave existing functionality untouched. Move new logic to ZipMigrationImporter.
repository/src/main/java/org/apache/atlas/repository/migration/ZipMigrator.java
Lines 33 (patched)
<https://reviews.apache.org/r/72039/#comment307757>
Rename: ZipMigrationImporter
repository/src/test/java/org/apache/atlas/repository/migration/MigrationBaseAsserts.java
Lines 67 (patched)
<https://reviews.apache.org/r/72039/#comment307758>
This won't be necessary.
- Ashutosh Mestry
On Jan. 31, 2020, 11:52 a.m., Nikhil Bonte wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72039/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2020, 11:52 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3595
> https://issues.apache.org/jira/browse/ATLAS-3595
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Data migration service added under ATLAS-2461 works with json files as input.
>
> This improvement will enable Data Migration service to import data using a zip file.
>
> Approach:
>
> * Property value for migration should contain full path of file, below is the example.
> * Data migration service will first check migration property if value refers to a zip file.
> * If true, it will call ZipMigrator to import data in zip file.
> * ZipMigrator internally uses existing ImportService to ingest zip.
> * Audit service & Notification service will also be started as a part of zip import.
>
>
> E.g. properties to enable zip data migration:
>
> * atlas.migration.data.filename=/path/fileName.zip
> * atlas.migration.mode.workers=5
> * atlas.migration.mode.batch.size=20
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/atlas/service/Services.java 7b36db5ff
> repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java 170136195
> repository/src/main/java/org/apache/atlas/repository/migration/ZipMigrator.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/migration/MigrationBaseAsserts.java 18e950e8c
>
>
> Diff: https://reviews.apache.org/r/72039/diff/6/
>
>
> Testing
> -------
>
> PC build - https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1614/
>
>
> Thanks,
>
> Nikhil Bonte
>
>