You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Nikhil Bonte (Jira)" <ji...@apache.org> on 2020/02/13 13:04:00 UTC
[jira] [Updated] (ATLAS-3595) Data Migration: Support zip file for
data migration
[ https://issues.apache.org/jira/browse/ATLAS-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nikhil Bonte updated ATLAS-3595:
--------------------------------
Description:
Data migration service added under [ATLAS-2461|http://example.com/] works with json files as input.
This improvement will enable Data Migration service to import data using a zip file.
Approach:
* Property value for migration should contain full path of file, below is the example.
* Data migration service will first check migration property if value refers to a zip file.
* If true, it will call ZipMigrationImporter to import data in zip file.
* ZipMigrationImporter internally uses existing ImportService to ingest zip.
* Audit service & Notification service will also be started as a part of zip import.
E.g. properties to enable zip data migration:
* atlas.migration.data.filename=/path/fileName.zip
atlas.migration.mode.workers=5
atlas.migration.mode.batch.size=20
was:
Data migration service added under [ATLAS-2461|http://example.com/] works with json files as input.
This improvement will enable Data Migration service to import data using a zip file.
Approach:
* Property value for migration should contain full path of file, below is the example.
* Data migration service will first check migration property if value refers to a zip file.
* If true, it will call ZipMigrator to import data in zip file.
* ZipMigrator internally uses existing ImportService to ingest zip.
* Audit service & Notification service will also be started as a part of zip import.
E.g. properties to enable zip data migration:
* atlas.migration.data.filename=/path/fileName.zip
atlas.migration.mode.workers=5
atlas.migration.mode.batch.size=20
> Data Migration: Support zip file for data migration
> ---------------------------------------------------
>
> Key: ATLAS-3595
> URL: https://issues.apache.org/jira/browse/ATLAS-3595
> Project: Atlas
> Issue Type: Improvement
> Reporter: Nikhil Bonte
> Assignee: Nikhil Bonte
> Priority: Major
> Attachments: ATLAS-3595-Migration-import-support-zip-file-for-dat-v6.patch
>
>
> Data migration service added under [ATLAS-2461|http://example.com/] works with json files as input.
> This improvement will enable Data Migration service to import data using a zip file.
> Approach:
> * Property value for migration should contain full path of file, below is the example.
> * Data migration service will first check migration property if value refers to a zip file.
> * If true, it will call ZipMigrationImporter to import data in zip file.
> * ZipMigrationImporter internally uses existing ImportService to ingest zip.
> * Audit service & Notification service will also be started as a part of zip import.
> E.g. properties to enable zip data migration:
> * atlas.migration.data.filename=/path/fileName.zip
> atlas.migration.mode.workers=5
> atlas.migration.mode.batch.size=20
--
This message was sent by Atlassian Jira
(v8.3.4#803005)