You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@atlas.apache.org by "Ashutosh Mestry (Jira)" <ji...@apache.org> on 2020/03/13 00:27:00 UTC

[jira] [Updated] (ATLAS-3663) Import Service: Migration Import: Provide Ability to Resume from Last Successfully Imported State

     [ https://issues.apache.org/jira/browse/ATLAS-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ashutosh Mestry updated ATLAS-3663:
-----------------------------------
    Attachment: ATLAS-3663-Auto-resume-of-ZipFileMigrationImporter-m.patch

> Import Service: Migration Import: Provide Ability to Resume from Last Successfully Imported State
> -------------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-3663
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3663
>             Project: Atlas
>          Issue Type: Improvement
>          Components:  atlas-core
>    Affects Versions: trunk
>            Reporter: Ashutosh Mestry
>            Assignee: Ashutosh Mestry
>            Priority: Major
>             Fix For: trunk
>
>         Attachments: ATLAS-3663-Auto-resume-of-ZipFileMigrationImporter-m.patch
>
>
> *Background*
> The new Migration Import implementation allows for importing large zip files. These imports can potentially run for long time. If the server shuts down while an import is in progress, there needs to be a mechanism so that the import resumes from the last imported entity.
> *Solution*
>  * Implement _DataMigrationStatusService_ this will maintain a special vertex one of whose property will be the last successfully imported status.
>  * Update this vertex on each successfully recorded import.
>  * At the start of _ZipFileBaseMigration_ check for existence of this vertex. If it exists and has a valid position, use this to set the _startPosition_ in the _AtlasImportRequest_. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)