You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry <am...@hortonworks.com> on 2018/04/20 05:03:01 UTC
Review Request 66728: Data Migration: HA Mode Initialization Fix
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66728/
-----------------------------------------------------------
Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath Subramanian.
Bugs: ATLAS-2589
https://issues.apache.org/jira/browse/ATLAS-2589
Repository: atlas
Description
-------
**Approach**
_AtlasTypeDefStoreInitializer_ is not reverted to the its original form. The earlier version initialized it incorrectly during migration mode.
_migration.ReaderStatusManager_ has been refactored to remove duplicate code and improve accuracy of restarts.
_migration.AtlasGraphSONReader_ additional logging.
_DataMigrationService_:
- Initializes _GraphBackedSearchIndexer_ followed by initalization of _AtlasTypeDefStoreInitializer_.
- Additional directory existence check added that will allow/disallow migration. Apporpirate log messages added.
Diffs
-----
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java 636e6e8cf
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManager.java 11732c426
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManagerTest.java 8c3d1576d
intg/src/main/java/org/apache/atlas/model/impexp/MigrationStatus.java 901494506
repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java a4f7a2fea
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java c56157dc2
repository/src/test/java/org/apache/atlas/repository/migration/MigrationProgressServiceTest.java a0e2e0373
Diff: https://reviews.apache.org/r/66728/diff/1/
Testing
-------
**Unit tests**
- Existing tests were executed.
- No additional tests added.
**Functional tests**
- Executed migration in multi-node environment.
Thanks,
Ashutosh Mestry
Re: Review Request 66728: Data Migration: HA Mode Initialization Fix
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66728/#review201610
-----------------------------------------------------------
Fix it, then Ship it!
repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java
Lines 129 (patched)
<https://reviews.apache.org/r/66728/#comment282934>
if (ret) {
LOG.info("will migrate data in directory {}", importDirectory);
} else {
LOG.error("cannot read migration data in directory {}", importDirectory);
}
- Madhan Neethiraj
On April 20, 2018, 5:03 a.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66728/
> -----------------------------------------------------------
>
> (Updated April 20, 2018, 5:03 a.m.)
>
>
> Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath Subramanian.
>
>
> Bugs: ATLAS-2589
> https://issues.apache.org/jira/browse/ATLAS-2589
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Approach**
> _AtlasTypeDefStoreInitializer_ is not reverted to the its original form. The earlier version initialized it incorrectly during migration mode.
> _migration.ReaderStatusManager_ has been refactored to remove duplicate code and improve accuracy of restarts.
> _migration.AtlasGraphSONReader_ additional logging.
> _DataMigrationService_:
> - Initializes _GraphBackedSearchIndexer_ followed by initalization of _AtlasTypeDefStoreInitializer_.
> - Additional directory existence check added that will allow/disallow migration. Apporpirate log messages added.
>
>
> Diffs
> -----
>
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java 636e6e8cf
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManager.java 11732c426
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManagerTest.java 8c3d1576d
> intg/src/main/java/org/apache/atlas/model/impexp/MigrationStatus.java 901494506
> repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java a4f7a2fea
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java c56157dc2
> repository/src/test/java/org/apache/atlas/repository/migration/MigrationProgressServiceTest.java a0e2e0373
>
>
> Diff: https://reviews.apache.org/r/66728/diff/1/
>
>
> Testing
> -------
>
> **Unit tests**
> - Existing tests were executed.
> - No additional tests added.
>
> **Functional tests**
> - Executed migration in multi-node environment.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 66728: Data Migration: HA Mode Initialization Fix
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66728/#review201670
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On April 20, 2018, 11:31 p.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66728/
> -----------------------------------------------------------
>
> (Updated April 20, 2018, 11:31 p.m.)
>
>
> Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath Subramanian.
>
>
> Bugs: ATLAS-2589
> https://issues.apache.org/jira/browse/ATLAS-2589
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Approach**
> _AtlasTypeDefStoreInitializer_ is not reverted to the its original form. The earlier version initialized it incorrectly during migration mode.
> _migration.ReaderStatusManager_ has been refactored to remove duplicate code and improve accuracy of restarts.
> _migration.AtlasGraphSONReader_ additional logging.
> _DataMigrationService_:
> - Initializes _GraphBackedSearchIndexer_ followed by initalization of _AtlasTypeDefStoreInitializer_.
> - Additional directory existence check added that will allow/disallow migration. Apporpirate log messages added.
>
>
> Diffs
> -----
>
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java 636e6e8cf
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/JsonNodeParsers.java cd6546003
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/JsonNodeProcessManager.java 6d246191f
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManager.java 11732c426
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManagerTest.java 8c3d1576d
> intg/src/main/java/org/apache/atlas/model/impexp/MigrationStatus.java 901494506
> repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java a4f7a2fea
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java c56157dc2
> repository/src/test/java/org/apache/atlas/repository/migration/MigrationProgressServiceTest.java a0e2e0373
>
>
> Diff: https://reviews.apache.org/r/66728/diff/4/
>
>
> Testing
> -------
>
> **Unit tests**
> - Existing tests were executed.
> - No additional tests added.
>
> **Functional tests**
> - Executed migration in multi-node environment.
>
> **Pre-commit**
> [Success](https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/292/)
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 66728: Data Migration: HA Mode Initialization Fix
Posted by Ashutosh Mestry <am...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66728/
-----------------------------------------------------------
(Updated April 20, 2018, 11:31 p.m.)
Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath Subramanian.
Changes
-------
Updates include:
- Updated logic for resuming.
- Minor refactoring.
Bugs: ATLAS-2589
https://issues.apache.org/jira/browse/ATLAS-2589
Repository: atlas
Description
-------
**Approach**
_AtlasTypeDefStoreInitializer_ is not reverted to the its original form. The earlier version initialized it incorrectly during migration mode.
_migration.ReaderStatusManager_ has been refactored to remove duplicate code and improve accuracy of restarts.
_migration.AtlasGraphSONReader_ additional logging.
_DataMigrationService_:
- Initializes _GraphBackedSearchIndexer_ followed by initalization of _AtlasTypeDefStoreInitializer_.
- Additional directory existence check added that will allow/disallow migration. Apporpirate log messages added.
Diffs (updated)
-----
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java 636e6e8cf
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/JsonNodeParsers.java cd6546003
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/JsonNodeProcessManager.java 6d246191f
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManager.java 11732c426
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManagerTest.java 8c3d1576d
intg/src/main/java/org/apache/atlas/model/impexp/MigrationStatus.java 901494506
repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java a4f7a2fea
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java c56157dc2
repository/src/test/java/org/apache/atlas/repository/migration/MigrationProgressServiceTest.java a0e2e0373
Diff: https://reviews.apache.org/r/66728/diff/4/
Changes: https://reviews.apache.org/r/66728/diff/3-4/
Testing
-------
**Unit tests**
- Existing tests were executed.
- No additional tests added.
**Functional tests**
- Executed migration in multi-node environment.
**Pre-commit**
[Success](https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/292/)
Thanks,
Ashutosh Mestry