You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2016/04/16 01:37:51 UTC
Review Request 46291: Moving db migration into LogStorage
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46291/
-----------------------------------------------------------
Review request for Aurora, Joshua Cohen and Bill Farner.
Repository: aurora
Description
-------
Our current placement of the `MigrationManager` does not let us use it for DB data backfill (e.g. cases when in addition to creating new tables we need to populate them with existing data). This is currently impossible due to catch-up transactions that are applied in LogStorage _after_ the snapshot is restored.
This change moves `MigrationManager.migrate()` call into the `LogStorage` to get called after `recover()` but before the `recovered` flag is set (to avoid pushing backfill changes into native log). This way we can backfill data right from migration scripts.
Diffs
-----
src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java d08873c88f159eb65b582840b48b7ff604862c31
src/main/java/org/apache/aurora/scheduler/storage/log/LogStorage.java 5143668c8dacb2d8695cf8299abf2e6fab536de2
src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java b6922e1d5bdd188304359b5646be437ed09ec8d1
src/test/java/org/apache/aurora/scheduler/storage/log/LogStorageTest.java bf9479d60531324578354c4a15fcfdac040e7ffc
src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java ff9c1d08a9a99a69e94d634c895d20fa6c8c2f88
Diff: https://reviews.apache.org/r/46291/diff/
Testing
-------
Thanks,
Maxim Khutornenko
Re: Review Request 46291: Moving db migration into LogStorage
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46291/#review129216
-----------------------------------------------------------
Ship it!
Master (81f52e4) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On April 15, 2016, 11:37 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46291/
> -----------------------------------------------------------
>
> (Updated April 15, 2016, 11:37 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Our current placement of the `MigrationManager` does not let us use it for DB data backfill (e.g. cases when in addition to creating new tables we need to populate them with existing data). This is currently impossible due to catch-up transactions that are applied in LogStorage _after_ the snapshot is restored.
>
> This change moves `MigrationManager.migrate()` call into the `LogStorage` to get called after `recover()` but before the `recovered` flag is set (to avoid pushing backfill changes into native log). This way we can backfill data right from migration scripts.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java d08873c88f159eb65b582840b48b7ff604862c31
> src/main/java/org/apache/aurora/scheduler/storage/log/LogStorage.java 5143668c8dacb2d8695cf8299abf2e6fab536de2
> src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java b6922e1d5bdd188304359b5646be437ed09ec8d1
> src/test/java/org/apache/aurora/scheduler/storage/log/LogStorageTest.java bf9479d60531324578354c4a15fcfdac040e7ffc
> src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java ff9c1d08a9a99a69e94d634c895d20fa6c8c2f88
>
> Diff: https://reviews.apache.org/r/46291/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 46291: Moving db migration into LogStorage
Posted by Joshua Cohen <jc...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46291/#review129359
-----------------------------------------------------------
Ship it!
Ship It!
- Joshua Cohen
On April 15, 2016, 11:37 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46291/
> -----------------------------------------------------------
>
> (Updated April 15, 2016, 11:37 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Our current placement of the `MigrationManager` does not let us use it for DB data backfill (e.g. cases when in addition to creating new tables we need to populate them with existing data). This is currently impossible due to catch-up transactions that are applied in LogStorage _after_ the snapshot is restored.
>
> This change moves `MigrationManager.migrate()` call into the `LogStorage` to get called after `recover()` but before the `recovered` flag is set (to avoid pushing backfill changes into native log). This way we can backfill data right from migration scripts.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/storage/backup/TemporaryStorage.java d08873c88f159eb65b582840b48b7ff604862c31
> src/main/java/org/apache/aurora/scheduler/storage/log/LogStorage.java 5143668c8dacb2d8695cf8299abf2e6fab536de2
> src/main/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl.java b6922e1d5bdd188304359b5646be437ed09ec8d1
> src/test/java/org/apache/aurora/scheduler/storage/log/LogStorageTest.java bf9479d60531324578354c4a15fcfdac040e7ffc
> src/test/java/org/apache/aurora/scheduler/storage/log/SnapshotStoreImplIT.java ff9c1d08a9a99a69e94d634c895d20fa6c8c2f88
>
> Diff: https://reviews.apache.org/r/46291/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Maxim Khutornenko
>
>