You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Jie Yu <yu...@gmail.com> on 2013/12/21 10:11:57 UTC
Re: Review Request 15802: Catch-up Replicated Log 3.1: Added log recovery
support.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15802/
-----------------------------------------------------------
(Updated Dec. 21, 2013, 9:11 a.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, Vinod Kone, and Jiang Yan Xu.
Changes
-------
Do not allow auto start-up for safety concerns.
Summary (updated)
-----------------
Catch-up Replicated Log 3.1: Added log recovery support.
Bugs: MESOS-736
https://issues.apache.org/jira/browse/MESOS-736
Repository: mesos-git
Description
-------
This is the last patch of a series of patches that implement catch-up replicated log.
Here is summary of this patch:
1) Introduced RecoverRequest/RecoverResponse.
2) Used Metadata in replica code.
3) A 2-phase empty log start-up algorithm.
4) Added a test to test two competing recover processes.
This is a joint work with Yan Xu.
Diffs (updated)
-----
src/Makefile.am ddc43bd
src/common/type_utils.hpp 3b05751
src/log/coordinator.hpp 3f6fb7c
src/log/coordinator.cpp 6e6466f
src/log/log.hpp 77edc7a
src/log/log.cpp d057925
src/log/recover.hpp PRE-CREATION
src/log/recover.cpp PRE-CREATION
src/log/replica.hpp d1f5ead
src/log/replica.cpp 82c2157
src/messages/log.proto e6460ab
src/tests/log_tests.cpp ff5f86c
Diff: https://reviews.apache.org/r/15802/diff/
Testing
-------
make check
bin/mesos-tests.sh --gtest_filter=LogTest.*:ReplicaTest.*:CoordinatorTest.*:LogZooKeeperTest.* --gtest_repeat=100
Thanks,
Jie Yu
Re: Review Request 15802: Catch-up Replicated Log 3.1: Added log recovery
support.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15802/
-----------------------------------------------------------
(Updated Jan. 16, 2014, 11:39 p.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, Vinod Kone, and Jiang Yan Xu.
Changes
-------
Rebased.
Bugs: MESOS-736
https://issues.apache.org/jira/browse/MESOS-736
Repository: mesos-git
Description
-------
This is the last patch of a series of patches that implement catch-up replicated log.
Here is summary of this patch:
1) Introduced RecoverRequest/RecoverResponse.
2) Used Metadata in replica code.
3) A 2-phase empty log start-up algorithm.
4) Added a test to test two competing recover processes.
This is a joint work with Yan Xu.
Diffs (updated)
-----
src/Makefile.am cf0c8c6
src/common/type_utils.hpp 3b05751
src/log/coordinator.hpp 3f6fb7c
src/log/coordinator.cpp 6e6466f
src/log/log.hpp 77edc7a
src/log/log.cpp d057925
src/log/recover.hpp PRE-CREATION
src/log/recover.cpp PRE-CREATION
src/log/replica.hpp d1f5ead
src/log/replica.cpp 82c2157
src/messages/log.proto e6460ab
src/tests/log_tests.cpp ff5f86c
Diff: https://reviews.apache.org/r/15802/diff/
Testing
-------
make check
bin/mesos-tests.sh --gtest_filter=LogTest.*:ReplicaTest.*:CoordinatorTest.*:LogZooKeeperTest.* --gtest_repeat=100
Thanks,
Jie Yu
Re: Review Request 15802: Catch-up Replicated Log 3.1: Added log recovery
support.
Posted by Jie Yu <yu...@gmail.com>.
> On Jan. 16, 2014, 4:18 p.m., Benjamin Hindman wrote:
> > src/log/recover.cpp, line 152
> > <https://reviews.apache.org/r/15802/diff/9/?file=424570#file424570line152>
> >
> > It's interesting that you don't use Network::watch here but that you do retry down below if broadcast returns an empty set. Do you want to use watch here too?
Added a new review: Catch-up Replicated Log 4.3.
- Jie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15802/#review32033
-----------------------------------------------------------
On Jan. 16, 2014, 6:40 a.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15802/
> -----------------------------------------------------------
>
> (Updated Jan. 16, 2014, 6:40 a.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Vinod Kone, and Jiang Yan Xu.
>
>
> Bugs: MESOS-736
> https://issues.apache.org/jira/browse/MESOS-736
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> This is the last patch of a series of patches that implement catch-up replicated log.
>
> Here is summary of this patch:
> 1) Introduced RecoverRequest/RecoverResponse.
> 2) Used Metadata in replica code.
> 3) A 2-phase empty log start-up algorithm.
> 4) Added a test to test two competing recover processes.
>
> This is a joint work with Yan Xu.
>
>
> Diffs
> -----
>
> src/Makefile.am cf0c8c6
> src/common/type_utils.hpp 3b05751
> src/log/coordinator.hpp 3f6fb7c
> src/log/coordinator.cpp 6e6466f
> src/log/log.hpp 77edc7a
> src/log/log.cpp d057925
> src/log/recover.hpp PRE-CREATION
> src/log/recover.cpp PRE-CREATION
> src/log/replica.hpp d1f5ead
> src/log/replica.cpp 82c2157
> src/messages/log.proto e6460ab
> src/tests/log_tests.cpp ff5f86c
>
> Diff: https://reviews.apache.org/r/15802/diff/
>
>
> Testing
> -------
>
> make check
>
> bin/mesos-tests.sh --gtest_filter=LogTest.*:ReplicaTest.*:CoordinatorTest.*:LogZooKeeperTest.* --gtest_repeat=100
>
>
> Thanks,
>
> Jie Yu
>
>
Re: Review Request 15802: Catch-up Replicated Log 3.1: Added log recovery
support.
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15802/#review32033
-----------------------------------------------------------
Ship it!
src/log/recover.cpp
<https://reviews.apache.org/r/15802/#comment60759>
It's interesting that you don't use Network::watch here but that you do retry down below if broadcast returns an empty set. Do you want to use watch here too?
- Benjamin Hindman
On Jan. 16, 2014, 6:40 a.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15802/
> -----------------------------------------------------------
>
> (Updated Jan. 16, 2014, 6:40 a.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Vinod Kone, and Jiang Yan Xu.
>
>
> Bugs: MESOS-736
> https://issues.apache.org/jira/browse/MESOS-736
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> This is the last patch of a series of patches that implement catch-up replicated log.
>
> Here is summary of this patch:
> 1) Introduced RecoverRequest/RecoverResponse.
> 2) Used Metadata in replica code.
> 3) A 2-phase empty log start-up algorithm.
> 4) Added a test to test two competing recover processes.
>
> This is a joint work with Yan Xu.
>
>
> Diffs
> -----
>
> src/Makefile.am cf0c8c6
> src/common/type_utils.hpp 3b05751
> src/log/coordinator.hpp 3f6fb7c
> src/log/coordinator.cpp 6e6466f
> src/log/log.hpp 77edc7a
> src/log/log.cpp d057925
> src/log/recover.hpp PRE-CREATION
> src/log/recover.cpp PRE-CREATION
> src/log/replica.hpp d1f5ead
> src/log/replica.cpp 82c2157
> src/messages/log.proto e6460ab
> src/tests/log_tests.cpp ff5f86c
>
> Diff: https://reviews.apache.org/r/15802/diff/
>
>
> Testing
> -------
>
> make check
>
> bin/mesos-tests.sh --gtest_filter=LogTest.*:ReplicaTest.*:CoordinatorTest.*:LogZooKeeperTest.* --gtest_repeat=100
>
>
> Thanks,
>
> Jie Yu
>
>
Re: Review Request 15802: Catch-up Replicated Log 3.1: Added log recovery
support.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15802/
-----------------------------------------------------------
(Updated Jan. 16, 2014, 6:40 a.m.)
Review request for mesos, Benjamin Hindman, Ben Mahler, Vinod Kone, and Jiang Yan Xu.
Changes
-------
Rebase. Addressed BenH's comments.
Removed 'strict' flag and 'STARTING' status.
Bugs: MESOS-736
https://issues.apache.org/jira/browse/MESOS-736
Repository: mesos-git
Description
-------
This is the last patch of a series of patches that implement catch-up replicated log.
Here is summary of this patch:
1) Introduced RecoverRequest/RecoverResponse.
2) Used Metadata in replica code.
3) A 2-phase empty log start-up algorithm.
4) Added a test to test two competing recover processes.
This is a joint work with Yan Xu.
Diffs (updated)
-----
src/Makefile.am cf0c8c6
src/common/type_utils.hpp 3b05751
src/log/coordinator.hpp 3f6fb7c
src/log/coordinator.cpp 6e6466f
src/log/log.hpp 77edc7a
src/log/log.cpp d057925
src/log/recover.hpp PRE-CREATION
src/log/recover.cpp PRE-CREATION
src/log/replica.hpp d1f5ead
src/log/replica.cpp 82c2157
src/messages/log.proto e6460ab
src/tests/log_tests.cpp ff5f86c
Diff: https://reviews.apache.org/r/15802/diff/
Testing
-------
make check
bin/mesos-tests.sh --gtest_filter=LogTest.*:ReplicaTest.*:CoordinatorTest.*:LogZooKeeperTest.* --gtest_repeat=100
Thanks,
Jie Yu
Re: Review Request 15802: Catch-up Replicated Log 3.1: Added log recovery
support.
Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15802/#review31934
-----------------------------------------------------------
src/log/recover.cpp
<https://reviews.apache.org/r/15802/#comment60692>
Scenarios:
(1) A and B and C all in VOTING, adding D, quorum of size 2:
D goes into RECOVERING, recovers then goes into VOTING.
(2) A and B in VOTING, adding D, quorum size of 2:
D goes into RECOVERING, recovers then goes into VOTING.
(3) A in VOTING, adding D, quorum size of 2:
D tries to recover forever.
(4) A in VOTING and B in EMPTY, adding D, quorum of size 2:
D tries to recover forever.
(5) A in VOTING and B in STARTING, adding D, quorum of size 2:
D goes into VOTING, does not recover from A, B does the same thing.
(6) A in VOTING and adding B which was initailized with STARTING, quorum of size 2:
B goes into VOTING, does not recover from A.
- Benjamin Hindman
On Dec. 21, 2013, 9:11 a.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15802/
> -----------------------------------------------------------
>
> (Updated Dec. 21, 2013, 9:11 a.m.)
>
>
> Review request for mesos, Benjamin Hindman, Ben Mahler, Vinod Kone, and Jiang Yan Xu.
>
>
> Bugs: MESOS-736
> https://issues.apache.org/jira/browse/MESOS-736
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> This is the last patch of a series of patches that implement catch-up replicated log.
>
> Here is summary of this patch:
> 1) Introduced RecoverRequest/RecoverResponse.
> 2) Used Metadata in replica code.
> 3) A 2-phase empty log start-up algorithm.
> 4) Added a test to test two competing recover processes.
>
> This is a joint work with Yan Xu.
>
>
> Diffs
> -----
>
> src/Makefile.am ddc43bd
> src/common/type_utils.hpp 3b05751
> src/log/coordinator.hpp 3f6fb7c
> src/log/coordinator.cpp 6e6466f
> src/log/log.hpp 77edc7a
> src/log/log.cpp d057925
> src/log/recover.hpp PRE-CREATION
> src/log/recover.cpp PRE-CREATION
> src/log/replica.hpp d1f5ead
> src/log/replica.cpp 82c2157
> src/messages/log.proto e6460ab
> src/tests/log_tests.cpp ff5f86c
>
> Diff: https://reviews.apache.org/r/15802/diff/
>
>
> Testing
> -------
>
> make check
>
> bin/mesos-tests.sh --gtest_filter=LogTest.*:ReplicaTest.*:CoordinatorTest.*:LogZooKeeperTest.* --gtest_repeat=100
>
>
> Thanks,
>
> Jie Yu
>
>