You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Jie Yu <yu...@gmail.com> on 2014/03/01 23:20:53 UTC

Re: Review Request 18600: Allowed replicated log to do auto initialization.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 1, 2014, 10:20 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Added/updated comments.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp 7f855f2 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 3403b47 
  src/tests/log_tests.cpp 0e2485f 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review36500
-----------------------------------------------------------


Patch looks great!

Reviews applied: [18584, 18600]

All tests passed.

- Mesos ReviewBot


On March 7, 2014, 5:40 a.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated March 7, 2014, 5:40 a.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 628a4b8 
>   src/log/log.cpp c427920 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp e611a4e 
>   src/tests/log_tests.cpp da827a1 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review36577
-----------------------------------------------------------


Patch looks great!

Reviews applied: [18600]

All tests passed.

- Mesos ReviewBot


On March 7, 2014, 9:44 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated March 7, 2014, 9:44 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 628a4b8 
>   src/log/log.cpp c427920 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp da827a1 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Benjamin Hindman <be...@berkeley.edu>.

> On April 15, 2014, 9:12 p.m., Ben Mahler wrote:
> > src/log/recover.cpp, line 113
> > <https://reviews.apache.org/r/18600/diff/9/?file=549174#file549174line113>
> >
> >     If we're auto-initializing, shouldn't we 'watch' for the cluster size as opposed to the quorum size to ensure we don't get stuck?

Beyond just watching for the cluster size to appear, what happens when this watch is triggered but before any messages are sent out the replicas die (or are stopped by an operator)? We don't want the replicas to get into a completely blocked state so we need some way of either retrying to do the auto-initialization after some timeouts and all together bailing after some number of retries.


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review40464
-----------------------------------------------------------


On April 4, 2014, 7:12 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated April 4, 2014, 7:12 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 6787c80 
>   src/log/log.cpp 9dd992f 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp 4f08927 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.

> On April 15, 2014, 9:12 p.m., Ben Mahler wrote:
> > src/log/recover.cpp, line 113
> > <https://reviews.apache.org/r/18600/diff/9/?file=549174#file549174line113>
> >
> >     If we're auto-initializing, shouldn't we 'watch' for the cluster size as opposed to the quorum size to ensure we don't get stuck?
> 
> Benjamin Hindman wrote:
>     Beyond just watching for the cluster size to appear, what happens when this watch is triggered but before any messages are sent out the replicas die (or are stopped by an operator)? We don't want the replicas to get into a completely blocked state so we need some way of either retrying to do the auto-initialization after some timeouts and all together bailing after some number of retries.

I thought about this again this morning. First, we CANNOT watch for the cluster size because we don't know if we are in auto-init case or not until we receive responses from replicas. If we are in catch-up case, we don't need all replicas to respond.

So I plan to do the following as BenH suggested: add a retry timeout for log recovery (retry the recovery after timeout occurs).


- Jie


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review40464
-----------------------------------------------------------


On April 4, 2014, 7:12 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated April 4, 2014, 7:12 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 6787c80 
>   src/log/log.cpp 9dd992f 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp 4f08927 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review40464
-----------------------------------------------------------



src/log/log.hpp
<https://reviews.apache.org/r/18600/#comment73487>

    Should this be the verb form: "autoInitialize"?



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment73488>

    If we're auto-initializing, shouldn't we 'watch' for the cluster size as opposed to the quorum size to ensure we don't get stuck?


- Ben Mahler


On April 4, 2014, 7:12 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated April 4, 2014, 7:12 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 6787c80 
>   src/log/log.cpp 9dd992f 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp 4f08927 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Benjamin Hindman <be...@berkeley.edu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review41090
-----------------------------------------------------------

Ship it!



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment74495>

    Can you add a similar comment as in 'timedout' here? That is let's reiterate the role 'terminating' plays and that it gets set in 'discard'.


- Benjamin Hindman


On April 21, 2014, 10:17 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated April 21, 2014, 10:17 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 6787c80 
>   src/log/log.cpp 9dd992f 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp 4f08927 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated April 21, 2014, 10:17 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Updated the "TODO".


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 6787c80 
  src/log/log.cpp 9dd992f 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 688da5f 
  src/tests/log_tests.cpp 4f08927 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated April 21, 2014, 10:11 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

BenM's comments.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 6787c80 
  src/log/log.cpp 9dd992f 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 688da5f 
  src/tests/log_tests.cpp 4f08927 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.

> On April 18, 2014, 12:46 a.m., Ben Mahler wrote:
> > src/log/recover.cpp, lines 239-240
> > <https://reviews.apache.org/r/18600/diff/10/?file=561411#file561411line239>
> >
> >     Maybe a good TODO would be on the Log interface? I'm curious why we would allow the 'quorum' to be specified explicitly when constructing a Log with a set of pids. Are there reasons for having larger than necessary quorum sizes?
> >     
> >     We don't seem to be validating the quorum size is large enough w.r.t 'pids'?

> I'm curious why we would allow the 'quorum' to be specified explicitly
> when constructing a Log with a set of pids. Are there reasons for
> having larger than necessary quorum sizes?

Technically, quorum size does not necessarily has to be "floor(cluster size/2) + 1".

For example, one may wanna choose to use quorum == cluster size in some cases (for high redundancy).


> On April 18, 2014, 12:46 a.m., Ben Mahler wrote:
> > src/log/recover.cpp, line 325
> > <https://reviews.apache.org/r/18600/diff/10/?file=561411#file561411line325>
> >
> >     Do we want to chain this new run behind 'future' transitioning from pending or is it safe to proceed immediately with the new protocol run?

Re-wrote it after the 'select' bug has been fixed in
https://reviews.apache.org/r/20533/


> On April 18, 2014, 12:46 a.m., Ben Mahler wrote:
> > src/tests/log_tests.cpp, lines 1749-1751
> > <https://reviews.apache.org/r/18600/diff/10/?file=561412#file561412line1749>
> >
> >     We have macros to do this in one line if you like:
> >     
> >     AWAIT_EQ
> >     AWAIT_ASSERT_EQ
> >     AWAIT_EXPECT_EQ
> >     
> >     Ditto elsewhere.

You'll gonna need AWAIT_EXPECT_SOME_EQ(...).

Will just leave it for now.


> On April 18, 2014, 12:46 a.m., Ben Mahler wrote:
> > src/log/recover.cpp, line 349
> > <https://reviews.apache.org/r/18600/diff/9-10/?file=549174#file549174line349>
> >
> >     10 Seconds seems like quite a lot when we expect operators to start replicas sequentially? What would be the consequences of a lower timeout?

We'll just retry more frequently. I'll keep 10 seconds for now (to be consistent with the catchup retry timeout) unless it becomes a problem.


- Jie


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review40724
-----------------------------------------------------------


On April 16, 2014, 11:43 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated April 16, 2014, 11:43 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 6787c80 
>   src/log/log.cpp 9dd992f 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp 4f08927 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review40724
-----------------------------------------------------------



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment73860>

    How about s/Running r/R/ and s/Retry/Retrying/ ?
    
    Would it be possible to have the timeout duration in this log message?



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment73861>

    10 Seconds seems like quite a lot when we expect operators to start replicas sequentially? What would be the consequences of a lower timeout?



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment73864>

    Maybe a good TODO would be on the Log interface? I'm curious why we would allow the 'quorum' to be specified explicitly when constructing a Log with a set of pids. Are there reasons for having larger than necessary quorum sizes?
    
    We don't seem to be validating the quorum size is large enough w.r.t 'pids'?



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment73863>

    Do we want to chain this new run behind 'future' transitioning from pending or is it safe to proceed immediately with the new protocol run?



src/tests/log_tests.cpp
<https://reviews.apache.org/r/18600/#comment73858>

    We have macros to do this in one line if you like:
    
    AWAIT_EQ
    AWAIT_ASSERT_EQ
    AWAIT_EXPECT_EQ
    
    Ditto elsewhere.


- Ben Mahler


On April 16, 2014, 11:43 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated April 16, 2014, 11:43 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 6787c80 
>   src/log/log.cpp 9dd992f 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp 4f08927 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated April 16, 2014, 11:43 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Review comments.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 6787c80 
  src/log/log.cpp 9dd992f 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 688da5f 
  src/tests/log_tests.cpp 4f08927 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review39595
-----------------------------------------------------------


Patch looks great!

Reviews applied: [18600]

All tests passed.

- Mesos ReviewBot


On April 4, 2014, 7:12 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated April 4, 2014, 7:12 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 6787c80 
>   src/log/log.cpp 9dd992f 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp 4f08927 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated April 4, 2014, 7:12 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

BenM's comments. Rebased.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 6787c80 
  src/log/log.cpp 9dd992f 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 688da5f 
  src/tests/log_tests.cpp 4f08927 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review37565
-----------------------------------------------------------


Patch looks great!

Reviews applied: [18600]

All tests passed.

- Mesos ReviewBot


On March 17, 2014, 11:30 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated March 17, 2014, 11:30 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp c2cb6cf 
>   src/log/log.cpp d9b2687 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp b368cde 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.

> On March 27, 2014, 11:38 p.m., Ben Mahler wrote:
> > src/log/log.hpp, line 193
> > <https://reviews.apache.org/r/18600/diff/8/?file=525771#file525771line193>
> >
> >     Can you update the documentation?
> >     
> >     I'm a bit hesitant about having an implicit 'auto-initialize=false', since it's less usable (from a developer/operator perspective). Maybe we should default to 'true' or require it to be specified?
> >     
> >     Up to you if you want to preserve the old API.

Yeah, let's preserve the old API in this patch, having another patch for changing the default.


> On March 27, 2014, 11:38 p.m., Ben Mahler wrote:
> > src/log/recover.cpp, lines 237-238
> > <https://reviews.apache.org/r/18600/diff/8/?file=525774#file525774line237>
> >
> >     Would you mind re-ordering to add EMPTY + STARTING just so that it matches your comments above? Ditto below for STARTING + VOTING.

Done.


> On March 27, 2014, 11:38 p.m., Ben Mahler wrote:
> > src/log/recover.cpp, lines 363-365
> > <https://reviews.apache.org/r/18600/diff/8/?file=525774#file525774line363>
> >
> >     Maybe we should make the transition to VOTING more explicit in the comment here?
> >     
> >     "If auto-initialization is enabled, all replicas will first go into STARTING status. Once *all* replicas have transitioned out of EMPTY status, the replicas will go into VOTING status."

Done.


> On March 27, 2014, 11:38 p.m., Ben Mahler wrote:
> > src/log/recover.cpp, line 430
> > <https://reviews.apache.org/r/18600/diff/8/?file=525774#file525774line430>
> >
> >     "re-running"?

Done.


- Jie


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review38826
-----------------------------------------------------------


On March 17, 2014, 11:30 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated March 17, 2014, 11:30 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp c2cb6cf 
>   src/log/log.cpp d9b2687 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp b368cde 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review38826
-----------------------------------------------------------


Hey Jie, this looks really good.

The only thing I would like to see before shipping is a test of the following:
  -Ensure that no-one transitions out of STARTING until all replicas are out of EMPTY.

You could probably fit this into your existing test by delaying the creation of the 3rd replica?


src/log/log.hpp
<https://reviews.apache.org/r/18600/#comment71155>

    Can you update the documentation?
    
    I'm a bit hesitant about having an implicit 'auto-initialize=false', since it's less usable (from a developer/operator perspective). Maybe we should default to 'true' or require it to be specified?
    
    Up to you if you want to preserve the old API.



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment71158>

    Great comments! s/then/them/



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment71159>

    Just for sanity: (2 * quorum) - 1



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment71160>

    Would you mind re-ordering to add EMPTY + STARTING just so that it matches your comments above? Ditto below for STARTING + VOTING.



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment71161>

    Maybe we should make the transition to VOTING more explicit in the comment here?
    
    "If auto-initialization is enabled, all replicas will first go into STARTING status. Once *all* replicas have transitioned out of EMPTY status, the replicas will go into VOTING status."



src/log/recover.cpp
<https://reviews.apache.org/r/18600/#comment71162>

    "re-running"?


- Ben Mahler


On March 17, 2014, 11:30 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated March 17, 2014, 11:30 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp c2cb6cf 
>   src/log/log.cpp d9b2687 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 688da5f 
>   src/tests/log_tests.cpp b368cde 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 17, 2014, 11:30 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Rebased.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp c2cb6cf 
  src/log/log.cpp d9b2687 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 688da5f 
  src/tests/log_tests.cpp b368cde 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 7, 2014, 9:44 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Rebased.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp c427920 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 688da5f 
  src/tests/log_tests.cpp da827a1 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 7, 2014, 5:40 a.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Rebased.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp c427920 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp e611a4e 
  src/tests/log_tests.cpp da827a1 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 6, 2014, 10:26 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Updated 'depends-on'


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp c427920 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp e611a4e 
  src/tests/log_tests.cpp da827a1 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 6, 2014, 10:25 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Rebased.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp c427920 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp e611a4e 
  src/tests/log_tests.cpp da827a1 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/#review36286
-----------------------------------------------------------


Patch looks great!

Reviews applied: [18794, 18368, 18584, 18600]

All tests passed.

- Mesos ReviewBot


On March 5, 2014, 8:20 p.m., Jie Yu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/18600/
> -----------------------------------------------------------
> 
> (Updated March 5, 2014, 8:20 p.m.)
> 
> 
> Review request for mesos, Benjamin Hindman and Ben Mahler.
> 
> 
> Bugs: MESOS-984
>     https://issues.apache.org/jira/browse/MESOS-984
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/log/log.hpp 628a4b8 
>   src/log/log.cpp 2d3a0cf 
>   src/log/recover.hpp 634bc06 
>   src/log/recover.cpp 3403b47 
>   src/tests/log_tests.cpp 7f2a740 
> 
> Diff: https://reviews.apache.org/r/18600/diff/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Jie Yu
> 
>


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 5, 2014, 8:20 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Updated 'depends-on' field.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp 2d3a0cf 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 3403b47 
  src/tests/log_tests.cpp 7f2a740 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 5, 2014, 8:19 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Resolved the conflict caused by returning const ref for Future:get() (exposed by review-bot!!!)


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp 2d3a0cf 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 3403b47 
  src/tests/log_tests.cpp 7f2a740 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 5, 2014, 6:18 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

updated 'depends on'


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp 7f855f2 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 3403b47 
  src/tests/log_tests.cpp 0e2485f 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu


Re: Review Request 18600: Allowed replicated log to do auto initialization.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18600/
-----------------------------------------------------------

(Updated March 1, 2014, 10:33 p.m.)


Review request for mesos, Benjamin Hindman and Ben Mahler.


Changes
-------

Dominic's comments.


Bugs: MESOS-984
    https://issues.apache.org/jira/browse/MESOS-984


Repository: mesos-git


Description
-------

See summary.


Diffs (updated)
-----

  src/log/log.hpp 628a4b8 
  src/log/log.cpp 7f855f2 
  src/log/recover.hpp 634bc06 
  src/log/recover.cpp 3403b47 
  src/tests/log_tests.cpp 0e2485f 

Diff: https://reviews.apache.org/r/18600/diff/


Testing
-------

make check


Thanks,

Jie Yu