You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Ilya Pronin <ip...@twopensource.com> on 2017/09/13 16:40:41 UTC
Review Request 62285: Added tombstone flag to NOP log action.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62285/
-----------------------------------------------------------
Review request for mesos and Jie Yu.
Bugs: MESOS-7973
https://issues.apache.org/jira/browse/MESOS-7973
Repository: mesos
Description
-------
NOP action is used as a filler for holes in the "fill" procedure and as
the action in a response to a promise request for a truncated position.
The tombstone flag is introduced so that replicas are able to
distinguish the truncated action from a real NOP.
Diffs
-----
src/log/leveldb.cpp 5310a123b0fb25f240429722b676fe46174cb2ce
src/log/replica.cpp 39f2879b2e37c1ca9a9f987ce0a3b83e8dbc9b43
src/messages/log.proto ca740bd011147f8f48cc5dfcacefa5fdbd95ccd0
src/tests/log_tests.cpp f9f9400c901152779ae0ebfe74cf8f7aac1d3396
Diff: https://reviews.apache.org/r/62285/diff/1/
Testing
-------
Added a test that verifies that a replica correctly handles the tombstone NOP. Ran `make check`.
Thanks,
Ilya Pronin
Re: Review Request 62285: Added tombstone flag to NOP log action.
Posted by Ilya Pronin <ip...@twopensource.com>.
> On Sept. 25, 2017, 5:12 a.m., Jie Yu wrote:
> > src/log/leveldb.cpp
> > Lines 247 (patched)
> > <https://reviews.apache.org/r/62285/diff/1/?file=1820903#file1820903line247>
> >
> > I would combine this condition with the condition in `else if` above.
Done!
> On Sept. 25, 2017, 5:12 a.m., Jie Yu wrote:
> > src/log/leveldb.cpp
> > Lines 364-366 (patched)
> > <https://reviews.apache.org/r/62285/diff/1/?file=1820903#file1820903line364>
> >
> > Can you explain this a bit more. When I see the code in `restore` above, I thought the invariant you want to maintain is: there is no tombstone entry in the replica's [begin,end]. Why the recovery code needs to see this position?
Yes, there should not be tombstones in replica's `[begin, end]` and the replica should recover the same `begin` position as it had when it terminated. While the latter is not technically required, I think it is nice to follow the principle of least surprise (especially when it doesn't require much effort): the replica has already learned that `begin > 0`, why does it recover with `begin = 0`?
Currently, `LevelDBStorage` recovery code calculates replica's `begin` position like this (C++ish pseudocode):
```c++
begin = 0;
for (const auto& action : actions)
if (action.type() == TRUNCATE)
begin = max(begin, action.truncate().to());
else if (action.type() == NOP)
begin = max(begin, action.position() + 1);
```
If it doesn't see a tombstone or a `TRUNCATE` action, it will recover `begin = 0`.
We can modify the recovery code to recover `begin` as the first position that we read from the disk. But we still need to persist the tombstone, because currently LevelDB truncations are best-effort.
> On Sept. 25, 2017, 5:12 a.m., Jie Yu wrote:
> > src/log/replica.cpp
> > Lines 735-736 (patched)
> > <https://reviews.apache.org/r/62285/diff/1/?file=1820904#file1820904line735>
> >
> > Ditto on combining considtions into a single `else if`
Done!
- Ilya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62285/#review186083
-----------------------------------------------------------
On Sept. 13, 2017, 5:40 p.m., Ilya Pronin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62285/
> -----------------------------------------------------------
>
> (Updated Sept. 13, 2017, 5:40 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Bugs: MESOS-7973
> https://issues.apache.org/jira/browse/MESOS-7973
>
>
> Repository: mesos
>
>
> Description
> -------
>
> NOP action is used as a filler for holes in the "fill" procedure and as
> the action in a response to a promise request for a truncated position.
> The tombstone flag is introduced so that replicas are able to
> distinguish the truncated action from a real NOP.
>
>
> Diffs
> -----
>
> src/log/leveldb.cpp 5310a123b0fb25f240429722b676fe46174cb2ce
> src/log/replica.cpp 39f2879b2e37c1ca9a9f987ce0a3b83e8dbc9b43
> src/messages/log.proto ca740bd011147f8f48cc5dfcacefa5fdbd95ccd0
> src/tests/log_tests.cpp f9f9400c901152779ae0ebfe74cf8f7aac1d3396
>
>
> Diff: https://reviews.apache.org/r/62285/diff/1/
>
>
> Testing
> -------
>
> Added a test that verifies that a replica correctly handles the tombstone NOP. Ran `make check`.
>
>
> Thanks,
>
> Ilya Pronin
>
>
Re: Review Request 62285: Added tombstone flag to NOP log action.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62285/#review186083
-----------------------------------------------------------
src/log/leveldb.cpp
Lines 247 (patched)
<https://reviews.apache.org/r/62285/#comment262505>
I would combine this condition with the condition in `else if` above.
src/log/leveldb.cpp
Lines 364-366 (patched)
<https://reviews.apache.org/r/62285/#comment262506>
Can you explain this a bit more. When I see the code in `restore` above, I thought the invariant you want to maintain is: there is no tombstone entry in the replica's [begin,end]. Why the recovery code needs to see this position?
src/log/replica.cpp
Lines 735-736 (patched)
<https://reviews.apache.org/r/62285/#comment262507>
Ditto on combining considtions into a single `else if`
- Jie Yu
On Sept. 13, 2017, 4:40 p.m., Ilya Pronin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62285/
> -----------------------------------------------------------
>
> (Updated Sept. 13, 2017, 4:40 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Bugs: MESOS-7973
> https://issues.apache.org/jira/browse/MESOS-7973
>
>
> Repository: mesos
>
>
> Description
> -------
>
> NOP action is used as a filler for holes in the "fill" procedure and as
> the action in a response to a promise request for a truncated position.
> The tombstone flag is introduced so that replicas are able to
> distinguish the truncated action from a real NOP.
>
>
> Diffs
> -----
>
> src/log/leveldb.cpp 5310a123b0fb25f240429722b676fe46174cb2ce
> src/log/replica.cpp 39f2879b2e37c1ca9a9f987ce0a3b83e8dbc9b43
> src/messages/log.proto ca740bd011147f8f48cc5dfcacefa5fdbd95ccd0
> src/tests/log_tests.cpp f9f9400c901152779ae0ebfe74cf8f7aac1d3396
>
>
> Diff: https://reviews.apache.org/r/62285/diff/1/
>
>
> Testing
> -------
>
> Added a test that verifies that a replica correctly handles the tombstone NOP. Ran `make check`.
>
>
> Thanks,
>
> Ilya Pronin
>
>
Re: Review Request 62285: Added tombstone flag to NOP log action.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62285/#review194367
-----------------------------------------------------------
Ship it!
Ship It!
- Jie Yu
On Sept. 25, 2017, 1:05 p.m., Ilya Pronin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62285/
> -----------------------------------------------------------
>
> (Updated Sept. 25, 2017, 1:05 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Bugs: MESOS-7973
> https://issues.apache.org/jira/browse/MESOS-7973
>
>
> Repository: mesos
>
>
> Description
> -------
>
> NOP action is used as a filler for holes in the "fill" procedure and as
> the action in a response to a promise request for a truncated position.
> The tombstone flag is introduced so that replicas are able to
> distinguish the truncated action from a real NOP.
>
>
> Diffs
> -----
>
> src/log/leveldb.cpp 5310a123b0fb25f240429722b676fe46174cb2ce
> src/log/replica.cpp 39f2879b2e37c1ca9a9f987ce0a3b83e8dbc9b43
> src/messages/log.proto ca740bd011147f8f48cc5dfcacefa5fdbd95ccd0
> src/tests/log_tests.cpp f9f9400c901152779ae0ebfe74cf8f7aac1d3396
>
>
> Diff: https://reviews.apache.org/r/62285/diff/2/
>
>
> Testing
> -------
>
> Added a test that verifies that a replica correctly handles the tombstone NOP. Ran `make check`.
>
>
> Thanks,
>
> Ilya Pronin
>
>
Re: Review Request 62285: Added tombstone flag to NOP log action.
Posted by Ilya Pronin <ip...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62285/
-----------------------------------------------------------
(Updated Sept. 25, 2017, 2:05 p.m.)
Review request for mesos and Jie Yu.
Changes
-------
Addressed review comments.
Bugs: MESOS-7973
https://issues.apache.org/jira/browse/MESOS-7973
Repository: mesos
Description
-------
NOP action is used as a filler for holes in the "fill" procedure and as
the action in a response to a promise request for a truncated position.
The tombstone flag is introduced so that replicas are able to
distinguish the truncated action from a real NOP.
Diffs (updated)
-----
src/log/leveldb.cpp 5310a123b0fb25f240429722b676fe46174cb2ce
src/log/replica.cpp 39f2879b2e37c1ca9a9f987ce0a3b83e8dbc9b43
src/messages/log.proto ca740bd011147f8f48cc5dfcacefa5fdbd95ccd0
src/tests/log_tests.cpp f9f9400c901152779ae0ebfe74cf8f7aac1d3396
Diff: https://reviews.apache.org/r/62285/diff/2/
Changes: https://reviews.apache.org/r/62285/diff/1-2/
Testing
-------
Added a test that verifies that a replica correctly handles the tombstone NOP. Ran `make check`.
Thanks,
Ilya Pronin