You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Joris Van Remoortere <jo...@gmail.com> on 2014/10/23 23:01:41 UTC
Review Request 27111: Block IO Isolator: usage metrics
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/
-----------------------------------------------------------
Review request for mesos, Benjamin Hindman and Ian Downes.
Bugs: MESOS-1977
https://issues.apache.org/jira/browse/MESOS-1977
Repository: mesos-git
Description
-------
Implement a block io isolator that just publishes read / write, bytes / operations per second.
A split of r25922 as suggested by Ian.
Diffs
-----
include/mesos/mesos.proto 6b93e90
src/Makefile.am 2617f77
src/linux/cgroups.hpp abf31df
src/linux/cgroups.cpp 62df4b7
src/linux/fs.hpp ac8b5f4
src/linux/fs.cpp b01d14c
src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
src/slave/containerizer/mesos/containerizer.cpp 9d08329
src/tests/isolator_tests.cpp 52b38a3
Diff: https://reviews.apache.org/r/27111/diff/
Testing
-------
make check
sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
Thanks,
Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Joris Van Remoortere <jo...@gmail.com>.
> On Oct. 25, 2014, 1:02 a.m., Ian Downes wrote:
> > src/slave/containerizer/isolators/cgroups/blkio.cpp, line 252
> > <https://reviews.apache.org/r/27111/diff/2/?file=731349#file731349line252>
> >
> > What's the difference between blkio.throttle.io_service_bytes and blkio.io_service_bytes?
>From my experimentation: blkio.io_service_bytes does not actually report the bytes read/written, whereas blkio.throttle.io_service_bytes does.
- Joris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/#review58418
-----------------------------------------------------------
On Oct. 25, 2014, 5:53 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27111/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2014, 5:53 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Ian Downes.
>
>
> Bugs: MESOS-1977
> https://issues.apache.org/jira/browse/MESOS-1977
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Implement a block io isolator that just publishes read / write, bytes / operations per second.
> A split of r25922 as suggested by Ian.
>
>
> Diffs
> -----
>
> include/mesos/mesos.proto 6b93e90
> src/Makefile.am 2617f77
> src/linux/cgroups.hpp abf31df
> src/linux/cgroups.cpp 62df4b7
> src/linux/fs.hpp ac8b5f4
> src/linux/fs.cpp b01d14c
> src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
> src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
> src/slave/containerizer/mesos/containerizer.cpp 9d08329
> src/tests/isolator_tests.cpp 52b38a3
>
> Diff: https://reviews.apache.org/r/27111/diff/
>
>
> Testing
> -------
>
> make check
> sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Ian Downes <ia...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/#review58418
-----------------------------------------------------------
Close to ready. Major points are cleaning up the cgroups statistics functions and making the test child safe.
src/linux/cgroups.hpp
<https://reviews.apache.org/r/27111/#comment99413>
s/The a pair/The pair/
src/linux/cgroups.cpp
<https://reviews.apache.org/r/27111/#comment99414>
We accept ">>" now.
src/linux/cgroups.cpp
<https://reviews.apache.org/r/27111/#comment99481>
I think it's error-prone to have the implicit <read, write> pairing and specifying the file.
What about hiding this function in an internal namespace and exposing the various statistics explicitly under a descriptive namespace, e.g., perhaps something like this:
Bytes blkio::statistics::bytes::{read,write}()
uint64_t blkio::statistics::iops::{read,write}()
Yes, converting to Bytes to return is unnecessary in your usage but it avoids any ambiguity for the user of this library function.
src/linux/fs.cpp
<https://reviews.apache.org/r/27111/#comment99419>
Any tests for this?
src/linux/fs.cpp
<https://reviews.apache.org/r/27111/#comment99417>
move this to after the file.fail() check?
src/slave/containerizer/isolators/cgroups/blkio.cpp
<https://reviews.apache.org/r/27111/#comment99473>
kill newline
src/slave/containerizer/isolators/cgroups/blkio.cpp
<https://reviews.apache.org/r/27111/#comment99482>
What's the difference between blkio.throttle.io_service_bytes and blkio.io_service_bytes?
src/tests/isolator_tests.cpp
<https://reviews.apache.org/r/27111/#comment99490>
s/alligned/aligned/
src/tests/isolator_tests.cpp
<https://reviews.apache.org/r/27111/#comment99492>
posix_memalign is definitely not async-signal-safe
src/tests/isolator_tests.cpp
<https://reviews.apache.org/r/27111/#comment99488>
s/ret/write/? or count or ...
something other than 'ret', please :-)
pwrite() is not listed as async-signal-safe: http://man7.org/linux/man-pages/man7/signal.7.html
src/tests/isolator_tests.cpp
<https://reviews.apache.org/r/27111/#comment99489>
ditto
src/tests/isolator_tests.cpp
<https://reviews.apache.org/r/27111/#comment99484>
This "setup" function must be async-signal-safe as it's called between the fork and exec, and we're forking from a threaded parent.
http://man7.org/linux/man-pages/man7/signal.7.html
Can you verify that everything in there is safe (and comment that it is) or put this code into a separate helper binary?
- Ian Downes
On Oct. 23, 2014, 2:58 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27111/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 2:58 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Ian Downes.
>
>
> Bugs: MESOS-1977
> https://issues.apache.org/jira/browse/MESOS-1977
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Implement a block io isolator that just publishes read / write, bytes / operations per second.
> A split of r25922 as suggested by Ian.
>
>
> Diffs
> -----
>
> include/mesos/mesos.proto 6b93e90
> src/Makefile.am 2617f77
> src/linux/cgroups.hpp abf31df
> src/linux/cgroups.cpp 62df4b7
> src/linux/fs.hpp ac8b5f4
> src/linux/fs.cpp b01d14c
> src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
> src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
> src/slave/containerizer/mesos/containerizer.cpp 9d08329
> src/tests/isolator_tests.cpp 52b38a3
>
> Diff: https://reviews.apache.org/r/27111/diff/
>
>
> Testing
> -------
>
> make check
> sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/#review58221
-----------------------------------------------------------
Patch looks great!
Reviews applied: [27111]
All tests passed.
- Mesos ReviewBot
On Oct. 23, 2014, 9:58 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27111/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 9:58 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Ian Downes.
>
>
> Bugs: MESOS-1977
> https://issues.apache.org/jira/browse/MESOS-1977
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Implement a block io isolator that just publishes read / write, bytes / operations per second.
> A split of r25922 as suggested by Ian.
>
>
> Diffs
> -----
>
> include/mesos/mesos.proto 6b93e90
> src/Makefile.am 2617f77
> src/linux/cgroups.hpp abf31df
> src/linux/cgroups.cpp 62df4b7
> src/linux/fs.hpp ac8b5f4
> src/linux/fs.cpp b01d14c
> src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
> src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
> src/slave/containerizer/mesos/containerizer.cpp 9d08329
> src/tests/isolator_tests.cpp 52b38a3
>
> Diff: https://reviews.apache.org/r/27111/diff/
>
>
> Testing
> -------
>
> make check
> sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/#review58524
-----------------------------------------------------------
Patch looks great!
Reviews applied: [27111]
All tests passed.
- Mesos ReviewBot
On Oct. 25, 2014, 5:53 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27111/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2014, 5:53 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Ian Downes.
>
>
> Bugs: MESOS-1977
> https://issues.apache.org/jira/browse/MESOS-1977
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Implement a block io isolator that just publishes read / write, bytes / operations per second.
> A split of r25922 as suggested by Ian.
>
>
> Diffs
> -----
>
> include/mesos/mesos.proto 6b93e90
> src/Makefile.am 2617f77
> src/linux/cgroups.hpp abf31df
> src/linux/cgroups.cpp 62df4b7
> src/linux/fs.hpp ac8b5f4
> src/linux/fs.cpp b01d14c
> src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
> src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
> src/slave/containerizer/mesos/containerizer.cpp 9d08329
> src/tests/isolator_tests.cpp 52b38a3
>
> Diff: https://reviews.apache.org/r/27111/diff/
>
>
> Testing
> -------
>
> make check
> sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/
-----------------------------------------------------------
(Updated Oct. 25, 2014, 5:53 p.m.)
Review request for mesos, Benjamin Hindman and Ian Downes.
Changes
-------
Add async-signal-safe comments.
Fix cosmetics.
Bugs: MESOS-1977
https://issues.apache.org/jira/browse/MESOS-1977
Repository: mesos-git
Description
-------
Implement a block io isolator that just publishes read / write, bytes / operations per second.
A split of r25922 as suggested by Ian.
Diffs (updated)
-----
include/mesos/mesos.proto 6b93e90
src/Makefile.am 2617f77
src/linux/cgroups.hpp abf31df
src/linux/cgroups.cpp 62df4b7
src/linux/fs.hpp ac8b5f4
src/linux/fs.cpp b01d14c
src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
src/slave/containerizer/mesos/containerizer.cpp 9d08329
src/tests/isolator_tests.cpp 52b38a3
Diff: https://reviews.apache.org/r/27111/diff/
Testing
-------
make check
sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
Thanks,
Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/
-----------------------------------------------------------
(Updated Oct. 25, 2014, 5:48 p.m.)
Review request for mesos, Benjamin Hindman and Ian Downes.
Changes
-------
Re-organize blkio stats.
Make blkio unit test setup async-signal-safe.
Bugs: MESOS-1977
https://issues.apache.org/jira/browse/MESOS-1977
Repository: mesos-git
Description
-------
Implement a block io isolator that just publishes read / write, bytes / operations per second.
A split of r25922 as suggested by Ian.
Diffs (updated)
-----
include/mesos/mesos.proto 6b93e90
src/Makefile.am 2617f77
src/linux/cgroups.hpp abf31df
src/linux/cgroups.cpp 62df4b7
src/linux/fs.hpp ac8b5f4
src/linux/fs.cpp b01d14c
src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
src/slave/containerizer/mesos/containerizer.cpp 9d08329
src/tests/isolator_tests.cpp 52b38a3
Diff: https://reviews.apache.org/r/27111/diff/
Testing
-------
make check
sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
Thanks,
Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Joris Van Remoortere <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/
-----------------------------------------------------------
(Updated Oct. 23, 2014, 9:58 p.m.)
Review request for mesos, Benjamin Hindman and Ian Downes.
Changes
-------
fix whitespace issues.
Bugs: MESOS-1977
https://issues.apache.org/jira/browse/MESOS-1977
Repository: mesos-git
Description
-------
Implement a block io isolator that just publishes read / write, bytes / operations per second.
A split of r25922 as suggested by Ian.
Diffs (updated)
-----
include/mesos/mesos.proto 6b93e90
src/Makefile.am 2617f77
src/linux/cgroups.hpp abf31df
src/linux/cgroups.cpp 62df4b7
src/linux/fs.hpp ac8b5f4
src/linux/fs.cpp b01d14c
src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
src/slave/containerizer/mesos/containerizer.cpp 9d08329
src/tests/isolator_tests.cpp 52b38a3
Diff: https://reviews.apache.org/r/27111/diff/
Testing
-------
make check
sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
Thanks,
Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Ben Mahler <be...@gmail.com>.
> On Oct. 23, 2014, 9:20 p.m., Mesos ReviewBot wrote:
> > Bad patch!
> >
> > Reviews applied: [27111]
> >
> > Failed command: ./support/apply-review.sh -n -r 27111
> >
> > Error:
> > --2014-10-23 21:20:04-- https://reviews.apache.org/r/27111/diff/raw/
> > Resolving reviews.apache.org (reviews.apache.org)... 140.211.11.74
> > Connecting to reviews.apache.org (reviews.apache.org)|140.211.11.74|:443... connected.
> > HTTP request sent, awaiting response... 200 OK
> > Length: 26290 (26K) [text/x-patch]
> > Saving to: '27111.patch'
> >
> > 0K .......... .......... ..... 100% 780K=0.03s
> >
> > 2014-10-23 21:20:04 (780 KB/s) - '27111.patch' saved [26290/26290]
> >
> > Successfully applied: Block IO Isolator: usage metrics
> >
> > Implement a block io isolator that just publishes read / write, bytes / operations per second.
> > A split of r25922 as suggested by Ian.
> >
> >
> > Review: https://reviews.apache.org/r/27111
> > Checking 8 files using filter --filter=-,+build/class,+build/deprecated,+build/endif_comment,+readability/todo,+readability/namespace,+runtime/vlog,+whitespace/blank_line,+whitespace/comma,+whitespace/end_of_line,+whitespace/ending_newline,+whitespace/forcolon,+whitespace/indent,+whitespace/line_length,+whitespace/operators,+whitespace/semicolon,+whitespace/tab,+whitespace/todo
> > src/tests/isolator_tests.cpp:910: Could not find a newline character at the end of the file. [whitespace/ending_newline] [5]
> > src/slave/containerizer/isolators/cgroups/blkio.cpp:217: Redundant blank line at the start of a code block should be deleted. [whitespace/blank_line] [2]
> > Total errors found: 2
> > Failed to commit patch
Are you using the commit hook in support/ ?
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/#review58109
-----------------------------------------------------------
On Oct. 23, 2014, 9:58 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27111/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 9:58 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Ian Downes.
>
>
> Bugs: MESOS-1977
> https://issues.apache.org/jira/browse/MESOS-1977
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Implement a block io isolator that just publishes read / write, bytes / operations per second.
> A split of r25922 as suggested by Ian.
>
>
> Diffs
> -----
>
> include/mesos/mesos.proto 6b93e90
> src/Makefile.am 2617f77
> src/linux/cgroups.hpp abf31df
> src/linux/cgroups.cpp 62df4b7
> src/linux/fs.hpp ac8b5f4
> src/linux/fs.cpp b01d14c
> src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
> src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
> src/slave/containerizer/mesos/containerizer.cpp 9d08329
> src/tests/isolator_tests.cpp 52b38a3
>
> Diff: https://reviews.apache.org/r/27111/diff/
>
>
> Testing
> -------
>
> make check
> sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
>
>
> Thanks,
>
> Joris Van Remoortere
>
>
Re: Review Request 27111: Block IO Isolator: usage metrics
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27111/#review58109
-----------------------------------------------------------
Bad patch!
Reviews applied: [27111]
Failed command: ./support/apply-review.sh -n -r 27111
Error:
--2014-10-23 21:20:04-- https://reviews.apache.org/r/27111/diff/raw/
Resolving reviews.apache.org (reviews.apache.org)... 140.211.11.74
Connecting to reviews.apache.org (reviews.apache.org)|140.211.11.74|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26290 (26K) [text/x-patch]
Saving to: '27111.patch'
0K .......... .......... ..... 100% 780K=0.03s
2014-10-23 21:20:04 (780 KB/s) - '27111.patch' saved [26290/26290]
Successfully applied: Block IO Isolator: usage metrics
Implement a block io isolator that just publishes read / write, bytes / operations per second.
A split of r25922 as suggested by Ian.
Review: https://reviews.apache.org/r/27111
Checking 8 files using filter --filter=-,+build/class,+build/deprecated,+build/endif_comment,+readability/todo,+readability/namespace,+runtime/vlog,+whitespace/blank_line,+whitespace/comma,+whitespace/end_of_line,+whitespace/ending_newline,+whitespace/forcolon,+whitespace/indent,+whitespace/line_length,+whitespace/operators,+whitespace/semicolon,+whitespace/tab,+whitespace/todo
src/tests/isolator_tests.cpp:910: Could not find a newline character at the end of the file. [whitespace/ending_newline] [5]
src/slave/containerizer/isolators/cgroups/blkio.cpp:217: Redundant blank line at the start of a code block should be deleted. [whitespace/blank_line] [2]
Total errors found: 2
Failed to commit patch
- Mesos ReviewBot
On Oct. 23, 2014, 9:01 p.m., Joris Van Remoortere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27111/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 9:01 p.m.)
>
>
> Review request for mesos, Benjamin Hindman and Ian Downes.
>
>
> Bugs: MESOS-1977
> https://issues.apache.org/jira/browse/MESOS-1977
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Implement a block io isolator that just publishes read / write, bytes / operations per second.
> A split of r25922 as suggested by Ian.
>
>
> Diffs
> -----
>
> include/mesos/mesos.proto 6b93e90
> src/Makefile.am 2617f77
> src/linux/cgroups.hpp abf31df
> src/linux/cgroups.cpp 62df4b7
> src/linux/fs.hpp ac8b5f4
> src/linux/fs.cpp b01d14c
> src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION
> src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION
> src/slave/containerizer/mesos/containerizer.cpp 9d08329
> src/tests/isolator_tests.cpp 52b38a3
>
> Diff: https://reviews.apache.org/r/27111/diff/
>
>
> Testing
> -------
>
> make check
> sudo ./mesos-tests --gtest_filter="BlkIOIsolatorTest*"
>
>
> Thanks,
>
> Joris Van Remoortere
>
>