You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by "Sam Whitlock (Created) (JIRA)" <ji...@apache.org> on 2011/11/17 00:05:51 UTC
[jira] [Created] (MESOS-89) create utilities to collect information
from the proc filesystem
create utilities to collect information from the proc filesystem
----------------------------------------------------------------
Key: MESOS-89
URL: https://issues.apache.org/jira/browse/MESOS-89
Project: Mesos
Issue Type: New Feature
Components: slave
Environment: This is specific to Linux, as OS X does not have a proc filesystem.
Reporter: Sam Whitlock
Assignee: Alex Degtiar
Priority: Minor
This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173818#comment-13173818 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2011-12-08 18:48:30, Benjamin Hindman wrote:
bq. >
Also, as discussed, I created a monitoring/linux folder to put all linux-specific source files.
bq. On 2011-12-08 18:48:30, Benjamin Hindman wrote:
bq. > src/monitoring/proc_utils.hpp, line 33
bq. > <https://reviews.apache.org/r/3050/diff/2/?file=63110#file63110line33>
bq. >
bq. > Brace on newline.
Done.
bq. On 2011-12-08 18:48:30, Benjamin Hindman wrote:
bq. > src/monitoring/proc_utils.cpp, line 43
bq. > <https://reviews.apache.org/r/3050/diff/2/?file=63111#file63111line43>
bq. >
bq. > Brace on newline.
Done.
bq. On 2011-12-08 18:48:30, Benjamin Hindman wrote:
bq. > src/monitoring/proc_utils.cpp, line 60
bq. > <https://reviews.apache.org/r/3050/diff/2/?file=63111#file63111line60>
bq. >
bq. > Double newline between functions.
Done.
bq. On 2011-12-08 18:48:30, Benjamin Hindman wrote:
bq. > src/monitoring/proc_utils.hpp, lines 38-39
bq. > <https://reviews.apache.org/r/3050/diff/2/?file=63110#file63110line38>
bq. >
bq. > Use seconds or milliseconds _abstraction_ from common/seconds.hpp.
Done.
bq. On 2011-12-08 18:48:30, Benjamin Hindman wrote:
bq. > src/monitoring/proc_utils.hpp, line 53
bq. > <https://reviews.apache.org/r/3050/diff/2/?file=63110#file63110line53>
bq. >
bq. > Seconds or milliseconds type, here and everywhere else.
Done. Went with seconds for consistency with rest of Mesos.
bq. On 2011-12-08 18:48:30, Benjamin Hindman wrote:
bq. > src/monitoring/proc_utils.cpp, line 76
bq. > <https://reviews.apache.org/r/3050/diff/2/?file=63111#file63111line76>
bq. >
bq. > Add more spaces between logical blocks.
Done.
- Alex
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review3745
-----------------------------------------------------------
On 2011-12-08 00:00:32, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-08 00:00:32)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/monitoring/proc_utils.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164856#comment-13164856 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/Makefile.in, line 137
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62806#file62806line137>
bq. >
bq. > MONITORING_OBJ should be set to empty string unconditionally first.
Done.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.hpp, line 39
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line39>
bq. >
bq. > Please state whether RSS or VSIZE or something else.
Done.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.cpp, line 22
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62808#file62808line22>
bq. >
bq. > include all system .h headers either before or after standard C++ headers.
Done.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.cpp, line 61
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62808#file62808line61>
bq. >
bq. > Please make these file-scoped (static or anonymous namespace) or put them in the header file.
Done.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.cpp, line 81
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62808#file62808line81>
bq. >
bq. > O is a pretty bad variable name.
Done. I was using outdated documentation where this field was a hard-coded 0 in proc. It is now num_threads.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.cpp, line 85
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62808#file62808line85>
bq. >
bq. > Check whether this read succeeds.
Done.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.hpp, line 37
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line37>
bq. >
bq. > Why milliseconds? libprocess uses seconds since some time, and I think USER_HZ usually (often?) isn't 1000.
Sam and I decided on milliseconds (since epoch when appropriate) for all measured times because it was the granularity closest to the times in various sources, and having one unit used consistently seemed cleaner. Would it make it more consistent with the rest of Mesos if we scale it up to seconds?
For reference, granularity of various times we used:
process start time (used for duration of initial read)- jiffies (4ms +/- depending on system HZ)
boot time (used for start time to make it since epoch)- seconds
current time (used for duration) - nanoseconds/milliseconds since epoch (depends on system)
cpu time (proc)- clock ticks (10ms +/- depending on system SC_CLK_TCK)
cpu time (lxc)- nanoseconds (unit returned in, not sure of actual granularity)
Period of measurement - potential lower bound on a single machine is probably in the millisecond range.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.hpp, line 33
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line33>
bq. >
bq. > Why strings? (and elsewhere)
bq. >
I was deciding between using unsigned integers, something like the pid_t type, and strings. I ultimately decided on strings because they supported special non-integer 'pids' proc lets you query (i.e. "self"). This was convenient for testing.
- Alex
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review3712
-----------------------------------------------------------
On 2011-12-08 00:00:32, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-08 00:00:32)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/monitoring/proc_utils.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165425#comment-13165425 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review3745
-----------------------------------------------------------
src/monitoring/proc_utils.hpp
<https://reviews.apache.org/r/3050/#comment8438>
Brace on newline.
src/monitoring/proc_utils.hpp
<https://reviews.apache.org/r/3050/#comment8439>
Use seconds or milliseconds _abstraction_ from common/seconds.hpp.
src/monitoring/proc_utils.hpp
<https://reviews.apache.org/r/3050/#comment8440>
Seconds or milliseconds type, here and everywhere else.
src/monitoring/proc_utils.cpp
<https://reviews.apache.org/r/3050/#comment8443>
Brace on newline.
src/monitoring/proc_utils.cpp
<https://reviews.apache.org/r/3050/#comment8444>
Double newline between functions.
src/monitoring/proc_utils.cpp
<https://reviews.apache.org/r/3050/#comment8445>
Add more spaces between logical blocks.
- Benjamin
On 2011-12-08 00:00:32, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-08 00:00:32)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/monitoring/proc_utils.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173819#comment-13173819 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On None, Alex Degtiar wrote:
bq. > src/monitoring/proc_utils.hpp, line 59
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line59>
bq. >
bq. > I'm not convinced this should be in proc_utils, since the time is not actually read from proc. It is also used in non-Linux parts of the code, so it would conflict with proc_utils only being built in Linux.
bq. >
bq. > Any suggestions for where it should go, or an alternative to use?
Resolved by using Clock::now().
- Alex
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review3698
-----------------------------------------------------------
On 2011-12-08 00:00:32, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-08 00:00:32)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/monitoring/proc_utils.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202589#comment-13202589 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review4868
-----------------------------------------------------------
src/Makefile.in
<https://reviews.apache.org/r/3050/#comment10719>
This needs to be merged into src/Makefile.am now.
src/tests/Makefile.in
<https://reviews.apache.org/r/3050/#comment10721>
This needs to be merged into src/Makefile.am
- Charles
On 2011-12-24 04:13:40, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-24 04:13:40)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/monitoring/linux/proc_utils.cpp PRE-CREATION
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/linux/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204057#comment-13204057 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2012-02-07 18:14:30, Charles Reiss wrote:
bq. >
bq.
bq. Sam Whitlock wrote:
bq. What target should these go into in Makefile.am? libmesos_no_third_party_la_SOURCES seems to be the one that all the source files are being added to.
bq.
bq. Charles Reiss wrote:
bq. libmesos_no_third_party_la would be fine for the non-test targets. (The test ones probably go into the mesos_tests_SOURCES.)
bq.
Because some of the monitoring stuff is linux-specific, should it be conditionally added to libmesos_no_third_party_la_SOURCES in the same way lxc_isolation_module.cpp is?
It seems like all the hpp files are unconditionally added to libmesos_no_third_party and linux-specific cpp files should only be added if OS_LINUX is true. Is this correct?
- Sam
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review4868
-----------------------------------------------------------
On 2011-12-24 04:13:40, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-24 04:13:40)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/monitoring/linux/proc_utils.cpp PRE-CREATION
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/linux/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203782#comment-13203782 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2012-02-07 18:14:30, Charles Reiss wrote:
bq. >
bq.
bq. Sam Whitlock wrote:
bq. What target should these go into in Makefile.am? libmesos_no_third_party_la_SOURCES seems to be the one that all the source files are being added to.
libmesos_no_third_party_la would be fine for the non-test targets. (The test ones probably go into the mesos_tests_SOURCES.)
- Charles
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review4868
-----------------------------------------------------------
On 2011-12-24 04:13:40, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-24 04:13:40)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/monitoring/linux/proc_utils.cpp PRE-CREATION
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/linux/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164868#comment-13164868 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.hpp, line 33
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line33>
bq. >
bq. > Why strings? (and elsewhere)
bq. >
bq.
bq. Alex Degtiar wrote:
bq. I was deciding between using unsigned integers, something like the pid_t type, and strings. I ultimately decided on strings because they supported special non-integer 'pids' proc lets you query (i.e. "self"). This was convenient for testing.
I don't think there's any special "PID" other than "self", and that's easy replaced with getpid() (which your test already needs).
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.hpp, line 37
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line37>
bq. >
bq. > Why milliseconds? libprocess uses seconds since some time, and I think USER_HZ usually (often?) isn't 1000.
bq.
bq. Alex Degtiar wrote:
bq. Sam and I decided on milliseconds (since epoch when appropriate) for all measured times because it was the granularity closest to the times in various sources, and having one unit used consistently seemed cleaner. Would it make it more consistent with the rest of Mesos if we scale it up to seconds?
bq.
bq. For reference, granularity of various times we used:
bq. process start time (used for duration of initial read)- jiffies (4ms +/- depending on system HZ)
bq. boot time (used for start time to make it since epoch)- seconds
bq. current time (used for duration) - nanoseconds/milliseconds since epoch (depends on system)
bq. cpu time (proc)- clock ticks (10ms +/- depending on system SC_CLK_TCK)
bq. cpu time (lxc)- nanoseconds (unit returned in, not sure of actual granularity)
bq. Period of measurement - potential lower bound on a single machine is probably in the millisecond range.
Assuming process::Clock::now() can be counted on to be in time since the epoch (I don't actually know if this is an API gaurentee), then I think you should have that replace getCurrentTime() and thus use seconds since the epoch for everything.
- Charles
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review3712
-----------------------------------------------------------
On 2011-12-08 00:00:32, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-08 00:00:32)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/monitoring/proc_utils.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164197#comment-13164197 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/
-----------------------------------------------------------
Review request for mesos.
Summary
-------
The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
This addresses bug MESOS-89.
https://issues.apache.org/jira/browse/MESOS-89
Diffs
-----
src/tests/Makefile.in ea943f7
src/tests/proc_utils_tests.cpp PRE-CREATION
src/Makefile.in 516f128
src/monitoring/proc_utils.hpp PRE-CREATION
src/monitoring/proc_utils.cpp PRE-CREATION
Diff: https://reviews.apache.org/r/3050/diff
Testing
-------
Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
Thanks,
Alex
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203759#comment-13203759 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2012-02-07 18:14:30, Charles Reiss wrote:
bq. >
What target should these go into in Makefile.am? libmesos_no_third_party_la_SOURCES seems to be the one that all the source files are being added to.
- Sam
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review4868
-----------------------------------------------------------
On 2011-12-24 04:13:40, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-24 04:13:40)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/monitoring/linux/proc_utils.cpp PRE-CREATION
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/linux/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173817#comment-13173817 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/
-----------------------------------------------------------
(Updated 2011-12-21 03:24:51.902604)
Review request for mesos.
Changes
-------
Thanks for the feedback. I've made the requested changes. Please take another look.
Summary
-------
The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
This addresses bug MESOS-89.
https://issues.apache.org/jira/browse/MESOS-89
Diffs (updated)
-----
src/Makefile.in 516f128
src/monitoring/linux/proc_utils.hpp PRE-CREATION
src/monitoring/linux/proc_utils.cpp PRE-CREATION
src/tests/Makefile.in ea943f7
src/tests/proc_utils_tests.cpp PRE-CREATION
Diff: https://reviews.apache.org/r/3050/diff
Testing
-------
Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
Thanks,
Alex
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MESOS-89) create utilities to collect information
from the proc filesystem
Posted by "Alex Degtiar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Degtiar updated MESOS-89:
------------------------------
Issue Type: Sub-task (was: New Feature)
Parent: MESOS-38
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204061#comment-13204061 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2012-02-07 18:14:30, Charles Reiss wrote:
bq. >
bq.
bq. Sam Whitlock wrote:
bq. What target should these go into in Makefile.am? libmesos_no_third_party_la_SOURCES seems to be the one that all the source files are being added to.
bq.
bq. Charles Reiss wrote:
bq. libmesos_no_third_party_la would be fine for the non-test targets. (The test ones probably go into the mesos_tests_SOURCES.)
bq.
bq.
bq. Sam Whitlock wrote:
bq. Because some of the monitoring stuff is linux-specific, should it be conditionally added to libmesos_no_third_party_la_SOURCES in the same way lxc_isolation_module.cpp is?
bq.
bq. It seems like all the hpp files are unconditionally added to libmesos_no_third_party and linux-specific cpp files should only be added if OS_LINUX is true. Is this correct?
Yes.
Yes, but don't forget to add it to EXTRA_DIST just ast in the lxc_isolation_module.cpp.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review4868
-----------------------------------------------------------
On 2011-12-24 04:13:40, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-24 04:13:40)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/monitoring/linux/proc_utils.cpp PRE-CREATION
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/linux/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164857#comment-13164857 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/
-----------------------------------------------------------
(Updated 2011-12-08 00:00:32.699023)
Review request for mesos.
Changes
-------
I've address the comments and updated the diff. PTAL.
Summary
-------
The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
This addresses bug MESOS-89.
https://issues.apache.org/jira/browse/MESOS-89
Diffs (updated)
-----
src/tests/Makefile.in ea943f7
src/tests/proc_utils_tests.cpp PRE-CREATION
src/monitoring/proc_utils.cpp PRE-CREATION
src/Makefile.in 516f128
src/monitoring/proc_utils.hpp PRE-CREATION
Diff: https://reviews.apache.org/r/3050/diff
Testing
-------
Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
Thanks,
Alex
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13197108#comment-13197108 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review4708
-----------------------------------------------------------
- Charles
On 2011-12-24 04:13:40, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-24 04:13:40)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/monitoring/linux/proc_utils.cpp PRE-CREATION
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/linux/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164700#comment-13164700 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review3712
-----------------------------------------------------------
src/Makefile.in
<https://reviews.apache.org/r/3050/#comment8323>
MONITORING_OBJ should be set to empty string unconditionally first.
src/monitoring/proc_utils.hpp
<https://reviews.apache.org/r/3050/#comment8319>
Why strings? (and elsewhere)
src/monitoring/proc_utils.hpp
<https://reviews.apache.org/r/3050/#comment8309>
Why milliseconds? libprocess uses seconds since some time, and I think USER_HZ usually (often?) isn't 1000.
src/monitoring/proc_utils.hpp
<https://reviews.apache.org/r/3050/#comment8320>
Please state whether RSS or VSIZE or something else.
src/monitoring/proc_utils.cpp
<https://reviews.apache.org/r/3050/#comment8316>
include all system .h headers either before or after standard C++ headers.
src/monitoring/proc_utils.cpp
<https://reviews.apache.org/r/3050/#comment8315>
Please make these file-scoped (static or anonymous namespace) or put them in the header file.
src/monitoring/proc_utils.cpp
<https://reviews.apache.org/r/3050/#comment8311>
O is a pretty bad variable name.
src/monitoring/proc_utils.cpp
<https://reviews.apache.org/r/3050/#comment8313>
Check whether this read succeeds.
- Charles
On 2011-12-07 07:08:06, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-07 07:08:06)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/proc_utils.hpp PRE-CREATION
bq. src/monitoring/proc_utils.cpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "Alex Degtiar (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13151735#comment-13151735 ]
Alex Degtiar commented on MESOS-89:
-----------------------------------
Except for minor tweaks (e.g. make sure error reporting, etc style is consistent with the rest of Mesos), this is complete. Will submit to Review Board soon.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: New Feature
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175652#comment-13175652 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/
-----------------------------------------------------------
(Updated 2011-12-24 04:13:40.408094)
Review request for mesos.
Changes
-------
I noticed a couple of unused imports. Fixed.
Summary
-------
The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
This addresses bug MESOS-89.
https://issues.apache.org/jira/browse/MESOS-89
Diffs (updated)
-----
src/monitoring/linux/proc_utils.cpp PRE-CREATION
src/tests/Makefile.in ea943f7
src/tests/proc_utils_tests.cpp PRE-CREATION
src/Makefile.in 516f128
src/monitoring/linux/proc_utils.hpp PRE-CREATION
Diff: https://reviews.apache.org/r/3050/diff
Testing
-------
Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
Thanks,
Alex
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MESOS-89) create utilities to collect
information from the proc filesystem
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MESOS-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173816#comment-13173816 ]
jiraposter@reviews.apache.org commented on MESOS-89:
----------------------------------------------------
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.hpp, line 37
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line37>
bq. >
bq. > Why milliseconds? libprocess uses seconds since some time, and I think USER_HZ usually (often?) isn't 1000.
bq.
bq. Alex Degtiar wrote:
bq. Sam and I decided on milliseconds (since epoch when appropriate) for all measured times because it was the granularity closest to the times in various sources, and having one unit used consistently seemed cleaner. Would it make it more consistent with the rest of Mesos if we scale it up to seconds?
bq.
bq. For reference, granularity of various times we used:
bq. process start time (used for duration of initial read)- jiffies (4ms +/- depending on system HZ)
bq. boot time (used for start time to make it since epoch)- seconds
bq. current time (used for duration) - nanoseconds/milliseconds since epoch (depends on system)
bq. cpu time (proc)- clock ticks (10ms +/- depending on system SC_CLK_TCK)
bq. cpu time (lxc)- nanoseconds (unit returned in, not sure of actual granularity)
bq. Period of measurement - potential lower bound on a single machine is probably in the millisecond range.
bq.
bq. Charles Reiss wrote:
bq. Assuming process::Clock::now() can be counted on to be in time since the epoch (I don't actually know if this is an API gaurentee), then I think you should have that replace getCurrentTime() and thus use seconds since the epoch for everything.
bq.
Done.
bq. On 2011-12-07 21:10:13, Charles Reiss wrote:
bq. > src/monitoring/proc_utils.hpp, line 33
bq. > <https://reviews.apache.org/r/3050/diff/1/?file=62807#file62807line33>
bq. >
bq. > Why strings? (and elsewhere)
bq. >
bq.
bq. Alex Degtiar wrote:
bq. I was deciding between using unsigned integers, something like the pid_t type, and strings. I ultimately decided on strings because they supported special non-integer 'pids' proc lets you query (i.e. "self"). This was convenient for testing.
bq.
bq. Charles Reiss wrote:
bq. I don't think there's any special "PID" other than "self", and that's easy replaced with getpid() (which your test already needs).
Replaced with pid_t and getpid().
- Alex
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3050/#review3712
-----------------------------------------------------------
On 2011-12-08 00:00:32, Alex Degtiar wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3050/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-12-08 00:00:32)
bq.
bq.
bq. Review request for mesos.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The first of several patches related to resource usage monitoring. This patch provides a collection of utilities for use on Linux for reading stats from proc. It is used by both the lxc and proc resource collectors.
bq.
bq.
bq. This addresses bug MESOS-89.
bq. https://issues.apache.org/jira/browse/MESOS-89
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/tests/Makefile.in ea943f7
bq. src/tests/proc_utils_tests.cpp PRE-CREATION
bq. src/monitoring/proc_utils.cpp PRE-CREATION
bq. src/Makefile.in 516f128
bq. src/monitoring/proc_utils.hpp PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/3050/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Sanity tests have been written in src/tests/proc_utils_tests.cpp for all utility functions, and functions have been tested ad hoc.
bq.
bq.
bq. Thanks,
bq.
bq. Alex
bq.
bq.
> create utilities to collect information from the proc filesystem
> ----------------------------------------------------------------
>
> Key: MESOS-89
> URL: https://issues.apache.org/jira/browse/MESOS-89
> Project: Mesos
> Issue Type: Sub-task
> Components: slave
> Environment: This is specific to Linux, as OS X does not have a proc filesystem.
> Reporter: Sam Whitlock
> Assignee: Alex Degtiar
> Priority: Minor
> Labels: monitoring
>
> This is a utility for MESOS-38 https://issues.apache.org/jira/browse/MESOS-38
> Create a collection of utilities to read information from the proc filesystem. This will be used in MESOS-38 to collect most of the information for the process-based isolation usage reporting mechanism. These utilities will also be used for certain functionality in monitoring lxc-based isolation because not all of the information needed is available via the cgroup interface, such as the start time of the container.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira