You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by jerrypeng <gi...@git.apache.org> on 2016/01/28 00:21:41 UTC
[GitHub] storm pull request: Documentation for cgroup support in Storm
GitHub user jerrypeng opened a pull request:
https://github.com/apache/storm/pull/1054
Documentation for cgroup support in Storm
Related to PR:
https://github.com/apache/storm/pull/1053
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerrypeng/storm asf-site
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1054.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1054
----
commit 45fbc7a9fcd4fe0370f847b32ad12d3e82670868
Author: Boyang Jerry Peng <je...@yahoo-inc.com>
Date: 2016-01-27T23:19:41Z
adding cgroups documentation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on the pull request:
https://github.com/apache/storm/pull/1054#issuecomment-175923449
Took a quick look and looks good. I found some typos which could be easily addressed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r54113951
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,65 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guarantee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms (kernel version 2.6.24 and above)**
+
+## Setup
+
+To use CGroups make sure to install cgroups and configure cgroups correctly. For more information about setting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
--- End diff --
I don't think its necessary since the link I provided below the sample cgconfig.conf contains the official detailed documentation on how to setup cgconfig.conf file. The official documentation is very detailed and provide examples.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by redsanket <gi...@git.apache.org>.
Github user redsanket commented on the pull request:
https://github.com/apache/storm/pull/1054#issuecomment-188304984
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51071834
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
--- End diff --
guaratee -> guarantee
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by redsanket <gi...@git.apache.org>.
Github user redsanket commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51293812
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guarantee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms (kernel version 2.6.24 and above)**
+
+## Setup
+
+To use CGroups make sure to install cgroups and configure cgroups correctly. For more information about setting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
--- End diff --
Looks good for the most part. Can we have a line explaining configs in the example just for the people who are not familiar with cgroups.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/1054
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on the pull request:
https://github.com/apache/storm/pull/1054#issuecomment-176292755
@HeartSaVioR @unsleepy22 @revans2 thanks for the review! I think I have address all of your comments
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51139132
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
--- End diff --
will fix
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by unsleepy22 <gi...@git.apache.org>.
Github user unsleepy22 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51076573
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
--- End diff --
will skipped -> will be skipped, very minor
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51139196
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
--- End diff --
will fix both
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51071856
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
--- End diff --
to to -> to
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51154587
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
--- End diff --
will add
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by unsleepy22 <gi...@git.apache.org>.
Github user unsleepy22 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51076449
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
--- End diff --
typo, seting -> setting
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51298506
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guarantee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms (kernel version 2.6.24 and above)**
+
+## Setup
+
+To use CGroups make sure to install cgroups and configure cgroups correctly. For more information about setting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
--- End diff --
I added a url that explains in detail what is in a cgconfig.conf file
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by wuchong <gi...@git.apache.org>.
Github user wuchong commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r54195364
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,65 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guarantee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms (kernel version 2.6.24 and above)**
+
+## Setup
+
+To use CGroups make sure to install cgroups and configure cgroups correctly. For more information about setting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
--- End diff --
Thanks for your explain , I get it.
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51154424
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
+| cgroup.storm.hierarchy.dir | The path to the cgroup hierarchy that storm will use. Default set to "/cgroup/storm_resources" |
+| cgroup.storm.resources | A list of subsystems that will be regulated by CGroups. Default set to cpu and memory |
+| cgroup.supervisor.rootdir | The root cgroup used by the supervisor. The path to the cgroup will be <cgroup.storm.hierarchy.dir>/<cgroup.supervisor.rootdir>. Default set to "storm" |
+| cgroup.cgexec.cmd | Absolute path to the cgexec command used to launch workers within a cgroup. Default set to "/bin/cgexec" |
+| cgroup.worker.memory.mb.limit | The memory limit in MB for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config memory.limit_in_bytes. For more details about memory.limit_in_bytes, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler |
+| cgroup.worker.cpu.limit | The cpu share for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config cpu.share. For more details about cpu.share, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler. |
+
+Since limiting CPU usuage via cpu.shares only limits the proportional CPU usage of a process, to limit the amout of CPU usage of all the worker processes on a supervisor node, please set the config supervisor.cpu.capacity. Where each increment represents 1% of a core thus if a user sets supervisor.cpu.capacity: 200, the user is indicating the use of 2 cores.
+
+## Integration with Resource Aware Scheduler
+
+CGroups can be used in conjunction with the Resource Aware Scheduler. CGroups will then enforce the resource usage of workers as allocated by the Resource Aware Scheduler. To use cgroups with the Resource Aware Scheduler, simply enable cgroups and be sure to NOT set cgroup.worker.memory.mb.limit and cgroup.worker.cpu.limit configs.
--- End diff --
will change
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51135789
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
--- End diff --
Can we also include a version number for when cgroup support was added?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51139298
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
+| cgroup.storm.hierarchy.dir | The path to the cgroup hierarchy that storm will use. Default set to "/cgroup/storm_resources" |
+| cgroup.storm.resources | A list of subsystems that will be regulated by CGroups. Default set to cpu and memory |
+| cgroup.supervisor.rootdir | The root cgroup used by the supervisor. The path to the cgroup will be <cgroup.storm.hierarchy.dir>/<cgroup.supervisor.rootdir>. Default set to "storm" |
+| cgroup.cgexec.cmd | Absolute path to the cgexec command used to launch workers within a cgroup. Default set to "/bin/cgexec" |
+| cgroup.worker.memory.mb.limit | The memory limit in MB for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config memory.limit_in_bytes. For more details about memory.limit_in_bytes, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler |
+| cgroup.worker.cpu.limit | The cpu share for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config cpu.share. For more details about cpu.share, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler. |
+
+Since limiting CPU usuage via cpu.shares only limits the proportional CPU usage of a process, to limit the amout of CPU usage of all the worker processes on a supervisor node, please set the config supervisor.cpu.capacity. Where each increment represents 1% of a core thus if a user sets supervisor.cpu.capacity: 200, the user is indicating the use of 2 cores.
--- End diff --
will fix
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51072040
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
+| cgroup.storm.hierarchy.dir | The path to the cgroup hierarchy that storm will use. Default set to "/cgroup/storm_resources" |
+| cgroup.storm.resources | A list of subsystems that will be regulated by CGroups. Default set to cpu and memory |
+| cgroup.supervisor.rootdir | The root cgroup used by the supervisor. The path to the cgroup will be <cgroup.storm.hierarchy.dir>/<cgroup.supervisor.rootdir>. Default set to "storm" |
+| cgroup.cgexec.cmd | Absolute path to the cgexec command used to launch workers within a cgroup. Default set to "/bin/cgexec" |
+| cgroup.worker.memory.mb.limit | The memory limit in MB for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config memory.limit_in_bytes. For more details about memory.limit_in_bytes, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler |
+| cgroup.worker.cpu.limit | The cpu share for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config cpu.share. For more details about cpu.share, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler. |
+
+Since limiting CPU usuage via cpu.shares only limits the proportional CPU usage of a process, to limit the amout of CPU usage of all the worker processes on a supervisor node, please set the config supervisor.cpu.capacity. Where each increment represents 1% of a core thus if a user sets supervisor.cpu.capacity: 200, the user is indicating the use of 2 cores.
--- End diff --
usuage -> usage
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51154779
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
+| cgroup.storm.hierarchy.dir | The path to the cgroup hierarchy that storm will use. Default set to "/cgroup/storm_resources" |
+| cgroup.storm.resources | A list of subsystems that will be regulated by CGroups. Default set to cpu and memory |
--- End diff --
will add
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by wuchong <gi...@git.apache.org>.
Github user wuchong commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r54042400
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,65 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guarantee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms (kernel version 2.6.24 and above)**
+
+## Setup
+
+To use CGroups make sure to install cgroups and configure cgroups correctly. For more information about setting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
--- End diff --
Can we tell users what dose `uid` and `gid` mean? and how to set them? Because it depends on the user who runs storm.
We can check `/etc/passwd` to find user's uid and gid. Such as in my `/etc/passwd` the `admin`'s uid and gid both are `505`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by redsanket <gi...@git.apache.org>.
Github user redsanket commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51309621
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guarantee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms (kernel version 2.6.24 and above)**
+
+## Setup
+
+To use CGroups make sure to install cgroups and configure cgroups correctly. For more information about setting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
--- End diff --
thats fine, i overlooked it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by unsleepy22 <gi...@git.apache.org>.
Github user unsleepy22 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51076681
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
+| cgroup.storm.hierarchy.dir | The path to the cgroup hierarchy that storm will use. Default set to "/cgroup/storm_resources" |
+| cgroup.storm.resources | A list of subsystems that will be regulated by CGroups. Default set to cpu and memory |
+| cgroup.supervisor.rootdir | The root cgroup used by the supervisor. The path to the cgroup will be <cgroup.storm.hierarchy.dir>/<cgroup.supervisor.rootdir>. Default set to "storm" |
+| cgroup.cgexec.cmd | Absolute path to the cgexec command used to launch workers within a cgroup. Default set to "/bin/cgexec" |
+| cgroup.worker.memory.mb.limit | The memory limit in MB for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config memory.limit_in_bytes. For more details about memory.limit_in_bytes, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler |
+| cgroup.worker.cpu.limit | The cpu share for each worker. This can be set on a per supervisor node basis. This config is used to set the cgroup config cpu.share. For more details about cpu.share, please visit: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html. Please note, if you are using the Resource Aware Scheduler, please do NOT set this config as this config will override the values calculated by the Resource Aware Scheduler. |
+
+Since limiting CPU usuage via cpu.shares only limits the proportional CPU usage of a process, to limit the amout of CPU usage of all the worker processes on a supervisor node, please set the config supervisor.cpu.capacity. Where each increment represents 1% of a core thus if a user sets supervisor.cpu.capacity: 200, the user is indicating the use of 2 cores.
+
+## Integration with Resource Aware Scheduler
+
+CGroups can be used in conjunction with the Resource Aware Scheduler. CGroups will then enforce the resource usage of workers as allocated by the Resource Aware Scheduler. To use cgroups with the Resource Aware Scheduler, simply enable cgroups and be sure to NOT set cgroup.worker.memory.mb.limit and cgroup.worker.cpu.limit configs.
--- End diff --
to NOT -> NOT to? very minor.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on the pull request:
https://github.com/apache/storm/pull/1054#issuecomment-185813428
@abhishekagarwal87 thanks for the review. I have fixed the type
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on the pull request:
https://github.com/apache/storm/pull/1054#issuecomment-180410973
@redsanket thanks for the review. I have addressed all your concerns
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51136522
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
+| cgroup.storm.hierarchy.dir | The path to the cgroup hierarchy that storm will use. Default set to "/cgroup/storm_resources" |
+| cgroup.storm.resources | A list of subsystems that will be regulated by CGroups. Default set to cpu and memory |
--- End diff --
What other subsystems are valid? If there are no other ones that are valid, simply indicate that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by abhishekagarwal87 <gi...@git.apache.org>.
Github user abhishekagarwal87 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r52448581
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,65 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guarantee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms (kernel version 2.6.24 and above)**
+
+## Setup
+
+To use CGroups make sure to install cgroups and configure cgroups correctly. For more information about setting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+For a more detailed explanation of the format and configs for the cgconfig.conf file, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html#The_cgconfig.conf_File
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| storm.cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will be skipped. Default set to "false" |
+| storm.cgroup.hierarchy.dir | The path to the cgroup hierarchy that storm will use. Default set to "/cgroup/storm_resources" |
+| storm.cgroup.resources | A list of subsystems that will be regulated by CGroups. Default set to cpu and memory. Currently only cpu and memory are supported |
+| storm.supervisor.cgroup.rootdir | The root cgroup used by the supervisor. The path to the cgroup will be <cgroup.storm.hierarchy.dir>/<cgroup.supervisor.rootdir>. Default set to "storm" |
--- End diff --
<cgroup.storm.hierarchy.dir> should be <storm.cgroup.hierarchy.dir>. I am assuming that supervisor root dir is actullay same as storm.cgroup.hierarchy.dir and not configurable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: Documentation for cgroup support in Storm
Posted by jerrypeng <gi...@git.apache.org>.
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/1054#discussion_r51139715
--- Diff: documentation/cgroups_in_storm.md ---
@@ -0,0 +1,61 @@
+# CGroups in Storm
+
+CGroups are used by Storm to limit the resource usage of workers to guaratee fairness and QOS.
+
+**Please note: CGroups is currently supported only on Linux platforms**
+
+## Setup
+
+To use CGroups make sure to to install cgroups and configure cgroups correctly. For more information about seting up and configuring, please visit:
+
+https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Using_Control_Groups.html
+
+A sample/default cgconfig.conf file is supplied in the <stormroot>/conf directory. The contents are as follows:
+
+```
+mount {
+ cpuset = /cgroup/cpuset;
+ cpu = /cgroup/storm_resources;
+ cpuacct = /cgroup/cpuacct;
+ memory = /cgroup/storm_resources;
+ devices = /cgroup/devices;
+ freezer = /cgroup/freezer;
+ net_cls = /cgroup/net_cls;
+ blkio = /cgroup/blkio;
+}
+
+group storm {
+ perm {
+ task {
+ uid = 500;
+ gid = 500;
+ }
+ admin {
+ uid = 500;
+ gid = 500;
+ }
+ }
+ cpu {
+ }
+}
+```
+
+# Settings Related To CGroups in Storm
+
+| Setting | Function |
+|-------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| cgroup.enable | This config is used to set whether or not cgroups will be used. Set "true" to enable use of cgroups. Set "false" to not use cgroups. When this config is set to false, unit tests related to cgroups will skipped. Default set to "false" |
--- End diff --
will fix
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---