You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Jie Yu <yu...@gmail.com> on 2014/08/15 00:53:55 UTC
Review Request 24719: Added document for network monitoring.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/
-----------------------------------------------------------
Review request for mesos, Ian Downes and Vinod Kone.
Repository: mesos-git
Description
-------
See summary
Diffs
-----
docs/network-monitoring.md PRE-CREATION
Diff: https://reviews.apache.org/r/24719/diff/
Testing
-------
checked the markdown syntax
Thanks,
Jie Yu
Re: Review Request 24719: Added document for network monitoring.
Posted by Chi Zhang <ch...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/#review50686
-----------------------------------------------------------
Ship it!
all minor stuff.
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88553>
[czhang@tw-172-25-24-214 linux]$ git describe 6a662719c9868b3d6c7d26b3a085f0cd3cc15e64
v3.15-rc1-55-g6a66271
[czhang@tw-172-25-24-214 linux]$ git describe 0d5edc68739f1c1e0519acbea1d3f0c1882a15d7
v3.15-rc1-56-g0d5edc6
[czhang@tw-172-25-24-214 linux]$ git describe e374c618b1465f0292047a9f4c244bd71ab5f1f0
v3.15-rc1-512-ge374c61
[czhang@tw-172-25-24-214 linux]$ git describe 7a9bc9b81a5bc6e44ebc80ef781332e4385083f2 <-- can probably skip.
v3.5-rc4-1092-g7a9bc9b
[czhang@tw-172-25-24-214 linux]$ git describe 25f929fbff0d1bcebf2e92656d33025cd330cbf8
v3.14-rc2-257-g25f929f
for the other 4, maybe rephrase to be more explict on 'a minimum 3.6 kernel + those patches _backported_'?
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88558>
for this whole dependency section, maybe just say something like 'network monitoring depends on libnl3 (url) >= 3.2.25. iproute2 (url) >= 2.6.39 is advised for debugging support ('ip netns' subcommands).
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88549>
iproute2 v2.6.39-8 starts to support netns
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88554>
s/world/network/
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88562>
s/executor/executors/
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88561>
add a shell-style comment at the end of the line to explain the effect? 'this reduces ephemeral ports available to host to only 57345-61000'?
- Chi Zhang
On Aug. 15, 2014, 12:13 a.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24719/
> -----------------------------------------------------------
>
> (Updated Aug. 15, 2014, 12:13 a.m.)
>
>
> Review request for mesos, Chi Zhang, Ian Downes, Vinod Kone, and Cong Wang.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> See summary
>
>
> Diffs
> -----
>
> docs/network-monitoring.md PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24719/diff/
>
>
> Testing
> -------
>
> checked the markdown syntax
>
>
> Thanks,
>
> Jie Yu
>
>
Re: Review Request 24719: Added document for network monitoring.
Posted by Cong Wang <cw...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/#review50740
-----------------------------------------------------------
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88598>
The first kernel commits (routing) listed here are not strictly needed since the bug can be workaround and mesos code already did.
- Cong Wang
On Aug. 15, 2014, 12:13 a.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24719/
> -----------------------------------------------------------
>
> (Updated Aug. 15, 2014, 12:13 a.m.)
>
>
> Review request for mesos, Chi Zhang, Ian Downes, Vinod Kone, and Cong Wang.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> See summary
>
>
> Diffs
> -----
>
> docs/network-monitoring.md PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24719/diff/
>
>
> Testing
> -------
>
> checked the markdown syntax
>
>
> Thanks,
>
> Jie Yu
>
>
Re: Review Request 24719: Added document for network monitoring.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/#review50668
-----------------------------------------------------------
Ship it!
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88519>
s/Our/The current/ ?
can you expand on "will not notice any difference"? you mean no difference w.r.t getting a private bridge, NAT or anything like that right?
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88520>
s/Prerequisite/Prerequisites/
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88521>
s/our code/Mesos/
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88522>
Can you update the "getting started doc" too w.r.t these pre-reqs for network monitoring?
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88523>
is there an authoritative doc that we can link to for more info on "network namespaces"?
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88524>
s/port/ports/ ?
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88526>
s/previously/ pre 0.20.0/
or maybe
s/previously/without network monitoring/
s/non of the/none of the/
s/host process is/host processes are/
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88527>
s/squeeze/squeeze and reboot/
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88528>
Also, note that they need to add "network/portmapping" to --isolation flag.
s/resource/resource (via '--resources' flag)/
s/wanna/want to/ :)
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88529>
s/this is/these are/
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88530>
give a random executor_id, executor_name and source instead of "aurora/thermos" :)
- Vinod Kone
On Aug. 15, 2014, 12:13 a.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24719/
> -----------------------------------------------------------
>
> (Updated Aug. 15, 2014, 12:13 a.m.)
>
>
> Review request for mesos, Chi Zhang, Ian Downes, Vinod Kone, and Cong Wang.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> See summary
>
>
> Diffs
> -----
>
> docs/network-monitoring.md PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24719/diff/
>
>
> Testing
> -------
>
> checked the markdown syntax
>
>
> Thanks,
>
> Jie Yu
>
>
Re: Review Request 24719: Added document for network monitoring.
Posted by Jie Yu <yu...@gmail.com>.
> On Aug. 15, 2014, 6:13 p.m., Ian Downes wrote:
> > docs/network-monitoring.md, line 9
> > <https://reviews.apache.org/r/24719/diff/2/?file=660980#file660980line9>
> >
> > Perhaps make explicit that they can reach other containers, including on the same host, and that they can still reach services running on the localhost of the host <--- This is important because you wouldn't usually expect this if you had container's running with separate network stacks.
I haven't mentioned network stack yet. Here, just given reader an idea that it's completely transparent. I'll way the local host thing later.
> On Aug. 15, 2014, 6:13 p.m., Ian Downes wrote:
> > docs/network-monitoring.md, line 45
> > <https://reviews.apache.org/r/24719/diff/2/?file=660980#file660980line45>
> >
> > Same public IP and same MAC?
> >
> > s/communicate with external world/make connections with other hosts/?
> >
> > Are you going to mention about multiple ICMP/ARP replies anywhere?
I don't wanna go into much details here, ICMP/ARP is hard to explain. We can have a blog later to explain this.
> On Aug. 15, 2014, 6:13 p.m., Ian Downes wrote:
> > docs/network-monitoring.md, line 52
> > <https://reviews.apache.org/r/24719/diff/2/?file=660980#file660980line52>
> >
> > Is it not more common to just use `echo "" > /path` while running as root? People may not be familiar with the tee trick.
> >
> > I think you should explain the reasoning and potential impact this change has. Clarify that this reduces the ephemeral range for any process that isn't is a container and that it limits the number of connections based on the 5-tuple and includes TIME_WAIT so highlight that this should be evaluated for scenarios which have a high number/churn of connections to services.
I'll leave the reasoning part in the blog.
- Jie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/#review50746
-----------------------------------------------------------
On Aug. 15, 2014, 12:13 a.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24719/
> -----------------------------------------------------------
>
> (Updated Aug. 15, 2014, 12:13 a.m.)
>
>
> Review request for mesos, Chi Zhang, Ian Downes, Vinod Kone, and Cong Wang.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> See summary
>
>
> Diffs
> -----
>
> docs/network-monitoring.md PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24719/diff/
>
>
> Testing
> -------
>
> checked the markdown syntax
>
>
> Thanks,
>
> Jie Yu
>
>
Re: Review Request 24719: Added document for network monitoring.
Posted by Ian Downes <ia...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/#review50746
-----------------------------------------------------------
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88614>
Perhaps make explicit that they can reach other containers, including on the same host, and that they can still reach services running on the localhost of the host <--- This is important because you wouldn't usually expect this if you had container's running with separate network stacks.
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88612>
Same public IP and same MAC?
s/communicate with external world/make connections with other hosts/?
Are you going to mention about multiple ICMP/ARP replies anywhere?
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88601>
s/By default, Linux uses ports [] for ephemeral ports./The default ephemeral port range on Linux is []/
Is [32768, 61000] the default range across all distributions?
s/used by the container as ephemeral ports/used as the ephemeral port range for the container's network stack/
Can you add a sentence that these ports are *directly mapped* into the container's port range, hence the naming network/port_mapping.
This ties into each container having its own /proc/sys/net/ipv4/ip_local_port_range.
s/need to make sure/need to ensure/
s/enforce the squeeze/ensure there are no connections using ports outside the new ephemeral range./
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88602>
Is it not more common to just use `echo "" > /path` while running as root? People may not be familiar with the tee trick.
I think you should explain the reasoning and potential impact this change has. Clarify that this reduces the ephemeral range for any process that isn't is a container and that it limits the number of connections based on the 5-tuple and includes TIME_WAIT so highlight that this should be evaluated for scenarios which have a high number/churn of connections to services.
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88605>
Explicitly state that the maximum number of containers on the slave will be limited by approximately |ephemeral_ports|/ephemeral_ports_per_container, subject to alignment etc.
E.g., for these numbers the slave is limited to 24 containers. This is an important limitation that should be made very explicit.
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88606>
The number of ephemeral ports is not power 2 *aligned*, it's just a power of 2.
Give an example here for power of two size to guide people, e.g., 512, 1024 or 2048.
s/what/which/
Explain that non power-2 sized will have some performance impact for handling packets.
docs/network-monitoring.md
<https://reviews.apache.org/r/24719/#comment88608>
Can you clean up the floating point madness here? e.g., cpus_limit: 0.35. It's not useful and is just distracting.
- Ian Downes
On Aug. 14, 2014, 5:13 p.m., Jie Yu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24719/
> -----------------------------------------------------------
>
> (Updated Aug. 14, 2014, 5:13 p.m.)
>
>
> Review request for mesos, Chi Zhang, Ian Downes, Vinod Kone, and Cong Wang.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> See summary
>
>
> Diffs
> -----
>
> docs/network-monitoring.md PRE-CREATION
>
> Diff: https://reviews.apache.org/r/24719/diff/
>
>
> Testing
> -------
>
> checked the markdown syntax
>
>
> Thanks,
>
> Jie Yu
>
>
Re: Review Request 24719: Added document for network monitoring.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/
-----------------------------------------------------------
(Updated Aug. 15, 2014, 12:13 a.m.)
Review request for mesos, Chi Zhang, Ian Downes, Vinod Kone, and Cong Wang.
Changes
-------
+chi and cong
Repository: mesos-git
Description
-------
See summary
Diffs
-----
docs/network-monitoring.md PRE-CREATION
Diff: https://reviews.apache.org/r/24719/diff/
Testing
-------
checked the markdown syntax
Thanks,
Jie Yu
Re: Review Request 24719: Added document for network monitoring.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24719/
-----------------------------------------------------------
(Updated Aug. 14, 2014, 10:59 p.m.)
Review request for mesos, Ian Downes and Vinod Kone.
Changes
-------
Updated.
Repository: mesos-git
Description
-------
See summary
Diffs (updated)
-----
docs/network-monitoring.md PRE-CREATION
Diff: https://reviews.apache.org/r/24719/diff/
Testing
-------
checked the markdown syntax
Thanks,
Jie Yu