You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by mprimi <gi...@git.apache.org> on 2017/04/06 22:10:55 UTC
[GitHub] zookeeper pull request #215: Four-letter words to voluntarily shed client co...
GitHub user mprimi opened a pull request:
https://github.com/apache/zookeeper/pull/215
Four-letter words to voluntarily shed client connections
Introduce 3 new 4-letter commands:
- shed: close all client connections
- sh25: close approximatively 1/4 of client connections
- sh50: close approximatively half of client connections
In clusters with large number of clients, a rolling bounce (or multiple) can lead to a situation of severe connection imbalance.
In these situations, restarting clients is the only solution to rebalance the load among servers.
This change introduces server admin command that voluntarily close client connections.
Clients should be able to migrate transparently to another server, just like in case of actual disconnection or server crash.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mprimi/zookeeper ZOOKEEPER-2748
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/215.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #215
----
commit f059259cce574803ccca6ee39d0d21ea01eaabc0
Author: Marco Primi <mp...@apple.com>
Date: 2017-03-01T22:19:47Z
Four-letter words to voluntarily shed client connections
Introduce 3 new 4-letter commands:
- shed: close all client connections
- sh25: close approximatively 1/4 of client connections
- sh50: close approximatively half of client connections
In clusters with large number of clients, a rolling bounce (or multiple) can lead to a situation of severe connection imbalance.
In these situations, restarting clients is the only solution to rebalance the load among servers.
This change introduces server admin command that voluntarily close client connections.
Clients should be able to migrate transparently to another server, just like in case of actual disconnection or server crash.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zookeeper pull request #215: Four-letter words to voluntarily shed client co...
Posted by shralex <gi...@git.apache.org>.
Github user shralex commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/215#discussion_r110710035
--- Diff: src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml ---
@@ -1974,6 +1974,31 @@ server.3=zoo3:2888:3888</programlisting>
</programlisting>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>shed</term>
+
+ <listitem>
+ <para>Closes all client connections, forcing them to move to a different server.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sh50</term>
+
+ <listitem>
+ <para>Probabilistically closes half of the client connections (i.e. each connection is closed with a probability of 1/2), forcing them to move to a different server.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sh50</term>
--- End diff --
looks like a typo - should this be sh25 ?
Is it possible to pass the % as parameter to the 4 letter words ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zookeeper issue #215: Four-letter words to voluntarily shed client connectio...
Posted by mprimi <gi...@git.apache.org>.
Github user mprimi commented on the issue:
https://github.com/apache/zookeeper/pull/215
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zookeeper pull request #215: Four-letter words to voluntarily shed client co...
Posted by mprimi <gi...@git.apache.org>.
Github user mprimi commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/215#discussion_r110959848
--- Diff: src/java/test/org/apache/zookeeper/test/ShedClientsTest.java ---
@@ -0,0 +1,161 @@
+/**
--- End diff --
Good idea, I could make better assertions with large number of clients.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zookeeper pull request #215: Four-letter words to voluntarily shed client co...
Posted by shralex <gi...@git.apache.org>.
Github user shralex commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/215#discussion_r110712848
--- Diff: src/java/test/org/apache/zookeeper/test/ShedClientsTest.java ---
@@ -0,0 +1,161 @@
+/**
--- End diff --
This test uses a single server and client, which doesn't seem ideal to test probabilities.
Consider doing something similar to src/java/test/org/apache/zookeeper/test/StaticHostProviderTest.java
which was developed as part of ZK-1355 for testing zk.updatedServerList()
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zookeeper pull request #215: Four-letter words to voluntarily shed client co...
Posted by mprimi <gi...@git.apache.org>.
Github user mprimi commented on a diff in the pull request:
https://github.com/apache/zookeeper/pull/215#discussion_r110959116
--- Diff: src/docs/src/documentation/content/xdocs/zookeeperAdmin.xml ---
@@ -1974,6 +1974,31 @@ server.3=zoo3:2888:3888</programlisting>
</programlisting>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term>shed</term>
+
+ <listitem>
+ <para>Closes all client connections, forcing them to move to a different server.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sh50</term>
+
+ <listitem>
+ <para>Probabilistically closes half of the client connections (i.e. each connection is closed with a probability of 1/2), forcing them to move to a different server.</para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>sh50</term>
--- End diff --
Actually this is correct, but the one below (line 1995) is wrong.
It is not trivial to pass the number because 4-letter commands are internally used as 4-byte integer and stored in a static map.
But if we go with ZooKeeperAdmin rather than 4-letter words, looks like we can parametrize.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---