You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@zookeeper.apache.org by GitBox <gi...@apache.org> on 2020/09/25 20:14:04 UTC
[GitHub] [zookeeper] hanm opened a new pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
hanm opened a new pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467
We'd like to understand how applications built on top of ZooKeeper behave under various faulty conditions, which is important to build resilient end to end solutions and avoid ZooKeeper being single point of failure. We'd also like to achieve this in both unit tests (in process) and integration tests (in and out of process). Traditional methods of using external fault injection mechanisms are non deterministic and requires non trivial set up and hard to integrate with unit tests, so here we introduce the ZooKeeperController service which solves both.
The basic idea here is to create a controllable ZooKeeperServer which accepts various control commands (such as - delay request, drop request, eat request, expire session, shutdown, trigger leader election, and so on), and reacting based on incoming commands. The controllable server and production server share the same underlying machineries (quorum peers, ZooKeeper server, etc) but code paths are separate, thus this feature has no production impact.
This controller system is currently composed of following pieces:
CommandClient: a convenient HTTP client to send control commands to controller service.
CommandListener: an embed HTTP server listening incoming commands and dispatch to controller service.
Controller Service: the service that's responsible to create controllable ZK server and the controller.
ZooKeeperServerController: the controller that changes the behavior of ZK server runtime.
Controllable Cnx / Factory: controllable connection that accept behavior change requests.
In future more control commands and controllable components can be added on top of this framework.
This can be used in either unit tests / integration tests as an in process embedded controllable ZooKeeper server, or as an out of process stand alone controllable ZooKeeper process.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] hanm commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
hanm commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-705968019
@eolivelli thanks for review. I will add documents in a separate pull request.
For license file of Jetty - ZooKeeper has been using Jetty server component for our admin server component since 2014 (implemented in ZOOKEEPER-1346), which is why I didn't bother to check license... but as I just checked, we indeed did not include the license / notice file for Jetty. Can you please double check and do you think if this is a problem for past releases as well? I am thinking if that's the case maybe it's better to fix the license issue in a separate ticket for visibility...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] eolivelli commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-707669709
+1
#shipit
I will merge later, otherwise @nkalmar please go ahead and merge
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] hanm commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
hanm commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-705975929
@eolivelli also see https://github.com/apache/zookeeper/blob/master/pom.xml#L595 - we already use Jetty so I would not consider lacking of Jetty license a blocker for this PR - though agree we should fix it (preferably done in a new ticket).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] hanm commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
hanm commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-707469481
@eolivelli gentle nudge for merging this... I have a few following up patches to contribute once this lands.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] hanm commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
hanm commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-705251511
@eolivelli @anmolnar @symat @nkalmar (and all curious readers): can i get a review for this? this will be a good addition to the testability of ZooKeeper (from our in house experience).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] hanm commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
hanm commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-705968019
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] nkalmar commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
nkalmar commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-707712843
merged to master, thanks @hanm
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] eolivelli commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-705977715
You are right.
This patch has been open for review for a while.
Let me merge it today later
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] eolivelli commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-705977715
You are right.
This patch has been open for review for a while.
Let me merge it today later
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] hanm commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
hanm commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-708138733
thanks @eolivelli and @nkalmar
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] asfgit closed pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] hanm commented on pull request #1467: ZOOKEEPER-3948: Introduce a deterministic runtime behavior injection framework for ZooKeeperServer testing.
Posted by GitBox <gi...@apache.org>.
hanm commented on pull request #1467:
URL: https://github.com/apache/zookeeper/pull/1467#issuecomment-699134053
Some of the controllable features here (such as reject connections, expire sessions etc) rely on some features not available in upstream, so those code / tests are disabled as part of this pull request.
Since this pull request itself is already fairly big, to make it easier for reviewers, the missing dependent features (also documents) will be submitted in separate pull requests once this landed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org