[GitHub] kafka pull request #4073: KAFKA-6060: Add workload generation capabilities t...

[cmccabe <gi...@git.apache.org>]

GitHub user cmccabe opened a pull request:

    https://github.com/apache/kafka/pull/4073

    KAFKA-6060: Add workload generation capabilities to Trogdor

    Previously, Trogdor only handled "Faults."  Now, Trogdor can handle
    "Tasks" which may be either faults, or workloads to execute in the
    background.
    
    The Agent and Coordinator have been refactored from a
    mutexes-and-condition-variables paradigm into a message passing
    paradigm.  No locks are necessary, because only one thread can access
    the task state or worker state.  This makes them a lot easier to reason
    about.
    
    The MockTime class can now handle mocking deferred message passing
    (adding a message to an ExecutorService with a delay).  I added a
    MockTimeTest.
    
    MiniTrogdorCluster now starts up Agent and Coordinator classes in
    paralle in order to minimize junit test time.
    
    RPC messages now inherit from a common Message.java class.  This class
    handles implementing serialization, equals, hashCode, etc.
    
    Remove FaultSet, since it is no longer necessary.
    
    Previously, if CoordinatorClient or AgentClient hit a networking
    problem, they would throw an exception.  They now retry several times
    before giving up.  Additionally, the REST RPCs to the Coordinator and
    Agent have been changed to be idempotent.  If a response is lost, and
    the request is resent, no harm will be done.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cmccabe/kafka KAFKA-6060

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/kafka/pull/4073.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4073
    
----
commit 936427509fec91cf33c798b8a7208dd6c8dc8ba5
Author: Colin P. Mccabe <cm...@confluent.io>
Date:   2017-09-26T01:30:13Z

    KAFKA-6060: Add workload generation capabilities to Trogdor
    
    Previously, Trogdor only handled "Faults."  Now, Trogdor can handle
    "Tasks" which may be either faults, or workloads to execute in the
    background.
    
    The Agent and Coordinator have been refactored from a
    mutexes-and-condition-variables paradigm into a message passing
    paradigm.  No locks are necessary, because only one thread can access
    the task state or worker state.  This makes them a lot easier to reason
    about.
    
    The MockTime class can now handle mocking deferred message passing
    (adding a message to an ExecutorService with a delay).  I added a
    MockTimeTest.
    
    MiniTrogdorCluster now starts up Agent and Coordinator classes in
    paralle in order to minimize junit test time.
    
    RPC messages now inherit from a common Message.java class.  This class
    handles implementing serialization, equals, hashCode, etc.
    
    Remove FaultSet, since it is no longer necessary.
    
    Previously, if CoordinatorClient or AgentClient hit a networking
    problem, they would throw an exception.  They now retry several times
    before giving up.  Additionally, the REST RPCs to the Coordinator and
    Agent have been changed to be idempotent.  If a response is lost, and
    the request is resent, no harm will be done.

----


---

[GitHub] kafka pull request #4073: KAFKA-6060: Add workload generation capabilities t...

[asfgit <gi...@git.apache.org>]

Github user asfgit closed the pull request at:

    https://github.com/apache/kafka/pull/4073


---