[jira] [Created] (KAFKA-5612) Replace JmxTool with a MetricsReporter in system tests

["Ewen Cheslack-Postava (JIRA)" <ji...@apache.org>]

Ewen Cheslack-Postava created KAFKA-5612:
--------------------------------------------

             Summary: Replace JmxTool with a MetricsReporter in system tests
                 Key: KAFKA-5612
                 URL: https://issues.apache.org/jira/browse/KAFKA-5612
             Project: Kafka
          Issue Type: Bug
          Components: system tests
    Affects Versions: 0.11.0.0
            Reporter: Ewen Cheslack-Postava


I marked this as affecting 0.11.0.0, but it affects all earlier versions as well, at least as far back as 0.10.1.

The discussion in https://github.com/apache/kafka/pull/3547 probably gives the clearest explanation, but the basic issue is that ever since JmxMixin was introduced to the system tests, we've faced race condition issues because the second process that performs the monitoring has various timing issues with the process it is monitoring. It can be both too fast and too slow, and the exact conditions it needs to wait for may not even be externally visible (e.g. that all metrics have been registered).

An alternative solution would be to introduce a MetricsReporter implementation that accomplishes the same thing, but just requires overriding some configs for the service that is utilizing JmxMixin. In particular, the reporter could output data to a simple file, ideally would not require all metrics that are reported to be available up front (i.e., no CSV format that requires a fixed header that cannot be changed), and wouldn't have any timing constraints (e.g., could at least guarantee that metrics are reported once at the beginning and end of the program).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)