You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ewen Cheslack-Postava (JIRA)" <ji...@apache.org> on 2018/03/06 03:38:00 UTC
[jira] [Commented] (KAFKA-5612) Replace JmxTool with a
MetricsReporter in system tests
[ https://issues.apache.org/jira/browse/KAFKA-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387219#comment-16387219 ]
Ewen Cheslack-Postava commented on KAFKA-5612:
----------------------------------------------
First pass at this was implemented in [https://github.com/apache/kafka/pull/4072.] This adds the implementation and migrates at least one test, but doesn't complete the migration, nor removes JmxTool as an option from our system tests. I would argue the issues with JmxTool have been so numerous that we should try to remove the jmx.py file entirely in order to discourage further use, regardless of whether JmxTool continues to exist in the source tree.
> Replace JmxTool with a MetricsReporter in system tests
> ------------------------------------------------------
>
> Key: KAFKA-5612
> URL: https://issues.apache.org/jira/browse/KAFKA-5612
> Project: Kafka
> Issue Type: Bug
> Components: system tests
> Affects Versions: 0.11.0.0
> Reporter: Ewen Cheslack-Postava
> Priority: Major
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> I marked this as affecting 0.11.0.0, but it affects all earlier versions as well, at least as far back as 0.10.1.
> The discussion in https://github.com/apache/kafka/pull/3547 probably gives the clearest explanation, but the basic issue is that ever since JmxMixin was introduced to the system tests, we've faced race condition issues because the second process that performs the monitoring has various timing issues with the process it is monitoring. It can be both too fast and too slow, and the exact conditions it needs to wait for may not even be externally visible (e.g. that all metrics have been registered).
> An alternative solution would be to introduce a MetricsReporter implementation that accomplishes the same thing, but just requires overriding some configs for the service that is utilizing JmxMixin. In particular, the reporter could output data to a simple file, ideally would not require all metrics that are reported to be available up front (i.e., no CSV format that requires a fixed header that cannot be changed), and wouldn't have any timing constraints (e.g., could at least guarantee that metrics are reported once at the beginning and end of the program).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)