You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Lukasz Gajowy (Jira)" <ji...@apache.org> on 2019/11/26 12:58:00 UTC
[jira] [Created] (BEAM-8826) Investigate possibility of system
metrics usage in portable performance tests
Lukasz Gajowy created BEAM-8826:
-----------------------------------
Summary: Investigate possibility of system metrics usage in portable performance tests
Key: BEAM-8826
URL: https://issues.apache.org/jira/browse/BEAM-8826
Project: Beam
Issue Type: Task
Components: testing
Reporter: Lukasz Gajowy
We currently use [TimeMonitor.java|https://github.com/apache/beam/blob/master/sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/metrics/TimeMonitor.java] and [MeasureTime.py|https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/sdks/python/apache_beam/testing/load_tests/load_test_metrics_utils.py#L406] DoFns to collect runtime in both portable and non-portable performance tests. However, in portable tests it seems to be possible to use [TOTAL_TIME_MSECS |https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/model/pipeline/src/main/proto/metrics.proto#L130]for collecting execution time. Other system metrics are available as well (size, bundle size etc).
It seems like a good way to simplify things and get more useful metrics from portable jobs so it is worth investigating ways of using it in performance tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)