You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/05/27 03:45:00 UTC
[jira] [Work logged] (BEAM-10073) Add dashboard for pubsub
performance tests to grafana
[ https://issues.apache.org/jira/browse/BEAM-10073?focusedWorklogId=437397&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-437397 ]
ASF GitHub Bot logged work on BEAM-10073:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/May/20 03:44
Start Date: 27/May/20 03:44
Worklog Time Spent: 10m
Work Description: kamilwu commented on a change in pull request #11809:
URL: https://github.com/apache/beam/pull/11809#discussion_r430015390
##########
File path: .test-infra/metrics/grafana/dashboards/perftests_metrics/Python_IO_IT_Tests_Dataflow.json
##########
@@ -35,7 +35,7 @@
},
"hiddenSeries": false,
"id": 2,
- "interval": "",
+ "interval": "15h",
Review comment:
Is there any particular reason why "15"?
##########
File path: .test-infra/metrics/grafana/dashboards/perftests_metrics/Python_IO_IT_Tests_Dataflow.json
##########
@@ -261,6 +261,250 @@
"align": false,
"alignLevel": null
}
+ },
+ {
+ "aliasColors": {},
+ "bars": false,
+ "cacheTimeout": null,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "BeamInfluxDB",
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 12,
+ "x": 0,
+ "y": 9
+ },
+ "hiddenSeries": false,
+ "id": 4,
+ "interval": "15h",
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": true,
+ "linewidth": 2,
+ "links": [],
+ "nullPointMode": "connected",
+ "options": {
+ "dataLinks": []
+ },
+ "percentage": false,
+ "pluginVersion": "6.7.2",
+ "pointradius": 2,
+ "points": true,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "read_time",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ }
+ ],
+ "measurement": "python_bqio_read",
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT mean(\"value\") FROM \"python_psio_2GB_results\" WHERE \"metric\" = \"pubsub_io_perf_read_runtime\" AND $timeFilter GROUP BY time($__interval), \"metric\"",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Reading 2GB of data | Pubsub IO",
Review comment:
We should also specify that the test is a streaming job and uses a native Dataflow IO. These are very important information for everyone seeing that chart.
##########
File path: .test-infra/metrics/grafana/dashboards/perftests_metrics/Python_IO_IT_Tests_Dataflow.json
##########
@@ -35,7 +35,7 @@
},
"hiddenSeries": false,
"id": 2,
- "interval": "",
+ "interval": "15h",
Review comment:
"H 15 * * *" means the job is executed daily at 15:00 (3 p.m.) UTC. So the interval should be 24h
##########
File path: .test-infra/metrics/grafana/dashboards/perftests_metrics/Python_IO_IT_Tests_Dataflow.json
##########
@@ -35,7 +35,7 @@
},
"hiddenSeries": false,
"id": 2,
- "interval": "",
+ "interval": "15h",
Review comment:
Take a look at the job history: https://builds.apache.org/view/A-D/view/Beam/view/All/job/beam_PubsubIO_Performance_Test_Python/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 437397)
Remaining Estimate: 0h
Time Spent: 10m
> Add dashboard for pubsub performance tests to grafana
> -----------------------------------------------------
>
> Key: BEAM-10073
> URL: https://issues.apache.org/jira/browse/BEAM-10073
> Project: Beam
> Issue Type: Task
> Components: sdk-py-core
> Affects Versions: Not applicable
> Reporter: Piotr Szuberski
> Assignee: Piotr Szuberski
> Priority: P1
> Fix For: Not applicable
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Pubsub IO performance tests are not published to influx and not displayed in grafana dashboards
--
This message was sent by Atlassian Jira
(v8.3.4#803005)