You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/06/03 15:00:47 UTC
[GitHub] [beam] kamilwu opened a new pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
kamilwu opened a new pull request #11909:
URL: https://github.com/apache/beam/pull/11909
Adds Grafana dashboards for Nexmark tests.
Dashboards can be previewed on http://metrics.beam.apache.org
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/)
Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python2_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PostCommit_Python35_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/)
XLang | --- | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/)
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website
--- | --- | --- | --- | ---
Non-portable | [](https://builds.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://builds.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/)
Portable | --- | [](https://builds.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia edited a comment on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
iemejia edited a comment on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-641060650
Hello, thanks for the already done fixes. For retention policy I think it matters for historical reasons. There are presentations also where we highlight that data that won't be available anymore.
**Notice that default views != retention of data.** We should definitely have shorter views by default but stored data should not disapear if there are no technical constraints that force us to do so, so I even question why have a retention policy? Aren't we all around big data anyway? And this is not going to be even filling an average hard disk size for a good time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-639569627
> It would be nice if we could attribute a consistent color to each runner so in every view we can know the color corresponds to the runner.
Done, each runner has one unique color now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-641922547
> We should definitely have shorter views by default but stored data should not disapear if there are no technical constraints that force us to do so
Ok, makes sense to me. I created an infinity retention policy. Data will not longer be deleted.
> Can you please add a banner with this warning on the top of the Nexmark performance dashboard.
Done.
I also changed the way we select multiple runners to be displayed on charts. Instead of clicking the legend, user is supposed to choose a combination of runners from a drop-down list "Runner" on the top of a dashboard. What's important, _all_ charts will be updated after selecting.
The issue of autoscaling Y axis is also fixed. If DataflowRunner is deselected, the results for other other runners are now more readable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-638789937
Wow I am impressed they look REALLY nice great job!
There are some things I would like to refine e.g. to be able to choose which runners to compare or just to choose one individual runner. (This in part because Dataflow runs with a lot more data so maybe the results are not ok, we need to assure some sort of uniformity in number of inputs).
About data retention, are there performance reasons to limit it? I am really REALLY interested on preserving the history to the point that I would love if possible if we could migrate the previous database into it (if possible). What are the constraints to do this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-641060650
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
iemejia commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-638790781
On extra nit pick: It would be nice if we could attribute a consistent color to each runner so in every view we can know the color corresponds to the runner.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] iemejia merged pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
iemejia merged pull request #11909:
URL: https://github.com/apache/beam/pull/11909
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-641926095
FYI: https://github.com/apache/beam/pull/11956
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-639376550
> I would like to refine e.g. to be able to choose which runners to compare or just to choose one individual runner
Just select one series from the legend. For selecting multiple series, hold shift while clicking.
> About data retention, are there performance reasons to limit it?
I'm pretty sure that we have enough free resources to handle more data (older than one year). However, the question is, do we really need such a long period? From my perspective, a yearly retention is good enough to detect and avoid regressions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #11909: [BEAM-8134] Grafana dashboards for Nexmark tests
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #11909:
URL: https://github.com/apache/beam/pull/11909#issuecomment-638261657
R: @iemejia @mxm Let me know what do you think. I put a link to dashboards in the description.
Our InfluxDB has yearly retention policy, which means data older than one year is automatically discarded. If you think this is not enough, I can adjust it (one database can have multiple retention policies at once).
The next step would be to set up a publisher in Nexmark code.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org