[jira] [Commented] (BEAM-7902) :beam-test-infra-metrics:test is consistently failing

["Mikhail Gryzykhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16900434#comment-16900434 ] 

Mikhail Gryzykhin commented on BEAM-7902:
-----------------------------------------

This shows 3 issues:
 # We should remove this test from build target, since it is an integration test that shows the healthiness of metrics website.
 # We need to fix beam_Prober_CommunityMetrics target to not hit cache. It always fetches fresh data from website even though binaries are the same.
 # There actually seem to be an issue with fetching latest data from github on metrics site. As you noted it is week old and test catches the issue.

 

> :beam-test-infra-metrics:test is consistently failing
> -----------------------------------------------------
>
>                 Key: BEAM-7902
>                 URL: https://issues.apache.org/jira/browse/BEAM-7902
>             Project: Beam
>          Issue Type: Bug
>          Components: test-failures
>            Reporter: Mark Liu
>            Assignee: Mikhail Gryzykhin
>            Priority: Major
>
> A failure gradle run from my local machine:  https://gradle.com/s/sqcwi7bo3a2nw. Stacktrace:
> {code}
> org.apache.beam.testinfra.metrics.ProberTests > CheckGrafanaStalenessAlerts FAILED
>     java.lang.AssertionError: Input data is stale! [id:1, dashboardId:5, dashboardUid:data-freshness, dashboardSlug:source-data-freshness, panelId:2, name:Source Data Freshness alert, state:alerting, newStateDate:2019-07-30T02:55:02Z, evalDate:0001-01-01T00:00:00Z, evalData:[evalMatches:[[metric:GitHub, tags:null, value:482.16617045000004]]], executionError:, url:/d/data-freshness/source-data-freshness]
>        See: http://104.154.241.245/d/data-freshness. Expression: (alert.state == ok)
>         at org.codehaus.groovy.runtime.InvokerHelper.assertFailed(InvokerHelper.java:406)
>         at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.assertFailed(ScriptBytecodeAdapter.java:650)
> ...
> {code}
> From the stacktrace suggestion, I went to http://104.154.241.245/d/data-freshness and got Github source is older than a week, which I guess is potentially the failure reason. However, no further instruction on what to do next. 
> After some investigations, I found this test `:beam-test-infra-metrics:test` is added to [beam_Prober_CommunityMetrics|https://builds.apache.org/view/A-D/view/Beam/view/All/job/beam_Prober_CommunityMetrics/] but never got passed (either failed or skipped due to cache). 
> Is there anything wrong with Github data refresh? Should we reevaluate the alert on dashboard or modify the test?



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)