You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/04/07 16:57:25 UTC
[GitHub] [pulsar-test-infra] nodece opened a new pull request, #28: fix: fix get failured checks
nodece opened a new pull request, #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28
Signed-off-by: Zixuan Liu <no...@gmail.com>
### Motivation
I tried to rerun the failed test, but the failed test cannot be rerun. I checked the https://github.com/apache/pulsar/runs/5869256539?check_suite_focus=true#step:5:314, no tests were run.
Using https://docs.github.com/en/rest/reference/actions#list-workflow-runs-for-a-repository is not working, this API cannot find my actions.
You can use this command to reproduce:
```
PR_USER=nodece PR_BRANCH=update_spring_library status=failure curl -s -H "Accept: application/vnd.github.v3+json" "https://api.github.com/repos/apache/pulsar/actions/runs?actor=${PR_USER}&branch=${PR_BRANCH}&status=${status}&per_page=100"
```
If we use the old way to get the failed checks, it is working:
```
curl -s -H "Accept: application/vnd.github.v3+json" "https://api.github.com/repos/apache/pulsar/commits/70266323cc7784c66c35044d64ef87e593559b7d/check-runs?per_page=1"
{
"total_count": 57,
"check_runs": [
{
"id": 5867399574,
"name": "Unit-PROXY Tests",
"node_id": "CR_kwDOA7PXtM8AAAABXblplg",
"head_sha": "70266323cc7784c66c35044d64ef87e593559b7d",
"external_id": "",
"url": "https://api.github.com/repos/apache/pulsar/check-runs/5867399574",
"html_url": "https://github.com/apache/pulsar/runs/5867399574",
"details_url": "https://github.com/apache/pulsar/runs/5867399574",
"status": "completed",
"conclusion": "success",
"started_at": "2022-04-07T11:22:16Z",
"completed_at": "2022-04-07T11:22:16Z",
"output": {
"title": "Unit-PROXY Tests ✔️",
"summary": "\n|Report|Passed|Failed|Skipped|Time|\n|:---|---:|---:|---:|---:|\n|[TEST-org.apache.pulsar.proxy.extensions.ProxyExtensionsTest.xml](https://github.com/apache/pulsar/runs/5867399574#r0)|5✔️|||3s|\n|[TEST-org.apache.pulsar.proxy.extensions.ProxyExtensionUtilsTest.xml](https://github.com/apache/pulsar/runs/5867399574#r1)|3✔️|||471ms|\n|[TEST-org.apache.pulsar.proxy.extensions.ProxyExtensionWithClassLoaderTest.xml](https://github.com/apache/pulsar/runs/5867399574#r2)|2✔️|||1s|\n|[TEST-org.apache.pulsar.proxy.extensions.SimpleProxyExtensionSeparateThreadPoolTest.xml](https://github.com/apache/pulsar/runs/5867399574#r3)|1✔️|||27ms|\n|[TEST-org.apache.pulsar.proxy.extensions.SimpleProxyExtensionSharedThreadPoolTest.xml](https://github.com/apache/pulsar/runs/5867399574#r4)|1✔️|||24ms|\n|[TEST-org.apache.pulsar.proxy.server.AdminProxyHandlerTest.xml](https://github.
com/apache/pulsar/runs/5867399574#r5)|3✔️|||1s|\n|[TEST-org.apache.pulsar.proxy.server.AuthedAdminProxyHandlerTest.xml](https://github.com/apache/pulsar/runs/5867399574#r6)|1✔️|||7s|\n|[TEST-org.apache.pulsar.proxy.server.BrokerProxyValidatorTest.xml](https://github.com/apache/pulsar/runs/5867399574#r7)|7✔️|||2s|\n|[TEST-org.apache.pulsar.proxy.server.FunctionWorkerRoutingTest.xml](https://github.com/apache/pulsar/runs/5867399574#r8)|1✔️|||5s|\n|[TEST-org.apache.pulsar.proxy.server.InvalidProxyConfigForAuthorizationTest.xml](https://github.com/apache/pulsar/runs/5867399574#r9)|1✔️|||102ms|\n|[TEST-org.apache.pulsar.proxy.server.ProxyAdditionalServletTest.xml](https://github.com/apache/pulsar/runs/5867399574#r10)|1✔️|||920ms|\n|[TEST-org.apache.pulsar.proxy.server.ProxyAuthenticatedProducerConsumerTest.xml](https://github.com/apache/pulsar/runs/5867399574#r11)|1✔️|||33s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyAuthenticationTest.xml](https://github.com
/apache/pulsar/runs/5867399574#r12)|1✔️|||6s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyConfigurationTest.xml](https://github.com/apache/pulsar/runs/5867399574#r13)|2✔️|||372ms|\n|[TEST-org.apache.pulsar.proxy.server.ProxyConnectionTest.xml](https://github.com/apache/pulsar/runs/5867399574#r14)|1✔️|||9ms|\n|[TEST-org.apache.pulsar.proxy.server.ProxyConnectionThrottlingTest.xml](https://github.com/apache/pulsar/runs/5867399574#r15)|1✔️|||5s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyEnableHAProxyProtocolTest.xml](https://github.com/apache/pulsar/runs/5867399574#r16)|1✔️|||3s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyForwardAuthDataTest.xml](https://github.com/apache/pulsar/runs/5867399574#r17)|1✔️|||69s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyIsAHttpProxyTest.xml](https://github.com/apache/pulsar/runs/5867399574#r18)|10✔️|||3s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyParserTest.xml](https://github.com/apache/pulsar/runs/5867399574#r19)|5✔
️|||10s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyPrometheusMetricsTest.xml](https://github.com/apache/pulsar/runs/5867399574#r20)|1✔️|||976ms|\n|[TEST-org.apache.pulsar.proxy.server.ProxyRolesEnforcementTest.xml](https://github.com/apache/pulsar/runs/5867399574#r21)|1✔️|||9s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyServiceStarterTest.xml](https://github.com/apache/pulsar/runs/5867399574#r22)|3✔️|||3s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyServiceTlsStarterTest.xml](https://github.com/apache/pulsar/runs/5867399574#r23)|2✔️|||3s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyStatsTest.xml](https://github.com/apache/pulsar/runs/5867399574#r24)|3✔️|||6s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyTest.xml](https://github.com/apache/pulsar/runs/5867399574#r25)|7✔️|||14s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyTlsTest.xml](https://github.com/apache/pulsar/runs/5867399574#r26)|2✔️|||5s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyWithAuthoriz
ationNegTest.xml](https://github.com/apache/pulsar/runs/5867399574#r27)|1✔️|||6s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyWithAuthorizationTest.xml](https://github.com/apache/pulsar/runs/5867399574#r28)|13✔️|||86s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyWithJwtAuthorizationTest.xml](https://github.com/apache/pulsar/runs/5867399574#r29)|3✔️|||14s|\n|[TEST-org.apache.pulsar.proxy.server.ProxyWithoutServiceDiscoveryTest.xml](https://github.com/apache/pulsar/runs/5867399574#r30)|1✔️|||39s|\n|[TEST-org.apache.pulsar.proxy.server.SuperUserAuthedAdminProxyHandlerTest.xml](https://github.com/apache/pulsar/runs/5867399574#r31)|3✔️|||19s|\n|[TEST-org.apache.pulsar.proxy.server.UnauthedAdminProxyHandlerTest.xml](https://github.com/apache/pulsar/runs/5867399574#r32)|2✔️|||405ms|\n|[TEST-org.apache.pulsar.proxy.util.CmdTest.xml](https://github.com/apache/pulsar/runs/5867399574#r33)|1✔️|||170ms|",
"text": null,
"annotations_count": 0,
"annotations_url": "https://api.github.com/repos/apache/pulsar/check-runs/5867399574/annotations"
},
"check_suite": {
"id": 5968925454
},
"app": {
"id": 15368,
"slug": "github-actions",
"node_id": "MDM6QXBwMTUzNjg=",
"owner": {
"login": "github",
"id": 9919,
"node_id": "MDEyOk9yZ2FuaXphdGlvbjk5MTk=",
"avatar_url": "https://avatars.githubusercontent.com/u/9919?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/github",
"html_url": "https://github.com/github",
"followers_url": "https://api.github.com/users/github/followers",
"following_url": "https://api.github.com/users/github/following{/other_user}",
"gists_url": "https://api.github.com/users/github/gists{/gist_id}",
"starred_url": "https://api.github.com/users/github/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/github/subscriptions",
"organizations_url": "https://api.github.com/users/github/orgs",
"repos_url": "https://api.github.com/users/github/repos",
"events_url": "https://api.github.com/users/github/events{/privacy}",
"received_events_url": "https://api.github.com/users/github/received_events",
"type": "Organization",
"site_admin": false
},
"name": "GitHub Actions",
"description": "Automate your workflow from idea to production",
"external_url": "https://help.github.com/en/actions",
"html_url": "https://github.com/apps/github-actions",
"created_at": "2018-07-30T09:30:17Z",
"updated_at": "2019-12-10T19:04:12Z",
"permissions": {
"actions": "write",
"administration": "read",
"checks": "write",
"contents": "write",
"deployments": "write",
"discussions": "write",
"issues": "write",
"metadata": "read",
"organization_packages": "write",
"packages": "write",
"pages": "write",
"pull_requests": "write",
"repository_hooks": "write",
"repository_projects": "write",
"security_events": "write",
"statuses": "write",
"vulnerability_alerts": "read"
},
"events": [
"branch_protection_rule",
"check_run",
"check_suite",
"create",
"delete",
"deployment",
"deployment_status",
"discussion",
"discussion_comment",
"fork",
"gollum",
"issues",
"issue_comment",
"label",
"milestone",
"page_build",
"project",
"project_card",
"project_column",
"public",
"pull_request",
"pull_request_review",
"pull_request_review_comment",
"push",
"registry_package",
"release",
"repository",
"repository_dispatch",
"status",
"watch",
"workflow_dispatch",
"workflow_run"
]
},
"pull_requests": [
]
}
]
}
````
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari commented on pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28#issuecomment-1104908056
I have fixed the rerunning in cases where the PR's head sha doesn't change (for example closing and reopening the PR to trigger a new build): https://github.com/apache/pulsar-test-infra/pull/33
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari commented on pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28#issuecomment-1092402909
> I have a question about the `get_runs failure` don't seem to get the correct data, this data includes the `conclusion: success` item. Do you notice this?
Yes. That happens when an actual build run hasn't failed. As explained in previous comments, the test reports added to the build are causing this.
The benefit of seeing the failed test reports is that we can start paying more attention to the flaky test problem and fixing them. Flaky tests might be indicating real production code issues.
> This is confusing, I'm looking forward your solution.
There's really no other solution than documenting this. We could remove the rendered test reports, but I think that integrating test reports to GitHub Actions UI is useful.
GitHub Actions UI has a bug that [test reports get attached to wrong workflows](https://github.com/dorny/test-reporter/issues/67) when there's more than one workflow that gets triggered by a build job. Perhaps some day GitHub fixes this issue and the test reports can be shown in the correct location, attached to each build job. That would help resolve the issue since the test reports wouldn't be a separate check and instead the test reports could be attached to the build job that runs the tests. That's currently not possible because of the GitHub Actions bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari commented on pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28#issuecomment-1092086661
In your case, it's the flaky tests report: https://github.com/apache/pulsar/pull/15065/checks?check_run_id=5867398812 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari closed pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
lhotari closed pull request #28: fix: fix get failured checks
URL: https://github.com/apache/pulsar-test-infra/pull/28
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari commented on pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28#issuecomment-1098935588
There's more details in https://lists.apache.org/thread/60x7sqg2p4mlssj5jtow6zwq3jksf6w3 .
Currently there's an issue with GitHub Actions that the workflow run search doesn't return all results and that's why rerunning failed jobs doesn't work in some cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] nodece commented on pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
nodece commented on PR #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28#issuecomment-1092396641
@lhotari Thanks for your explanation!
> There's a misunderstanding. The new CI requires using the new API for re-running just failed jobs and not the complete workflow. Please report what is broken first and we can think of a solution.
I have a question about the `get_runs failure` don't seem to get the correct data, this data includes the `conclusion: success` item. Do you notice this?
> I explained some of the changes in https://lists.apache.org/thread/n05xp17krb4f3cc0jjkh65ytqk1w6130 .
> "The limitation is that you need to wait for all jobs to complete before failed jobs can be re-run."
>
> You can also provide feedback on the mailing list.
This is detailed.
> With the new CI there's another case where you see a red mark. It's when flaky tests fail, but later pass in a retry, the failed test result will get recorded to a test report. That doesn't block merging, but will show up so that the failures can be inspected. It's a bit confusing since GitHub Actions has a bug that the test reports get attached randomly to a workflow when there are multiple workflows executing. I'll write to the dev list explaining this since I have received a few questions about this.
> It might appear that re-running doesn't work because of the new test reports. Those failures won't clear off, but they don't block merging either.
This is confusing, I'm looking forward your solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari commented on pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28#issuecomment-1092083499
With the new CI there's another case where you see a red mark. It's when flaky tests fail, but later pass in a retry, the failed test result will get recorded to a test report. That doesn't block merging, but will show up so that the failures can be inspected. It's a bit confusing since GitHub Actions has a bug that the test reports get attached randomly to a workflow when there are multiple workflows executing. I'll write to the dev list explaining this since I have received a few questions about this.
It might appear that re-running doesn't work because of the new test reports. Those failures won't clear off, but they don't block merging either.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari commented on pull request #28: fix: fix get failured checks
Posted by GitBox <gi...@apache.org>.
lhotari commented on PR #28:
URL: https://github.com/apache/pulsar-test-infra/pull/28#issuecomment-1092079756
I explained some of the changes in https://lists.apache.org/thread/n05xp17krb4f3cc0jjkh65ytqk1w6130 .
You can also provide feedback on the mailing list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org