You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Ryan Skraba (Jira)" <ji...@apache.org> on 2022/09/09 08:35:00 UTC
[jira] [Commented] (FLINK-29198) RetryExtension doesn't make the test fail if the retries are exhausted
[ https://issues.apache.org/jira/browse/FLINK-29198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17602202#comment-17602202 ]
Ryan Skraba commented on FLINK-29198:
-------------------------------------
{{@RetryOnFailure}} works as as expected, but I can reproduce this for {{@RetryOnException}} :
{code:java}
@TestTemplate
@RetryOnException(times = 3 exception = IllegalArgumentException.class)
void testFailForever() {
throw new IllegalArgumentException();
} {code}
This test never passes, but it never fails either. The test case is aborted and is counted as "skipped". According to the [Javadoc|https://github.com/apache/flink/blob/8a4538ae4551713591bbe92fd3ec46b1f212e36e/flink-test-utils-parent/flink-test-utils-junit/src/main/java/org/apache/flink/testutils/junit/RetryOnException.java#L47], it should fail after 4 tries.
It looks like the new RetryExtension explicitly does not fail on the given exception class ever. I think it might be surprising to some developers!
Can you assign this to me?
> RetryExtension doesn't make the test fail if the retries are exhausted
> ----------------------------------------------------------------------
>
> Key: FLINK-29198
> URL: https://issues.apache.org/jira/browse/FLINK-29198
> Project: Flink
> Issue Type: Improvement
> Components: Tests
> Affects Versions: 1.15.0, 1.16.0, 1.17.0
> Reporter: Matthias Pohl
> Assignee: Ryan Skraba
> Priority: Major
> Labels: starter
>
> FLINK-24627 introduced retry functionality for JUnit5-based tests. It appears that the retry mechanism doesn't have the desired behavior: If the retries are exhausted without the test ever succeeding will result in the test being ignored. I would expect the test to fail in that case. Otherwise, a CI run would succeed without anyone noticing the malfunctioning of the test.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)