You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zhu Zhu (Jira)" <ji...@apache.org> on 2019/09/09 08:29:01 UTC
[jira] [Comment Edited] (FLINK-13953) Facilitate enabling new
Scheduler in MiniCluster Tests
[ https://issues.apache.org/jira/browse/FLINK-13953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925457#comment-16925457 ]
Zhu Zhu edited comment on FLINK-13953 at 9/9/19 8:28 AM:
---------------------------------------------------------
Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it.
3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime.
Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different profiles.
For per-commit tests only the default scheduler("legacy" currently and "ng" later) is used. And scheduled travis builds can be added for other schedulers.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng".
What do you think?
was (Author: zhuzh):
Hi [~gjy], I did some investigation on how to get different MiniCluster test methods to run multiple times for different schedulers, so that we can do all the tests in one maven tests pass. (the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] requires one more tests pass to run tests for schedulerNG tests)
1. The parameterized tests mechanism of JUnit 4.12(currently used) only supports the same parameters for all tests in the same class.
2. The parameterized tests mechanism of JUnit 5 can support to execute a single test method multiple times with different parameters. However, I'm not sure if anything would broke if we'd like to upgrade it.
3. I also found a lib [JUnitParams|[https://github.com/Pragmatists/JUnitParams]] which supports method level test parameters for JUnit 4.12. However, the project has no new commit for almost 1 year. So I'm not sure if it's a good idea to use it as it might have been out of maintenance.
One concern to use parameterized tests is that we may need to change the existing MiniCluster tests to generate, parse and handle the parameters, even though the tests target for verifying nothing related with schedulers. And later anyone developing an IT case needs to tag and parameterize it as well. This is a bit weird and is hard to guarantee as some IT cases are out of the control of runtime.
Considering a long term solution to verify different schedulers, I think the [prototype|[https://github.com/tillrohrmann/flink/tree/introduceSchedulerSwitch]] from [~till.rohrmann] would be better.
With it we can run MiniCluster tests for different schedulers with different profiles.
For per-commit tests only the default scheduler("legacy" currently and "ng" later) is used. And scheduled travis builds can be added for other schedulers.
A category named "MiniClusterTest" (rather than "AlsoRunWithSchedulerNG") can be used to tag the all MiniCluster tests so that it still works when we switch the default scheduler to "ng".
What do you think?
> Facilitate enabling new Scheduler in MiniCluster Tests
> ------------------------------------------------------
>
> Key: FLINK-13953
> URL: https://issues.apache.org/jira/browse/FLINK-13953
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Tests
> Reporter: Gary Yao
> Assignee: Zhu Zhu
> Priority: Major
>
> Currently, tests using the {{MiniCluster}} use the legacy scheduler by default. Once the new scheduler is implemented, we should run tests with the new scheduler enabled. However, it is not expected that all tests will pass immediately. Therefore, it should be possible to enable the new scheduler for a subset of tests.
> *Acceptance Criteria*
> * Subset of tests using {{MiniCluster}} can be run on a per-commit basis (on Travis) against new scheduler and also legacy scheduler
--
This message was sent by Atlassian Jira
(v8.3.2#803003)