[jira] [Comment Edited] (IGNITE-11708) Unable to run tests in IgniteConfigVariationsAbstractTest subclasses

["Ivan Fedotov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-11708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818759#comment-16818759 ] 

Ivan Fedotov edited comment on IGNITE-11708 at 4/16/19 8:23 AM:
----------------------------------------------------------------

[~Pavlukhin], I added test for check {{IgniteConfigVariationsAbstractTest}} execution according to your suggestions. May be add similar check that before/after JUnit methods also work, what do you think?

According to fix {{IgniteConfigVariationsAbstractTest}} rules workflow. I decided to simplify code and remove outer rule and ruleChain from {{IgniteConfigVariationsAbstractTest}} [1]. We can implement the same idea if we just put {{testCfg}} initilization in {{beforeTestsStarted}}. Now all tests work - you can try to start {{InterceptorCacheConfigVariationsFullApiTestSuite}} [2] and see that methods work indeed (it takes time unlike the master branch).

But now TeamCity bot shows us a lot of failures [3]. I tried to figure out and it seems that failures are related to what tests check, but not JUnit error. I mean that in master tests are green just because nothing happens, but indeed tests can fail. I can not figure out in each tests quickly, but in nutshell, according to logs the failures are caused errors in tests functionality. Moreover, 450 failures on 15_000 tests are just 3%, so it could be true.

[1] [https://github.com/apache/ignite/pull/6434/files]
 [2] [https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java]
 [3] [https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest]


was (Author: ivanan.fed):
[~Pavlukhin], I added test for check {{IgniteConfigVariationsAbstractTest}} execution according to your suggestions. May be add similar check that before/after JUnit methods also work, what do you think?

According to fix {{IgniteConfigVariationsAbstractTest}} rules workflow. I decided to simplify code and remove outer rule and ruleChain from {{IgniteConfigVariationsAbstractTest}} [1]. We can implement the same idea if we just put {{testCfg}} initilization in {{beforeTestsStarted}}. Now all tests work - you can try to start {{InterceptorCacheConfigVariationsFullApiTestSuite}} [2] and see that methods work indeed (it takes time unlike the master branch). 

But now TeamCity bot shows us a lot of failures [3]. I tried to figure out and it seems that failures are related to what tests check, but not JUnit error. I mean that in master tests are green just because nothing happens, but indeed tests can fail. I can not figure out in each tests quickly, but in nutshell, according to logs the failures are caused errors in tests functionality. Moreover, 300 failures on 15_000 tests are just 2%, so it could be true.


[1] https://github.com/apache/ignite/pull/6434
[2] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testsuites/InterceptorCacheConfigVariationsFullApiTestSuite.java
[3] https://mtcga.gridgain.com/pr.html?serverId=apache&suiteId=IgniteTests24Java8_RunAll&branchForTc=pull/6434/head&action=Latest



> Unable to run tests in IgniteConfigVariationsAbstractTest subclasses
> --------------------------------------------------------------------
>
>                 Key: IGNITE-11708
>                 URL: https://issues.apache.org/jira/browse/IGNITE-11708
>             Project: Ignite
>          Issue Type: Bug
>            Reporter: Ivan Fedotov
>            Assignee: Ivan Fedotov
>            Priority: Major
>              Labels: iep30
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> It seems that test classes that extend from IgniteConfigVariationsAbstractTest cannot be started with JUnit4 @Test annotation. 
> It is easy to check: if throw exception in any test methods, nothing will happen.
> Reason can be in rule chain in IgniteConfigVariationsAbstractTest class [1], maybe it destroys existing test workflow.
> [1] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/testframework/junits/IgniteConfigVariationsAbstractTest.java#L62



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)