[jira] [Updated] (AMBARI-8060) [Test] The framework executes the tests in random order, which makes some tests fail inconsistently

["Florian Barca (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-8060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Florian Barca updated AMBARI-8060:
----------------------------------
    Attachment: AMBARI-8060.0.patch

The shuffle(tests) calls inside the unitTests.py scripts contribute to test results inconsistencies.

+ Commented the shuffle(tests) calls, to allow the test results to become predictable.

> [Test] The framework executes the tests in random order, which makes some tests fail inconsistently
> ---------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-8060
>                 URL: https://issues.apache.org/jira/browse/AMBARI-8060
>             Project: Ambari
>          Issue Type: Task
>          Components: ambari-agent, ambari-server
>    Affects Versions: branch-windows-dev
>            Reporter: Florian Barca
>            Assignee: Florian Barca
>             Fix For: branch-windows-dev
>
>         Attachments: AMBARI-8060.0.patch, AMBARI-8060.0.patch
>
>   Original Estimate: 6h
>  Remaining Estimate: 6h
>
> The Python unit testing framework randomizes the tests' order of execution. This exposes glitches in the tests, like below:
> FAIL: test_service_check_default (test_service_check.TestServiceCheck)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/vagrant/unittests/ambari/ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py", line 33, in test_service_check_default
>     self.assert_service_check()
>   File "/vagrant/unittests/ambari/ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py", line 56, in assert_service_check
>     try_sleep = 5,
>   File "/vagrant/unittests/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py", line 176, in assertResourceCalled
>     self.assertEquals(kwargs, resource.arguments)
> AssertionError: {'logoutput': True, 'path': ['/usr/bin:/usr/bin'], 'tries': 3, 'command': '/tmp/oozieSmoke2.sh suse /etc/oozie/conf /usr/bin /etc/hadoop/conf /usr/bin ambari-qa False', 'try_sleep': 5} != {'logoutput': True, 'path': ['/usr/bin:/usr/bin'], 'tries': 3, 'command': u'/tmp/oozieSmoke2.sh redhat /etc/oozie/conf /usr/bin /etc/hadoop/conf /usr/bin ambari-qa False', 'try_sleep': 5}
> FAIL: test_service_check_secured (test_service_check.TestServiceCheck)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/vagrant/unittests/ambari/ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py", line 43, in test_service_check_secured
>     self.assert_service_check()
>   File "/vagrant/unittests/ambari/ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py", line 56, in assert_service_check
>     try_sleep = 5,
>   File "/vagrant/unittests/ambari/ambari-server/src/test/python/stacks/utils/RMFTestCase.py", line 176, in assertResourceCalled
>     self.assertEquals(kwargs, resource.arguments)
> AssertionError: {'logoutput': True, 'path': ['/usr/bin:/usr/bin'], 'tries': 3, 'command': '/tmp/oozieSmoke2.sh suse /etc/oozie/conf /usr/bin /etc/hadoop/conf /usr/bin ambari-qa False', 'try_sleep': 5} != {'logoutput': True, 'path': ['/usr/bin:/usr/bin'], 'tries': 3, 'command': u'/tmp/oozieSmoke2.sh redhat /etc/oozie/conf /usr/bin /etc/hadoop/conf /usr/bin ambari-qa False', 'try_sleep': 5}
> FAIL: test_check_postgre_up (TestAmbariServer.TestAmbariServer)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/vagrant/unittests/ambari/ambari-common/src/test/python/mock/mock.py", line 1199, in patched
>     return func(*args, **keywargs)
>   File "/vagrant/unittests/ambari/ambari-server/src/test/python/TestAmbariServer.py", line 2090, in test_check_postgre_up
>     self.assertEqual(4, retcode)
> AssertionError: 4 != 0
> This occurs because files that are not normally executed first are being sometimes executed first, which exposes out-of-order mock import directives.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)