[GitHub] perdasilva edited a comment on issue #14146: Disables flaky test test_operator_gpu.test_deconvolution

[GitBox <gi...@apache.org>]

perdasilva edited a comment on issue #14146: Disables flaky test test_operator_gpu.test_deconvolution
URL: https://github.com/apache/incubator-mxnet/pull/14146#issuecomment-464861199
 
 
   I think as a general statement, what you are saying makes sense. But in this case (#14144) - which I would suggest you look at - the changes were to the test script for the installation documentation. I find it difficult to believe that a bash script run by CI would trigger the (flaky) test_operator_gpu.test_deconvolution. Especially since, a) the PR was merged after running the job a second time (on the same code) and ended up green and b) the error report from the jenkins job (bellow) has all the hallmarks of a flaky test, namely miniscule floating point variations.
   
   ```
   Items are not equal:
   
   Error 1.290297 exceeds tolerance rtol=0.001000, atol=0.001000.  Location of maximum error:(31, 2, 7, 11), a=0.181059, b=0.182585
   
    a: array([[[[  71.53303956,   27.22900263,  -81.94053338, ...,  131.83555384,
                0.14714987, -125.67693774],
            [  71.65034077, -127.16477248,   97.06501721, ...,  224.75550074,...
   
    b: array([[[[  71.53299713,   27.22902107,  -81.94055176, ...,  131.83551025,
                0.14712699, -125.6769104 ],
            [  71.65033722, -127.1647644 ,   97.06500244, ...,  224.75546265,...
   ```
   
   I would argue that the CI is in place to catch potential errors of merging you code in to the code base. If  it catches an error on one run, but not on a second re-run of the exact same code, then unreliably failing test is flaky by definition and CI isn't doing its job properly.
   
   Given all the data above, I don't feel the need to spend time reproducing this in master.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services