You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/02/28 07:53:33 UTC

[GitHub] zheng-da commented on issue #9844: Flaky test_operator_gpu.test_binary_op @ Python3: MKLDNN-GPU

zheng-da commented on issue #9844: Flaky test_operator_gpu.test_binary_op @ Python3: MKLDNN-GPU
URL: https://github.com/apache/incubator-mxnet/issues/9844#issuecomment-369151122
 
 
   It seems this bug is caused by the implementation of bmod. We can easily reproduce the error as below.
   
   ```
   ubuntu@ip-172-31-7-213:~/incubator-mxnet$ export MXNET_TEST_SEED=1138777814
   ubuntu@ip-172-31-7-213:~/incubator-mxnet$ for i in {1..10}; do nosetests -v tests/python/gpu/test_operator_gpu.py:test_binary_op; done
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=2101295148 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=2101295148 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.466s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1174927805 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1174927805 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.031s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=951474900 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=951474900 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.079s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1915316160 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1915316160 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.157s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=284489543 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=284489543 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.091s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=938567636 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=938567636 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.172s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=154247196 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=154247196 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.233s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=44397712 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=44397712 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.141s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1796974521 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1796974521 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.049s
   
   FAILED (failures=1)
   [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=290456305 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   test_operator_gpu.test_binary_op ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   FAIL
   
   ======================================================================
   FAIL: test_operator_gpu.test_binary_op
   ----------------------------------------------------------------------
   Traceback (most recent call last):
     File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
       self.test(*self.arg)
     File "/usr/lib/python2.7/dist-packages/nose/util.py", line 620, in newfunc
       return func(*arg, **kw)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/common.py", line 155, in test_new
       orig_test(*args, **kwargs)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1377, in test_binary_op
       test_bmod(a, b)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1353, in test_bmod
       lambda g_out, a, b: (g_out, - g_out * (np.float32(a) // np.float32(b))), gen_binary_data)
     File "/home/ubuntu/incubator-mxnet/tests/python/gpu/../unittest/test_operator.py", line 1319, in check_binary_op_backward
       assert_allclose(y_2.asnumpy(), x_2, rtol=rtol, atol=atol)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 1395, in assert_allclose
       verbose=verbose, header=header, equal_nan=equal_nan)
     File "/usr/local/lib/python2.7/dist-packages/numpy/testing/utils.py", line 778, in assert_array_compare
       raise AssertionError(msg)
   AssertionError: 
   Not equal to tolerance rtol=0.001, atol=1e-05
   
   (mismatch 0.222222222222%)
    x: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
    y: array([[[[[ -0.000000e+00,  -0.000000e+00,  -0.000000e+00],
             [ -6.009688e-01,  -0.000000e+00,  -1.463857e+00]],
   ...
   -------------------- >> begin captured logging << --------------------
   common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=290456305 to reproduce.
   common: WARNING: *** test-level seed set: all "@with_seed()" tests run deterministically ***
   common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1138777814 to reproduce.
   --------------------- >> end captured logging << ---------------------
   
   ----------------------------------------------------------------------
   Ran 1 test in 4.250s
   
   FAILED (failures=1)
   
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services