[GitHub] [incubator-mxnet] eric-haibin-lin commented on a change in pull request #17002: Multi_sum_sq review, AtomicAdd removal

[GitBox <gi...@apache.org>]

eric-haibin-lin commented on a change in pull request #17002: Multi_sum_sq review, AtomicAdd removal
URL: https://github.com/apache/incubator-mxnet/pull/17002#discussion_r355085744
 
 

 ##########
 File path: tests/python/gpu/test_operator_gpu.py
 ##########
 @@ -310,6 +310,35 @@ def check_fast_lars(w_dtype, g_dtype, shapes, ctx, tol1, tol2):
             ref_new_lrs[i] = lrs[i]
     assert_almost_equal(ref_new_lrs.asnumpy(), mx_new_lrs.asnumpy(), atol=tol2, rtol=tol2)
 
+def check_multi_sum_sq(dtype, shapes, ctx, tol1, tol2):
+    values_arr = [np.random.rand(*shape).astype(dtype) * 10. for shape in shapes]
+
+    mx_vals = _make_ndarrays(values_arr, ctx=ctx)
+    sum_sq = mx.nd.multi_sum_sq(*mx_vals, num_arrays=len(shapes))
+
+    ref_sum_sq = mx.nd.array([(v.astype('float32') ** 2).sum() for v in values_arr],
+                            dtype='float32', ctx=ctx)
+
+    assert_almost_equal(ref_sum_sq.asnumpy(), sum_sq.asnumpy(), atol=tol1, rtol=tol1)
+
+@with_seed()
+def test_multi_sum_sq():
+    min_nparam = 390
+    max_nparam = 400
+    mindim = 50000
+    maxdim = 3200000
+    maxndim = 1
+
+    dtypes = ['float16','float32', 'float64']
+    for ctx in [mx.gpu(0)]:
+        for dtype in dtypes:
+            nparam = np.random.randint(min_nparam + 1, max_nparam + 1)
+            shapes = [np.random.randint(mindim, maxdim + 1, size=maxndim) for i in range(nparam)]
+            lowTol = ctx == mx.cpu(0) and ('float16'in [dtype])
+            tol1 = 1e-3 if lowTol else 1e-5
+            tol2 = 1e-6 if lowTol else 1e-7
+            check_multi_sum_sq(dtype, shapes, ctx, tol1, tol2)
 
 Review comment:
   since this PR fixes the non-determinism, is it possible to add a test that calculates squaresum twice and assert the output is deterministic? 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services