You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/07/30 07:36:35 UTC
[GitHub] [incubator-mxnet] ChaiBapchya opened a new pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
ChaiBapchya opened a new pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825
## Description ##
Tests mx.nd.linalg.gemm operator for large size tensor [not large dimension]
## Results ##
```
python3 -m pytest -s --exitfirst --verbose tests/nightly/test_large_array.py::test_linalg
============================================================================================================================= test session starts ==============================================================================================================================
platform linux -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1 -- /home/ubuntu/anaconda3/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/ubuntu/chai-mxnet/.hypothesis/examples')
rootdir: /home/ubuntu/chai-mxnet
plugins: remotedata-0.3.2, openfiles-0.4.0, astropy-header-0.1.2, hypothesis-5.8.3, arraydiff-0.3, doctestplus-0.5.0
collected 1 item
tests/nightly/test_large_array.py::test_linalg
```
## Comments
- FIxes indent in linalg.gemm2 test
- Removes hardcode & replaces with "perturbation" for gradient checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] access2rohit commented on a change in pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
access2rohit commented on a change in pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825#discussion_r462740140
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
Review comment:
nit: space around '-'
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
+ inp1_grad, inp2_grad, inp3_grad, out= run_gemm(inp1, inp2, inp3)
Review comment:
nit: space before =
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
+ inp1_grad, inp2_grad, inp3_grad, out= run_gemm(inp1, inp2, inp3)
+ assert_almost_equal(out.asnumpy()[0][0][0], MEDIUM_X+perturbation)
+ assert out.shape == inp3.shape
+ out.backward()
+ assert inp1_grad.shape == (MEDIUM_X, SMALL_Y, MEDIUM_X)
+ assert inp2_grad.shape == (MEDIUM_X, SMALL_Y, MEDIUM_X)
+ assert inp3_grad.shape == (MEDIUM_X, SMALL_Y, SMALL_Y)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0][0], SMALL_Y-(1-perturbation))
Review comment:
nit: space around '-'
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
+ inp1_grad, inp2_grad, inp3_grad, out= run_gemm(inp1, inp2, inp3)
Review comment:
still
```
out= run_gemm
```
can you do ?
```
out = run_gemm
```
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y - (1 - perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
Review comment:
spaces around `=`
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y - (1 - perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
Review comment:
spaces around `=`
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y - (1 - perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
Review comment:
same
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
+ inp1_grad, inp2_grad, inp3_grad, out= run_gemm(inp1, inp2, inp3)
Review comment:
```
out= run_gemm
```
to
```
out = run_gemm
```
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
+ inp1_grad, inp2_grad, inp3_grad, out= run_gemm(inp1, inp2, inp3)
Review comment:
```
out= run_gemm
```
to
```
out = run_gemm
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] access2rohit commented on pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
access2rohit commented on pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825#issuecomment-666119663
left nitpicky comments. Rest LGTM !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] ChaiBapchya commented on pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825#issuecomment-666212039
@mxnet-bot run ci [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] ChaiBapchya commented on a change in pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on a change in pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825#discussion_r462741389
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
Review comment:
Addressed
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
+ inp1_grad, inp2_grad, inp3_grad, out= run_gemm(inp1, inp2, inp3)
Review comment:
Addressed
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y-(1-perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
+ inp1_grad, inp2_grad, inp3_grad, out= run_gemm(inp1, inp2, inp3)
+ assert_almost_equal(out.asnumpy()[0][0][0], MEDIUM_X+perturbation)
+ assert out.shape == inp3.shape
+ out.backward()
+ assert inp1_grad.shape == (MEDIUM_X, SMALL_Y, MEDIUM_X)
+ assert inp2_grad.shape == (MEDIUM_X, SMALL_Y, MEDIUM_X)
+ assert inp3_grad.shape == (MEDIUM_X, SMALL_Y, SMALL_Y)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0][0], SMALL_Y-(1-perturbation))
Review comment:
Addressed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] ChaiBapchya commented on a change in pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on a change in pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825#discussion_r462822933
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y - (1 - perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
Review comment:
It's incorrect according to PEP8 style guide Python
https://www.python.org/dev/peps/pep-0008/
<img width="895" alt="Screen Shot 2020-07-30 at 1 05 56 AM" src="https://user-images.githubusercontent.com/10992635/88897924-e7dbdc00-d200-11ea-987c-e36c6ac58fde.png">
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y - (1 - perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ perturbation = 0.2
+ inp1[0][0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
+ inp3 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, SMALL_Y))
Review comment:
It's incorrect according to PEP8 style guide Python
https://www.python.org/dev/peps/pep-0008/
<img width="895" alt="Screen Shot 2020-07-30 at 1 05 56 AM" src="https://user-images.githubusercontent.com/10992635/88897924-e7dbdc00-d200-11ea-987c-e36c6ac58fde.png">
##########
File path: tests/nightly/test_large_array.py
##########
@@ -1211,23 +1211,49 @@ def check_syrk_batch():
assert_almost_equal(A.grad[1,0,0], nd.array([0.4]), rtol=1e-3, atol=1e-5)
def check_gemm2():
- def run_gemm2(inp1,inp2):
+ def run_gemm2(inp1, inp2):
inp1.attach_grad()
inp2.attach_grad()
with mx.autograd.record():
- out = mx.nd.linalg.gemm2(inp1,inp2)
+ out = mx.nd.linalg.gemm2(inp1, inp2)
return inp1.grad, inp2.grad, out
- inp1=mx.nd.ones(shape=(SMALL_Y, LARGE_X))
- inp1[0][0]=0.1
- inp2=mx.nd.ones(shape=(LARGE_X, SMALL_Y))
- inp1_grad, inp2_grad, out= run_gemm2(inp1,inp2)
+ inp1 = mx.nd.ones(shape=(SMALL_Y, LARGE_X))
+ perturbation = 0.2
+ inp1[0][0] = perturbation
+ inp2 = mx.nd.ones(shape=(LARGE_X, SMALL_Y))
+ inp1_grad, inp2_grad, out = run_gemm2(inp1, inp2)
assert out.asnumpy()[0][0] == LARGE_X
assert out.shape == (SMALL_Y, SMALL_Y)
out.backward()
assert inp1_grad.shape == (SMALL_Y, LARGE_X)
assert inp2_grad.shape == (LARGE_X, SMALL_Y)
- assert_almost_equal(inp2_grad.asnumpy()[0][0],49.1)
+ assert_almost_equal(inp1_grad.asnumpy()[0][0], SMALL_Y)
+ assert_almost_equal(inp2_grad.asnumpy()[0][0], SMALL_Y - (1 - perturbation))
+
+ def check_gemm():
+ def run_gemm(inp1,inp2, inp3):
+ inp1.attach_grad()
+ inp2.attach_grad()
+ inp3.attach_grad()
+ with mx.autograd.record():
+ out = mx.nd.linalg.gemm(inp1, inp2, inp3, transpose_b=True)
+ return inp1.grad, inp2.grad, inp3.grad, out
+
+ inp1 = mx.nd.ones(shape=(MEDIUM_X, SMALL_Y, MEDIUM_X))
Review comment:
It's incorrect according to PEP8 style guide Python
https://www.python.org/dev/peps/pep-0008/
<img width="895" alt="Screen Shot 2020-07-30 at 1 05 56 AM" src="https://user-images.githubusercontent.com/10992635/88898201-3db08400-d201-11ea-8296-8dbdf9922569.png">
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] sandeep-krishnamurthy merged pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
sandeep-krishnamurthy merged pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825#issuecomment-666212089
Jenkins CI successfully triggered : [centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #18825: [v1.x][LT] Add forward & backward linalg.gemm test for large size
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #18825:
URL: https://github.com/apache/incubator-mxnet/pull/18825#issuecomment-666015006
Hey @ChaiBapchya , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [website, windows-cpu, sanity, centos-cpu, edge, miscellaneous, unix-cpu, clang, centos-gpu, windows-gpu, unix-gpu]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org