You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@singa.apache.org by GitBox <gi...@apache.org> on 2020/01/30 10:05:35 UTC
[GitHub] [singa] joddiy opened a new pull request #586: Add GlobalAVGPool
operator for autograd and onnx
joddiy opened a new pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586
Add GlobalAVGPool operator for autograd and onnx
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] joddiy commented on a change in pull request #586: Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#discussion_r372985947
##########
File path: python/singa/autograd.py
##########
@@ -2760,3 +2760,42 @@ def backward(self, dy):
def reciprocal(x):
return Reciprocal()(x)[0]
+
+
+class GlobalAveragePool(Operation):
+ def __init__(self, data_format='channels_first'):
+ super(GlobalAveragePool, self).__init__()
+ self.data_format = data_format
+
+ def forward(self, x):
+ if training:
+ self.mask = singa.Tensor(list(x.shape()), x.device())
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] nudles commented on a change in pull request #586: Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#discussion_r372954175
##########
File path: test/python/test_operation.py
##########
@@ -2745,5 +2745,97 @@ def test_prelu_broadcast_cpu(self):
np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx0)), grad0, decimal=5)
np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx1)), grad1, decimal=5)
+ def test_globalaveragepool_cpu(self):
+ X = np.array([[[
+ [1, 2, 3],
+ [4, 5, 6],
+ [7, 8, 9],
+ ]]]).astype(np.float32)
+ XT = np.array([[[[5]]]]).astype(np.float32)
+ DY = np.ones((1, 1, 1, 1), dtype=np.float32)
+
+ x = tensor.from_numpy(X)
+ x.to_device(cpu_dev)
+ dy = tensor.from_numpy(DY)
+ dy.to_device(cpu_dev)
+
+ result = autograd.globalaveragepool(x)
+ dx = result.creator.backward(dy.data)
+
+ DX = np.ones(X.shape, dtype=np.float32)
+ DX = np.multiply(DX, DY) / np.prod(X.shape[2:])
+
+ np.testing.assert_array_almost_equal(tensor.to_numpy(result), XT, decimal=5)
+ np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx)), DX, decimal=5)
+
+ def test_globalaveragepool_cpu_channel_last(self):
+ X = np.array([[
+ [[1], [2], [3]],
+ [[4], [5], [6]],
+ [[7], [8], [9]],
+ ]]).astype(np.float32)
+ XT = np.array([[[[5]]]]).astype(np.float32)
+ DY = np.ones((1, 1, 1, 1), dtype=np.float32)
+
+ x = tensor.from_numpy(X)
+ x.to_device(cpu_dev)
+ dy = tensor.from_numpy(DY)
+ dy.to_device(cpu_dev)
+
+ result = autograd.globalaveragepool(x, 'channel_last')
+ dx = result.creator.backward(dy.data)
+
+ DX = np.ones(X.shape, dtype=np.float32)
+ DX = np.multiply(DX, DY) / np.prod(X.shape[1:-1])
+
+ np.testing.assert_array_almost_equal(tensor.to_numpy(result), XT, decimal=5)
+ np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx)), DX, decimal=5)
+
+ def test_globalaveragepool_gpu(self):
+ X = np.array([[[
Review comment:
can you merge the code for gpu and cpu test?
many lines are repeated..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] nudles merged pull request #586: SINGA-503 Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
nudles merged pull request #586: SINGA-503 Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] nudles commented on a change in pull request #586: Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#discussion_r372954319
##########
File path: python/singa/autograd.py
##########
@@ -2760,3 +2760,42 @@ def backward(self, dy):
def reciprocal(x):
return Reciprocal()(x)[0]
+
+
+class GlobalAveragePool(Operation):
+ def __init__(self, data_format='channels_first'):
+ super(GlobalAveragePool, self).__init__()
+ self.data_format = data_format
+
+ def forward(self, x):
+ if training:
+ self.mask = singa.Tensor(list(x.shape()), x.device())
Review comment:
can remove list()?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] joddiy commented on issue #586: Add GlobalAVGPool operator
for autograd and onnx
Posted by GitBox <gi...@apache.org>.
joddiy commented on issue #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#issuecomment-580290087
> need to add some docstrings for the new operators.
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] nudles commented on a change in pull request #586: Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#discussion_r372953025
##########
File path: python/singa/autograd.py
##########
@@ -2760,3 +2760,42 @@ def backward(self, dy):
def reciprocal(x):
return Reciprocal()(x)[0]
+
+
+class GlobalAveragePool(Operation):
+ def __init__(self, data_format='channels_first'):
+ super(GlobalAveragePool, self).__init__()
+ self.data_format = data_format
+
+ def forward(self, x):
+ if training:
+ self.mask = singa.Tensor(list(x.shape()), x.device())
+
+ shape = list(x.shape())
+
+ # (N x C x H x W) for channels_first
+ if self.data_format == 'channels_first':
+ axes = tuple(i for i in range(2, len(shape)))
+ self.shape_divisor = 1/np.prod(shape[2:])
+ else: # (N x H x W x C) for channels_last
+ axes = tuple(i for i in range(1, len(shape)-1))
+ self.shape_divisor = 1/np.prod(shape[1:-1])
+
+ # output shape
+ # (N x C x 1 x 1) for channels_first
+ # (N x 1 x 1 x C) for channels_last
+ for i in axes:
+ shape[i] = 1
+
+ x = tensor.from_raw_tensor(x)
+ x = tensor.sum(x, axis=axes)
Review comment:
the raw tensor should also have the sum function?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] nudles commented on issue #586: Add GlobalAVGPool operator
for autograd and onnx
Posted by GitBox <gi...@apache.org>.
nudles commented on issue #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#issuecomment-580259721
need to add some docstrings for the new operators.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] joddiy commented on a change in pull request #586: Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#discussion_r372985903
##########
File path: test/python/test_operation.py
##########
@@ -2745,5 +2745,97 @@ def test_prelu_broadcast_cpu(self):
np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx0)), grad0, decimal=5)
np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx1)), grad1, decimal=5)
+ def test_globalaveragepool_cpu(self):
+ X = np.array([[[
+ [1, 2, 3],
+ [4, 5, 6],
+ [7, 8, 9],
+ ]]]).astype(np.float32)
+ XT = np.array([[[[5]]]]).astype(np.float32)
+ DY = np.ones((1, 1, 1, 1), dtype=np.float32)
+
+ x = tensor.from_numpy(X)
+ x.to_device(cpu_dev)
+ dy = tensor.from_numpy(DY)
+ dy.to_device(cpu_dev)
+
+ result = autograd.globalaveragepool(x)
+ dx = result.creator.backward(dy.data)
+
+ DX = np.ones(X.shape, dtype=np.float32)
+ DX = np.multiply(DX, DY) / np.prod(X.shape[2:])
+
+ np.testing.assert_array_almost_equal(tensor.to_numpy(result), XT, decimal=5)
+ np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx)), DX, decimal=5)
+
+ def test_globalaveragepool_cpu_channel_last(self):
+ X = np.array([[
+ [[1], [2], [3]],
+ [[4], [5], [6]],
+ [[7], [8], [9]],
+ ]]).astype(np.float32)
+ XT = np.array([[[[5]]]]).astype(np.float32)
+ DY = np.ones((1, 1, 1, 1), dtype=np.float32)
+
+ x = tensor.from_numpy(X)
+ x.to_device(cpu_dev)
+ dy = tensor.from_numpy(DY)
+ dy.to_device(cpu_dev)
+
+ result = autograd.globalaveragepool(x, 'channel_last')
+ dx = result.creator.backward(dy.data)
+
+ DX = np.ones(X.shape, dtype=np.float32)
+ DX = np.multiply(DX, DY) / np.prod(X.shape[1:-1])
+
+ np.testing.assert_array_almost_equal(tensor.to_numpy(result), XT, decimal=5)
+ np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx)), DX, decimal=5)
+
+ def test_globalaveragepool_gpu(self):
+ X = np.array([[[
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] nudles commented on a change in pull request #586: Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#discussion_r372956298
##########
File path: test/python/test_operation.py
##########
@@ -2745,5 +2745,97 @@ def test_prelu_broadcast_cpu(self):
np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx0)), grad0, decimal=5)
np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx1)), grad1, decimal=5)
+ def test_globalaveragepool_cpu(self):
+ X = np.array([[[
+ [1, 2, 3],
+ [4, 5, 6],
+ [7, 8, 9],
+ ]]]).astype(np.float32)
+ XT = np.array([[[[5]]]]).astype(np.float32)
+ DY = np.ones((1, 1, 1, 1), dtype=np.float32)
+
+ x = tensor.from_numpy(X)
+ x.to_device(cpu_dev)
+ dy = tensor.from_numpy(DY)
+ dy.to_device(cpu_dev)
+
+ result = autograd.globalaveragepool(x)
+ dx = result.creator.backward(dy.data)
+
+ DX = np.ones(X.shape, dtype=np.float32)
+ DX = np.multiply(DX, DY) / np.prod(X.shape[2:])
+
+ np.testing.assert_array_almost_equal(tensor.to_numpy(result), XT, decimal=5)
+ np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx)), DX, decimal=5)
+
+ def test_globalaveragepool_cpu_channel_last(self):
+ X = np.array([[
+ [[1], [2], [3]],
+ [[4], [5], [6]],
+ [[7], [8], [9]],
+ ]]).astype(np.float32)
+ XT = np.array([[[[5]]]]).astype(np.float32)
+ DY = np.ones((1, 1, 1, 1), dtype=np.float32)
+
+ x = tensor.from_numpy(X)
+ x.to_device(cpu_dev)
+ dy = tensor.from_numpy(DY)
+ dy.to_device(cpu_dev)
+
+ result = autograd.globalaveragepool(x, 'channel_last')
+ dx = result.creator.backward(dy.data)
+
+ DX = np.ones(X.shape, dtype=np.float32)
+ DX = np.multiply(DX, DY) / np.prod(X.shape[1:-1])
+
+ np.testing.assert_array_almost_equal(tensor.to_numpy(result), XT, decimal=5)
+ np.testing.assert_array_almost_equal(tensor.to_numpy(tensor.from_raw_tensor(dx)), DX, decimal=5)
+
+ def test_globalaveragepool_gpu(self):
+ X = np.array([[[
Review comment:
like this [test](https://github.com/apache/singa/pull/515/files#diff-c0a014dc6b3a07acd5e7c10001de82c9R1483)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa] joddiy commented on a change in pull request #586: Add
GlobalAVGPool operator for autograd and onnx
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #586: Add GlobalAVGPool operator for autograd and onnx
URL: https://github.com/apache/singa/pull/586#discussion_r372986616
##########
File path: python/singa/autograd.py
##########
@@ -2760,3 +2760,42 @@ def backward(self, dy):
def reciprocal(x):
return Reciprocal()(x)[0]
+
+
+class GlobalAveragePool(Operation):
+ def __init__(self, data_format='channels_first'):
+ super(GlobalAveragePool, self).__init__()
+ self.data_format = data_format
+
+ def forward(self, x):
+ if training:
+ self.mask = singa.Tensor(list(x.shape()), x.device())
+
+ shape = list(x.shape())
+
+ # (N x C x H x W) for channels_first
+ if self.data_format == 'channels_first':
+ axes = tuple(i for i in range(2, len(shape)))
+ self.shape_divisor = 1/np.prod(shape[2:])
+ else: # (N x H x W x C) for channels_last
+ axes = tuple(i for i in range(1, len(shape)-1))
+ self.shape_divisor = 1/np.prod(shape[1:-1])
+
+ # output shape
+ # (N x C x 1 x 1) for channels_first
+ # (N x 1 x 1 x C) for channels_last
+ for i in axes:
+ shape[i] = 1
+
+ x = tensor.from_raw_tensor(x)
+ x = tensor.sum(x, axis=axes)
Review comment:
it seems the singa.Sum does not support sum over axis larger than 1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services