You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/01/07 20:10:58 UTC
[GitHub] [incubator-mxnet] guanxinq opened a new pull request #17242: add
RandomApply in gluon's transforms
guanxinq opened a new pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242
## Description ##
Added RandomApply API in gluon's transforms which applies randomly a list of transformations with a given probability.
Unit test was added.
This fixes https://github.com/apache/incubator-mxnet/issues/17085.
## Checklist ##
### Essentials ###
Please feel free to remove inapplicable items for your PR.
- [ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant [JIRA issue](https://issues.apache.org/jira/projects/MXNET/issues) created (except PRs with tiny changes)
- [X] Changes are complete (i.e. I finished coding on this PR)
- [X] All changes have test coverage:
- Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
- Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
- Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
- [X] Code is well-documented:
- For user-facing API changes, API doc string has been updated.
- For new C++ functions in header files, their functionalities and arguments are documented.
- For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
- Check the API doc at https://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
- [X] To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
### Changes ###
- [ ] Feature1, tests, (and when applicable, API doc)
- [ ] Feature2, tests, (and when applicable, API doc)
## Comments ##
- If this change is a backward incompatible change, why must this change be made.
- Interesting edge cases to note here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363952628
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
+
+ img = mx.nd.ones((245, 480, 3), dtype='uint8')
+ iteration = 1000
+ num_apply = 0
+ for i in range(iteration):
Review comment:
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364355692
##########
File path: python/mxnet/gluon/data/vision/transforms.py
##########
@@ -581,3 +581,28 @@ def hybrid_forward(self, F, x):
if is_np_array():
F = F.npx
return F.image.random_lighting(x, self._alpha)
+
+class RandomApply(Sequential):
+ """Apply a list of transformations randomly given probability
+
+ Parameters
+ ----------
+ Inputs:
+ - **transforms**: list of transformations
+ - **p**: probability
+
+ Outputs:
+ Transformed image.
+ """
+
+ def __init__(self, transform, p=0.5):
Review comment:
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363957815
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
+
+ img = mx.nd.ones((245, 480, 3), dtype='uint8')
Review comment:
It may be not necessary. The test here is to check the frequency of the composed transforms applied. Also, in the new PR, I will reduce the img dimension.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363966064
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364893422
##########
File path: python/mxnet/gluon/data/vision/transforms.py
##########
@@ -581,3 +582,28 @@ def hybrid_forward(self, F, x):
if is_np_array():
F = F.npx
return F.image.random_lighting(x, self._alpha)
+
+
+class RandomApply(Sequential):
+ """Apply a list of transformations randomly given probability
+
+ Parameters
+ ----------
+ Inputs:
+ - **transforms**: list of transformations
+ - **p**: probability
Review comment:
Yes. Updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364388417
##########
File path: python/mxnet/gluon/data/vision/transforms.py
##########
@@ -581,3 +581,28 @@ def hybrid_forward(self, F, x):
if is_np_array():
F = F.npx
return F.image.random_lighting(x, self._alpha)
+
+class RandomApply(Sequential):
+ """Apply a list of transformations randomly given probability
+
+ Parameters
+ ----------
+ Inputs:
+ - **transforms**: list of transformations
+ - **p**: probability
+
+ Outputs:
+ Transformed image.
+ """
+
+ def __init__(self, transforms, p=0.5):
+ super(RandomApply, self).__init__()
+ self.transforms = transforms
+ self.p = p
+
+ def forward(self, x):
+ import random
Review comment:
nit: move this generic import to the beginning of the file
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363945372
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
+
+ img = mx.nd.ones((245, 480, 3), dtype='uint8')
+ iteration = 1000
+ num_apply = 0
+ for i in range(iteration):
Review comment:
replace unused i with _
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363952366
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
+
+ img = mx.nd.ones((245, 480, 3), dtype='uint8')
+ iteration = 1000
+ num_apply = 0
+ for i in range(iteration):
+ out = transform(img)
+ if out.shape[0] == 224:
+ num_apply += 1
+ assert(abs(num_apply/float(iteration)-0.5) < 1e-1)
Review comment:
Sure. Did not notice that threshold could be passed into assert_almost_equal().
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364026031
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,22 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
Review comment:
add an underscore between random and transforms
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest merged pull request #17242: add
RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest merged pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363944967
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
+
+ img = mx.nd.ones((245, 480, 3), dtype='uint8')
+ iteration = 1000
+ num_apply = 0
+ for i in range(iteration):
+ out = transform(img)
+ if out.shape[0] == 224:
+ num_apply += 1
+ assert(abs(num_apply/float(iteration)-0.5) < 1e-1)
Review comment:
Use `assert_almost_equal` from test_util? https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/test_utils.py
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363945779
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
Review comment:
line too long.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364392036
##########
File path: python/mxnet/gluon/data/vision/transforms.py
##########
@@ -581,3 +581,28 @@ def hybrid_forward(self, F, x):
if is_np_array():
F = F.npx
return F.image.random_lighting(x, self._alpha)
+
Review comment:
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363945636
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
+
+ img = mx.nd.ones((245, 480, 3), dtype='uint8')
Review comment:
can you use a for loop to test different shapes and dimensions?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] anirudhacharya commented on a change in pull
request #17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
anirudhacharya commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364580836
##########
File path: python/mxnet/gluon/data/vision/transforms.py
##########
@@ -581,3 +582,28 @@ def hybrid_forward(self, F, x):
if is_np_array():
F = F.npx
return F.image.random_lighting(x, self._alpha)
+
+
+class RandomApply(Sequential):
+ """Apply a list of transformations randomly given probability
+
+ Parameters
+ ----------
+ Inputs:
+ - **transforms**: list of transformations
+ - **p**: probability
Review comment:
- aren't these two supposed to be parameters? also it would be useful to specify the datatype of parameters( eg. float for probability).
- inputs will be the input image( data).
( as a reference please see other transforms in line 562)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364388775
##########
File path: python/mxnet/gluon/data/vision/transforms.py
##########
@@ -581,3 +581,28 @@ def hybrid_forward(self, F, x):
if is_np_array():
F = F.npx
return F.image.random_lighting(x, self._alpha)
+
Review comment:
Add one more blank line between class
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364391526
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,22 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] guanxinq commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
guanxinq commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r363957815
##########
File path: tests/python/unittest/test_gluon_data_vision.py
##########
@@ -229,6 +229,21 @@ def test_transformer():
transform(mx.nd.ones((245, 480, 3), dtype='uint8')).wait_to_read()
+@with_seed()
+def test_randomtransforms():
+ from mxnet.gluon.data.vision import transforms
+
+ transform = transforms.Compose([transforms.RandomApply(transforms.Compose([transforms.Resize(300), transforms.RandomResizedCrop(224)]), 0.5)])
+
+ img = mx.nd.ones((245, 480, 3), dtype='uint8')
Review comment:
It may be not necessary. The test here is to check the frequency of the composed transforms applied. Also, in the new PR, I will use a smaller image.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] apeforest commented on a change in pull request
#17242: add RandomApply in gluon's transforms
Posted by GitBox <gi...@apache.org>.
apeforest commented on a change in pull request #17242: add RandomApply in gluon's transforms
URL: https://github.com/apache/incubator-mxnet/pull/17242#discussion_r364025852
##########
File path: python/mxnet/gluon/data/vision/transforms.py
##########
@@ -581,3 +581,28 @@ def hybrid_forward(self, F, x):
if is_np_array():
F = F.npx
return F.image.random_lighting(x, self._alpha)
+
+class RandomApply(Sequential):
+ """Apply a list of transformations randomly given probability
+
+ Parameters
+ ----------
+ Inputs:
+ - **transforms**: list of transformations
+ - **p**: probability
+
+ Outputs:
+ Transformed image.
+ """
+
+ def __init__(self, transform, p=0.5):
Review comment:
Should the variable be transforms instead?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services