You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by gi...@git.apache.org on 2017/08/23 18:04:44 UTC
[GitHub] thomasmooon commented on issue #7490: How to train with input layer dropout?
thomasmooon commented on issue #7490: How to train with input layer dropout?
URL: https://github.com/apache/incubator-mxnet/issues/7490#issuecomment-324415949
Unfortunately this is getting frustrating in R, mainly by the limited documentation.
Using python this seems easy and straightforward: http://mxnet.io/api/python/symbol.html#mxnet.symbol.Dropout
I rewrote the net using symbolic language and tested it by adding `mx.symbol.Dropout()` in 2nd line.
But this has no effect.
The commented lines is how it's done in python. Issue #1008 doesn't help either.
Beside of that [#3821](https://github.com/apache/incubator-mxnet/issues/3821) considers only "prediction with dropout on" and not "training with dropout on"
```
create_drop_net <- function(num_hidden=500, p = 0.2, is.train) {
data <- mx.symbol.Variable("data")
dropout <- mx.symbol.Dropout(data, p=p)
# exec <- mx.simple.bind(data = data)
# mx.exec.forward(exec, is.train = is.train)
label <- mx.symbol.Variable('label')
fc1 <- mx.symbol.FullyConnected(data, name = "fc1", num_hidden = num_hidden)
act1 <- mx.symbol.Activation(fc1, name = "relu1", act_type = "relu")
fc2 <- mx.symbol.FullyConnected(act1, name = "fc2", num_hidden = num_hidden)
act2 <- mx.symbol.Activation(fc2, name = "relu2", act_type = "relu")
fc3 <- mx.symbol.FullyConnected(act2, name = "fc3", num_hidden = num_hidden)
act3 <- mx.symbol.Activation(fc3, name = "relu3", act_type = "relu")
recon <- mx.symbol.FullyConnected(act3, name = "reconstruction", num_hidden = nrow(train))
net <- mx.symbol.LinearRegressionOutput(data = recon, label = label)
return(net)
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services