[GitHub] [incubator-mxnet] ngupta23 commented on issue #5979: Same prediction values for linear regression while using mxnet in R

[GitBox <gi...@apache.org>]

ngupta23 commented on issue #5979: Same prediction values for linear regression while using mxnet in R
URL: https://github.com/apache/incubator-mxnet/issues/5979#issuecomment-528492660
 
 
   I had a similar problem with all outputs predicting the same value. For my case, there were a couple of things that I had to change to fix this. 
   
   Change the architecture of the neural network (number of neurons in the layers). There was no fixed rule that worked for me. In some cases, when I increased the number of neurons, the predictions were same for all observations. In other cases, when I increased the neurons further, the predictions were better.
   
   What also helped was increasing the number of epochs (num.round). I was initially using the default 10, after increasing it to 100 and above, it gave better results. Maybe 10 epochs was not enough to update the weights enough from the random initialization.
   
   Another thing that impacted the results was the learning rate. Decreasing it too  much (1e-5 for my dataset) caused me to get the same predictions for all observations. I had to keep it at around 1e-3 to make it work.
   
   All the above changes were made orthogonally (make change to a single hyperparameter and observe the change in the predictions). It is possible that changing these hyperparameters simultaneously might lead to other conclusions. But the bottom line is that changing the architecture and hyperparaneter values will solve the issue, just that it might take a while to figure out what is the right range for the hyperparameters.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services