You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by zh...@apache.org on 2018/08/22 21:42:01 UTC
[incubator-mxnet] branch master updated: fixed broken links (#12293)
This is an automated email from the ASF dual-hosted git repository.
zhasheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-mxnet.git
The following commit(s) were added to refs/heads/master by this push:
new adc57d1 fixed broken links (#12293)
adc57d1 is described below
commit adc57d188239da19eb5da23807c256f22ba4f931
Author: Aaron Markham <ma...@amazon.com>
AuthorDate: Wed Aug 22 14:41:50 2018 -0700
fixed broken links (#12293)
---
.../gluon/logistic_regression_explained.md | 54 +++++++++++-----------
1 file changed, 27 insertions(+), 27 deletions(-)
diff --git a/docs/tutorials/gluon/logistic_regression_explained.md b/docs/tutorials/gluon/logistic_regression_explained.md
index 8e5e4a5..577a914 100644
--- a/docs/tutorials/gluon/logistic_regression_explained.md
+++ b/docs/tutorials/gluon/logistic_regression_explained.md
@@ -55,9 +55,9 @@ val_dataloader = DataLoader(val_dataset, batch_size=batch_size, shuffle=True)
## Defining and training the model
-The only requirement for the logistic regression is that the last layer of the network must be a single neuron. Apache MXNet allows us to do so by using [Dense](https://mxnet.incubator.apache.org/api/python/gluon/nn.html#mxnet.gluon.nn.Dense) layer and specifying the number of units to 1. The rest of the network can be arbitrarily complex.
+The only requirement for the logistic regression is that the last layer of the network must be a single neuron. Apache MXNet allows us to do so by using [Dense](https://mxnet.incubator.apache.org/api/python/gluon/nn.html#mxnet.gluon.nn.Dense) layer and specifying the number of units to 1. The rest of the network can be arbitrarily complex.
-Below, we define a model which has an input layer of 10 neurons, a couple of inner layers of 10 neurons each, and output layer of 1 neuron. We stack the layers using [HybridSequential](https://mxnet.incubator.apache.org/api/python/gluon/gluon.html#mxnet.gluon.nn.HybridSequential) block and initialize parameters of the network using [Xavier](https://mxnet.incubator.apache.org/api/python/optimization/optimization.html#mxnet.initializer.Xavier) initialization.
+Below, we define a model which has an input layer of 10 neurons, a couple of inner layers of 10 neurons each, and output layer of 1 neuron. We stack the layers using [HybridSequential](https://mxnet.incubator.apache.org/api/python/gluon/gluon.html#mxnet.gluon.nn.HybridSequential) block and initialize parameters of the network using [Xavier](https://mxnet.incubator.apache.org/api/python/optimization/optimization.html#mxnet.initializer.Xavier) initialization.
```python
@@ -78,14 +78,14 @@ Loss function is used to calculate how the output of the network differs from th
Trainer object allows to specify the method of training to be used. For our tutorial we use [Stochastic Gradient Descent (SGD)](https://mxnet.incubator.apache.org/api/python/optimization/optimization.html#mxnet.optimizer.SGD). For more information on SGD refer to [the following tutorial](https://gluon.mxnet.io/chapter06_optimization/gd-sgd-scratch.html). We also need to parametrize it with learning rate value, which defines the weight updates, and weight decay, which is used for regularization.
-Metric helps us to estimate how good our model is in terms of a problem we are trying to solve. Where loss function has more importance for the training process, a metric is usually the thing we are trying to improve and reach maximum value. We also can use more than one metric, to measure various aspects of our model. In our example, we are using [Accuracy](https://mxnet.incubator.apache.org/api/python/model.html#mxnet.metric.Accuracy) and [F1 score](https://mxnet.incubator.apache.org/a [...]
+Metric helps us to estimate how good our model is in terms of a problem we are trying to solve. Where loss function has more importance for the training process, a metric is usually the thing we are trying to improve and reach maximum value. We also can use more than one metric, to measure various aspects of our model. In our example, we are using [Accuracy](https://mxnet.incubator.apache.org/api/python/metric/metric.html?highlight=metric.acc#mxnet.metric.Accuracy) and [F1 score](http:// [...]
Below we define these objects.
```python
loss = gluon.loss.SigmoidBinaryCrossEntropyLoss()
-trainer = Trainer(params=net.collect_params(), optimizer='sgd',
+trainer = Trainer(params=net.collect_params(), optimizer='sgd',
optimizer_params={'learning_rate': 0.1})
accuracy = mx.metric.Accuracy()
f1 = mx.metric.F1()
@@ -97,16 +97,16 @@ The next step is to define the training function in which we iterate over all ba
```python
def train_model():
cumulative_train_loss = 0
-
+
for i, (data, label) in enumerate(train_dataloader):
with autograd.record():
# Do forward pass on a batch of training data
output = net(data)
-
+
# Calculate loss for the training data batch
loss_result = loss(output, label)
- # Calculate gradients
+ # Calculate gradients
loss_result.backward()
# Update parameters of the network
@@ -114,15 +114,15 @@ def train_model():
# sum losses of every batch
cumulative_train_loss += nd.sum(loss_result).asscalar()
-
+
return cumulative_train_loss
```
## Validating the model
-Our validation function is very similar to the training one. The main difference is that we want to calculate accuracy of the model. We use [Accuracy metric](https://mxnet.incubator.apache.org/api/python/model.html#mxnet.metric.Accuracy) to do so.
+Our validation function is very similar to the training one. The main difference is that we want to calculate accuracy of the model. We use [Accuracy metric](https://mxnet.incubator.apache.org/api/python/metric/metric.html?highlight=metric.acc#mxnet.metric.Accuracy) to do so.
-`Accuracy` metric requires 2 arguments: 1) a vector of ground-truth classes and 2) A vector or matrix of predictions. When predictions are of the same shape as the vector of ground-truth classes, `Accuracy` class assumes that prediction vector contains predicted classes. So, it converts the vector to `Int32` and compare each item of ground-truth classes to prediction vector.
+`Accuracy` metric requires 2 arguments: 1) a vector of ground-truth classes and 2) A vector or matrix of predictions. When predictions are of the same shape as the vector of ground-truth classes, `Accuracy` class assumes that prediction vector contains predicted classes. So, it converts the vector to `Int32` and compare each item of ground-truth classes to prediction vector.
Because of the behaviour above, you will get an unexpected result if you just apply [Sigmoid](https://mxnet.incubator.apache.org/api/python/ndarray/ndarray.html#mxnet.ndarray.sigmoid) function to the network result and pass it to `Accuracy` metric. As mentioned before, we need to apply `Sigmoid` function to the output of the neuron to get a probability of belonging to the class 1. But `Sigmoid` function produces output in range [0; 1], and all numbers in that range are going to be casted [...]
@@ -146,27 +146,27 @@ Then we pass this stacked matrix to `F1` score.
```python
def validate_model(threshold):
cumulative_val_loss = 0
-
+
for i, (val_data, val_ground_truth_class) in enumerate(val_dataloader):
# Do forward pass on a batch of validation data
output = net(val_data)
-
+
# Similar to cumulative training loss, calculate cumulative validation loss
cumulative_val_loss += nd.sum(loss(output, val_ground_truth_class)).asscalar()
-
+
# getting prediction as a sigmoid
prediction = net(val_data).sigmoid()
-
+
# Converting neuron outputs to classes
predicted_classes = mx.nd.ceil(prediction - threshold)
-
+
# Update validation accuracy
- accuracy.update(val_ground_truth_class, predicted_classes.reshape(-1))
-
+ accuracy.update(val_ground_truth_class, predicted_classes.reshape(-1))
+
# calculate probabilities of belonging to different classes. F1 metric works only with this notation
prediction = prediction.reshape(-1)
probabilities = mx.nd.stack(1 - prediction, prediction, axis=1)
-
+
f1.update(val_ground_truth_class, probabilities)
return cumulative_val_loss
@@ -184,8 +184,8 @@ threshold = 0.5
for e in range(epochs):
avg_train_loss = train_model() / train_data_size
avg_val_loss = validate_model(threshold) / val_data_size
-
- print("Epoch: %s, Training loss: %.2f, Validation loss: %.2f, Validation accuracy: %.2f, F1 score: %.2f" %
+
+ print("Epoch: %s, Training loss: %.2f, Validation loss: %.2f, Validation accuracy: %.2f, F1 score: %.2f" %
(e, avg_train_loss, avg_val_loss, accuracy.get()[1], f1.get()[1]))
# we reset accuracy, so the new epoch's accuracy would be calculated from the blank state
@@ -203,13 +203,13 @@ for e in range(epochs):
Epoch: 4, Training loss: 0.06, Validation loss: 0.09, Validation accuracy: 0.97, F1 score: 0.58 <!--notebook-skip-line-->
Epoch: 5, Training loss: 0.04, Validation loss: 0.12, Validation accuracy: 0.97, F1 score: 0.59 <!--notebook-skip-line-->
-
+
Epoch: 6, Training loss: 0.05, Validation loss: 0.09, Validation accuracy: 0.99, F1 score: 0.62 <!--notebook-skip-line-->
-
+
Epoch: 7, Training loss: 0.05, Validation loss: 0.10, Validation accuracy: 0.97, F1 score: 0.62 <!--notebook-skip-line-->
-
+
Epoch: 8, Training loss: 0.05, Validation loss: 0.12, Validation accuracy: 0.95, F1 score: 0.63 <!--notebook-skip-line-->
-
+
Epoch: 9, Training loss: 0.04, Validation loss: 0.09, Validation accuracy: 0.98, F1 score: 0.65 <!--notebook-skip-line-->
@@ -217,7 +217,7 @@ In our case we hit the accuracy of 0.98 and F1 score of 0.65.
## Tip 1: Use only one neuron in the output layer
-Despite that there are 2 classes, there should be only one output neuron, because `SigmoidBinaryCrossEntropyLoss` accepts only one feature as an input.
+Despite that there are 2 classes, there should be only one output neuron, because `SigmoidBinaryCrossEntropyLoss` accepts only one feature as an input.
## Tip 2: Encode classes as 0 and 1
@@ -225,7 +225,7 @@ For `SigmoidBinaryCrossEntropyLoss` to work it is required that classes were enc
## Tip 3: Use SigmoidBinaryCrossEntropyLoss instead of LogisticRegressionOutput
-NDArray API has two options to calculate logistic regression loss: [SigmoidBinaryCrossEntropyLoss](https://mxnet.incubator.apache.org/api/python/gluon/loss.html#mxnet.gluon.loss.SigmoidBinaryCrossEntropyLoss) and [LogisticRegressionOutput](https://mxnet.incubator.apache.org/api/python/ndarray/ndarray.html#mxnet.ndarray.LogisticRegressionOutput). `LogisticRegressionOutput` is designed to be an output layer when using the Module API, and is not supposed to be used when using Gluon API.
+NDArray API has two options to calculate logistic regression loss: [SigmoidBinaryCrossEntropyLoss](https://mxnet.incubator.apache.org/api/python/gluon/loss.html#mxnet.gluon.loss.SigmoidBinaryCrossEntropyLoss) and [LogisticRegressionOutput](https://mxnet.incubator.apache.org/api/python/ndarray/ndarray.html#mxnet.ndarray.LogisticRegressionOutput). `LogisticRegressionOutput` is designed to be an output layer when using the Module API, and is not supposed to be used when using Gluon API.
## Conclusion
@@ -235,4 +235,4 @@ In this tutorial I explained some potential pitfalls to be aware of. When doing
1. Use `SigmoidBinaryCrossEntropyLoss`
1. Convert probabilities to classes before calculating Accuracy
-<!-- INSERT SOURCE DOWNLOAD BUTTONS -->
\ No newline at end of file
+<!-- INSERT SOURCE DOWNLOAD BUTTONS -->