[GitHub] piiswrong commented on a change in pull request #7304: [WIP] gluon bce & ctc losses

[gi...@git.apache.org]

piiswrong commented on a change in pull request #7304: [WIP] gluon bce & ctc losses
URL: https://github.com/apache/incubator-mxnet/pull/7304#discussion_r132318239
 
 

 ##########
 File path: python/mxnet/gluon/loss.py
 ##########
 @@ -148,13 +150,48 @@ def __init__(self, weight=None, batch_axis=0, **kwargs):
         super(L1Loss, self).__init__(weight, batch_axis, **kwargs)
 
     def hybrid_forward(self, F, output, label, sample_weight=None):
-        if F is ndarray:
-            loss = ndarray.abs(output - label.reshape(output.shape))
+        label = _reshape_label_as_output(F, output, label)
+        loss = F.abs(output - label)
+        loss = _apply_weighting(F, loss, self._weight, sample_weight)
+        return F.mean(loss, axis=self._batch_axis, exclude=True)
+
+
+class BinaryCrossEntropyLoss(Loss):
+    r"""The cross-entropy loss for binary classification.
+
+    BCE loss is useful when training logistic regression.
+
+    .. math::
+        loss(o, t) = - 1/n \sum_i (t[i] * log(o[i]) + (1 - t[i]) * log(1 - o[i]))
+
+
+    Parameters
+    ----------
+    from_sigmoid : bool, default is `False`
+        Whether the input is from the output of sigmoid. Set this to false will make
+        the loss calculate sigmoid and then BCE, which is more numerically stable through
+        log-sum-exp trick.
+    weight : float or None
+        Global scalar weight for loss.
+    sample_weight : Symbol or None
+        Per sample weighting. Must be broadcastable to
+        the same shape as loss. For example, if loss has
+        shape (64, 10) and you want to weight each sample
+        in the batch, `sample_weight` should have shape (64, 1).
+    batch_axis : int, default 0
+        The axis that represents mini-batch.
+    """
+    def __init__(self, from_sigmoid=False, weight=None, batch_axis=0, **kwargs):
+        super(BinaryCrossEntropyLoss, self).__init__(weight, batch_axis, **kwargs)
+        self._from_sigmoid = from_sigmoid
+
+    def hybrid_forward(self, F, output, label, sample_weight=None):
+        label = _reshape_label_as_output(F, output, label)
+        if not self._from_sigmoid:
+            max_val = F.maximum(-output, 0)
+            loss = output - output*label + max_val + F.log(F.exp(-max_val)+F.exp(-output-max_val))
 
 Review comment:
   add test to compare against softmaxcrossentropy with 2 classes
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services