[GitHub] [incubator-mxnet] acphile commented on issue #18046: Proposal to mxnet.metric

[GitBox <gi...@apache.org>]

acphile commented on issue #18046:
URL: https://github.com/apache/incubator-mxnet/issues/18046#issuecomment-619826986


   Here are the updated changes to be made:
   ### 1. improve Class MAE, MSE, RMSE
   a. **UPD: remove “macro” supports which represents average per batch**
   b. Rewrite RMSE to inherit from MSE
   ### 2. improve Class _BinaryClassification
   a. **UPD: including parameter “class_type” in [‘binary’, ‘multiclass’, ‘multilabel’]**
   b. support the situation len(pred.shape)==1 for class_type='binary'
        i. for binary classification, we only need to output a confidence score of being positive, like: pred=[0.1,0.3,0.7] or like pred=[[0.1],[0.3],[0.7]]
   c. including parameter “threshold”, default: threshold=0.5
        i. sometimes we may need to define a threshold that when confidence(positive) > threshold, we classify it as positive, otherwise negative
        ii. used when class_type in [‘binary’, ‘multilabel’]
   d. including parameter “beta” default: beta=1
        i. updating “fscore” calculation with F-beta= (1+beta^2)\*precision\*recall/(beta^2\*precision+recall), which is more general
   e. **UPD: add cases for multillabel/multiclass**
        i. including paramater ‘class_type’ in [‘binary’, ‘multilabel’, ‘multiclass’]
        ii. For ‘multilabel’, pred should be (N, ..., C) and label should be (N, ..., C)
        iii. For ‘multiclass’, pred should be (N, ..., C) and label should be (N, ...)
   f. **UPD: replace global_fscore with micro_fscore**
   ### 3. add Class BinaryAccuracy(threshold=0.5)
   ### 4. add Class MeanCosineSimilarity(axis=-1, eps=1e-12)
   ### 5. add Class MeanPairwiseDistance(p=2)
   ### 6. improve Class F1:
   a. F1(class_type="binary", threshold=0.5, average="micro")
   b. **average in [“binary”, “micro”, “macro”]:**
        i. "macro": Calculate metrics for each label and return unweighted mean of f1.
        ii. "micro": Calculate metrics globally by counting the total TP, FN and FP.
        iii. None: Return f1 scores for each class (numpy.ndarray) .
   ### 7. add Class Fbeta(class_type="binary", beta=1, threshold=0.5, average="micro")
   ### 8. **UPD: using mxnet.numpy instead of numpy**


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org