You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/10/08 05:42:19 UTC
[jira] [Resolved] (SPARK-24431) wrong areaUnderPR calculation in
BinaryClassificationEvaluator
[ https://issues.apache.org/jira/browse/SPARK-24431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-24431.
----------------------------------
Resolution: Incomplete
> wrong areaUnderPR calculation in BinaryClassificationEvaluator
> ---------------------------------------------------------------
>
> Key: SPARK-24431
> URL: https://issues.apache.org/jira/browse/SPARK-24431
> Project: Spark
> Issue Type: Bug
> Components: ML
> Affects Versions: 2.2.0
> Reporter: Xinyong Tian
> Priority: Major
> Labels: bulk-closed
>
> My problem, I am using CrossValidator(estimator=LogisticRegression(...), ..., evaluator=BinaryClassificationEvaluator(metricName='areaUnderPR')) to select best model. when the regParam in logistict regression is very high, no variable will be selected (no model), ie every row 's prediction is same ,eg. equal event rate (baseline frequency). But at this point, BinaryClassificationEvaluator set the areaUnderPR highest. As a result best model seleted is a no model.
> the reason is following. at time of no model, precision recall curve will be only two points: at recall =0, precision should be set to zero , while the software set it to 1. at recall=1, precision is the event rate. As a result, the areaUnderPR will be close 0.5 (my even rate is very low), which is maximum .
> the solution is to set precision =0 when recall =0.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org