You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by robert-dodier <gi...@git.apache.org> on 2015/03/14 03:19:36 UTC
[GitHub] spark pull request: [SPARK-6332] [MLlib] compute calibration curve...
GitHub user robert-dodier opened a pull request:
https://github.com/apache/spark/pull/5025
[SPARK-6332] [MLlib] compute calibration curve for binary classifier
This PR contains an implementation of a calibration method in the class BinaryClassificationMetrics. The code was adapted from the method for ROC curve construction. Tests on small data sets have been added to BinaryClassificationMetricsSuite, and the current version of the code passes those tests.
In this implementation, the return value of the new method is an RDD[((Double, Double), (Double, Long))]. The first pair describes each bin and the second pair describes the content of each bin. In the first pair, the two values are the least and greatest scores in the bin. In the second pair, the two values are the proportion of positive examples in the bin, and the number of examples in the bin. I chose this representation in order to keep as much information as possible. However, a simpler representation might be better; let's talk about that if anyone is interested.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/robert-dodier/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5025.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5025
----
commit 1450509ef32eae77484cb1a85e55c0d3943e415f
Author: Robert Dodier <ro...@users.sourceforge.net>
Date: 2015-03-13T17:53:39Z
Change command line arguments in order to get tests to run.
commit d1aacfb084bcccb68e9c04ac7ec835c9710f5e36
Author: Robert Dodier <ro...@users.sourceforge.net>
Date: 2015-03-13T17:58:04Z
Initial, incomplete work towards calibration for binary classifiers.
o ProbabilisticClassifier.scala:
mention calibration in comments
o BinaryClassificationMetrics.scala:
adapting code for ROC to calibration; incomplete and commented
out for now
o BinaryClassificationMetricsSuite.scala:
tests for calibration
commit 87382e5e6fe238824ee9ddb50c6a4c2e9a858cec
Author: Robert Dodier <ro...@users.sourceforge.net>
Date: 2015-03-13T21:08:14Z
Initial attempt to implement calibration; compiles, not tested yet.
commit d3c81cd61c685a132c29cae86f48e05c37569d3c
Author: Robert Dodier <ro...@users.sourceforge.net>
Date: 2015-03-13T21:26:32Z
Change (..., (Double, Int)) to (..., (Double, Long)) to match
types to what calibration actually returns.
commit 951817f94921f42cf3031a15a97f787d032dcd92
Author: Robert Dodier <ro...@users.sourceforge.net>
Date: 2015-03-14T00:48:56Z
Adjust JVM command line to get tests to run.
commit 07e807a8176b76ec5b26f93bb826b6e7b696306b
Author: Robert Dodier <ro...@users.sourceforge.net>
Date: 2015-03-14T00:50:11Z
Adjust bin size to prevent final bin from being very small compared to others.
commit c526891c2eaa372f0dbf9835f3420819a967085c
Author: Robert Dodier <ro...@users.sourceforge.net>
Date: 2015-03-14T02:14:30Z
Revert local changes to pom.xml.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6332] [MLlib] compute calibration curve...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/5025
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6332] [MLlib] compute calibration curve...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/5025#issuecomment-96778662
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6332] [MLlib] compute calibration curve...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5025#issuecomment-79715325
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6332] [MLlib] compute calibration curve...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/5025#issuecomment-168112611
I'm going to close this pull request. If this is still relevant and you are interested in pushing it forward, please open a new pull request. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6332] [MLlib] compute calibration curve...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/5025#issuecomment-80624475
@robert-dodier Thanks for the PR! I added a couple of clarification questions to the JIRA
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6332] [MLlib] compute calibration curve...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5025#issuecomment-96769007
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org