You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by hhbyyh <gi...@git.apache.org> on 2017/04/16 01:07:45 UTC
[GitHub] spark pull request #17645: [SPARK-20348] [ML] Support squared hinge loss (L2...
GitHub user hhbyyh opened a pull request:
https://github.com/apache/spark/pull/17645
[SPARK-20348] [ML] Support squared hinge loss (L2 loss) for LinearSVC
## What changes were proposed in this pull request?
While Hinge loss is the standard loss function for linear SVM, Squared hinge loss (a.k.a. L2 loss) is also popular in practice. L2-SVM is differentiable and imposes a bigger (quadratic vs. linear) loss for points which violate the margin. Some introduction can be found from http://mccormickml.com/2015/01/06/what-is-an-l2-svm/
Liblinear and scikit learn both offer squared hinge loss as the default loss function for linear SVM.
## How was this patch tested?
strengthen existing unit test and add new unit test for comparison.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hhbyyh/spark svml2loss
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17645.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17645
----
commit 6541f69401653d7e9ebfff9d573d87daba108084
Author: Yuhao Yang <yu...@intel.com>
Date: 2017-04-16T00:38:21Z
add l2 loss
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17645: [SPARK-20348] [ML] Support squared hinge loss (L2...
Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/17645#discussion_r115280207
--- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala ---
@@ -42,15 +44,35 @@ import org.apache.spark.sql.functions.{col, lit}
/** Params for linear SVM Classifier. */
private[classification] trait LinearSVCParams extends ClassifierParams with HasRegParam
with HasMaxIter with HasFitIntercept with HasTol with HasStandardization with HasWeightCol
- with HasThreshold with HasAggregationDepth
+ with HasThreshold with HasAggregationDepth {
+
+ /**
+ * Specifies the loss function. Currently "hinge" and "squared_hinge" are supported.
+ * "hinge" is the standard SVM loss (a.k.a. L1 loss) while "squared_hinge" is the square of
+ * the hinge loss (a.k.a. L2 loss).
+ *
+ * @see <a href="https://en.wikipedia.org/wiki/Hinge_loss">Hinge loss (Wikipedia)</a>
+ *
+ * @group param
+ */
+ @Since("2.3.0")
+ final val lossFunction: Param[String] = new Param(this, "lossFunction", "Specifies the loss " +
--- End diff --
OK, let leave as is and refactor in the future. One minor issue: What about renaming it to ```loss```? I found the name of corresponding params in ```sklearn.svm.linearSVC``` is ```loss```. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17645
ping @hhbyyh, where are we on this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17645
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17645: [SPARK-20348] [ML] Support squared hinge loss (L2...
Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on a diff in the pull request:
https://github.com/apache/spark/pull/17645#discussion_r112984129
--- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala ---
@@ -42,15 +44,35 @@ import org.apache.spark.sql.functions.{col, lit}
/** Params for linear SVM Classifier. */
private[classification] trait LinearSVCParams extends ClassifierParams with HasRegParam
with HasMaxIter with HasFitIntercept with HasTol with HasStandardization with HasWeightCol
- with HasThreshold with HasAggregationDepth
+ with HasThreshold with HasAggregationDepth {
+
+ /**
+ * Specifies the loss function. Currently "hinge" and "squared_hinge" are supported.
+ * "hinge" is the standard SVM loss (a.k.a. L1 loss) while "squared_hinge" is the square of
+ * the hinge loss (a.k.a. L2 loss).
+ *
+ * @see <a href="https://en.wikipedia.org/wiki/Hinge_loss">Hinge loss (Wikipedia)</a>
+ *
+ * @group param
+ */
+ @Since("2.3.0")
+ final val lossFunction: Param[String] = new Param(this, "lossFunction", "Specifies the loss " +
--- End diff --
I'd prefer to move this out to shared params, since it can be used by other algorithms as well. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17645
I took out this in the list. Though, should we maybe close this for now and reopen again when it's ready if it takes quite long? It'd be probably better than leaving this open without further updates for long time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by hhbyyh <gi...@git.apache.org>.
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/17645
OK. I'll close it for now and try to merge it with https://github.com/apache/spark/pull/17862.
Thanks for the comment from @yanboliang
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by hhbyyh <gi...@git.apache.org>.
Github user hhbyyh commented on the issue:
https://github.com/apache/spark/pull/17645
Hi @HyukjinKwon I think this is a feature we need, but currently we are still having some discussion about optimizer interface.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17645
**[Test build #75830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75830/testReport)** for PR 17645 at commit [`6541f69`](https://github.com/apache/spark/commit/6541f69401653d7e9ebfff9d573d87daba108084).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/17645
**[Test build #75830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75830/testReport)** for PR 17645 at commit [`6541f69`](https://github.com/apache/spark/commit/6541f69401653d7e9ebfff9d573d87daba108084).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/17645
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75830/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17645: [SPARK-20348] [ML] Support squared hinge loss (L2...
Posted by hhbyyh <gi...@git.apache.org>.
Github user hhbyyh closed the pull request at:
https://github.com/apache/spark/pull/17645
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #17645: [SPARK-20348] [ML] Support squared hinge loss (L2 loss) ...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17645
ping @hhbyyh. WDYT?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #17645: [SPARK-20348] [ML] Support squared hinge loss (L2...
Posted by hhbyyh <gi...@git.apache.org>.
Github user hhbyyh commented on a diff in the pull request:
https://github.com/apache/spark/pull/17645#discussion_r113836900
--- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LinearSVC.scala ---
@@ -42,15 +44,35 @@ import org.apache.spark.sql.functions.{col, lit}
/** Params for linear SVM Classifier. */
private[classification] trait LinearSVCParams extends ClassifierParams with HasRegParam
with HasMaxIter with HasFitIntercept with HasTol with HasStandardization with HasWeightCol
- with HasThreshold with HasAggregationDepth
+ with HasThreshold with HasAggregationDepth {
+
+ /**
+ * Specifies the loss function. Currently "hinge" and "squared_hinge" are supported.
+ * "hinge" is the standard SVM loss (a.k.a. L1 loss) while "squared_hinge" is the square of
+ * the hinge loss (a.k.a. L2 loss).
+ *
+ * @see <a href="https://en.wikipedia.org/wiki/Hinge_loss">Hinge loss (Wikipedia)</a>
+ *
+ * @group param
+ */
+ @Since("2.3.0")
+ final val lossFunction: Param[String] = new Param(this, "lossFunction", "Specifies the loss " +
--- End diff --
Sure we can do it.
But I'm thinking maybe we should conduct an integrated refactor about the common optimization parameters some time in the future, either through shared params or other trait or abstract class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org