[jira] [Updated] (SPARK-20810) ML LinearSVC vs MLlib SVMWithSGD output different solution

["Yanbo Liang (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-20810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yanbo Liang updated SPARK-20810:
--------------------------------
    Description: 
Fitting with SVM classification model on the same dataset, ML {{LinearSVC}} produces different solution compared with MLlib {{SVMWithSGD}}. I understand they use different optimization solver (OWLQN vs SGD), does it make sense to converge to different solution?
AFAIK, both of them use Hinge loss which is convex but not differentiable function. Since the derivative of the hinge loss at certain place is non-deterministic, should we switch to use squared hinge loss which is the default loss function of {{sklearn.svm.LinearSVC}}?
This issue is very easy to reproduce, you can paste the following code snippet to {{LinearSVCSuite}} and then click run in Intellij IDE.
{code}
test("LinearSVC vs SVMWithSGD") {
    import org.apache.spark.mllib.linalg.{Vectors => OldVectors}
    import org.apache.spark.mllib.regression.{LabeledPoint => OldLabeledPoint}

    val trainer1 = new LinearSVC()
      .setRegParam(0.00002)
      .setMaxIter(200)
      .setTol(1e-4)
    val model1 = trainer1.fit(binaryDataset)

    println(model1.coefficients)
    println(model1.intercept)

    val oldData = binaryDataset.rdd.map { case Row(label: Double, features: Vector) =>
        OldLabeledPoint(label, OldVectors.fromML(features))
    }
    val trainer2 = new SVMWithSGD().setIntercept(true)
    trainer2.optimizer.setRegParam(0.00002).setNumIterations(200).setConvergenceTol(1e-4)

    val model2 = trainer2.run(oldData)

    println(model2.weights)
    println(model2.intercept)
  }
{code} 

  was:
Fitting with SVM classification model on the same dataset, ML {{LinearSVC}} produces different solution compared with MLlib {{SVMWithSGD}}. I understand they use different optimization solver (OWLQN vs SGD), does it make sense to converge to different solution?
AFAIK, both of them use Hinge loss which is convex but not differentiable function. Since the derivative of the hinge loss at certain place is non-deterministic, should we switch to use squared hinge loss? 


> ML LinearSVC vs MLlib SVMWithSGD output different solution
> ----------------------------------------------------------
>
>                 Key: SPARK-20810
>                 URL: https://issues.apache.org/jira/browse/SPARK-20810
>             Project: Spark
>          Issue Type: Question
>          Components: ML, MLlib
>    Affects Versions: 2.2.0
>            Reporter: Yanbo Liang
>
> Fitting with SVM classification model on the same dataset, ML {{LinearSVC}} produces different solution compared with MLlib {{SVMWithSGD}}. I understand they use different optimization solver (OWLQN vs SGD), does it make sense to converge to different solution?
> AFAIK, both of them use Hinge loss which is convex but not differentiable function. Since the derivative of the hinge loss at certain place is non-deterministic, should we switch to use squared hinge loss which is the default loss function of {{sklearn.svm.LinearSVC}}?
> This issue is very easy to reproduce, you can paste the following code snippet to {{LinearSVCSuite}} and then click run in Intellij IDE.
> {code}
> test("LinearSVC vs SVMWithSGD") {
>     import org.apache.spark.mllib.linalg.{Vectors => OldVectors}
>     import org.apache.spark.mllib.regression.{LabeledPoint => OldLabeledPoint}
>     val trainer1 = new LinearSVC()
>       .setRegParam(0.00002)
>       .setMaxIter(200)
>       .setTol(1e-4)
>     val model1 = trainer1.fit(binaryDataset)
>     println(model1.coefficients)
>     println(model1.intercept)
>     val oldData = binaryDataset.rdd.map { case Row(label: Double, features: Vector) =>
>         OldLabeledPoint(label, OldVectors.fromML(features))
>     }
>     val trainer2 = new SVMWithSGD().setIntercept(true)
>     trainer2.optimizer.setRegParam(0.00002).setNumIterations(200).setConvergenceTol(1e-4)
>     val model2 = trainer2.run(oldData)
>     println(model2.weights)
>     println(model2.intercept)
>   }
> {code} 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org