You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by yanboliang <gi...@git.apache.org> on 2017/12/15 20:08:35 UTC
[GitHub] spark pull request #19994: [SPARK-22810][ML][PySpark] Expose Python API for ...
GitHub user yanboliang opened a pull request:
https://github.com/apache/spark/pull/19994
[SPARK-22810][ML][PySpark] Expose Python API for LinearRegression with huber loss.
## What changes were proposed in this pull request?
Expose Python API for _LinearRegression_ with _huber_ loss.
## How was this patch tested?
Unit test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yanboliang/spark spark-22810
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19994.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19994
----
commit 1ed46a2ea0fe28e173df4bc9bfec301beafc1acd
Author: Yanbo Liang <yb...@gmail.com>
Date: 2017-12-15T19:58:55Z
Expose Python API for LinearRegression with huber loss.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by yanboliang <gi...@git.apache.org>.
Github user yanboliang commented on the issue:
https://github.com/apache/spark/pull/19994
Merged into master, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19994: [SPARK-22810][ML][PySpark] Expose Python API for ...
Posted by WeichenXu123 <gi...@git.apache.org>.
Github user WeichenXu123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19994#discussion_r157391801
--- Diff: python/pyspark/ml/tests.py ---
@@ -1725,6 +1725,27 @@ def test_offset(self):
self.assertTrue(np.isclose(model.intercept, -1.561613, atol=1E-4))
+class LinearRegressionTest(SparkSessionTestCase):
+
+ def test_linear_regression_with_huber_loss(self):
+
+ data_path = "data/mllib/sample_linear_regression_data.txt"
+ df = self.spark.read.format("libsvm").load(data_path)
+
+ lir = LinearRegression(loss="huber")
--- End diff --
The testcase should include `setEpsilon`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19994
**[Test build #84977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84977/testReport)** for PR 19994 at commit [`1ed46a2`](https://github.com/apache/spark/commit/1ed46a2ea0fe28e173df4bc9bfec301beafc1acd).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19994: [SPARK-22810][ML][PySpark] Expose Python API for ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19994
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19994
**[Test build #85210 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85210/testReport)** for PR 19994 at commit [`e02d60f`](https://github.com/apache/spark/commit/e02d60f5057d74e1bfa18d6121762827a853c12b).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19994
**[Test build #85210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85210/testReport)** for PR 19994 at commit [`e02d60f`](https://github.com/apache/spark/commit/e02d60f5057d74e1bfa18d6121762827a853c12b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19994
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85210/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by WeichenXu123 <gi...@git.apache.org>.
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/19994
LGTM.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19994
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19994
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19994: [SPARK-22810][ML][PySpark] Expose Python API for ...
Posted by WeichenXu123 <gi...@git.apache.org>.
Github user WeichenXu123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19994#discussion_r157391859
--- Diff: python/pyspark/ml/regression.py ---
@@ -155,6 +183,14 @@ def intercept(self):
"""
return self._call_java("intercept")
+ @property
+ @since("2.3.0")
+ def scale(self):
+ """
+ The value by which \|y - X'w\| is scaled down when loss is "huber".
--- End diff --
add doc "When square loss the value is 1.0"
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19994
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84977/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19994: [SPARK-22810][ML][PySpark] Expose Python API for LinearR...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19994
**[Test build #84977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84977/testReport)** for PR 19994 at commit [`1ed46a2`](https://github.com/apache/spark/commit/1ed46a2ea0fe28e173df4bc9bfec301beafc1acd).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class HasLoss(Params):`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org