You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by huaxingao <gi...@git.apache.org> on 2018/08/17 18:23:27 UTC
[GitHub] spark pull request #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSi...
GitHub user huaxingao opened a pull request:
https://github.com/apache/spark/pull/22136
[SPARK-25124][ML]VectorSizeHint setSize and getSize don't return values
## What changes were proposed in this pull request?
In feature.py, VectorSizeHint setSize and getSize don't return value. Add return.
## How was this patch tested?
I tested the changes on my local.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/huaxingao/spark spark-25124
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22136.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22136
----
commit 91a819af424778d284d66893cc6b11e1015720e0
Author: Huaxin Gao <hu...@...>
Date: 2018-08-17T18:15:53Z
[SPARK-25124]VectorSizeHint setSize and getSize don't return values
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22136
**[Test build #95120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95120/testReport)** for PR 22136 at commit [`a211d51`](https://github.com/apache/spark/commit/a211d51ccedbe29f87e476e77a8d2f7134811f88).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSi...
Posted by huaxingao <gi...@git.apache.org>.
Github user huaxingao commented on a diff in the pull request:
https://github.com/apache/spark/pull/22136#discussion_r212088986
--- Diff: python/pyspark/ml/tests.py ---
@@ -844,6 +844,28 @@ def test_string_indexer_from_labels(self):
.select(model_default.getOrDefault(model_default.outputCol)).collect()
self.assertEqual(len(transformed_list), 5)
+ def test_vector_size_hint(self):
--- End diff --
@jkbradley Sorry, my bad. I added set/getSize and removed VectorAssembler from the test to simply.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSi...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22136
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94903/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSi...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/22136#discussion_r212069129
--- Diff: python/pyspark/ml/tests.py ---
@@ -844,6 +844,28 @@ def test_string_indexer_from_labels(self):
.select(model_default.getOrDefault(model_default.outputCol)).collect()
self.assertEqual(len(transformed_list), 5)
+ def test_vector_size_hint(self):
--- End diff --
This test doesn't test the 2 functions which were buggy. Could you please test those (and simplify the test if possible)? Thanks!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22136
**[Test build #94903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94903/testReport)** for PR 22136 at commit [`91a819a`](https://github.com/apache/spark/commit/91a819af424778d284d66893cc6b11e1015720e0).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/22136
LGTM
Merging with master. I'll try to backport it to 2.3 too.
Thanks!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2286/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by thunterdb <gi...@git.apache.org>.
Github user thunterdb commented on the issue:
https://github.com/apache/spark/pull/22136
@huaxingao thank you for your pull request. Can you please add a test to make sure this does not regress?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22136
**[Test build #95120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95120/testReport)** for PR 22136 at commit [`a211d51`](https://github.com/apache/spark/commit/a211d51ccedbe29f87e476e77a8d2f7134811f88).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22136
**[Test build #95043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95043/testReport)** for PR 22136 at commit [`94c6702`](https://github.com/apache/spark/commit/94c67024445831b6328c4026d49279700ee33038).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22136
**[Test build #95043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95043/testReport)** for PR 22136 at commit [`94c6702`](https://github.com/apache/spark/commit/94c67024445831b6328c4026d49279700ee33038).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSi...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/22136#discussion_r212483950
--- Diff: python/pyspark/ml/tests.py ---
@@ -844,6 +844,28 @@ def test_string_indexer_from_labels(self):
.select(model_default.getOrDefault(model_default.outputCol)).collect()
self.assertEqual(len(transformed_list), 5)
+ def test_vector_size_hint(self):
--- End diff --
Thanks! FYI this still isn't really testing the return value of setSize, but I think it's OK since we don't really do that anywhere else : P and I'm confident in the above change.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95043/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2392/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by jkbradley <gi...@git.apache.org>.
Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/22136
Well, this merged successfully with master but not with 2.3; it seemed to pull in code from another PR, strangely. Would you mind sending a backport PR against branch-2.3? Thank you!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2458/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95120/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22136
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22136
**[Test build #94903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94903/testReport)** for PR 22136 at commit [`91a819a`](https://github.com/apache/spark/commit/91a819af424778d284d66893cc6b11e1015720e0).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org