You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by chie8842 <gi...@git.apache.org> on 2017/07/10 10:05:34 UTC
[GitHub] spark pull request #18586: [Spark 21358][Examples] Argument of repartitionan...
GitHub user chie8842 opened a pull request:
https://github.com/apache/spark/pull/18586
[Spark 21358][Examples] Argument of repartitionandsortwithinpartitions at pyspark
## What changes were proposed in this pull request?
At example of repartitionAndSortWithinPartitions at rdd.py, third argument should be True or False.
I proposed fix of example code.
## How was this patch tested?
* I rename test_repartitionAndSortWithinPartitions to test_repartitionAndSortWIthinPartitions_asc to specify boolean argument.
* I added test_repartitionAndSortWithinPartitions_desc to test False pattern at third argument.
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chie8842/spark SPARK-21358
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18586.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18586
----
commit 6a70aa7a42c9aa840f9e83aa41a984acd9fe21a5
Author: chie hayashida <ch...@retty.me>
Date: 2017-07-10T09:46:28Z
fixed ascending argument at repartitionandsortwithinpartitions
commit fac7a700331b34551ccdb80e615798b7f87b6af0
Author: chie hayashida <ch...@retty.me>
Date: 2017-07-10T09:57:38Z
insert ascending argument at test_repartitionAndSortWithinPartitions_asc and added test_repartitionAndSortWithinPartitions_desc
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [SPARK-21358][Examples] Argument of repartitionandsortwi...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18586
**[Test build #3841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3841/testReport)** for PR 18586 at commit [`f5804fe`](https://github.com/apache/spark/commit/f5804feb2f81b927ad9fa7c582b2ded4d6e00d16).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [Spark 21358][Examples] Argument of repartitionandsortwi...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18586
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [SPARK-21358][Examples] Argument of repartitionandsortwi...
Posted by chie8842 <gi...@git.apache.org>.
Github user chie8842 commented on the issue:
https://github.com/apache/spark/pull/18586
@srowen Thank you for pointing out about test fail. I fixed it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [SPARK-21358][Examples] Argument of repartitionandsortwi...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18586
Your new test looks reasonable, though it failed, because I believe the sorting is only by key. It wouldn't necessarily cause them to be ordered by the second value in the tuple. The example is a little bit misleading because it sort of looks like it is sorting by value too. But, I think you could just adjust the test to expect the actual order as it happens here. The output was sorted by key, correctly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [SPARK-21358][Examples] Argument of repartitionandsortwi...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18586
**[Test build #3840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3840/testReport)** for PR 18586 at commit [`fac7a70`](https://github.com/apache/spark/commit/fac7a700331b34551ccdb80e615798b7f87b6af0).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18586: [SPARK-21358][Examples] Argument of repartitionan...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18586
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [SPARK-21358][Examples] Argument of repartitionandsortwi...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18586
**[Test build #3841 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3841/testReport)** for PR 18586 at commit [`f5804fe`](https://github.com/apache/spark/commit/f5804feb2f81b927ad9fa7c582b2ded4d6e00d16).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [Spark 21358][Examples] Argument of repartitionandsortwi...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18586
The extra test is nice too, yeah. By the way if you change the title to include `[SPARK-21358]` instead, it will automatically link to the JIRA.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [Spark 21358][Examples] Argument of repartitionandsortwi...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18586
**[Test build #3840 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3840/testReport)** for PR 18586 at commit [`fac7a70`](https://github.com/apache/spark/commit/fac7a700331b34551ccdb80e615798b7f87b6af0).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18586: [SPARK-21358][Examples] Argument of repartitionandsortwi...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/18586
Merging in master. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org