You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by asokadiggs <gi...@git.apache.org> on 2015/09/28 22:58:44 UTC
[GitHub] spark pull request: Update dropDuplicates() documentation
GitHub user asokadiggs opened a pull request:
https://github.com/apache/spark/pull/8930
Update dropDuplicates() documentation
Documentation for dropDuplicates() and drop_duplicates() is one and the same. Resolved the error in the example for drop_duplicates using the same approach used for groupby and groupBy, by indicating that dropDuplicates and drop_duplicates are aliases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/asokadiggs/spark jira-10782
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8930.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8930
----
commit 279d6207be25cead5ef5b1c04f1719bd15f2dc9a
Author: asokadiggs <as...@intel.com>
Date: 2015-09-28T20:51:03Z
Update dataframe.py
Documentation for dropDuplicates() and drop_duplicates() is one and the same. Resolved the error in the example for drop_duplicates using the same approach used for groupby and groupBy, by indicating that dropDuplicates and drop_duplicates are aliases.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-144178294
[Test build #1825 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1825/consoleFull) for PR 8930 at commit [`279d620`](https://github.com/apache/spark/commit/279d6207be25cead5ef5b1c04f1719bd15f2dc9a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-144178055
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] Update dropDuplicates documentat...
Posted by asokadiggs <gi...@git.apache.org>.
Github user asokadiggs commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-143875054
FYI - I've used spark/python/docs/make html to generate the updated HTML versions of the files and visually verified the result in the resulting documentation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-143928835
[Test build #1822 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1822/consoleFull) for PR 8930 at commit [`279d620`](https://github.com/apache/spark/commit/279d6207be25cead5ef5b1c04f1719bd15f2dc9a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-143928985
I was going to say, I don't think this addresses the problem you identified, but then I realized there's no separate declaration of `drop_duplicates`? so I probably don't understand where that's coming from. If it must necessarily clone `dropDuplicates` docs then I agree with your note. Looks like this needs a style fix though. See the test result.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8930
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Update dropDuplicates() documentation
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-143872898
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by asokadiggs <gi...@git.apache.org>.
Github user asokadiggs commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-144106274
Correct - there is no separate declaration for drop_duplicates. There is a section in dataframe.py at approximately line 1280-1285 that sets drop_duplicates = dropDuplicates and groupby = groupBy for Pandas compatibility. So there isn't an option to separately document drop_duplicates or groupby.
I implemented the same solution for drop_duplicates as is already implemented for groupby. At least for me it's readable and understandable, so it works.
I found the test result - I'll figure out the white space error and resubmit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-143928873
[Test build #1822 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1822/console) for PR 8930 at commit [`279d620`](https://github.com/apache/spark/commit/279d6207be25cead5ef5b1c04f1719bd15f2dc9a).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-10782] [Python] Update dropDuplicates d...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8930#issuecomment-144183857
[Test build #1825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1825/console) for PR 8930 at commit [`279d620`](https://github.com/apache/spark/commit/279d6207be25cead5ef5b1c04f1719bd15f2dc9a).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org