You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by mhnatiuk <gi...@git.apache.org> on 2016/04/01 14:03:26 UTC
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
GitHub user mhnatiuk opened a pull request:
https://github.com/apache/spark/pull/12111
Modified NAMESPACE to allow user to access lapply and map actions on …
## What changes were proposed in this pull request?
I made lapply, lapplyParition, map and reduce available as methods that operate on Spark DataFrame. This is needed for common operations, such as calculating statistic over columns of DataFrame
## How was this patch tested?
No specific unit test needed. run-test.sh positive
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mhnatiuk/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12111.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12111
----
commit 3ca55eb585904b6e5b3406597b39a91629d0c84f
Author: mhnatiuk <mh...@ip-10-123-18-109.ec2.internal>
Date: 2016-04-01T12:00:35Z
Modified NAMESPACE to allow user to access lapply and map actions on DaaFrames
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by mhnatiuk <gi...@git.apache.org>.
Github user mhnatiuk commented on the pull request:
https://github.com/apache/spark/pull/12111#issuecomment-205296909
Please close this PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/12111#issuecomment-204465670
Have a look at https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark and consider asking a question first rather than making a PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by mhnatiuk <gi...@git.apache.org>.
Github user mhnatiuk commented on the pull request:
https://github.com/apache/spark/pull/12111#issuecomment-204374916
I made also manual test. It seems that I didn't fully understand how SparkR lapply works. My research shows that it operates on the list of rows rather than columns as normal lapply works. Can anybody verify this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/12111#issuecomment-212139962
@mhnatiuk We don't have permission to close this PR, but you should have permission to do so
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12111#issuecomment-204370675
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by DeepSparkBot <gi...@git.apache.org>.
Github user DeepSparkBot commented on the pull request:
https://github.com/apache/spark/pull/12111#issuecomment-204477716
**Missing JIRA detected.**
Please file one and attach it to the title of this PR. For further instructions, see https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12111
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: Modified NAMESPACE to allow user to access lap...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/12111#issuecomment-204664301
Actually we want to remove the RDD-like APIs in R. The only reason we haven't done it yet (they are only hidden but not removed right now) is because we haven't implemented UDFs in R yet.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org