You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by wangmiao1981 <gi...@git.apache.org> on 2016/05/24 22:36:55 UTC
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
GitHub user wangmiao1981 opened a pull request:
https://github.com/apache/spark/pull/13284
[SPARK-15439][SparkR]:Failed to run unit test in SparkR
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
There are some failures when running SparkR unit tests.
In this PR, I fixed two of these failures in test_context.R and test_sparkSQL.R
The first one is due to different masked name. I added missed names in the expected arrays.
The second one is because one PR removed the logic of a previous fix of missing subset method.
The file privilege issue is still there. I am debugging it. SparkR shell can run the test case successfully.
test_that("pipeRDD() on RDDs", {
actual <- collect(pipeRDD(rdd, "more"))
When using run-test script, it complains no such directories as below:
cannot open file '/tmp/Rtmp4FQbah/filee2273f9d47f7': No such file or directory
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
Manually test it
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wangmiao1981/spark R
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13284.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13284
----
commit b3850093a7ae1c3c356afb64aa78b2f069127618
Author: wm624@hotmail.com <wm...@hotmail.com>
Date: 2016-05-24T22:29:49Z
fix Unit test error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221475769
@felixcheung
> conflicts(detail = TRUE)
$.GlobalEnv
[1] "df"
$`package:SparkR`
[1] "alias" "cov" "filter" "fitted"
[5] "glm" "lag" "na.omit" "predict"
[9] "sd" "var" "window" "head"
[13] "str" "show" "%in%" "as.data.frame"
[17] "atan2" "attach" "colnames" "colnames<-"
[21] "drop" "endsWith" "factorial" "ifelse"
[25] "intersect" "mean" "merge" "ncol"
[29] "nrow" "rank" "rbind" "sample"
[33] "startsWith" "subset" "substr" "summary"
[37] "transform" "unique" "with"
$`package:stats`
[1] "alias" "cov" "df" "filter" "fitted" "glm" "lag"
[8] "na.omit" "predict" "sd" "var" "window"
$`package:utils`
[1] "head" "str"
$`package:methods`
[1] "show" "body<-" "kronecker"
$`package:base`
[1] "%in%" "as.data.frame" "atan2" "attach"
[5] "body<-" "colnames" "colnames<-" "drop"
[9] "endsWith" "factorial" "ifelse" "intersect"
[13] "kronecker" "mean" "merge" "ncol"
[17] "nrow" "rank" "rbind" "sample"
[21] "startsWith" "subset" "substr" "summary"
[25] "transform" "unique" "with"
See the $`package:SparkR` above
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221422898
**[Test build #59228 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59228/consoleFull)** for PR 13284 at commit [`b385009`](https://github.com/apache/spark/commit/b3850093a7ae1c3c356afb64aa78b2f069127618).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by sun-rui <gi...@git.apache.org>.
Github user sun-rui commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221505799
I did try to solve the PipeRDD issue, but just as @wangmiao1981 said, "For the failure pipedRDD test case, if I copy & paste it in the SparkR, it works fine." It's so weird. I guess it is related to recent R changes, possibly 3.2.4 or else.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221426332
**[Test build #59228 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59228/consoleFull)** for PR 13284 at commit [`b385009`](https://github.com/apache/spark/commit/b3850093a7ae1c3c356afb64aa78b2f069127618).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/13284
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221477791
so startsWith endsWith were definitely added in 3.3.0.
I think we should be able to change the generic to match the signature to get it to work (have not tried it yet, since I don't have 3.3.0)
```
startsWith(x, prefix)
endsWith(x, suffix)
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221486545
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59260/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/13284#discussion_r64513464
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1445,7 +1445,11 @@ setMethod("[", signature(x = "SparkDataFrame"),
#' }
setMethod("subset", signature(x = "SparkDataFrame"),
function(x, subset, select, drop = F, ...) {
- x[subset, select, drop = drop]
+ if (missing(subset)) {
+ x[, select, drop = drop]
--- End diff --
I think we should keep the ..., like this
`x[, select, drop = drop, ...]`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221478607
**[Test build #59256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59256/consoleFull)** for PR 13284 at commit [`5ac9b9b`](https://github.com/apache/spark/commit/5ac9b9ba3094714f1d153a3f96690fa312b794ae).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221483871
Ok - lets just fix the masking and the subset ones in this PR. Once we understand what is the problem with `pipeRDD` we can fix it in a separate PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221668874
@shivaram I am debugging and try to find a hint.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221486544
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221477942
as for subset - maybe tighter signature matching checks in R 3.3.0? I couldn't find any reference on that though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221641136
I comment out the failed case. There is no other failures. As mentioned by @shivaram , we can try to fix the pipeRDD in a separate JIRA. I suspect it is related to operating system. It passed when I use ubuntu, but always fails if I use my mac.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/13284#discussion_r64513429
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1445,7 +1445,11 @@ setMethod("[", signature(x = "SparkDataFrame"),
#' }
setMethod("subset", signature(x = "SparkDataFrame"),
function(x, subset, select, drop = F, ...) {
- x[subset, select, drop = drop]
+ if (missing(subset)) {
+ x[, select, drop = drop]
+ } else {
--- End diff --
could you pls fix the extra space: `else {`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221483353
@shivaram @felixcheung For the failure pipedRDD test case, if I copy & paste it in the SparkR, it works fine.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221772558
Yeah thats a good idea @wangmiao1981 can you open a JIRA to not mask `startsWith` and `endsWith` by updating our generics ?
LGTM - Merging this to master and branch-2.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221476116
Hmm looks like `startsWith` and `endsWith` were added in R 3.3.0 - See http://www.r-statistics.com/2016/05/r-3-3-0-is-released/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221786544
@shivaram I will create a JIRA soon. Thursday and Friday, I will be on travel to NYC. Will do it on Saturday.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221489377
**[Test build #59263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59263/consoleFull)** for PR 13284 at commit [`3028e2c`](https://github.com/apache/spark/commit/3028e2c419e55e4cfa2c743361ca8b0beb004de3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221476748
Continuing from my previous message, we can't add or remove `endsWith` and `startsWith` as we want to support all R versions from 3.1.0 onwards. We could get the R version at run time (using ` R.Version()$major`) and only add them for 3.3.0 and higher
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221477072
@shivaram I will make the change with R version check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221483796
**[Test build #59260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59260/consoleFull)** for PR 13284 at commit [`4a187ca`](https://github.com/apache/spark/commit/4a187cacc04bb43b9399698fa95eb4e322b706e7).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221772080
looks fine - I think we should really try to make startsWith and endsWith work though, but that could be a follow up.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221489457
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59263/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221476475
**[Test build #59256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59256/consoleFull)** for PR 13284 at commit [`5ac9b9b`](https://github.com/apache/spark/commit/5ac9b9ba3094714f1d153a3f96690fa312b794ae).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by vectorijk <gi...@git.apache.org>.
Github user vectorijk commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221485421
We may also investigate why unit test run differently on Jenkins and local.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/13284#discussion_r64520757
--- Diff: R/pkg/inst/tests/testthat/test_context.R ---
@@ -24,10 +24,18 @@ test_that("Check masked functions", {
func <- lapply(masked, function(x) { capture.output(showMethods(x))[[1]] })
funcSparkROrEmpty <- grepl("\\(package SparkR\\)$|^$", func)
maskedBySparkR <- masked[funcSparkROrEmpty]
- namesOfMasked <- c("describe", "cov", "filter", "lag", "na.omit", "predict", "sd", "var",
- "colnames", "colnames<-", "intersect", "rank", "rbind", "sample", "subset",
- "summary", "transform", "drop", "window", "as.data.frame",
- "endsWith", "startsWith")
+ if (as.numeric(R.version$major) == 3 && as.numeric(R.version$minor) > 2) {
--- End diff --
can we just append to the list ? i.e. create `namesOfMasked` outside this and if the version number is greater than 2 we can do
`namesOfMasked <- c("endsWith", "startsWith", namesOfMasked)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221478629
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221478631
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59256/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221656579
Re-tested on Ubuntu, the pipedRDD test case still fails. R version 3.3.0 beta (2016-03-30 r70404)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221489454
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by vectorijk <gi...@git.apache.org>.
Github user vectorijk commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221486634
> @shivaram I will make the change with R version check.
@wangmiao1981 FYI, I have switched to R 3.1.3 before on Mac. It seems fail too. Do you mind try it again?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221486456
**[Test build #59260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59260/consoleFull)** for PR 13284 at commit [`4a187ca`](https://github.com/apache/spark/commit/4a187cacc04bb43b9399698fa95eb4e322b706e7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221462859
@wangmiao1981 Thanks for investigating this. Do you know why these are not failing in Jenkins though ? (the subset test and the pipedRDD one)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221469622
I'm seeing the error with `subset` as well.
As for "endsWith", "startsWith" - could you check what you have in your environment - you could run `conflicts(detail = TRUE)` - AFAIK I'm not seeing this.
Which version of R do you have? And package versions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221486881
**[Test build #59263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59263/consoleFull)** for PR 13284 at commit [`3028e2c`](https://github.com/apache/spark/commit/3028e2c419e55e4cfa2c743361ca8b0beb004de3).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221426358
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on a diff in the pull request:
https://github.com/apache/spark/pull/13284#discussion_r64520919
--- Diff: R/pkg/inst/tests/testthat/test_context.R ---
@@ -24,10 +24,18 @@ test_that("Check masked functions", {
func <- lapply(masked, function(x) { capture.output(showMethods(x))[[1]] })
funcSparkROrEmpty <- grepl("\\(package SparkR\\)$|^$", func)
maskedBySparkR <- masked[funcSparkROrEmpty]
- namesOfMasked <- c("describe", "cov", "filter", "lag", "na.omit", "predict", "sd", "var",
- "colnames", "colnames<-", "intersect", "rank", "rbind", "sample", "subset",
- "summary", "transform", "drop", "window", "as.data.frame",
- "endsWith", "startsWith")
+ if (as.numeric(R.version$major) == 3 && as.numeric(R.version$minor) > 2) {
--- End diff --
Sounds good. Let me revise it. Just started learning R.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221426362
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59228/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221477450
Ok - so that explains one of the problems. Does anybody know what the problem in `subset` is ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221475397
R version 3.3.0 (2016-05-03) -- "Supposedly Educational"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)
> installed.packages()
Package
SparkR "SparkR"
base "base"
boot "boot"
class "class"
cluster "cluster"
codetools "codetools"
compiler "compiler"
crayon "crayon"
datasets "datasets"
digest "digest"
foreign "foreign"
graphics "graphics"
grDevices "grDevices"
grid "grid"
KernSmooth "KernSmooth"
lattice "lattice"
magrittr "magrittr"
MASS "MASS"
Matrix "Matrix"
memoise "memoise"
methods "methods"
mgcv "mgcv"
nlme "nlme"
nnet "nnet"
parallel "parallel"
praise "praise"
R6 "R6"
rpart "rpart"
spatial "spatial"
splines "splines"
stats "stats"
stats4 "stats4"
survival "survival"
tcltk "tcltk"
testthat "testthat"
tools "tools"
utils "utils"
LibPath
SparkR "/Users/mwang/spark_ws_0904/R/lib"
base "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
boot "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
class "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
cluster "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
codetools "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
compiler "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
crayon "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
datasets "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
digest "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
foreign "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
graphics "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
grDevices "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
grid "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
KernSmooth "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
lattice "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
magrittr "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
MASS "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
Matrix "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
memoise "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
methods "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
mgcv "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
nlme "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
nnet "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
parallel "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
praise "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
R6 "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
rpart "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
spatial "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
splines "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
stats "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
stats4 "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
survival "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
tcltk "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
testthat "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
tools "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
utils "/Library/Frameworks/R.framework/Versions/3.3/Resources/library"
Version Priority
SparkR "2.0.0" NA
base "3.3.0" "base"
boot "1.3-18" "recommended"
class "7.3-14" "recommended"
cluster "2.0.4" "recommended"
codetools "0.2-14" "recommended"
compiler "3.3.0" "base"
crayon "1.3.1" NA
datasets "3.3.0" "base"
digest "0.6.9" NA
foreign "0.8-66" "recommended"
graphics "3.3.0" "base"
grDevices "3.3.0" "base"
grid "3.3.0" "base"
KernSmooth "2.23-15" "recommended"
lattice "0.20-33" "recommended"
magrittr "1.5" NA
MASS "7.3-45" "recommended"
Matrix "1.2-6" "recommended"
memoise "1.0.0" NA
methods "3.3.0" "base"
mgcv "1.8-12" "recommended"
nlme "3.1-127" "recommended"
nnet "7.3-12" "recommended"
parallel "3.3.0" "base"
praise "1.0.0" NA
R6 "2.1.2" NA
rpart "4.1-10" "recommended"
spatial "7.3-11" "recommended"
splines "3.3.0" "base"
stats "3.3.0" "base"
stats4 "3.3.0" "base"
survival "2.39-2" "recommended"
tcltk "3.3.0" "base"
testthat "1.0.2" NA
tools "3.3.0" "base"
utils "3.3.0" "base"
Depends
SparkR "\nR (>= 3.0),\nmethods,"
base NA
boot "R (>= 3.0.0), graphics, stats"
class "R (>= 3.0.0), stats, utils"
cluster "R (>= 3.0.1)"
codetools "R (>= 2.1)"
compiler NA
crayon NA
datasets NA
digest "R (>= 2.4.1)"
foreign "R (>= 3.0.0)"
graphics NA
grDevices NA
grid NA
KernSmooth "R (>= 2.5.0), stats"
lattice "R (>= 3.0.0)"
magrittr NA
MASS "R (>= 3.1.0), grDevices, graphics, stats, utils"
Matrix "R (>= 3.0.1)"
memoise NA
methods NA
mgcv "R (>= 2.14.0), nlme (>= 3.1-64)"
nlme "R (>= 3.0.2)"
nnet "R (>= 2.14.0), stats, utils"
parallel NA
praise NA
R6 "R (>= 3.0)"
rpart "R (>= 2.15.0), graphics, stats, grDevices"
spatial "R (>= 3.0.0), graphics, stats, utils"
splines NA
stats NA
stats4 NA
survival "R (>= 2.13.0)"
tcltk NA
testthat "R (>= 3.1.0)"
tools NA
utils NA
Imports LinkingTo
SparkR "\nmethods" NA
base NA NA
boot NA NA
class "MASS" NA
cluster "graphics, grDevices, stats, utils" NA
codetools NA NA
compiler NA NA
crayon "memoise, grDevices, methods, utils" NA
datasets NA NA
digest NA NA
foreign "methods, utils, stats" NA
graphics "grDevices" NA
grDevices NA NA
grid "grDevices, utils" NA
KernSmooth NA NA
lattice "grid, grDevices, graphics, stats, utils" NA
magrittr NA NA
MASS "methods" NA
Matrix "methods, graphics, grid, stats, utils, lattice" NA
memoise "digest (>= 0.6.3)" NA
methods "utils, stats" NA
mgcv "methods, stats, graphics, Matrix" NA
nlme "graphics, stats, utils, lattice" NA
nnet NA NA
parallel "tools" NA
praise NA NA
R6 NA NA
rpart NA NA
spatial NA NA
splines "graphics, stats" NA
stats "utils, grDevices, graphics" NA
stats4 "graphics, methods, stats" NA
survival "graphics, Matrix, methods, splines, stats, utils" NA
tcltk "utils" NA
testthat "digest, crayon, praise, magrittr, R6, methods" NA
tools NA NA
utils NA NA
Suggests
SparkR "\ntestthat,\ne1071,\nsurvival"
base "methods"
boot "MASS, survival"
class NA
cluster "MASS"
codetools NA
compiler NA
crayon "testthat"
datasets NA
digest "knitr, rmarkdown"
foreign NA
graphics NA
grDevices "KernSmooth"
grid "lattice"
KernSmooth "MASS"
lattice "KernSmooth, MASS"
magrittr "testthat, knitr"
MASS "lattice, nlme, nnet, survival"
Matrix "expm, MASS"
memoise "testthat"
methods "codetools"
mgcv "splines, parallel, survival, MASS"
nlme "Hmisc, MASS"
nnet "MASS"
parallel "methods"
praise "testthat"
R6 "knitr, microbenchmark, pryr, testthat, ggplot2, scales"
rpart "survival"
spatial "MASS"
splines "Matrix, methods"
stats "MASS, Matrix, SuppDists, methods, stats4"
stats4 NA
survival NA
tcltk NA
testthat "devtools, withr, covr"
tools "codetools, methods, xml2"
utils "methods, XML"
Enhances License
SparkR NA "Apache License (== 2.0)"
base NA "Part of R 3.3.0"
boot NA "Unlimited"
class NA "GPL-2 | GPL-3"
cluster NA "GPL (>= 2)"
codetools NA "GPL"
compiler NA "Part of R 3.3.0"
crayon NA "MIT + file LICENSE"
datasets NA "Part of R 3.3.0"
digest NA "GPL-2"
foreign NA "GPL (>= 2)"
graphics NA "Part of R 3.3.0"
grDevices NA "Part of R 3.3.0"
grid NA "Part of R 3.3.0"
KernSmooth NA "Unlimited"
lattice "chron" "GPL (>= 2)"
magrittr NA "MIT + file LICENSE"
MASS NA "GPL-2 | GPL-3"
Matrix "MatrixModels, graph, SparseM, sfsmisc" "GPL (>= 2) | file LICENCE"
memoise NA "MIT + file LICENSE"
methods NA "Part of R 3.3.0"
mgcv NA "GPL (>= 2)"
nlme NA "GPL (>= 2)"
nnet NA "GPL-2 | GPL-3"
parallel "snow, nws, Rmpi" "Part of R 3.3.0"
praise NA "MIT + file LICENSE"
R6 NA "MIT + file LICENSE"
rpart NA "GPL-2 | GPL-3"
spatial NA "GPL-2 | GPL-3"
splines NA "Part of R 3.3.0"
stats NA "Part of R 3.3.0"
stats4 NA "Part of R 3.3.0"
survival NA "LGPL (>= 2)"
tcltk NA "Part of R 3.3.0"
testthat NA "MIT + file LICENSE"
tools NA "Part of R 3.3.0"
utils NA "Part of R 3.3.0"
License_is_FOSS License_restricts_use OS_type MD5sum
SparkR NA NA NA NA
base NA NA NA NA
boot NA NA NA NA
class NA NA NA NA
cluster NA NA NA NA
codetools NA NA NA NA
compiler NA NA NA NA
crayon NA NA NA NA
datasets NA NA NA NA
digest NA NA NA NA
foreign NA NA NA NA
graphics NA NA NA NA
grDevices NA NA NA NA
grid NA NA NA NA
KernSmooth NA NA NA NA
lattice NA NA NA NA
magrittr NA NA NA NA
MASS NA NA NA NA
Matrix NA NA NA NA
memoise NA NA NA NA
methods NA NA NA NA
mgcv NA NA NA NA
nlme NA NA NA NA
nnet NA NA NA NA
parallel NA NA NA NA
praise NA NA NA NA
R6 NA NA NA NA
rpart NA NA NA NA
spatial NA NA NA NA
splines NA NA NA NA
stats NA NA NA NA
stats4 NA NA NA NA
survival NA NA NA NA
tcltk NA NA NA NA
testthat NA NA NA NA
tools NA NA NA NA
utils NA NA NA NA
NeedsCompilation Built
SparkR NA "3.3.0"
base NA "3.3.0"
boot "no" "3.3.0"
class "yes" "3.3.0"
cluster "yes" "3.3.0"
codetools "no" "3.3.0"
compiler NA "3.3.0"
crayon "no" "3.3.0"
datasets NA "3.3.0"
digest "yes" "3.3.0"
foreign "yes" "3.3.0"
graphics NA "3.3.0"
grDevices NA "3.3.0"
grid NA "3.3.0"
KernSmooth "yes" "3.3.0"
lattice "yes" "3.3.0"
magrittr "no" "3.3.0"
MASS "yes" "3.3.0"
Matrix "yes" "3.3.0"
memoise "no" "3.3.0"
methods NA "3.3.0"
mgcv "yes" "3.3.0"
nlme "yes" "3.3.0"
nnet "yes" "3.3.0"
parallel NA "3.3.0"
praise "no" "3.3.0"
R6 "no" "3.3.0"
rpart "yes" "3.3.0"
spatial "yes" "3.3.0"
splines NA "3.3.0"
stats NA "3.3.0"
stats4 NA "3.3.0"
survival "yes" "3.3.0"
tcltk NA "3.3.0"
testthat "yes" "3.3.0"
tools NA "3.3.0"
utils NA "3.3.0"
@vectorijk also see this failure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221486087
@vectorijk As we discussed above, R version is different on local and Jenkins. We installed R 3.3.0 in local while Jenkins still uses the old version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/13284#discussion_r64513468
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1445,7 +1445,11 @@ setMethod("[", signature(x = "SparkDataFrame"),
#' }
setMethod("subset", signature(x = "SparkDataFrame"),
function(x, subset, select, drop = F, ...) {
- x[subset, select, drop = drop]
+ if (missing(subset)) {
+ x[, select, drop = drop]
+ } else {
+ x[subset, select, drop = drop]
--- End diff --
ditto
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221502917
now I'm seeing the pipeRDD test failure...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221487340
@vectorijk please run conflicts(detail = TRUE) and check the $package:SparkR with test_context.R, namesOfMasked.
3.2.0 should have more methods than 3.1.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221476011
@shivaram The pipedRDD one seems working when using sudo in Linux. My mac does not work though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15439][SparkR]:Failed to run unit test ...
Posted by shivaram <gi...@git.apache.org>.
Github user shivaram commented on the pull request:
https://github.com/apache/spark/pull/13284#issuecomment-221663730
@wangmiao1981 Lets continue the pipeRDD debugging on the JIRA. This change LGTM for the subset and the masking tests
@felixcheung any other comments ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org