You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by srowen <gi...@git.apache.org> on 2014/03/04 19:38:34 UTC
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/77
SPARK-1181. 'mvn test' fails out of the box since sbt assembly does not necessarily exist
The test suite requires that "sbt assembly" has been run in order for some tests (like DriverSuite) to pass. The tests themselves say as much.
This means that a "mvn test" from a fresh clone fails.
There's a pretty simple fix, to have Maven's test-compile phase invoke "sbt assembly". I suppose the only downside is re-invoking "sbt assembly" each time tests are run.
I'm open to ideas about how to set this up more intelligently but it would be a generally good thing if the Maven build's tests passed out of the box.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srowen/spark SPARK-1181
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/77.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #77
----
commit 3cf7ddc9e3762baecce5e62e9fdc9606519cb0bb
Author: Sean Owen <so...@cloudera.com>
Date: 2014-03-04T18:37:12Z
Invoke "sbt assembly" during test-compile phase to ensure tests that require the assembly to pass
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/77#issuecomment-36663340
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/77#issuecomment-36669615
One or more automated tests failed
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/12997/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/77#issuecomment-36686522
OK that works, to package and then test. In the canonical Maven lifecycle, packaging comes after test, so test would not depend on packaging. In practice this is at worst a fine workaround.
I agree that making these integration tests is likely the right-est answer. I will close this however as I think that's for another day; other bigger build changes might make this point irrelevant.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/77#issuecomment-36663341
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/77#issuecomment-36669614
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by markhamstra <gi...@git.apache.org>.
Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/77#issuecomment-36659947
The standard maven build procedure should be to run `mvn -DskipTests package` first (which builds the assembly) and then `mvn test`. The "Building Spark with Maven" page should be updated to clearly explain that procedure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/77#issuecomment-36665859
In Maven, you can run tests that depend on packages/assemblies during Maven's `integration-test` phase, which automatically runs after the Maven `package` phase. I'm not sure what the SBT equivalent of `integration-test` is.
This approach would require us to move the integration tests into a separate test directory or package.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1181. 'mvn test' fails out of the box si...
Posted by srowen <gi...@git.apache.org>.
Github user srowen closed the pull request at:
https://github.com/apache/spark/pull/77
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---