You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by dongjoon-hyun <gi...@git.apache.org> on 2018/10/25 21:10:01 UTC
[GitHub] spark pull request #22840: [SPARK-25840][BUILD] `make-distribution.sh` shoul...
GitHub user dongjoon-hyun opened a pull request:
https://github.com/apache/spark/pull/22840
[SPARK-25840][BUILD] `make-distribution.sh` should not fail due to missing LICENSE-binary
## What changes were proposed in this pull request?
We vote for the artifacts. All releases are in the form of the source materials needed to make changes to the software being released. (http://www.apache.org/legal/release-policy.html#artifacts)
From Spark 2.4.0, the source artifact and binary artifact starts to contain own proper LICENSE files (LICENSE, LICENSE-binary). It's great to have them. However, unfortunately, `dev/make-distribution.sh` inside source artifacts start to fail because it expects `LICENSE-binary` and source artifact have only the LICENSE file.
https://dist.apache.org/repos/dist/dev/spark/v2.4.0-rc4-bin/spark-2.4.0.tgz
`dev/make-distribution.sh` is used during the voting phase because we are voting on that source artifact instead of GitHub repository. Individual contributor usually don't have the downstream repository and starts to try build the voting source artifacts. (Personally, I did before.)
This PR aims to recover that script to work in any way. This doesn't aim for source artifacts to reproduce the compiled artifacts.
## How was this patch tested?
Manual.
```
$ rm LICENSE-binary
$ dev/make-distribution.sh
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dongjoon-hyun/spark SPARK-25840
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22840.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22840
----
commit 0950bb86ac028655c665687398c7dcfce1853f04
Author: Dongjoon Hyun <do...@...>
Date: 2018-10-25T21:06:41Z
[SPARK-25840][BUILD] `make-distribution.sh` should not fail due to missing LICENSE-binary
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22840
Hm, I'm not dead-set against it, though it seems a little deceptive; you won't actually get the exact binary distro out, and that's what the script purports to do. (Arguably, why do we bundle the script then?) Is it needed to test the source release? I'd expect we compile and run tests, as those are the things that are meant to work from the source release.
I'm not against it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22840
cc @srowen
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22840
Hm, but then you haven't created a completely valid binary release; it's missing licenses. As I say, I don't know that the binary release tarball must be exactly recreatable from the source artifact. The compiled bits? yes, must be exactly recreateable, and they are as far as I know
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22840
@srowen . It's a documented feature.
- http://spark.apache.org/docs/latest/building-spark.html#building-a-runnable-distribution
I know that you're not against it, but Spark 2.4.0 had better respect the document.
Are we going to document it; From Spark 2.4.0 source distribution, we cannot build it from the source.?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22840
Yeah ... I read that as being about the source tree from the repo. But it does say only "... like the ones distributed by the project". OK I think this is fine, for 2.4.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22840: [SPARK-25840][BUILD] `make-distribution.sh` shoul...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22840
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22840
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22840
**[Test build #98051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98051/testReport)** for PR 22840 at commit [`0950bb8`](https://github.com/apache/spark/commit/0950bb86ac028655c665687398c7dcfce1853f04).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22840
**[Test build #98051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98051/testReport)** for PR 22840 at commit [`0950bb8`](https://github.com/apache/spark/commit/0950bb86ac028655c665687398c7dcfce1853f04).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22840
Oops. My bad. I'll monitor the branch.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22840
I noted that this doesn't aim to reproduce the provided binary release.
This aims to help individual contributor participate to test the source artifacts during voting phase.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22840
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98051/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22840
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22840
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4506/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22840: [SPARK-25840][BUILD] `make-distribution.sh` should not f...
Posted by dongjoon-hyun <gi...@git.apache.org>.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22840
Thank you, @srowen and @HyukjinKwon .
Merged to master/branch-2.4.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org