You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Aki Tanaka (JIRA)" <ji...@apache.org> on 2018/11/05 22:33:00 UTC
[jira] [Comment Edited] (SPARK-24654) Update, fix LICENSE and
NOTICE, and specialize for source vs binary
[ https://issues.apache.org/jira/browse/SPARK-24654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675830#comment-16675830 ]
Aki Tanaka edited comment on SPARK-24654 at 11/5/18 10:32 PM:
--------------------------------------------------------------
Hello,
I think this patch (to dev/make-distribution.sh) causes a build failure when we download a distribution tgz ([http://apache.osuosl.org/spark/spark-2.4.0/]) and build a Spark rpm using bigtop.
[https://github.com/apache/spark/commit/30aa37fca45ec0ad4f30076bc855d1a201cfc097#diff-ba2c046d92a1d2b5b417788bfb5cb5f8]
In the commit above, a command 'mkdir' -p "$DISTDIR/licenses" was removed. This means that 'licenses' directory will not be created when licenses-binary does not exist, I think. On the other hand, Spark RPM spec file requires "$DISTDIR/licenses" directory [1]. So, when "$DISTDIR/licenses" directory does not exist, building RPM will fail with the following error.
{code:java}
Processing files: spark-core-2.4.0.noarch
error: File not found: /workspace/xxx/build/spark/rpm/BUILDROOT/spark-core-2.4.0/usr/lib/spark/licenses
File not found: /workspace/workspace/xxx/build/spark/rpm/BUILDROOT/spark-core-2.4.0/usr/lib/spark/licenses
RPM build errors:
:spark-rpm FAILED
{code}
[1] [https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/spark/SPECS/spark.spec#L215]
was (Author: tanakahda):
Hello,
I think this patch (to dev/make-distribution.sh) causes a build failure when we download a distribution tgz (http://apache.osuosl.org/spark/spark-2.4.0/) and build a Spark rpm using bigtop.
https://github.com/apache/spark/commit/30aa37fca45ec0ad4f30076bc855d1a201cfc097#diff-ba2c046d92a1d2b5b417788bfb5cb5f8
In the commit above, a command 'mkdir' -p "$DISTDIR/licenses" was removed. This means that 'licenses' directory will not be created when licenses-binary does not exist, I think. On the other hand, Spark RPM spec files requires "$DISTDIR/licenses" directory [1]. So, when "$DISTDIR/licenses" directory does not exist, building RPM will fail with the following error.
{code:java}
Processing files: spark-core-2.4.0.noarch
error: File not found: /workspace/xxx/build/spark/rpm/BUILDROOT/spark-core-2.4.0/usr/lib/spark/licenses
File not found: /workspace/workspace/xxx/build/spark/rpm/BUILDROOT/spark-core-2.4.0/usr/lib/spark/licenses
RPM build errors:
:spark-rpm FAILED
{code}
[1] https://github.com/apache/bigtop/blob/master/bigtop-packages/src/rpm/spark/SPECS/spark.spec#L215
> Update, fix LICENSE and NOTICE, and specialize for source vs binary
> -------------------------------------------------------------------
>
> Key: SPARK-24654
> URL: https://issues.apache.org/jira/browse/SPARK-24654
> Project: Spark
> Issue Type: Task
> Components: Build
> Affects Versions: 2.3.1
> Reporter: Sean Owen
> Assignee: Sean Owen
> Priority: Major
> Fix For: 2.4.0
>
>
> The message at [http://mail-archives.apache.org/mod_mbox/spark-dev/201806.mbox/%3C4D5001D8-D24A-4FC9-A942-50616A8EB13B%40classsoftware.com%3E] brings up several issues with how LICENSE and NOTICE are structured:
> * They aren't specialized for source vs binary release
> * They seem to include items that shouldn't be in a release (test deps)
> * They may be missing some entries
> * Some info is not in its recommended location in NOTICE vs LICENSE
> I propose to regenerate all of this info, to bring it up to date and address these issues, and compile notes on what I did for the future.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org