[jira] [Commented] (FLINK-9582) dist assemblies access jars outside of flink-dist

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-9582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16512531#comment-16512531 ] 

ASF GitHub Bot commented on FLINK-9582:
---------------------------------------

GitHub user zentol opened a pull request:

    https://github.com/apache/flink/pull/6167

    [FLINK-9582][build] Rework flink-dist assemblies

    ## What is the purpose of the change
    
    This PR reworks the `flink-dist` assemblies to not access specific jars outside of the `flink-dist` module.
    This allows compiling `flink-dist` without having built any other module (bar `flink-shaded-hadoop2-uber`, but that's another story`).
    
    Instead, the flink jars that flink-dist depends on are copied to `target/dependencies` using the `maven-dependency-plugin`, from which we assemble the final distribution.
    
    ## Brief change log
    
    * add `maven-dependency-plugin` execution to flink-dist for copying dependencies to `target/dependencies`
    * modify all entries in `flink-dist` assemblies to specific flink jars to use copied jars instead
    * added `jar-with-dependencies` classifier to several dependencies of flink-dist to accurately reflect what flink-dist actually requires
    
    
    ## Verifying this change
    
    The general functionality is verified by travis.
    
    I have manually verified that building flink-dist + some other module(s) correctly puts the newly compiled modules in the target directory, instead of downloaded artifacts.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 9582

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/6167.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #6167
    
----
commit c1ea8980bf359fcbd6a89dcacf1f5ade3b9a9779
Author: zentol <ch...@...>
Date:   2018-06-13T21:52:59Z

    [FLINK-9582][build] Rework flink-dist assemblies

----


> dist assemblies access jars outside of flink-dist
> -------------------------------------------------
>
>                 Key: FLINK-9582
>                 URL: https://issues.apache.org/jira/browse/FLINK-9582
>             Project: Flink
>          Issue Type: Bug
>          Components: Build System
>    Affects Versions: 1.5.0, 1.6.0
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>            Priority: Major
>             Fix For: 1.6.0
>
>
> The flink-dist assemblies access compiled jars outside of flink-dist, for example like this:
> {code:java}
> <source>../flink-libraries/flink-cep/target/flink-cep_${scala.binary.version}-${project.version}.jar</source>{code}
> As usual, accessing files outside of the module that you're building is a terrible idea.
> It's brittle as it relies on paths that aren't guaranteed to be stable, and requires these modules to be built beforehand. There's also an inherent potential for dependency conflicts when building flink-dist on it's own, as maven may download certain snapshot artifacts, but the assemblies ignore these and bundle jars present in Flink.
> We can use the maven-dependency plugin to copy required dependencies into the {{target}} directory of flink-dist, and point the assemblies to these jars.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)