[jira] [Commented] (BIGTOP-1906) Optimize and enahnce build/CI process to reuse artifacts

["Konstantin Boudnik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609165#comment-14609165 ] 

Konstantin Boudnik commented on BIGTOP-1906:
--------------------------------------------

I actually have no idea what Bigtop client jars are, so I've asked ;)
Thanks for clarification of your point!

> Optimize and enahnce build/CI process to reuse artifacts
> --------------------------------------------------------
>
>                 Key: BIGTOP-1906
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1906
>             Project: Bigtop
>          Issue Type: Sub-task
>          Components: build
>    Affects Versions: 1.0.0
>         Environment: all (host as well as any container based building)
>            Reporter: Nate DAmico
>              Labels: build, containers, maven
>             Fix For: 1.1.0
>
>
> This was generated from latest "future bigtop" thread on mailing list, capturing core details here from [~oflebbe]
> cc: [~evans_ye]
> ==============================================
> However we have a substantial problem hidden deep in th CI „2.0“ approach using containers
> You may know that we place artifacts (i.e. jars) we built with bigtop into the local maven cache ~/.m2. (look for mvn install in do-component-build). The idea is that later maven builds will pick these artifacts and use them rather downloading them from maven central.
> Placing artifacts into ~/.m2 will not have any effect if we use CI containers the way we do now: The maven cache ~/.m2 is lost when the container ends.
> [This triggered misfeature in JIRA https://issues.apache.org/jira/browse/BIGTOP-1893 , BTW:  gradle rpm/apt behaved differently from a container build with artifacts from maven central.]
> Option 1)  Remove mvn install from all do-component-builds
> Results:
> + We compile projects the way the upstream-developer does.
> - local fixes and configurations will not propagated
> Questions:
> If we do not try to reuse our build-artifacts within compile we have to ask ourself "why do we compile projects at all?“.
> We can build a great test wether someone else has touched / manipulated the maven central cache if we compare artifacts, but is this the really the point of compiling ourselves?
> Option 2) Use mvn install and reuse artifacts even in containers.
> Consequences:
> - Containers are not stateless any more
> - We have to add depencies to CI jobs so they run in order
> - single components may break the whole compile process.
> - Compile does not scale any more
> My Opinion:
> The way we do now "mvn install“ ,  simply tainting the maven cache seems not a really controlled way to propagate artifacts to me.
> Option 3) Use 1) but reuse artifacts in packages by placing symlinks and dependencies between them.
> - Packages will break with subtile problems if we do symlink artifacts  from different releases.
> ----
> Neither Option 1, Option 2 nor Option 3 seems a clever way to fix the problem. Would like to hear comments regarding this issue:
> In my humble opinion we should follow Option 2 with all the grave consequences. But maybe reworking mvn install by placing the artifacts with a bigtop specific name / groupid into the maven cache and upload them to maven central .
> Olaf



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)