You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Rick Hillegas (JIRA)" <ji...@apache.org> on 2019/08/18 15:57:00 UTC
[jira] [Commented] (DERBY-7052) Reordering and (mildly)
reorganizing build.xml Ant targets around buildsource
[ https://issues.apache.org/jira/browse/DERBY-7052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16909997#comment-16909997 ]
Rick Hillegas commented on DERBY-7052:
--------------------------------------
Thanks for this comprehensive analysis, Davide. I think that this is a welcome cleanup.
I have applied your patch and verified clean builds when invoking the targets you modified. Now I have done a full build. I will run full tests both with the classpath and the modulepath.
> Reordering and (mildly) reorganizing build.xml Ant targets around buildsource
> -----------------------------------------------------------------------------
>
> Key: DERBY-7052
> URL: https://issues.apache.org/jira/browse/DERBY-7052
> Project: Derby
> Issue Type: Improvement
> Components: Build tools
> Affects Versions: 10.15.1.3
> Reporter: Davide Grandi
> Priority: Minor
> Attachments: build-11.buildsource-expanded.pdf, build-11.buildsource.pdf, build-11.xml, build-11.xml.pdf, build@1863766.buildsource-expanded.pdf, build@1863766.buildsource.pdf, build@1863766.xml, build@1863766.xml.pdf, dot.zip, intermediate.zip, patch.txt
>
>
> Module dependencies should be reflected by Ant targets dependencies.
> And Ant targets execution follows the rules stated in
> [https://ant.apache.org/manual/targets.html]
> so a dependency could be covered (being redundant) by applying transitivity rule
> to every dependency's dependecies (please look at the dot+pdf diagrams).
> --------------------------------------------------------------------------------
> Modules dependencies, by the docs, are :
> ||module depends||on||
> |org.apache.derby.client|org.apache.derby.commons|
> |org.apache.derby.engine|org.apache.derby.commons|
> |org.apache.derby.locale_*|org.apache.derby.commons|
> | | |
> |org.apache.derby.tools|org.apache.derby.client|
> |org.apache.derby.tools|org.apache.derby.engine|
> | | |
> |org.apache.derby.optionaltools|org.apache.derby.tools|
> | | |
> |org.apache.derby.server|org.apache.derby.tools|
> | | |
> |org.apache.derby.runner|org.apache.derby.server|
> | | |
> |org.apache.derby.engine|org.osgi.framework|
> but actual ant dependencies are (partial target => module map) :
>
> ||target||depends||and module||depends on||
> |buildsource|client| | |
> |buildsource|shared| | |
> |buildsource|felixStubs| |org.osgi.framework|
> |engine|shared|org.apache.derby.engine|org.apache.derby.commons|
> |client|shared|org.apache.derby.client|org.apache.derby.commons|
> |tools|engine|org.apache.derby.tools|org.apache.derby.engine|
> |drda|engine| | |
> |runner|optional| | |
> |optional|engine| | |
> so we expect that building 'engine' target alone will fails due to the lack of
> 'felixstubs' dependency.
> And it fails ("01 ant engine.log") quoting "requires static org.osgi.framework;"
> So the best try is to delete (on copy "build-01.xml" of "build.xml") the
> dependency
> buildsource => felixstubs
> and to put it as last dependency of 'engine' target
> Result : success ("02 ant -f build-01.xml engine.txt")
> Similarly we can expect that also 'tools' fails, since it really depends on
> 'client' but this is not explicited in "build.xml".
> And it fails ("03 ant -f build-01.xml tools.log") with:
> [javac] requires static org.apache.derby.client;
> So the best try is to switch the 'client' dependency (in copy "build-02.xml" of
> "build-01.xml") from 'buildsource' to 'tools'.
> Result: success ("04 ant -f build-02.xml tools.log").
> The same thing for 'optional' that should depends on 'tools' but depends only on
> 'engine'.
> And it fails ("05 ant -f build-02.xml optional.log") with
> [javac] requires org.apache.derby.tools;
> So we switch the 'tools' dependency from 'buildsource' to 'optional' (in copy
> "build-03.xml" of "build.02.xml")
> And it succeeds ("06 ant -f build-03.xml optional.log").
> Same as 'drda' (aka server) that require 'tools' ("07 ant -f build-03.xml drda.log")
> [javac] requires org.apache.derby.tools;
> that succeds after adding 'tools' dependency to 'drda'
> ("08 ant -f build-04.xml drda.log")
> and also for 'runner' ("09 ant -f build-04.xml runner.log") that requires 'drda'
> [javac] requires org.apache.derby.server;
> and succeeds after adding it into copy "build-05.xml" of "build-04.xml".
> ("10 ant -f build-05.xml runner.log")
> After graphing "build-05.xml" (in "build-05.xml.pdf") one can notice that,
> apart from redundant prerequisites (covered later), there's an extra dependency
> on 'runner', that in module hierararchy needs only 'drda'.
> After duplicating "build-05.xml" in "build-06.xml" and edited, removing
> 'runner' -> 'optional' prerequisite, build of 'runner' succeeds
> ("11 ant -f build-06.xml runner.log")
> The next step is removing all the extra dependencies.
> Given the transitivity of dependencies in this set
> A => C (a depends on C)
> A => B (a depends on B)
> B => C (B depends on C)
> the first dependendency is already covered by second and third one, and ant
> will ignore it.
> After graphing "build-06.xml" (in "build-06.pdf") and focusing on the subset
> from 'buildsource' to 'shared' we can reduce it (in "build-07.xml")applying the
> previous rule and deleting these dependencies:
> - buildSource => shared
> - buildsource => drda
> - buildsource => engine
> - drda => engine
> - optional => engine
> before continuing there are some test that can be passed :
> - single target checking : a serie of deep clean + single target builds e.g. :
> ant -f build-07.xml clobber
> svn st --no-ignore
> del/rmdir items marked with "I"
> rmdir /Q /S classes.pptesting
> rmdir /Q /S jars
> del tools\java\junit.jar
> ant -f build-07.xml shared
> ... for shared, felixstub, engine, client, tools, drda, storeless, optional,
> runner
> so the full build list, refactored in a windows batch command, is
> --------------------------------------------------------------------------------
> @echo off
> call ant clobber -file %1 -quiet -silent > nul 2>&1
> if errorlevel 1 (
> echo error %ERRORLEVEL% during first clobber
> exit /B 1
> )
> for %%t in (
> shared
> felixStubs
> engine
> client
> tools
> drda
> storeless
> optional
> runner
> buildsource
> ) do (
> call ant clobber -file %1 -quiet -silent > nul 2>&1
> rmdir /Q /S classes.pptesting > nul 2>&1
> rmdir /Q /S jars > nul 2>&1
> del tools\java\junit.jar > nul 2>&1
> rmdir /Q /S classes > nul 2>&1
> rmdir /Q /S generated > nul 2>&1
> rmdir /Q /S release > nul 2>&1
> svn st --no-ignore | find "I "
> echo ant -file %1 %%t
> call ant -file %1 -quiet -silent %%t
> if errorlevel 1 exit /B 1
> echo.
> )
> --------------------------------------------------------------------------------
> Executing
> "ant -f build-07.xml -quiet felixstubs"
> we get an unexpected fail (but tests exist for this) and looking at
> "13 ant -f build-07.xml felixStubs.log"
> we can notice that the root cause is an undefined variable.
> adding a dependency on 'state' to 'felixStubs' (on "build-08.xml") the build succeed
> ("14 ant -f build-08.xml felixStubs.log")
> Graphing "build-08.xml" (in "build.08.pdf") we notice that there's still a
> redundant dependency :
> shared => init
> covered by
> shared => state => showenv => init
> and, just on the right, another redundancy
> init => makeOutDirs
> already covered by
> init => setCompilerProperties => prebuild => makeOutDirs
> and, finally, a dependency
> engine => state
> already covered by
> engine => felixStubs => state
> and
> engine => shared => state
> so these can be removed.
> Building "build-09.xml" succeeds, and so the run test for all the targets
> ("15 test targets.cmd build-09.xml")
> --------------------------------------------------------------------------------
> The last tests are
> a) log comparison
> b) file comparison
> we define a cleaning sequence
> –
> call ant clobber -f build-09.xml -quiet -silent
> rmdir /Q /S classes.pptesting
> rmdir /Q /S jars
> rmdir /Q /S classes
> del tools\java\junit.jar
> svn st --no-ignore
> –
> and, for the standard build sequence
> –
> <cleaning sequence>
> ant -quiet all
> ant -quiet buildjars
> –
> we generate a standard log, a verbose log
> <cleaning sequence>
> ant all -f build.xml > "all - build.xml.log"
> ant buildjars -f build.xml >> "all - build.xml.log"
> <cleaning sequence>
> ant all -f build.xml -verbose > "all - build.xml - verbose.log"
> ant buildjars -f build.xml -verbose >> "all - build.xml - verbose.log"
> <cleaning sequence>
> ant all -f build-09.xml > "all - build-09.xml.log"
> ant buildjars -f build-09.xml >> "all - build-09.xml.log"
> <cleaning sequence>
> ant all -f build-09.xml -verbose > "all - build-09.xml - verbose.log"
> ant buildjars -f build-09.xml -verbose >> "all - build-09.xml - verbose.log"
> all differences seems ok.
> --------------------------------------------------------------------------------
> After looking at "build-09.xml.pdf" we notice that
> buildsource => init
> is already covered by many path, eg
> buildsource => optional => tools => client => shared => state => showenv =>
> => init
> and also
> buildsource => prebuild
> is covered by
> ... => init => setCompilerProperties => prebuild
> and also
> buildsource => setCompilerProperties
> by the same path(s), so 'init', 'prebuild' and 'setCompilerProperties' can be
> deleted from 'buildsource' prerequisites in "build-10.xml".
> Tests and binary compare are ok.
> --------------------------------------------------------------------------------
> Looking at "build-10.xml.pdf" and "build-10.buildsource-expanded-01.pdf" we
> notice that 'build' target, 5th buildsource dependency, is ant-called anyway
> from 'engine' and can be deleted from 'buildsource' dependencies in
> "build-11.xml". Tests and binary compares pass.
> The only problem remaining is the 'generateSecurityPolicies' target that fails
> if built alone (due to a deep buried dependency)
> --------------------------------------------------------------------------------
> Summary of included files (maybe zipped) :
> ||File||Content||
> |build@1863766.buildsource.dot|actual build graphing - focused on buildsource|
> |build@1863766.buildsource.pdf | |
> |build@1863766.buildsource-expanded.dot|same as above but with external build.xml expanded|
> |build@1863766.buildsource-expanded.pdf| |
> |build@1863766.xml|actual build.xml|
> |build@1863766.xml.dot|rendered in dot+pdf|
> |build@1863766.xml.pdf| |
> | | |
> |build-11.buildsource.dot|final build-11 graphing|
> |build-11.buildsource.pdf| |
> |build-11.buildsource-expanded.dot|same as above but with external build.xml expanded|
> |build-11.buildsource-expanded.pdf| |
> |build-11.xml|final build-11.xml|
> |build-11.xml.dot|rendered in dot+pdf|
> |build-11.xml.pdf | |
> | | |
> |patch.txt|the tiny 65 lines patch|
> Bye !
> Davide Grandi
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)