You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Davide Grandi (JIRA)" <ji...@apache.org> on 2019/08/16 17:22:00 UTC
[jira] [Created] (DERBY-7052) Reordering and (mildly) reorganizing
build.xml Ant targets around buildsource
Davide Grandi created DERBY-7052:
------------------------------------
Summary: Reordering and (mildly) reorganizing build.xml Ant targets around buildsource
Key: DERBY-7052
URL: https://issues.apache.org/jira/browse/DERBY-7052
Project: Derby
Issue Type: Improvement
Components: Build tools
Affects Versions: 10.15.1.3
Reporter: Davide Grandi
Module dependencies should be reflected by Ant targets dependencies.
And Ant targets execution follows the rules stated in
https://ant.apache.org/manual/targets.html
so a dependency could be covered (being redundant) by applying transitivity rule
to every dependency's dependecies (please look at the dot+pdf diagrams).
--------------------------------------------------------------------------------
Modules dependencies, by the docs, are :
||module depends||on||
|org.apache.derby.client|org.apache.derby.commons|
|org.apache.derby.engine|org.apache.derby.commons|
|org.apache.derby.locale_*|org.apache.derby.commons|
| | |
|org.apache.derby.tools|org.apache.derby.client|
|org.apache.derby.tools|org.apache.derby.engine|
| | |
|org.apache.derby.optionaltools|org.apache.derby.tools|
| | |
|org.apache.derby.server|org.apache.derby.tools|
| | |
|org.apache.derby.runner|org.apache.derby.server|
| | |
|org.apache.derby.engine|org.osgi.framework|
but actual ant dependencies are (partial target => module map) :
||target||depends||and module||depends on||
|buildsource|client| | |
|buildsource |shared| | |
|buildsource|felixStubs| |org.osgi.framework|
|engine|shared|org.apache.derby.engine|org.apache.derby.commons|
|client|shared|org.apache.derby.client|org.apache.derby.commons|
|tools|engine|org.apache.derby.tools|org.apache.derby.engine|
|drda|engine| | |
|runner|optional| | |
|optional|engine| | |
so we expect that building 'engine' target alone will fails due to the lack of
'felixstubs' dependency.
And it fails ("01 ant engine.log") quoting "requires static org.osgi.framework;"
So the best try is to delete (on copy "build-01.xml" of "build.xml") the
dependency
buildsource => felixstubs
and to put it as last dependency of 'engine' target
Result : success ("02 ant -f build-01.xml engine.txt")
Similarly we can expect that also 'tools' fails, since it really depends on
'client' but this is not explicited in "build.xml".
And it fails ("03 ant -f build-01.xml tools.log") with:
[javac] requires static org.apache.derby.client;
So the best try is to switch the 'client' dependency (in copy "build-02.xml" of
"build-01.xml") from 'buildsource' to 'tools'.
Result: success ("04 ant -f build-02.xml tools.log").
The same thing for 'optional' that should depends on 'tools' but depends only on
'engine'.
And it fails ("05 ant -f build-02.xml optional.log") with
[javac] requires org.apache.derby.tools;
So we switch the 'tools' dependency from 'buildsource' to 'optional' (in copy
"build-03.xml" of "build.02.xml")
And it succeeds ("06 ant -f build-03.xml optional.log").
Same as 'drda' (aka server) that require 'tools' ("07 ant -f build-03.xml drda.log")
[javac] requires org.apache.derby.tools;
that succeds after adding 'tools' dependency to 'drda'
("08 ant -f build-04.xml drda.log")
and also for 'runner' ("09 ant -f build-04.xml runner.log") that requires 'drda'
[javac] requires org.apache.derby.server;
and succeeds after adding it into copy "build-05.xml" of "build-04.xml".
("10 ant -f build-05.xml runner.log")
After graphing "build-05.xml" (in "build-05.xml.pdf") one can notice that,
apart from redundant prerequisites (covered later), there's an extra dependency
on 'runner', that in module hierararchy needs only 'drda'.
After duplicating "build-05.xml" in "build-06.xml" and edited, removing
'runner' -> 'optional' prerequisite, build of 'runner' succeeds
("11 ant -f build-06.xml runner.log")
The next step is removing all the extra dependencies.
Given the transitivity of dependencies in this set
A => C (a depends on C)
A => B (a depends on B)
B => C (B depends on C)
the first dependendency is already covered by second and third one, and ant
will ignore it.
After graphing "build-06.xml" (in "build-06.pdf") and focusing on the subset
from 'buildsource' to 'shared' we can reduce it (in "build-07.xml")applying the
previous rule and deleting these dependencies:
- buildSource => shared
- buildsource => drda
- buildsource => engine
- drda => engine
- optional => engine
before continuing there are some test that can be passed :
- single target checking : a serie of deep clean + single target builds e.g. :
ant -f build-07.xml clobber
svn st --no-ignore
del/rmdir items marked with "I"
rmdir /Q /S classes.pptesting
rmdir /Q /S jars
del tools\java\junit.jar
ant -f build-07.xml shared
... for shared, felixstub, engine, client, tools, drda, storeless, optional,
runner
so the full build list, refactored in a windows batch command, is
--------------------------------------------------------------------------------
@echo off
call ant clobber -file %1 -quiet -silent > nul 2>&1
if errorlevel 1 (
echo error %ERRORLEVEL% during first clobber
exit /B 1
)
for %%t in (
shared
felixStubs
engine
client
tools
drda
storeless
optional
runner
buildsource
) do (
call ant clobber -file %1 -quiet -silent > nul 2>&1
rmdir /Q /S classes.pptesting > nul 2>&1
rmdir /Q /S jars > nul 2>&1
del tools\java\junit.jar > nul 2>&1
rmdir /Q /S classes > nul 2>&1
rmdir /Q /S generated > nul 2>&1
rmdir /Q /S release > nul 2>&1
svn st --no-ignore | find "I "
echo ant -file %1 %%t
call ant -file %1 -quiet -silent %%t
if errorlevel 1 exit /B 1
echo.
)
--------------------------------------------------------------------------------
Executing
"ant -f build-07.xml -quiet felixstubs"
we get an unexpected fail (but tests exist for this) and looking at
"13 ant -f build-07.xml felixStubs.log"
we can notice that the root cause is an undefined variable.
adding a dependency on 'state' to 'felixStubs' (on "build-08.xml") the build succeed
("14 ant -f build-08.xml felixStubs.log")
Graphing "build-08.xml" (in "build.08.pdf") we notice that there's still a
redundant dependency :
shared => init
covered by
shared => state => showenv => init
and, just on the right, another redundancy
init => makeOutDirs
already covered by
init => setCompilerProperties => prebuild => makeOutDirs
and, finally, a dependency
engine => state
already covered by
engine => felixStubs => state
and
engine => shared => state
so these can be removed.
Building "build-09.xml" succeeds, and so the run test for all the targets
("15 test targets.cmd build-09.xml")
--------------------------------------------------------------------------------
The last tests are
a) log comparison
b) file comparison
we define a cleaning sequence
--
call ant clobber -f build-09.xml -quiet -silent
rmdir /Q /S classes.pptesting
rmdir /Q /S jars
rmdir /Q /S classes
del tools\java\junit.jar
svn st --no-ignore
--
and, for the standard build sequence
--
<cleaning sequence>
ant -quiet all
ant -quiet buildjars
--
we generate a standard log, a verbose log
<cleaning sequence>
ant all -f build.xml > "all - build.xml.log"
ant buildjars -f build.xml >> "all - build.xml.log"
<cleaning sequence>
ant all -f build.xml -verbose > "all - build.xml - verbose.log"
ant buildjars -f build.xml -verbose >> "all - build.xml - verbose.log"
<cleaning sequence>
ant all -f build-09.xml > "all - build-09.xml.log"
ant buildjars -f build-09.xml >> "all - build-09.xml.log"
<cleaning sequence>
ant all -f build-09.xml -verbose > "all - build-09.xml - verbose.log"
ant buildjars -f build-09.xml -verbose >> "all - build-09.xml - verbose.log"
all differences seems ok.
--------------------------------------------------------------------------------
After looking at "build-09.xml.pdf" we notice that
buildsource => init
is already covered by many path, eg
buildsource => optional => tools => client => shared => state => showenv =>
=> init
and also
buildsource => prebuild
is covered by
... => init => setCompilerProperties => prebuild
and also
buildsource => setCompilerProperties
by the same path(s), so 'init', 'prebuild' and 'setCompilerProperties' can be
deleted from 'buildsource' prerequisites in "build-10.xml".
Tests and binary compare are ok.
--------------------------------------------------------------------------------
Looking at "build-10.xml.pdf" and "build-10.buildsource-expanded-01.pdf" we
notice that 'build' target, 5th buildsource dependency, is ant-called anyway
from 'engine' and can be deleted from 'buildsource' dependencies in
"build-11.xml". Tests and binary compares pass.
The only problem remaining is the 'generateSecurityPolicies' target that fails
if built alone (due to a deep buried dependency)
--------------------------------------------------------------------------------
Summary of included files (maybe zipped) :
||File||Content||
|build@1863766.buildsource.dot|actual build graphing - focused on buildsource|
|build@1863766.buildsource.pdf | |
|build@1863766.buildsource-expanded.dot|same as above but with external build.xml expanded|
|build@1863766.buildsource-expanded.pdf| |
|build@1863766.xml|actual build.xml|
|build@1863766.xml.dot|rendered in dot+pdf|
|build@1863766.xml.pdf| |
| | |
|build-11.buildsource.dot|final build-11 graphing|
|build-11.buildsource.pdf| |
|build-11.buildsource-expanded.dot|same as above but with external build.xml expanded|
|build-11.buildsource-expanded.pdf| |
|build-11.xml|final build-11.xml|
|build-11.xml.dot|rendered in dot+pdf|
|build-11.xml.pdf | |
| | |
|patch.txt|the tiny 65 lines patch|
... I'll attach asap ...
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)