gbuild development update 2018-11

[Damjan Jovanovic <da...@apache.org>]

Hi

Out of our 183 total modules, 94 are now in gbuild (51.37%).

That seems like a small number, given the amount of time it has taken to
get there. Right?

Well, of the remaining modules, 37 of them (another 20.22% of total) are
"externals" (zlib, jpeg, etc.) which we merely unzip and patch with dmake,
but then chain-build with their own build systems. It is relatively easy to
port the unzipping/patching logic to gbuild, but then there are many custom
actions in each module. Potentially, for modules that don't need patching,
we could also download binaries instead of patching and building from
source?

The rest are our own internal dmake modules, 52 of them (another 28.42% of
total), which build with dmake all the way, and which have to be converted
fully. From my perspective, the main challenges with those currently are:
* XCU and XHP files are somehow gathered and processed in many modules. The
dmake logic for this is complex.
* Perl scripts are called from many modules. It's not clear how to call
Perl from gbuild's declarative syntax.
* Windows is used in many modules, and the Windows build is the slowest and
most painful to work with.
* .NET tools are used in many modules, which I am unfamiliar with.
* ATL is required in some modules but seems like a huge mission to install.
* XSLT processing is done in many modules, which again requires calling out
to other tools somehow.
* Quite specialized custom build logic is found in a few modules, and it's
not clear whether to convert it to gbuild or how to deal with it. Do we
make gbuild scripts to build UNO extensions just to cater to main/mysqlc?
* Some dmake code is unused, but discovering that takes a while.
* Mac-only modules (apple_remote) need a Mac to test on.

On the bright side, even with so much left, we've come a long way. All the
"main" applications (writer, calc, impress, base) are in gbuild. Every type
of binary (eg. "unoverlibs" with "3" in their names and symbolic links to
those without the 3) is correctly built and delivered. Much of the tooling
and code generation, for example yacc and flex, has already been ported to
gbuild. Ant modules can open in Java IDEs, with all their dependencies, and
be compiled and tested from within the IDE, with even their IDL code
generation that involves having Ant call UNO tools in the shell, all
working, without "make" involved at all - an unprecedented development
[LibreOffice can only build Java modules with their gbuild, in the shell;
Java IDEs certainly can't parse gbuild's eval-filled non-deterministic
makefiles, so their Java development must be quite a pain ;)].

Today I even got gbuild to extract dependencies from Ant, and use them to
build Ant submodules in the correct order. We no longer need build.pl to
order building of Ant submodules, and in a similar way, eventually we won't
need it to order building of gbuild modules either.

Changing build systems is not the most exciting development possible, but a
bad build system is a barrier to entry for new developers and a pain for
existing ones, and the innovation and cleanups possible while changing to a
new one, all have benefits.

If anyone wants to help, let me know and I'll teach you how it's done.

Damjan