You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by janI <ja...@apache.org> on 2013/10/18 11:32:19 UTC
[proposal] replace build.pl with a central Makefile.
Hi.
due to the discussion in thread "Mentor a new build system", I have made a
proposal for a central Makefile located in main.
It has been roughly tested it, thanks to a clever utility from andre.
As discussed build.pl contains a lot of options, which need to be
considered in a makefile.
My suggestion is on
http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile
Please feel free to edit/comment on the page. I have reduced to options a
lot, and some of them might be in use.
thanks in advance for your comments.
Re: [proposal] replace build.pl with a central Makefile.
Posted by janI <ja...@apache.org>.
On 21 October 2013 10:50, Andre Fischer <aw...@gmail.com> wrote:
> On 18.10.2013 19:54, janI wrote:
>
>> On 18 October 2013 16:52, Andre Fischer <aw...@gmail.com> wrote:
>>
>> On 18.10.2013 15:58, janI wrote:
>>>
>>> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
>>>>
>>>> On 18.10.2013 14:02, janI wrote:
>>>>
>>>>> sd
>>>>>
>>>>>>
>>>>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>>>>>
>>>>>> On 18.10.2013 11:32, janI wrote:
>>>>>>
>>>>>> Hi.
>>>>>>>
>>>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>>>>> made a
>>>>>>>> proposal for a central Makefile located in main.
>>>>>>>>
>>>>>>>> Hi Jan,
>>>>>>>>
>>>>>>>> it is great that you are going to improve this part of the build
>>>>>>> system.
>>>>>>> But I think that we need more details about how the proposed
>>>>>>> build
>>>>>>> system
>>>>>>> works. Without them I can not really evaluate the proposal.
>>>>>>>
>>>>>>> First of all, I agree with juergens remarks that this should be
>>>>>>>
>>>>>>> discussed
>>>>>> before implemented, hence the wiki page.
>>>>>>
>>>>>> Secondly this has nothing directly to do with the proposed build
>>>>>> system,
>>>>>> its a simple replacement of build.pl in the current system.
>>>>>>
>>>>>> Yes, that is how I understood it. I just did not know how to call
>>>>>> the
>>>>>>
>>>>> build.pl replacement.
>>>>>
>>>>>
>>>>>
>>>>> I know that build.pl works, but having a Makefile in main, would
>>>>> make
>>>>>
>>>>>> us
>>>>>> one step closer on being compatible with the distros. To me this job
>>>>>> is
>>>>>> a
>>>>>> simple cleanup, not something we deadly need, but nice to have.
>>>>>>
>>>>>>
>>>>>> Some remarks regarding the missing options:
>>>>>>
>>>>>> --from <module>
>>>>>>> This is one of the more important options and one that I use
>>>>>>> frequently
>>>>>>> (also in the form --all:<module>).
>>>>>>> Note that if you are in <moduleA> and call 'make --from
>>>>>>> <moduleB>'
>>>>>>> then
>>>>>>> all modules are built
>>>>>>> a) which <moduleA> depends on
>>>>>>> b) but not those that <moduleB> depends on
>>>>>>> c) Both <moduleA> and <moduleB> are built.
>>>>>>>
>>>>>>> I have changed the documentation.
>>>>>>>
>>>>>>> I use the --all:<module> myself very often, and have changed the
>>>>>> documentation, because it is of course supported.
>>>>>>
>>>>>> The difference is that you do the call in main, but that is a minor
>>>>>> detail
>>>>>> that can be easily corrected (have <module>/Makefile calling
>>>>>> main/Makefile.
>>>>>>
>>>>>> I have also changed documentation on --html due to juergens comments.
>>>>>>
>>>>>> I am not sure that we understand --from and --since in the same way
>>>>>> so
>>>>>>
>>>>> I
>>>>> will try to explain what I think they do.
>>>>>
>>>>> Let's imagine that we have a simple project with modules A, B, C, D and
>>>>> E.
>>>>> where B depends on A, C on B, D on C, and E on D.
>>>>> A ' make all' would mean 'make E'. The dependencies would then lead to
>>>>> building modules A, B, C, D, E in this order.
>>>>> If I am in E and call 'make --from C' then only C, D, and E should be
>>>>> built. A 'make --since C' would only build D and E.
>>>>>
>>>>> If I am in D and call 'make --from B' then modules B, C, and D are
>>>>> built.
>>>>> Call 'make --since B' to build only C and D.
>>>>> Note that 'make --from' accepts more than one module name (while 'make
>>>>> --all:<module>' does not).
>>>>> Note also that in the above case (stand in D, call 'make --from B')
>>>>> module
>>>>> A is not built, regardless of whether there are changes in A or not.
>>>>> Whereas a simple call to make (still standing in D) would build all
>>>>> modules that D depends on, directly or indirectly. Thus the options
>>>>> '--from' and '--since' exist to actively exclude modules from being
>>>>> built.
>>>>>
>>>>> The whole thing becomes a little bit more complicated with multiple
>>>>> options to '--from' (I never use '--since' and also don't know a valid
>>>>> use
>>>>> case so I will ignore it for now) and more complex dependencies then in
>>>>> the
>>>>> simple example above. Let's say that if we stand in instsetoo_native
>>>>> and
>>>>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This
>>>>> would
>>>>> build svx, sfx2 and all modules that depend (directly or indirectly) on
>>>>> svx
>>>>> OR sfx2.
>>>>>
>>>>> got it, now I just have one problem, why would you not build the
>>>>>
>>>> dependent
>>>> modules, if they needed to be built, thats a scenario I dont understand.
>>>> With a central makefile, <module>/makefile will not be called so we do
>>>> not
>>>> waste cpu cycles.
>>>>
>>>> With the .done files, we know when a module was last built and all
>>>> modules
>>>> that depend it should be rebuilt which the rule
>>>> <module>.done : <module_depend>.done
>>>>
>>>> will ensure, so If we have A -> B -> C -> D
>>>>
>>>> I go in B, and call make, then when I go in D and make, B,C,D will be
>>>> made.
>>>>
>>>> If we have A -> B -> D C -> D
>>>> and do the same then only D will be made.
>>>>
>>>> So --from is not really saving anything ?
>>>>
>>>> a) In your example you first go into B then, in a second step, into D.
>>> The '--from' option lets you do the same (well, not really the same,
>>> but
>>> see below) just from D.
>>>
>>> b) You go first to B and call make. This makes A, if necessary, then B.
>>> The making of A is exactly the thing that you want to prevent with the
>>> '--from' option. Go into D and call 'make --from B'. A is not built.
>>>
>>> Actually I go to main and say "make D", that will cause B but not A to
>> be
>> built.
>>
>
> I guess that calling 'make D' from main/ would be equivalent to call it
> from instsetoo_native/. But the --from option also works from inside other
> directories. It can then be used to build a 'range' of modules, ie. go
> into module Y/ and call 'make --from X' and all modules from X/ to Y/ will
> be built. A real life example for using this method could be:
>
> 1. Make some changes to documentation but not code in sfx2/.
> 2. Make some changes in svx/.
> 3. Then go to sw/ and call 'make --from svx'
>
> This prevents the changes in sfx2/ to trigger recompilation. It builds
> only modules svx/ and everything that depends on it until sw/ is reached.
> The costly recreation of install sets (especially when there are multiple
> languages) is omitted.
>
> With .done files you could do this in main/ with
> make done_before_sfx2
> make sw
> The 'done_before_svx' could be implemented similar to the regular svx
> target like this:
>
> done_before_svx : done_l10n ... done_libxslt
> # nothing to be done for done_before...
>
> done_l10n : ...
> touch <some-path>/l10n.done
yes this would be piece of cake to implement, more editing than thinking.
>
>
>
>>
>> c) After the discussion with you I am not sure if we still need --from
>>> because the two reasons I know for its existence my not be relevant with
>>> the new approach.
>>>
>>> Thx, I think the discussion is important. I am always afraid (still
>> feeling
>> newbee) to miss something, and at the same time many other say "we are
>> used
>> to". I think the right thing to do is somewhere in the middle, and we can
>> only reach it by discussing it.
>>
>
> I am now working for about 13 years on OpenOffice but due to its size and
> complexity I know only a small part (mostly Impress UI and slideshow).
> Everywhere else I am a newbee, too.
> And there is always the possibility to miss something. I think it is
> important to prepare as good as possible but at some point you just have to
> implement your changes/features and handle the consequences. That means
> that there will be errors. But those can be fixed. Part of being a
> developer in an open source project is to find this point when planning
> stops and development starts and when development is finished to 'weather
> the storm' by fixing bugs and handling criticism.
And keeping the "storm" in a branch, helps against criticism.
By the way, when we talk about development (excluding another theme) even
though I often talk "loudly" I do not feel critized, I merely feel many of
us have strong feelings and sometimes express them a bit bombastic.
Actually this discussion is to me a perfect example of how open source
developments works when its best.
rgds
jan I.
>
>
>> I find discussion where e.g. you say "I need option X because" very
>> fruitfull, because they are based on actual need, and not just "nice to
>> have".
>>
>>
>> c1) With the '--from' option you can tweak the dependency rules at
>>> runtime
>>> (a bit). This allows you to exclude projects from being built when you
>>> know that that is not necessary. But from experience I know that can
>>> lead
>>> to very subtle errors. Letting the system determine what to built is
>>> usually more reliable.
>>>
>>> you are a pro. and I am getting there, but all newcommers are not, and
>> to
>> attracts them we need to make it easy and fault tolerant !
>>
>
> Yes, absolutely. I am convinced that providing a system that does
> everything the pros need is more easy then make the same system reliable,
> fault tolerant and easy to use.
> Therefore focusing on the last three is a good strategy.
>
>
>
>> I am sure, I dont need to tell you how to tweek .done file to achieve what
>> you want :-)
>>
>>
>> c2) With build.pl a 'build --all' still builds every module on which the
>>> one you are standing in depends on. When those modules have been built
>>> previously, then no compilation takes place. But calling dmake for a
>>> couple of directories for close to 200 modules (when you stand in
>>> instsetoo_native) takes a lot of time (several minutes on Windows), even
>>> when no file has to be compiled. This wasteful way of doing nothing can
>>> be
>>> prevented with the --from option.
>>> However, with the new approach and its .done flag files you can determine
>>> which modules need to be built much faster. You don't have to call dmake
>>> on directories that where already built. Hm, but this again, does only
>>> work if your .done files have dependencies on all relevant source files.
>>> That is something that is missing at least from my script.
>>>
>>> Hmmm, I had a different idea, let me try that on you. If someone
>> changes a
>> source file, but does not do a make, its not intented to be public.
>> Meaning
>> we do not depend on the source files, but the actual run of make. Is that
>> a
>> wrong idea ?
>>
>
> I am not quite sure that I understand you. Let's look at an (abstract)
> example. There is a file code.cxx in module X/. I can build the whole
> module by calling 'make X' in main/. The question is, what happens on
> 'make all'? Currently, build.pl uses a brute force method and goes into
> every module, and in case of dmake modules, into every of their
> directories, and calls 'make'. This takes a couple of minutes (on
> Windows) when no files has been changed. That is why the --from option,
> and others like it, where created.
> The alternative is to use dependencies to let 'make' figure out what to
> build. These dependencies do not yet exist. There are dependencies
> between files, even when they are in different modules. So, when I change
> a file in sfx2/ and then build sw/ then some (or many) files in sw/ will be
> compiled. But there are no dependencies that tell us that sw/ has to be
> built. For this we need dependencies of .done files on the source file.
> When a source file changes then these dependencies will trigger the
> corresponding .done file to be made. This would then include building the
> module and touching the .done file.
>
> But that would create a lot of dependencies, at least one for each of our
> source files. And they all have to be loaded whenever 'make' is called.
> Loading the dependencies might take a few seconds. Determining the time
> stamps of all files involved might take longer. But gbuild when used
> Office wide would introduce more dependencies. But we still have to find
> out if the time it takes is acceptable when spend on every call of 'make'
> or if we need some shortcuts for the daily work.
>
>
>
>>
>>
>>> So, reasons for the existence of '--from' are a result of old/slow
>>> computers, slow files systems (still valid on Windows), missing global
>>> dependencies (which we now have for gbuild) and impatient developers.
>>>
>>>
>>>
>>> While this is easy to do with eg Perl I am not sure how to handle this
>>>>
>>>>> with just a Makefile. The straightforward approach with handling
>>>>> <module>.done files does not work. And that is one of the reasons why
>>>>> I
>>>>> don't think that (GNU) makefiles are a good solution for any problem.
>>>>> Most
>>>>> of us are used to program object oriented/imperative. Makefiles
>>>>> require
>>>>> a
>>>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>>>> Maybe
>>>>> it would be better to reimplement build.pl with a lot fewer options
>>>>> and
>>>>> with better readable code.
>>>>>
>>>>> I agree that makefiles are nowhere near a good solution to many of
>>>>> these
>>>>>
>>>> problems, but its like windows, I dont like it, but everybody uses it.
>>>>
>>>> We could easily write a new build.pl, that also took care of the local
>>>> makefiles, but our build system would not be in the mainstream, and e.g.
>>>> the distros would not like to integrate AOO.
>>>>
>>>> I have over the last years followed research in building systems, and
>>>> there
>>>> are (sadly enough) nobody that tries a real object oriented aproach.
>>>> Also
>>>> if you look at packages like visual studio, QT, eclipse they all use the
>>>> principle of makefiles with declarative approach.
>>>>
>>>> So my simple question is, do we want to approach the main road
>>>> (makefiles
>>>> for unix, visual studio for windows/mac) or do we want to have better
>>>> but
>>>> non standard system.
>>>>
>>>> Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
>>> ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
>>> pragmatist says to use the make. It is good enough for others, it is
>>> good
>>> enough for use.
>>> On the other hand, when I start a new project I usually start with the
>>> question of what are the best tools for the problem at hand. And make
>>> does
>>> not seem to be the first or the best answer. Look at our dmake or gbuild
>>> system. Both don't use make in a standard way. gmake even defines its
>>> own
>>> language, object oriented and imperative, on top of the GNU make macros.
>>> That is, for me, an act of desperation.
>>>
>>> I just wonder did you see "gütterdämmerungen" (I hope I spelled it
>> right,
>>
> almost, it is Götterdämmerung
>
> its 2 years since I lived in Vienna so its a bit rusty), I saw it on
>> danube, 2 nights in a row.
>>
> I have not yet have much exposure to Wagner. But seeing this in Vienna on
> the Danube must have been an unforgettable spectacle.
>
>
>
>>
>> I have made experiments with an alternative approach, a domain specific
>>> language somewhat similar to Java. I personally like this approach
>>> because
>>> a) it uses the paradigm that I already use when writing C++ code. That
>>> means that I can apply my existing knowledge to the build process and
>>> that
>>> I don't have to remember all the tricks and pitfalls of makefiles.
>>> b) as expected it was much easier to handle file dependencies and
>>> parallel
>>> processing of build jobs in Java than adding object orientation and
>>> imperative control flow to makefiles.
>>>
>>> I like your approach, and would like to help on that, BUT I think we
>> benefit as a community to stay on the main road.
>>
>
> Good point.
>
>
>
>>
>> If I had the time and if I would be the one working on it then I would
>>> prefer an non-Makefile approach. But maybe I am just suffering too much
>>> from one of the 'three great virtues of a programmer': hubris.
>>>
>>> No you are like me, just not understaning why the world isnt perfect :-)
>>
>
> :-)
>
>
>>
>> You are the one who leads the build project changes, so you have to
>>> decide
>>> which approach to use. I am not trying to make your life harder (than
>>> necessary), I am only trying to point out some of the pitfalls that I
>>> have
>>> encountered in the past. And to prevent you from removing features that
>>> I
>>> use :-)
>>>
>>
>> I do not decide things, I think about solutions, throw them at the
>> community, and see if the float. If they float I help implement them.
>>
>> I am a doer, but also a listener. Implementing what I think is right,
>> would
>> be dead wrong, but I pave the way and make us discuss/think sometimes by
>> provoking a bit, which I think is good.
>>
>
> A good approach in general, but I don't think that it would be wrong when
> you implemented what you think is right, if you do it after a discussion.
>
> -Andre
>
>
>>
>>
>>>
>>>
>>> rgds
>>>> jan I.
>>>>
>>>> Ps. its always refreshing to discuss with you, you often have an
>>>> alternative approach, which is not just a dream but doable.
>>>>
>>>> Thanks. That makes two of us.
>>>
>>> Have a nice weekend,
>>>
>>> same to you !
>>
>> rgds
>> jan I.
>>
>>
>> Andre
>>>
>>>
>>> -Andre
>>>>
>>>>>
>>>>> --prepare
>>>>>
>>>>>> Also one option that is important for our every day work. Use
>>>>>>> case:
>>>>>>> You make changes in <module> and are not sure if these changes are
>>>>>>> compatible/incompatible. To be on the safe side you discard the
>>>>>>> output
>>>>>>> of
>>>>>>> all depending modules. To save time you keep the output of all other
>>>>>>> modules.
>>>>>>>
>>>>>>> Often used together with '--from' like 'make --prepare --from
>>>>>>> svx' to
>>>>>>> prepare a build after making changes in svx.
>>>>>>>
>>>>>>> Documentation changed, funny thing is that svx does not clear
>>>>>>> correctly
>>>>>>>
>>>>>>> on
>>>>>> my ubuntu build.
>>>>>>
>>>>>>
>>>>>> --since <module>
>>>>>>
>>>>>> A variant of '--from'. The only difference is that <module>
>>>>>>> itself
>>>>>>> is
>>>>>>> not built.
>>>>>>>
>>>>>>> If your proposed approach is similar to what my script produces
>>>>>>> then
>>>>>>> it
>>>>>>> is not too difficult to support --from/--since. I made some
>>>>>>> experiments
>>>>>>> in
>>>>>>> this direction but was to lazy to finish them.
>>>>>>>
>>>>>>> My approach is very similar, but I failed to see how --since is
>>>>>>>
>>>>>>> supported.
>>>>>> And question is if its real important.
>>>>>>
>>>>>>
>>>>>> --job
>>>>>>
>>>>>> --pre_job
>>>>>>> --post_job
>>>>>>> These are sometimes handy to run a non-standard command for all
>>>>>>> modules.
>>>>>>>
>>>>>>> I have added them, they are by the way a good example why we need
>>>>>>> a
>>>>>>>
>>>>>>> discussion I have never used them.
>>>>>>
>>>>>> However maybe the real discussion is "do we want to replace build and
>>>>>> have
>>>>>> a main/Makefile instead?"
>>>>>>
>>>>>>
>>>>>>
>>>>>> - I have not used the rest of the unsupported options and would not
>>>>>> miss
>>>>>>
>>>>>> them. Others may have other sets of options that are important to
>>>>>>> them.
>>>>>>>
>>>>>>>
>>>>>>> Some general remarks:
>>>>>>>
>>>>>>> - Why keep one makefile per module? Why not put all the inter-module
>>>>>>> dependencies into one file (like my script does)?
>>>>>>>
>>>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>>>
>>>>>>> "main/Makefile"
>>>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today
>>>>>> just
>>>>>> will call the old makefiles as described in prj/build.lst
>>>>>>
>>>>>> - Why not use the oportunity to move (a part of) the build environment
>>>>>> out
>>>>>>
>>>>>> of the way to, say, build/ ?
>>>>>>
>>>>>>> You have guessed my next step.
>>>>>>>
>>>>>>> - How are dependencies between modules handled (just the manual
>>>>>>
>>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>>>> introduced
>>>>>>> by
>>>>>>> gmake).
>>>>>>>
>>>>>>> See doc. on --from. Its done with <module>.done files
>>>>>>>
>>>>>>> - How is the output of the individual calls to dmake or GNU make
>>>>>>
>>>>>> handled/made accessible. Ie. if there is a build error, how can I
>>>>>>> look
>>>>>>> up
>>>>>>> the corresponding build output?
>>>>>>>
>>>>>>> see doc. script make_log
>>>>>>>
>>>>>>> - Are the gmake makefiles included (run in the same process) or
>>>>>> is GNU
>>>>>>
>>>>>> make started for them it its own process?
>>>>>>>
>>>>>>> For a start they would be called (own process), but its something
>>>>>>> where
>>>>>>>
>>>>>>> I
>>>>>> have no strong opinions.
>>>>>>
>>>>>> Please (just to be sure), this proposal has nothing to do with the
>>>>>> students
>>>>>> work, its simply because I saw a positive discussion on removing
>>>>>> build.pl
>>>>>> ,
>>>>>> and spent a couple of hours looking at it. If there is a preference
>>>>>> not
>>>>>> to
>>>>>> remove build.pl I will simply forget it.
>>>>>>
>>>>>> rgds
>>>>>> jan I.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>>> Andre
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> It has been roughly tested it, thanks to a clever utility from
>>>>>>> andre.
>>>>>>>
>>>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>>>> considered in a makefile.
>>>>>>>>
>>>>>>>> My suggestion is on
>>>>>>>> http://wiki.openoffice.org/********wiki/Build_System_Analysis:****<http://wiki.openoffice.org/******wiki/Build_System_Analysis:**>
>>>>>>>> <http://wiki.openoffice.org/******wiki/Build_System_**Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>>> >
>>>>>>>> **<http://wiki.openoffice.org/******wiki/Build_System_**Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>>> <http://wiki.**openoffice.org/**wiki/Build_**System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>>>> >
>>>>>>>> build.pl_versus_makefile<http:******//wiki.openoffice.org/**
>>>>>>>> wiki/**** <http://wiki.openoffice.org/wiki/****><http://wiki.**
>>>>>>>> openoffice.org/wiki/** <http://wiki.openoffice.org/wiki/**>>
>>>>>>>> Build_System_Analysis:build.******pl_versus_makefile<http://**
>>>>>>>> wiki.openoffice.org/wiki/****Build_System_Analysis:build.**<http://wiki.openoffice.org/wiki/**Build_System_Analysis:build.**>
>>>>>>>> pl_versus_makefile<http://**wiki.openoffice.org/wiki/**
>>>>>>>> Build_System_Analysis:build.**pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>>>>> >
>>>>>>>>
>>>>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>>>> options
>>>>>>>> a
>>>>>>>> lot, and some of them might be in use.
>>>>>>>>
>>>>>>>> thanks in advance for your comments.
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------********------------------------**
>>>>>>>> --**
>>>>>>>> --**
>>>>>>>>
>>>>>>>> --**---------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.******a**
>>>>>>> pache.org<
>>>>>>> http://apache.org**>
>>>>>>> <dev-unsubscribe@**openoffice.****apache.org<http://**
>>>>>>> openoffice.apache.org <http://openoffice.apache.org>>
>>>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>>>> <de...@openoffice.apache.org>
>>>>>>> >
>>>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------******--------------------------**
>>>>>>> --**
>>>>>>>
>>>>>> --**---------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>> http://apache.org**>
>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>> <de...@openoffice.apache.org>
>>>>> >
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------****----------------------------**
>>> --**---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>> <de...@openoffice.apache.org>
>>> >
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: [proposal] replace build.pl with a central Makefile.
Posted by Andre Fischer <aw...@gmail.com>.
On 18.10.2013 19:54, janI wrote:
> On 18 October 2013 16:52, Andre Fischer <aw...@gmail.com> wrote:
>
>> On 18.10.2013 15:58, janI wrote:
>>
>>> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
>>>
>>> On 18.10.2013 14:02, janI wrote:
>>>> sd
>>>>>
>>>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>>>>
>>>>> On 18.10.2013 11:32, janI wrote:
>>>>>
>>>>>> Hi.
>>>>>>
>>>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>>>> made a
>>>>>>> proposal for a central Makefile located in main.
>>>>>>>
>>>>>>> Hi Jan,
>>>>>>>
>>>>>> it is great that you are going to improve this part of the build
>>>>>> system.
>>>>>> But I think that we need more details about how the proposed build
>>>>>> system
>>>>>> works. Without them I can not really evaluate the proposal.
>>>>>>
>>>>>> First of all, I agree with juergens remarks that this should be
>>>>>>
>>>>> discussed
>>>>> before implemented, hence the wiki page.
>>>>>
>>>>> Secondly this has nothing directly to do with the proposed build system,
>>>>> its a simple replacement of build.pl in the current system.
>>>>>
>>>>> Yes, that is how I understood it. I just did not know how to call the
>>>> build.pl replacement.
>>>>
>>>>
>>>>
>>>> I know that build.pl works, but having a Makefile in main, would make
>>>>> us
>>>>> one step closer on being compatible with the distros. To me this job is
>>>>> a
>>>>> simple cleanup, not something we deadly need, but nice to have.
>>>>>
>>>>>
>>>>> Some remarks regarding the missing options:
>>>>>
>>>>>> --from <module>
>>>>>> This is one of the more important options and one that I use
>>>>>> frequently
>>>>>> (also in the form --all:<module>).
>>>>>> Note that if you are in <moduleA> and call 'make --from <moduleB>'
>>>>>> then
>>>>>> all modules are built
>>>>>> a) which <moduleA> depends on
>>>>>> b) but not those that <moduleB> depends on
>>>>>> c) Both <moduleA> and <moduleB> are built.
>>>>>>
>>>>>> I have changed the documentation.
>>>>>>
>>>>> I use the --all:<module> myself very often, and have changed the
>>>>> documentation, because it is of course supported.
>>>>>
>>>>> The difference is that you do the call in main, but that is a minor
>>>>> detail
>>>>> that can be easily corrected (have <module>/Makefile calling
>>>>> main/Makefile.
>>>>>
>>>>> I have also changed documentation on --html due to juergens comments.
>>>>>
>>>>> I am not sure that we understand --from and --since in the same way so
>>>> I
>>>> will try to explain what I think they do.
>>>>
>>>> Let's imagine that we have a simple project with modules A, B, C, D and
>>>> E.
>>>> where B depends on A, C on B, D on C, and E on D.
>>>> A ' make all' would mean 'make E'. The dependencies would then lead to
>>>> building modules A, B, C, D, E in this order.
>>>> If I am in E and call 'make --from C' then only C, D, and E should be
>>>> built. A 'make --since C' would only build D and E.
>>>>
>>>> If I am in D and call 'make --from B' then modules B, C, and D are built.
>>>> Call 'make --since B' to build only C and D.
>>>> Note that 'make --from' accepts more than one module name (while 'make
>>>> --all:<module>' does not).
>>>> Note also that in the above case (stand in D, call 'make --from B')
>>>> module
>>>> A is not built, regardless of whether there are changes in A or not.
>>>> Whereas a simple call to make (still standing in D) would build all
>>>> modules that D depends on, directly or indirectly. Thus the options
>>>> '--from' and '--since' exist to actively exclude modules from being
>>>> built.
>>>>
>>>> The whole thing becomes a little bit more complicated with multiple
>>>> options to '--from' (I never use '--since' and also don't know a valid
>>>> use
>>>> case so I will ignore it for now) and more complex dependencies then in
>>>> the
>>>> simple example above. Let's say that if we stand in instsetoo_native and
>>>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This would
>>>> build svx, sfx2 and all modules that depend (directly or indirectly) on
>>>> svx
>>>> OR sfx2.
>>>>
>>>> got it, now I just have one problem, why would you not build the
>>> dependent
>>> modules, if they needed to be built, thats a scenario I dont understand.
>>> With a central makefile, <module>/makefile will not be called so we do not
>>> waste cpu cycles.
>>>
>>> With the .done files, we know when a module was last built and all modules
>>> that depend it should be rebuilt which the rule
>>> <module>.done : <module_depend>.done
>>>
>>> will ensure, so If we have A -> B -> C -> D
>>>
>>> I go in B, and call make, then when I go in D and make, B,C,D will be
>>> made.
>>>
>>> If we have A -> B -> D C -> D
>>> and do the same then only D will be made.
>>>
>>> So --from is not really saving anything ?
>>>
>> a) In your example you first go into B then, in a second step, into D.
>> The '--from' option lets you do the same (well, not really the same, but
>> see below) just from D.
>>
>> b) You go first to B and call make. This makes A, if necessary, then B.
>> The making of A is exactly the thing that you want to prevent with the
>> '--from' option. Go into D and call 'make --from B'. A is not built.
>>
> Actually I go to main and say "make D", that will cause B but not A to be
> built.
I guess that calling 'make D' from main/ would be equivalent to call it
from instsetoo_native/. But the --from option also works from inside
other directories. It can then be used to build a 'range' of modules,
ie. go into module Y/ and call 'make --from X' and all modules from X/
to Y/ will be built. A real life example for using this method could be:
1. Make some changes to documentation but not code in sfx2/.
2. Make some changes in svx/.
3. Then go to sw/ and call 'make --from svx'
This prevents the changes in sfx2/ to trigger recompilation. It builds
only modules svx/ and everything that depends on it until sw/ is
reached. The costly recreation of install sets (especially when there
are multiple languages) is omitted.
With .done files you could do this in main/ with
make done_before_sfx2
make sw
The 'done_before_svx' could be implemented similar to the regular svx
target like this:
done_before_svx : done_l10n ... done_libxslt
# nothing to be done for done_before...
done_l10n : ...
touch <some-path>/l10n.done
>
>
>> c) After the discussion with you I am not sure if we still need --from
>> because the two reasons I know for its existence my not be relevant with
>> the new approach.
>>
> Thx, I think the discussion is important. I am always afraid (still feeling
> newbee) to miss something, and at the same time many other say "we are used
> to". I think the right thing to do is somewhere in the middle, and we can
> only reach it by discussing it.
I am now working for about 13 years on OpenOffice but due to its size
and complexity I know only a small part (mostly Impress UI and
slideshow). Everywhere else I am a newbee, too.
And there is always the possibility to miss something. I think it is
important to prepare as good as possible but at some point you just have
to implement your changes/features and handle the consequences. That
means that there will be errors. But those can be fixed. Part of being
a developer in an open source project is to find this point when
planning stops and development starts and when development is finished
to 'weather the storm' by fixing bugs and handling criticism.
>
> I find discussion where e.g. you say "I need option X because" very
> fruitfull, because they are based on actual need, and not just "nice to
> have".
>
>
>> c1) With the '--from' option you can tweak the dependency rules at runtime
>> (a bit). This allows you to exclude projects from being built when you
>> know that that is not necessary. But from experience I know that can lead
>> to very subtle errors. Letting the system determine what to built is
>> usually more reliable.
>>
> you are a pro. and I am getting there, but all newcommers are not, and to
> attracts them we need to make it easy and fault tolerant !
Yes, absolutely. I am convinced that providing a system that does
everything the pros need is more easy then make the same system
reliable, fault tolerant and easy to use.
Therefore focusing on the last three is a good strategy.
>
> I am sure, I dont need to tell you how to tweek .done file to achieve what
> you want :-)
>
>
>> c2) With build.pl a 'build --all' still builds every module on which the
>> one you are standing in depends on. When those modules have been built
>> previously, then no compilation takes place. But calling dmake for a
>> couple of directories for close to 200 modules (when you stand in
>> instsetoo_native) takes a lot of time (several minutes on Windows), even
>> when no file has to be compiled. This wasteful way of doing nothing can be
>> prevented with the --from option.
>> However, with the new approach and its .done flag files you can determine
>> which modules need to be built much faster. You don't have to call dmake
>> on directories that where already built. Hm, but this again, does only
>> work if your .done files have dependencies on all relevant source files.
>> That is something that is missing at least from my script.
>>
> Hmmm, I had a different idea, let me try that on you. If someone changes a
> source file, but does not do a make, its not intented to be public. Meaning
> we do not depend on the source files, but the actual run of make. Is that a
> wrong idea ?
I am not quite sure that I understand you. Let's look at an (abstract)
example. There is a file code.cxx in module X/. I can build the whole
module by calling 'make X' in main/. The question is, what happens on
'make all'? Currently, build.pl uses a brute force method and goes into
every module, and in case of dmake modules, into every of their
directories, and calls 'make'. This takes a couple of minutes (on
Windows) when no files has been changed. That is why the --from option,
and others like it, where created.
The alternative is to use dependencies to let 'make' figure out what to
build. These dependencies do not yet exist. There are dependencies
between files, even when they are in different modules. So, when I
change a file in sfx2/ and then build sw/ then some (or many) files in
sw/ will be compiled. But there are no dependencies that tell us that
sw/ has to be built. For this we need dependencies of .done files on
the source file. When a source file changes then these dependencies will
trigger the corresponding .done file to be made. This would then
include building the module and touching the .done file.
But that would create a lot of dependencies, at least one for each of
our source files. And they all have to be loaded whenever 'make' is
called. Loading the dependencies might take a few seconds. Determining
the time stamps of all files involved might take longer. But gbuild
when used Office wide would introduce more dependencies. But we still
have to find out if the time it takes is acceptable when spend on every
call of 'make' or if we need some shortcuts for the daily work.
>
>
>>
>> So, reasons for the existence of '--from' are a result of old/slow
>> computers, slow files systems (still valid on Windows), missing global
>> dependencies (which we now have for gbuild) and impatient developers.
>>
>>
>>
>>> While this is easy to do with eg Perl I am not sure how to handle this
>>>> with just a Makefile. The straightforward approach with handling
>>>> <module>.done files does not work. And that is one of the reasons why I
>>>> don't think that (GNU) makefiles are a good solution for any problem.
>>>> Most
>>>> of us are used to program object oriented/imperative. Makefiles require
>>>> a
>>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>>> Maybe
>>>> it would be better to reimplement build.pl with a lot fewer options and
>>>> with better readable code.
>>>>
>>>> I agree that makefiles are nowhere near a good solution to many of these
>>> problems, but its like windows, I dont like it, but everybody uses it.
>>>
>>> We could easily write a new build.pl, that also took care of the local
>>> makefiles, but our build system would not be in the mainstream, and e.g.
>>> the distros would not like to integrate AOO.
>>>
>>> I have over the last years followed research in building systems, and
>>> there
>>> are (sadly enough) nobody that tries a real object oriented aproach. Also
>>> if you look at packages like visual studio, QT, eclipse they all use the
>>> principle of makefiles with declarative approach.
>>>
>>> So my simple question is, do we want to approach the main road (makefiles
>>> for unix, visual studio for windows/mac) or do we want to have better but
>>> non standard system.
>>>
>> Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
>> ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
>> pragmatist says to use the make. It is good enough for others, it is good
>> enough for use.
>> On the other hand, when I start a new project I usually start with the
>> question of what are the best tools for the problem at hand. And make does
>> not seem to be the first or the best answer. Look at our dmake or gbuild
>> system. Both don't use make in a standard way. gmake even defines its own
>> language, object oriented and imperative, on top of the GNU make macros.
>> That is, for me, an act of desperation.
>>
> I just wonder did you see "gütterdämmerungen" (I hope I spelled it right,
almost, it is Götterdämmerung
> its 2 years since I lived in Vienna so its a bit rusty), I saw it on
> danube, 2 nights in a row.
I have not yet have much exposure to Wagner. But seeing this in Vienna
on the Danube must have been an unforgettable spectacle.
>
>
>> I have made experiments with an alternative approach, a domain specific
>> language somewhat similar to Java. I personally like this approach because
>> a) it uses the paradigm that I already use when writing C++ code. That
>> means that I can apply my existing knowledge to the build process and that
>> I don't have to remember all the tricks and pitfalls of makefiles.
>> b) as expected it was much easier to handle file dependencies and parallel
>> processing of build jobs in Java than adding object orientation and
>> imperative control flow to makefiles.
>>
> I like your approach, and would like to help on that, BUT I think we
> benefit as a community to stay on the main road.
Good point.
>
>
>> If I had the time and if I would be the one working on it then I would
>> prefer an non-Makefile approach. But maybe I am just suffering too much
>> from one of the 'three great virtues of a programmer': hubris.
>>
> No you are like me, just not understaning why the world isnt perfect :-)
:-)
>
>
>> You are the one who leads the build project changes, so you have to decide
>> which approach to use. I am not trying to make your life harder (than
>> necessary), I am only trying to point out some of the pitfalls that I have
>> encountered in the past. And to prevent you from removing features that I
>> use :-)
>
> I do not decide things, I think about solutions, throw them at the
> community, and see if the float. If they float I help implement them.
>
> I am a doer, but also a listener. Implementing what I think is right, would
> be dead wrong, but I pave the way and make us discuss/think sometimes by
> provoking a bit, which I think is good.
A good approach in general, but I don't think that it would be wrong
when you implemented what you think is right, if you do it after a
discussion.
-Andre
>
>
>>
>>
>>
>>> rgds
>>> jan I.
>>>
>>> Ps. its always refreshing to discuss with you, you often have an
>>> alternative approach, which is not just a dream but doable.
>>>
>> Thanks. That makes two of us.
>>
>> Have a nice weekend,
>>
> same to you !
>
> rgds
> jan I.
>
>
>> Andre
>>
>>
>>> -Andre
>>>>
>>>> --prepare
>>>>>> Also one option that is important for our every day work. Use
>>>>>> case:
>>>>>> You make changes in <module> and are not sure if these changes are
>>>>>> compatible/incompatible. To be on the safe side you discard the output
>>>>>> of
>>>>>> all depending modules. To save time you keep the output of all other
>>>>>> modules.
>>>>>>
>>>>>> Often used together with '--from' like 'make --prepare --from
>>>>>> svx' to
>>>>>> prepare a build after making changes in svx.
>>>>>>
>>>>>> Documentation changed, funny thing is that svx does not clear
>>>>>> correctly
>>>>>>
>>>>> on
>>>>> my ubuntu build.
>>>>>
>>>>>
>>>>> --since <module>
>>>>>
>>>>>> A variant of '--from'. The only difference is that <module>
>>>>>> itself
>>>>>> is
>>>>>> not built.
>>>>>>
>>>>>> If your proposed approach is similar to what my script produces
>>>>>> then
>>>>>> it
>>>>>> is not too difficult to support --from/--since. I made some
>>>>>> experiments
>>>>>> in
>>>>>> this direction but was to lazy to finish them.
>>>>>>
>>>>>> My approach is very similar, but I failed to see how --since is
>>>>>>
>>>>> supported.
>>>>> And question is if its real important.
>>>>>
>>>>>
>>>>> --job
>>>>>
>>>>>> --pre_job
>>>>>> --post_job
>>>>>> These are sometimes handy to run a non-standard command for all
>>>>>> modules.
>>>>>>
>>>>>> I have added them, they are by the way a good example why we need a
>>>>>>
>>>>> discussion I have never used them.
>>>>>
>>>>> However maybe the real discussion is "do we want to replace build and
>>>>> have
>>>>> a main/Makefile instead?"
>>>>>
>>>>>
>>>>>
>>>>> - I have not used the rest of the unsupported options and would not
>>>>> miss
>>>>>
>>>>>> them. Others may have other sets of options that are important to
>>>>>> them.
>>>>>>
>>>>>>
>>>>>> Some general remarks:
>>>>>>
>>>>>> - Why keep one makefile per module? Why not put all the inter-module
>>>>>> dependencies into one file (like my script does)?
>>>>>>
>>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>>
>>>>> "main/Makefile"
>>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today just
>>>>> will call the old makefiles as described in prj/build.lst
>>>>>
>>>>> - Why not use the oportunity to move (a part of) the build environment
>>>>> out
>>>>>
>>>>> of the way to, say, build/ ?
>>>>>> You have guessed my next step.
>>>>>>
>>>>> - How are dependencies between modules handled (just the manual
>>>>>
>>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>>> introduced
>>>>>> by
>>>>>> gmake).
>>>>>>
>>>>>> See doc. on --from. Its done with <module>.done files
>>>>>>
>>>>> - How is the output of the individual calls to dmake or GNU make
>>>>>
>>>>>> handled/made accessible. Ie. if there is a build error, how can I look
>>>>>> up
>>>>>> the corresponding build output?
>>>>>>
>>>>>> see doc. script make_log
>>>>>>
>>>>> - Are the gmake makefiles included (run in the same process) or is GNU
>>>>>
>>>>>> make started for them it its own process?
>>>>>>
>>>>>> For a start they would be called (own process), but its something
>>>>>> where
>>>>>>
>>>>> I
>>>>> have no strong opinions.
>>>>>
>>>>> Please (just to be sure), this proposal has nothing to do with the
>>>>> students
>>>>> work, its simply because I saw a positive discussion on removing
>>>>> build.pl
>>>>> ,
>>>>> and spent a couple of hours looking at it. If there is a preference not
>>>>> to
>>>>> remove build.pl I will simply forget it.
>>>>>
>>>>> rgds
>>>>> jan I.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>> Andre
>>>>>>
>>>>>>
>>>>>>
>>>>>> It has been roughly tested it, thanks to a clever utility from andre.
>>>>>>
>>>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>>> considered in a makefile.
>>>>>>>
>>>>>>> My suggestion is on
>>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****<http://wiki.openoffice.org/wiki/**>
>>>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://**
>>>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.**
>>>>>>> pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>>>>
>>>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>>> options
>>>>>>> a
>>>>>>> lot, and some of them might be in use.
>>>>>>>
>>>>>>> thanks in advance for your comments.
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------******--------------------------**
>>>>>>> --**
>>>>>>>
>>>>>> --**---------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>>> http://apache.org**>
>>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>>> <de...@openoffice.apache.org>
>>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------****----------------------------**
>>>> --**---------
>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>>> <de...@openoffice.apache.org>
>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>
>>>>
>>>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [proposal] replace build.pl with a central Makefile.
Posted by janI <ja...@apache.org>.
On 18 October 2013 16:52, Andre Fischer <aw...@gmail.com> wrote:
> On 18.10.2013 15:58, janI wrote:
>
>> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
>>
>> On 18.10.2013 14:02, janI wrote:
>>>
>>> sd
>>>>
>>>>
>>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>>>
>>>> On 18.10.2013 11:32, janI wrote:
>>>>
>>>>> Hi.
>>>>>
>>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>>> made a
>>>>>> proposal for a central Makefile located in main.
>>>>>>
>>>>>> Hi Jan,
>>>>>>
>>>>> it is great that you are going to improve this part of the build
>>>>> system.
>>>>> But I think that we need more details about how the proposed build
>>>>> system
>>>>> works. Without them I can not really evaluate the proposal.
>>>>>
>>>>> First of all, I agree with juergens remarks that this should be
>>>>>
>>>> discussed
>>>> before implemented, hence the wiki page.
>>>>
>>>> Secondly this has nothing directly to do with the proposed build system,
>>>> its a simple replacement of build.pl in the current system.
>>>>
>>>> Yes, that is how I understood it. I just did not know how to call the
>>> build.pl replacement.
>>>
>>>
>>>
>>> I know that build.pl works, but having a Makefile in main, would make
>>>> us
>>>> one step closer on being compatible with the distros. To me this job is
>>>> a
>>>> simple cleanup, not something we deadly need, but nice to have.
>>>>
>>>>
>>>> Some remarks regarding the missing options:
>>>>
>>>>> --from <module>
>>>>> This is one of the more important options and one that I use
>>>>> frequently
>>>>> (also in the form --all:<module>).
>>>>> Note that if you are in <moduleA> and call 'make --from <moduleB>'
>>>>> then
>>>>> all modules are built
>>>>> a) which <moduleA> depends on
>>>>> b) but not those that <moduleB> depends on
>>>>> c) Both <moduleA> and <moduleB> are built.
>>>>>
>>>>> I have changed the documentation.
>>>>>
>>>> I use the --all:<module> myself very often, and have changed the
>>>> documentation, because it is of course supported.
>>>>
>>>> The difference is that you do the call in main, but that is a minor
>>>> detail
>>>> that can be easily corrected (have <module>/Makefile calling
>>>> main/Makefile.
>>>>
>>>> I have also changed documentation on --html due to juergens comments.
>>>>
>>>> I am not sure that we understand --from and --since in the same way so
>>> I
>>> will try to explain what I think they do.
>>>
>>> Let's imagine that we have a simple project with modules A, B, C, D and
>>> E.
>>> where B depends on A, C on B, D on C, and E on D.
>>> A ' make all' would mean 'make E'. The dependencies would then lead to
>>> building modules A, B, C, D, E in this order.
>>> If I am in E and call 'make --from C' then only C, D, and E should be
>>> built. A 'make --since C' would only build D and E.
>>>
>>> If I am in D and call 'make --from B' then modules B, C, and D are built.
>>> Call 'make --since B' to build only C and D.
>>> Note that 'make --from' accepts more than one module name (while 'make
>>> --all:<module>' does not).
>>> Note also that in the above case (stand in D, call 'make --from B')
>>> module
>>> A is not built, regardless of whether there are changes in A or not.
>>> Whereas a simple call to make (still standing in D) would build all
>>> modules that D depends on, directly or indirectly. Thus the options
>>> '--from' and '--since' exist to actively exclude modules from being
>>> built.
>>>
>>> The whole thing becomes a little bit more complicated with multiple
>>> options to '--from' (I never use '--since' and also don't know a valid
>>> use
>>> case so I will ignore it for now) and more complex dependencies then in
>>> the
>>> simple example above. Let's say that if we stand in instsetoo_native and
>>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This would
>>> build svx, sfx2 and all modules that depend (directly or indirectly) on
>>> svx
>>> OR sfx2.
>>>
>>> got it, now I just have one problem, why would you not build the
>> dependent
>> modules, if they needed to be built, thats a scenario I dont understand.
>> With a central makefile, <module>/makefile will not be called so we do not
>> waste cpu cycles.
>>
>> With the .done files, we know when a module was last built and all modules
>> that depend it should be rebuilt which the rule
>> <module>.done : <module_depend>.done
>>
>> will ensure, so If we have A -> B -> C -> D
>>
>> I go in B, and call make, then when I go in D and make, B,C,D will be
>> made.
>>
>> If we have A -> B -> D C -> D
>> and do the same then only D will be made.
>>
>> So --from is not really saving anything ?
>>
>
> a) In your example you first go into B then, in a second step, into D.
> The '--from' option lets you do the same (well, not really the same, but
> see below) just from D.
>
> b) You go first to B and call make. This makes A, if necessary, then B.
> The making of A is exactly the thing that you want to prevent with the
> '--from' option. Go into D and call 'make --from B'. A is not built.
>
Actually I go to main and say "make D", that will cause B but not A to be
built.
>
> c) After the discussion with you I am not sure if we still need --from
> because the two reasons I know for its existence my not be relevant with
> the new approach.
>
Thx, I think the discussion is important. I am always afraid (still feeling
newbee) to miss something, and at the same time many other say "we are used
to". I think the right thing to do is somewhere in the middle, and we can
only reach it by discussing it.
I find discussion where e.g. you say "I need option X because" very
fruitfull, because they are based on actual need, and not just "nice to
have".
>
> c1) With the '--from' option you can tweak the dependency rules at runtime
> (a bit). This allows you to exclude projects from being built when you
> know that that is not necessary. But from experience I know that can lead
> to very subtle errors. Letting the system determine what to built is
> usually more reliable.
>
you are a pro. and I am getting there, but all newcommers are not, and to
attracts them we need to make it easy and fault tolerant !
I am sure, I dont need to tell you how to tweek .done file to achieve what
you want :-)
>
> c2) With build.pl a 'build --all' still builds every module on which the
> one you are standing in depends on. When those modules have been built
> previously, then no compilation takes place. But calling dmake for a
> couple of directories for close to 200 modules (when you stand in
> instsetoo_native) takes a lot of time (several minutes on Windows), even
> when no file has to be compiled. This wasteful way of doing nothing can be
> prevented with the --from option.
> However, with the new approach and its .done flag files you can determine
> which modules need to be built much faster. You don't have to call dmake
> on directories that where already built. Hm, but this again, does only
> work if your .done files have dependencies on all relevant source files.
> That is something that is missing at least from my script.
>
Hmmm, I had a different idea, let me try that on you. If someone changes a
source file, but does not do a make, its not intented to be public. Meaning
we do not depend on the source files, but the actual run of make. Is that a
wrong idea ?
>
>
> So, reasons for the existence of '--from' are a result of old/slow
> computers, slow files systems (still valid on Windows), missing global
> dependencies (which we now have for gbuild) and impatient developers.
>
>
>
>>
>> While this is easy to do with eg Perl I am not sure how to handle this
>>> with just a Makefile. The straightforward approach with handling
>>> <module>.done files does not work. And that is one of the reasons why I
>>> don't think that (GNU) makefiles are a good solution for any problem.
>>> Most
>>> of us are used to program object oriented/imperative. Makefiles require
>>> a
>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>> Maybe
>>> it would be better to reimplement build.pl with a lot fewer options and
>>> with better readable code.
>>>
>>> I agree that makefiles are nowhere near a good solution to many of these
>> problems, but its like windows, I dont like it, but everybody uses it.
>>
>> We could easily write a new build.pl, that also took care of the local
>> makefiles, but our build system would not be in the mainstream, and e.g.
>> the distros would not like to integrate AOO.
>>
>> I have over the last years followed research in building systems, and
>> there
>> are (sadly enough) nobody that tries a real object oriented aproach. Also
>> if you look at packages like visual studio, QT, eclipse they all use the
>> principle of makefiles with declarative approach.
>>
>> So my simple question is, do we want to approach the main road (makefiles
>> for unix, visual studio for windows/mac) or do we want to have better but
>> non standard system.
>>
>
> Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
> ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
> pragmatist says to use the make. It is good enough for others, it is good
> enough for use.
> On the other hand, when I start a new project I usually start with the
> question of what are the best tools for the problem at hand. And make does
> not seem to be the first or the best answer. Look at our dmake or gbuild
> system. Both don't use make in a standard way. gmake even defines its own
> language, object oriented and imperative, on top of the GNU make macros.
> That is, for me, an act of desperation.
>
I just wonder did you see "gütterdämmerungen" (I hope I spelled it right,
its 2 years since I lived in Vienna so its a bit rusty), I saw it on
danube, 2 nights in a row.
> I have made experiments with an alternative approach, a domain specific
> language somewhat similar to Java. I personally like this approach because
> a) it uses the paradigm that I already use when writing C++ code. That
> means that I can apply my existing knowledge to the build process and that
> I don't have to remember all the tricks and pitfalls of makefiles.
> b) as expected it was much easier to handle file dependencies and parallel
> processing of build jobs in Java than adding object orientation and
> imperative control flow to makefiles.
>
I like your approach, and would like to help on that, BUT I think we
benefit as a community to stay on the main road.
>
> If I had the time and if I would be the one working on it then I would
> prefer an non-Makefile approach. But maybe I am just suffering too much
> from one of the 'three great virtues of a programmer': hubris.
>
No you are like me, just not understaning why the world isnt perfect :-)
>
> You are the one who leads the build project changes, so you have to decide
> which approach to use. I am not trying to make your life harder (than
> necessary), I am only trying to point out some of the pitfalls that I have
> encountered in the past. And to prevent you from removing features that I
> use :-)
I do not decide things, I think about solutions, throw them at the
community, and see if the float. If they float I help implement them.
I am a doer, but also a listener. Implementing what I think is right, would
be dead wrong, but I pave the way and make us discuss/think sometimes by
provoking a bit, which I think is good.
>
>
>
>
>> rgds
>> jan I.
>>
>> Ps. its always refreshing to discuss with you, you often have an
>> alternative approach, which is not just a dream but doable.
>>
>
> Thanks. That makes two of us.
>
> Have a nice weekend,
>
same to you !
rgds
jan I.
> Andre
>
>
>>
>> -Andre
>>>
>>>
>>> --prepare
>>>>
>>>>> Also one option that is important for our every day work. Use
>>>>> case:
>>>>> You make changes in <module> and are not sure if these changes are
>>>>> compatible/incompatible. To be on the safe side you discard the output
>>>>> of
>>>>> all depending modules. To save time you keep the output of all other
>>>>> modules.
>>>>>
>>>>> Often used together with '--from' like 'make --prepare --from
>>>>> svx' to
>>>>> prepare a build after making changes in svx.
>>>>>
>>>>> Documentation changed, funny thing is that svx does not clear
>>>>> correctly
>>>>>
>>>> on
>>>> my ubuntu build.
>>>>
>>>>
>>>> --since <module>
>>>>
>>>>> A variant of '--from'. The only difference is that <module>
>>>>> itself
>>>>> is
>>>>> not built.
>>>>>
>>>>> If your proposed approach is similar to what my script produces
>>>>> then
>>>>> it
>>>>> is not too difficult to support --from/--since. I made some
>>>>> experiments
>>>>> in
>>>>> this direction but was to lazy to finish them.
>>>>>
>>>>> My approach is very similar, but I failed to see how --since is
>>>>>
>>>> supported.
>>>> And question is if its real important.
>>>>
>>>>
>>>> --job
>>>>
>>>>> --pre_job
>>>>> --post_job
>>>>> These are sometimes handy to run a non-standard command for all
>>>>> modules.
>>>>>
>>>>> I have added them, they are by the way a good example why we need a
>>>>>
>>>> discussion I have never used them.
>>>>
>>>> However maybe the real discussion is "do we want to replace build and
>>>> have
>>>> a main/Makefile instead?"
>>>>
>>>>
>>>>
>>>> - I have not used the rest of the unsupported options and would not
>>>> miss
>>>>
>>>>> them. Others may have other sets of options that are important to
>>>>> them.
>>>>>
>>>>>
>>>>> Some general remarks:
>>>>>
>>>>> - Why keep one makefile per module? Why not put all the inter-module
>>>>> dependencies into one file (like my script does)?
>>>>>
>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>
>>>> "main/Makefile"
>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today just
>>>> will call the old makefiles as described in prj/build.lst
>>>>
>>>> - Why not use the oportunity to move (a part of) the build environment
>>>> out
>>>>
>>>> of the way to, say, build/ ?
>>>>>
>>>>> You have guessed my next step.
>>>>>
>>>>
>>>> - How are dependencies between modules handled (just the manual
>>>>
>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>> introduced
>>>>> by
>>>>> gmake).
>>>>>
>>>>> See doc. on --from. Its done with <module>.done files
>>>>>
>>>>
>>>> - How is the output of the individual calls to dmake or GNU make
>>>>
>>>>> handled/made accessible. Ie. if there is a build error, how can I look
>>>>> up
>>>>> the corresponding build output?
>>>>>
>>>>> see doc. script make_log
>>>>>
>>>> - Are the gmake makefiles included (run in the same process) or is GNU
>>>>
>>>>> make started for them it its own process?
>>>>>
>>>>> For a start they would be called (own process), but its something
>>>>> where
>>>>>
>>>> I
>>>> have no strong opinions.
>>>>
>>>> Please (just to be sure), this proposal has nothing to do with the
>>>> students
>>>> work, its simply because I saw a positive discussion on removing
>>>> build.pl
>>>> ,
>>>> and spent a couple of hours looking at it. If there is a preference not
>>>> to
>>>> remove build.pl I will simply forget it.
>>>>
>>>> rgds
>>>> jan I.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Regards,
>>>>> Andre
>>>>>
>>>>>
>>>>>
>>>>> It has been roughly tested it, thanks to a clever utility from andre.
>>>>>
>>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>> considered in a makefile.
>>>>>>
>>>>>> My suggestion is on
>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>> >
>>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****<http://wiki.openoffice.org/wiki/**>
>>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://**
>>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.**
>>>>>> pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>>> >
>>>>>>
>>>>>>
>>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>> options
>>>>>> a
>>>>>> lot, and some of them might be in use.
>>>>>>
>>>>>> thanks in advance for your comments.
>>>>>>
>>>>>>
>>>>>> ------------------------------******--------------------------**
>>>>>> --**
>>>>>>
>>>>> --**---------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>> http://apache.org**>
>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>> <de...@openoffice.apache.org>
>>>>> >
>>>>>
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------****----------------------------**
>>> --**---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>> <de...@openoffice.apache.org>
>>> >
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: [proposal] replace build.pl with a central Makefile.
Posted by janI <ja...@apache.org>.
On 21 October 2013 15:52, Andre Fischer <aw...@gmail.com> wrote:
> On 21.10.2013 12:35, janI wrote:
>
>> [...]
>>
>> I have the following plan:
>> a) rename all Makefile to Makefile.gb, and correct prj/makefile.mk to the
>> new name
>>
>
> prj/makefile.mk usually only contains a simple forwarding. This would
> mean that making a gbuild module would require two forwards, one from the
> global Makefile to prj/makefile.mk and one from there to ./Makefile.
> Maybe prj/makefile.mk could be removed and ./Makefile be called directly
> from the global Makefile?
Would be a lot easier, I have looked at tweaking build.pl to accept make
along with nmake, and then call make instead of dmake, but I have had a lot
of success. Any help would be apriciated.
>
>
> b) for each <module> add a new Makefile, that basically call the existing
>> makefile.mk, as per your script (and build.lst) and update build.lst
>>
>
> I would suggest to do this forwarding with a macro, so that when there are
> any changes in the future, we only have to change the macro once instead of
> every forwarding call.
> A while back I modified the forwarding calls for gbuild modules (from prj/
> makefile.mk to ./Makefile) to include the debug flag. Would have been
> easier when all had used the same macro.
good idea.
>
>
> c) add deliver.lst to Makefile and update build.lst
>>
>
> Are deliver.lst and build.lst still used after this change? If not, can
> you delete them?
yes, I do that at a later step, when I am sure the new Makefile works.
>
>
>
>> once done, both build.pl and main/Makefile will work in the branch.
>>
>> d) have all of you test, and critic the idea.
>> e) remove */prj and build.pl
>> f) integrate in trunk.
>>
>
> Sounds good to me, with or without changes according to my suggestions.
>
thx.
rgds
jan I.
>
> -Andre
>
>
>> rgds
>> jan I.
>>
>>
>>
>>
>> -Andre
>>>
>>>
>>>
>>> hope you like it.
>>>> rgds
>>>> jan I.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> While this is easy to do with eg Perl I am not sure how to handle
>>>>>>
>>>>>>> this
>>>>>>>
>>>>>>> with just a Makefile. The straightforward approach with handling
>>>>>>>> <module>.done files does not work. And that is one of the reasons
>>>>>>>> why
>>>>>>>>
>>>>>>>> I
>>>>>>>
>>>>>> don't think that (GNU) makefiles are a good solution for any problem.
>>>>>>
>>>>>>> Most
>>>>>>>> of us are used to program object oriented/imperative. Makefiles
>>>>>>>>
>>>>>>>> require
>>>>>>>
>>>>>> a
>>>>>>
>>>>>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>>>>>>> Maybe
>>>>>>>> it would be better to reimplement build.pl with a lot fewer options
>>>>>>>>
>>>>>>>> and
>>>>>>>
>>>>>> with better readable code.
>>>>>>
>>>>>>> I agree that makefiles are nowhere near a good solution to many of
>>>>>>>>
>>>>>>>> these
>>>>>>>
>>>>>> problems, but its like windows, I dont like it, but everybody uses it.
>>>>>>
>>>>>>> We could easily write a new build.pl, that also took care of the
>>>>>>> local
>>>>>>> makefiles, but our build system would not be in the mainstream, and
>>>>>>> e.g.
>>>>>>> the distros would not like to integrate AOO.
>>>>>>>
>>>>>>> I have over the last years followed research in building systems, and
>>>>>>> there
>>>>>>> are (sadly enough) nobody that tries a real object oriented aproach.
>>>>>>>
>>>>>>> Also
>>>>>> if you look at packages like visual studio, QT, eclipse they all use
>>>>>> the
>>>>>>
>>>>>>> principle of makefiles with declarative approach.
>>>>>>>
>>>>>>> So my simple question is, do we want to approach the main road
>>>>>>>
>>>>>>> (makefiles
>>>>>> for unix, visual studio for windows/mac) or do we want to have better
>>>>>> but
>>>>>> non standard system.
>>>>>>
>>>>>>> Good analysis. Maybe I should answer with Faust: "Zwei Seelen
>>>>>>> wohnen,
>>>>>>>
>>>>>> ach! in meiner Brust" (two souls alas! are dwelling in my breast).
>>>>>> The
>>>>>> pragmatist says to use the make. It is good enough for others, it is
>>>>>>
>>>>>> good
>>>>>
>>>>> enough for use.
>>>>>> On the other hand, when I start a new project I usually start with the
>>>>>> question of what are the best tools for the problem at hand. And make
>>>>>>
>>>>>> does
>>>>>
>>>>> not seem to be the first or the best answer. Look at our dmake or
>>>>>> gbuild
>>>>>> system. Both don't use make in a standard way. gmake even defines its
>>>>>>
>>>>>> own
>>>>>
>>>>> language, object oriented and imperative, on top of the GNU make
>>>>>> macros.
>>>>>> That is, for me, an act of desperation.
>>>>>> I have made experiments with an alternative approach, a domain
>>>>>> specific
>>>>>> language somewhat similar to Java. I personally like this approach
>>>>>>
>>>>>> because
>>>>>
>>>>> a) it uses the paradigm that I already use when writing C++ code. That
>>>>>> means that I can apply my existing knowledge to the build process and
>>>>>>
>>>>>> that
>>>>>
>>>>> I don't have to remember all the tricks and pitfalls of makefiles.
>>>>>> b) as expected it was much easier to handle file dependencies and
>>>>>>
>>>>>> parallel
>>>>>
>>>>> processing of build jobs in Java than adding object orientation and
>>>>>> imperative control flow to makefiles.
>>>>>>
>>>>>> If I had the time and if I would be the one working on it then I would
>>>>>> prefer an non-Makefile approach. But maybe I am just suffering too
>>>>>> much
>>>>>> from one of the 'three great virtues of a programmer': hubris.
>>>>>>
>>>>>> You are the one who leads the build project changes, so you have to
>>>>>>
>>>>>> decide
>>>>>
>>>>> which approach to use. I am not trying to make your life harder (than
>>>>>> necessary), I am only trying to point out some of the pitfalls that I
>>>>>>
>>>>>> have
>>>>>
>>>>> encountered in the past. And to prevent you from removing features
>>>>>> that
>>>>>>
>>>>>> I
>>>>>
>>>>> use :-)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> rgds
>>>>>>
>>>>>>> jan I.
>>>>>>>
>>>>>>> Ps. its always refreshing to discuss with you, you often have an
>>>>>>> alternative approach, which is not just a dream but doable.
>>>>>>>
>>>>>>> Thanks. That makes two of us.
>>>>>>>
>>>>>> Have a nice weekend,
>>>>>> Andre
>>>>>>
>>>>>>
>>>>>>
>>>>>> -Andre
>>>>>>
>>>>>>> --prepare
>>>>>>>>
>>>>>>>> Also one option that is important for our every day work.
>>>>>>>>> Use
>>>>>>>>>
>>>>>>>>>> case:
>>>>>>>>>> You make changes in <module> and are not sure if these changes are
>>>>>>>>>> compatible/incompatible. To be on the safe side you discard the
>>>>>>>>>>
>>>>>>>>>> output
>>>>>>>>>
>>>>>>>> of
>>>>>>
>>>>>>> all depending modules. To save time you keep the output of all
>>>>>>>>>> other
>>>>>>>>>> modules.
>>>>>>>>>>
>>>>>>>>>> Often used together with '--from' like 'make --prepare
>>>>>>>>>> --from
>>>>>>>>>> svx' to
>>>>>>>>>> prepare a build after making changes in svx.
>>>>>>>>>>
>>>>>>>>>> Documentation changed, funny thing is that svx does not clear
>>>>>>>>>> correctly
>>>>>>>>>>
>>>>>>>>>> on
>>>>>>>>>>
>>>>>>>>> my ubuntu build.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --since <module>
>>>>>>>>>
>>>>>>>>> A variant of '--from'. The only difference is that
>>>>>>>>> <module>
>>>>>>>>>
>>>>>>>>>> itself
>>>>>>>>>> is
>>>>>>>>>> not built.
>>>>>>>>>>
>>>>>>>>>> If your proposed approach is similar to what my script
>>>>>>>>>> produces
>>>>>>>>>> then
>>>>>>>>>> it
>>>>>>>>>> is not too difficult to support --from/--since. I made some
>>>>>>>>>> experiments
>>>>>>>>>> in
>>>>>>>>>> this direction but was to lazy to finish them.
>>>>>>>>>>
>>>>>>>>>> My approach is very similar, but I failed to see how --since
>>>>>>>>>> is
>>>>>>>>>>
>>>>>>>>>> supported.
>>>>>>>>>>
>>>>>>>>> And question is if its real important.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --job
>>>>>>>>>
>>>>>>>>> --pre_job
>>>>>>>>>
>>>>>>>>>> --post_job
>>>>>>>>>> These are sometimes handy to run a non-standard command for
>>>>>>>>>> all
>>>>>>>>>> modules.
>>>>>>>>>>
>>>>>>>>>> I have added them, they are by the way a good example why we
>>>>>>>>>> need a
>>>>>>>>>>
>>>>>>>>>> discussion I have never used them.
>>>>>>>>>>
>>>>>>>>> However maybe the real discussion is "do we want to replace build
>>>>>>>>> and
>>>>>>>>> have
>>>>>>>>> a main/Makefile instead?"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - I have not used the rest of the unsupported options and would
>>>>>>>>> not
>>>>>>>>> miss
>>>>>>>>>
>>>>>>>>> them. Others may have other sets of options that are important
>>>>>>>>> to
>>>>>>>>>
>>>>>>>>>> them.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Some general remarks:
>>>>>>>>>>
>>>>>>>>>> - Why keep one makefile per module? Why not put all the
>>>>>>>>>> inter-module
>>>>>>>>>> dependencies into one file (like my script does)?
>>>>>>>>>>
>>>>>>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>>>>>>
>>>>>>>>>> "main/Makefile"
>>>>>>>>>>
>>>>>>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today
>>>>>>>>>
>>>>>>>>> just
>>>>>>>>
>>>>>>> will call the old makefiles as described in prj/build.lst
>>>>>>
>>>>>>> - Why not use the oportunity to move (a part of) the build
>>>>>>>>> environment
>>>>>>>>> out
>>>>>>>>>
>>>>>>>>> of the way to, say, build/ ?
>>>>>>>>>
>>>>>>>>> You have guessed my next step.
>>>>>>>>>>
>>>>>>>>>> - How are dependencies between modules handled (just the
>>>>>>>>>> manual
>>>>>>>>>>
>>>>>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>>>>>>
>>>>>>>>>> introduced
>>>>>>>>>> by
>>>>>>>>>> gmake).
>>>>>>>>>>
>>>>>>>>>> See doc. on --from. Its done with <module>.done files
>>>>>>>>>>
>>>>>>>>>> - How is the output of the individual calls to dmake or GNU
>>>>>>>>>> make
>>>>>>>>>>
>>>>>>>>> handled/made accessible. Ie. if there is a build error, how can
>>>>>>>>> I
>>>>>>>>> look
>>>>>>>>>
>>>>>>>> up
>>>>>>
>>>>>>> the corresponding build output?
>>>>>>>>>>
>>>>>>>>>> see doc. script make_log
>>>>>>>>>>
>>>>>>>>>> - Are the gmake makefiles included (run in the same process)
>>>>>>>>>> or
>>>>>>>>>>
>>>>>>>>> is
>>>>>>>>>
>>>>>>>>> GNU
>>>>>>>>
>>>>>>> make started for them it its own process?
>>>>>>
>>>>>>> For a start they would be called (own process), but its something
>>>>>>>>>> where
>>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>>
>>>>>>>>> have no strong opinions.
>>>>>>>>>
>>>>>>>>> Please (just to be sure), this proposal has nothing to do with the
>>>>>>>>> students
>>>>>>>>> work, its simply because I saw a positive discussion on removing
>>>>>>>>> build.pl
>>>>>>>>> ,
>>>>>>>>> and spent a couple of hours looking at it. If there is a preference
>>>>>>>>>
>>>>>>>>> not
>>>>>>>>
>>>>>>> to
>>>>>>
>>>>>>> remove build.pl I will simply forget it.
>>>>>>>>>
>>>>>>>>> rgds
>>>>>>>>> jan I.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Andre
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It has been roughly tested it, thanks to a clever utility from
>>>>>>>>>>
>>>>>>>>>> andre.
>>>>>>>>>
>>>>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>>
>>>>>>> considered in a makefile.
>>>>>>>>>>>
>>>>>>>>>>> My suggestion is on
>>>>>>>>>>> http://wiki.openoffice.org/**********wiki/Build_System_**
>>>>>>>>>>> Analysis:****<http://wiki.openoffice.org/********wiki/Build_System_Analysis:****>
>>>>>>>>>>> <http://wiki.**openoffice.org/******wiki/**
>>>>>>>>>>> Build_System_Analysis:**<http://wiki.openoffice.org/******wiki/Build_System_Analysis:**>
>>>>>>>>>>> >
>>>>>>>>>>> <
>>>>>>>>>>>
>>>>>>>>>>> http://wiki.openoffice.org/********wiki/Build_System_Analysis:*
>>>>>>>>>> ***<http://wiki.openoffice.org/******wiki/Build_System_Analysis:**>
>>>>>>>>>> <http://wiki.openoffice.org/******wiki/Build_System_**Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>> **>
>>>>>
>>>>> **<http://wiki.openoffice.org/********wiki/Build_System_****
>>>>>> Analysis:**<http://wiki.openoffice.org/******wiki/Build_System_**Analysis:**>
>>>>>> <http://wiki.**openoffice.org/****wiki/Build_**System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>> >
>>>>>>
>>>>>> <
>>>>>>>>>>>
>>>>>>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>> build.pl_versus_makefile<http:********//wiki.openoffice.org/****
>>>>>> wiki/**** <http://wiki.openoffice.org/**wiki/****><http://wiki.**
>>>>>> openoffice.org/wiki/**** <http://wiki.openoffice.org/wiki/****>>
>>>>>>
>>>>>>> <
>>>>>>>>>>>
>>>>>>>>>>> http://wiki.openoffice.org/****wiki/**<http://wiki.openoffice.org/**wiki/**>
>>>>>>>>>> <http://wiki.**openoffice.org/wiki/**<http://wiki.openoffice.org/wiki/**>
>>>>>>>>>> >
>>>>>>>>>>
>>>>>>>>> Build_System_Analysis:build.********pl_versus_makefile<http://****
>>>>>>
>>>>>>> wiki.openoffice.org/wiki/******Build_System_Analysis:build.**<http://wiki.openoffice.org/wiki/****Build_System_Analysis:build.**>
>>>>>>>>>>> **<http://wiki.openoffice.org/**wiki/**Build_System_Analysis:**
>>>>>>>>>>> build.**<http://wiki.openoffice.org/wiki/**Build_System_Analysis:build.**>
>>>>>>>>>>> >
>>>>>>>>>>> pl_versus_makefile<
>>>>>>>>>>>
>>>>>>>>>>> http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>>>>>>
>>>>>>>>> build.pl_versus_makefile<http:**//wiki.openoffice.org/wiki/**
>>>>> Build_System_Analysis:build.**pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>> >
>>>>>
>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>>
>>>>>>> options
>>>>>>>>>>> a
>>>>>>>>>>> lot, and some of them might be in use.
>>>>>>>>>>>
>>>>>>>>>>> thanks in advance for your comments.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------**
>>>>>>>>>>> ********----------------------**--*
>>>>>>>>>>> *--**
>>>>>>>>>>> --**
>>>>>>>>>>>
>>>>>>>>>>> --**---------
>>>>>>>>>>>
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.********a**
>>>>>>>>>> pache.org<
>>>>>>>>>> http://apache.org**>
>>>>>>>>>> <dev-unsubscribe@**openoffice.******apache.org<
>>>>>>>>>>
>>>>>>>>>> http://openoffice.apache.org>
>>>>>>>>>
>>>>>>>> <dev-unsubscribe@**openoffice.****apache.org<http://**
>>>>>> openoffice.apache.org <http://openoffice.apache.org>>
>>>>>>
>>>>>> <
>>>>>>>>>>
>>>>>>>>>> dev-unsubscribe@openoffice.**a**pache.org <http://apache.org><
>>>>>>>>> dev-unsubscribe@**openoffice.apache.org<de...@openoffice.apache.org>
>>>>>>>>> >
>>>>>>>>>
>>>>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>
>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------********------------------------
>>>>>>>>>> **--**
>>>>>>>>>> --**
>>>>>>>>>>
>>>>>>>>>> --**---------
>>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.******a**
>>>>>>>> pache.org<
>>>>>>>>
>>>>>>>> http://apache.org>
>>>>>>>
>>>>>> <dev-unsubscribe@**openoffice.****apache.org<http://**
>>>>>> openoffice.apache.org <http://openoffice.apache.org>>
>>>>>>
>>>>>> <
>>>>>>>>
>>>>>>>> dev-unsubscribe@openoffice.**a**pache.org <http://apache.org><
>>>>>>> dev-unsubscribe@**openoffice.apache.org<de...@openoffice.apache.org>
>>>>>>> >
>>>>>>>
>>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------******--------------------------**
>>>>>>>> --**
>>>>>>>>
>>>>>>> --**---------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>>> http://apache.org**>
>>>>>> <
>>>>>>
>>>>>> dev-unsubscribe@openoffice.**a**pache.org <http://apache.org><
>>>>> dev-unsubscribe@**openoffice.apache.org<de...@openoffice.apache.org>
>>>>> >
>>>>>
>>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>
>>>>> ------------------------------****----------------------------**--**
>>>>> ------------------------------****-------
>>>>>
>>>>> MzK
>>>>>
>>>>> "Truth is stranger than fiction, but it is because Fiction is obliged
>>>>> to stick to possibilities. Truth isn't."
>>>>> -- "Following the Equator", Mark Twain
>>>>>
>>>>>
>>>>> ------------------------------****----------------------------**
>>> --**---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>> <de...@openoffice.apache.org>
>>> >
>>>
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: [proposal] replace build.pl with a central Makefile.
Posted by Andre Fischer <aw...@gmail.com>.
On 21.10.2013 12:35, janI wrote:
> [...]
> I have the following plan:
> a) rename all Makefile to Makefile.gb, and correct prj/makefile.mk to the
> new name
prj/makefile.mk usually only contains a simple forwarding. This would
mean that making a gbuild module would require two forwards, one from
the global Makefile to prj/makefile.mk and one from there to
./Makefile. Maybe prj/makefile.mk could be removed and ./Makefile be
called directly from the global Makefile?
> b) for each <module> add a new Makefile, that basically call the existing
> makefile.mk, as per your script (and build.lst) and update build.lst
I would suggest to do this forwarding with a macro, so that when there
are any changes in the future, we only have to change the macro once
instead of every forwarding call.
A while back I modified the forwarding calls for gbuild modules (from
prj/makefile.mk to ./Makefile) to include the debug flag. Would have
been easier when all had used the same macro.
> c) add deliver.lst to Makefile and update build.lst
Are deliver.lst and build.lst still used after this change? If not, can
you delete them?
>
> once done, both build.pl and main/Makefile will work in the branch.
>
> d) have all of you test, and critic the idea.
> e) remove */prj and build.pl
> f) integrate in trunk.
Sounds good to me, with or without changes according to my suggestions.
-Andre
>
> rgds
> jan I.
>
>
>
>
>> -Andre
>>
>>
>>
>>> hope you like it.
>>> rgds
>>> jan I.
>>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>>> While this is easy to do with eg Perl I am not sure how to handle
>>>>>> this
>>>>>>
>>>>>>> with just a Makefile. The straightforward approach with handling
>>>>>>> <module>.done files does not work. And that is one of the reasons why
>>>>>>>
>>>>>> I
>>>>> don't think that (GNU) makefiles are a good solution for any problem.
>>>>>>> Most
>>>>>>> of us are used to program object oriented/imperative. Makefiles
>>>>>>>
>>>>>> require
>>>>> a
>>>>>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>>>>>> Maybe
>>>>>>> it would be better to reimplement build.pl with a lot fewer options
>>>>>>>
>>>>>> and
>>>>> with better readable code.
>>>>>>> I agree that makefiles are nowhere near a good solution to many of
>>>>>>>
>>>>>> these
>>>>> problems, but its like windows, I dont like it, but everybody uses it.
>>>>>> We could easily write a new build.pl, that also took care of the local
>>>>>> makefiles, but our build system would not be in the mainstream, and
>>>>>> e.g.
>>>>>> the distros would not like to integrate AOO.
>>>>>>
>>>>>> I have over the last years followed research in building systems, and
>>>>>> there
>>>>>> are (sadly enough) nobody that tries a real object oriented aproach.
>>>>>>
>>>>> Also
>>>>> if you look at packages like visual studio, QT, eclipse they all use the
>>>>>> principle of makefiles with declarative approach.
>>>>>>
>>>>>> So my simple question is, do we want to approach the main road
>>>>>>
>>>>> (makefiles
>>>>> for unix, visual studio for windows/mac) or do we want to have better
>>>>> but
>>>>> non standard system.
>>>>>> Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
>>>>> ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
>>>>> pragmatist says to use the make. It is good enough for others, it is
>>>>>
>>>> good
>>>>
>>>>> enough for use.
>>>>> On the other hand, when I start a new project I usually start with the
>>>>> question of what are the best tools for the problem at hand. And make
>>>>>
>>>> does
>>>>
>>>>> not seem to be the first or the best answer. Look at our dmake or
>>>>> gbuild
>>>>> system. Both don't use make in a standard way. gmake even defines its
>>>>>
>>>> own
>>>>
>>>>> language, object oriented and imperative, on top of the GNU make macros.
>>>>> That is, for me, an act of desperation.
>>>>> I have made experiments with an alternative approach, a domain specific
>>>>> language somewhat similar to Java. I personally like this approach
>>>>>
>>>> because
>>>>
>>>>> a) it uses the paradigm that I already use when writing C++ code. That
>>>>> means that I can apply my existing knowledge to the build process and
>>>>>
>>>> that
>>>>
>>>>> I don't have to remember all the tricks and pitfalls of makefiles.
>>>>> b) as expected it was much easier to handle file dependencies and
>>>>>
>>>> parallel
>>>>
>>>>> processing of build jobs in Java than adding object orientation and
>>>>> imperative control flow to makefiles.
>>>>>
>>>>> If I had the time and if I would be the one working on it then I would
>>>>> prefer an non-Makefile approach. But maybe I am just suffering too much
>>>>> from one of the 'three great virtues of a programmer': hubris.
>>>>>
>>>>> You are the one who leads the build project changes, so you have to
>>>>>
>>>> decide
>>>>
>>>>> which approach to use. I am not trying to make your life harder (than
>>>>> necessary), I am only trying to point out some of the pitfalls that I
>>>>>
>>>> have
>>>>
>>>>> encountered in the past. And to prevent you from removing features that
>>>>>
>>>> I
>>>>
>>>>> use :-)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> rgds
>>>>>> jan I.
>>>>>>
>>>>>> Ps. its always refreshing to discuss with you, you often have an
>>>>>> alternative approach, which is not just a dream but doable.
>>>>>>
>>>>>> Thanks. That makes two of us.
>>>>> Have a nice weekend,
>>>>> Andre
>>>>>
>>>>>
>>>>>
>>>>> -Andre
>>>>>>> --prepare
>>>>>>>
>>>>>>>> Also one option that is important for our every day work. Use
>>>>>>>>> case:
>>>>>>>>> You make changes in <module> and are not sure if these changes are
>>>>>>>>> compatible/incompatible. To be on the safe side you discard the
>>>>>>>>>
>>>>>>>> output
>>>>> of
>>>>>>>>> all depending modules. To save time you keep the output of all
>>>>>>>>> other
>>>>>>>>> modules.
>>>>>>>>>
>>>>>>>>> Often used together with '--from' like 'make --prepare --from
>>>>>>>>> svx' to
>>>>>>>>> prepare a build after making changes in svx.
>>>>>>>>>
>>>>>>>>> Documentation changed, funny thing is that svx does not clear
>>>>>>>>> correctly
>>>>>>>>>
>>>>>>>>> on
>>>>>>>> my ubuntu build.
>>>>>>>>
>>>>>>>>
>>>>>>>> --since <module>
>>>>>>>>
>>>>>>>> A variant of '--from'. The only difference is that <module>
>>>>>>>>> itself
>>>>>>>>> is
>>>>>>>>> not built.
>>>>>>>>>
>>>>>>>>> If your proposed approach is similar to what my script
>>>>>>>>> produces
>>>>>>>>> then
>>>>>>>>> it
>>>>>>>>> is not too difficult to support --from/--since. I made some
>>>>>>>>> experiments
>>>>>>>>> in
>>>>>>>>> this direction but was to lazy to finish them.
>>>>>>>>>
>>>>>>>>> My approach is very similar, but I failed to see how --since is
>>>>>>>>>
>>>>>>>>> supported.
>>>>>>>> And question is if its real important.
>>>>>>>>
>>>>>>>>
>>>>>>>> --job
>>>>>>>>
>>>>>>>> --pre_job
>>>>>>>>> --post_job
>>>>>>>>> These are sometimes handy to run a non-standard command for all
>>>>>>>>> modules.
>>>>>>>>>
>>>>>>>>> I have added them, they are by the way a good example why we
>>>>>>>>> need a
>>>>>>>>>
>>>>>>>>> discussion I have never used them.
>>>>>>>> However maybe the real discussion is "do we want to replace build and
>>>>>>>> have
>>>>>>>> a main/Makefile instead?"
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> - I have not used the rest of the unsupported options and would
>>>>>>>> not
>>>>>>>> miss
>>>>>>>>
>>>>>>>> them. Others may have other sets of options that are important to
>>>>>>>>> them.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Some general remarks:
>>>>>>>>>
>>>>>>>>> - Why keep one makefile per module? Why not put all the
>>>>>>>>> inter-module
>>>>>>>>> dependencies into one file (like my script does)?
>>>>>>>>>
>>>>>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>>>>>
>>>>>>>>> "main/Makefile"
>>>>>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today
>>>>>>>>
>>>>>>> just
>>>>> will call the old makefiles as described in prj/build.lst
>>>>>>>> - Why not use the oportunity to move (a part of) the build
>>>>>>>> environment
>>>>>>>> out
>>>>>>>>
>>>>>>>> of the way to, say, build/ ?
>>>>>>>>
>>>>>>>>> You have guessed my next step.
>>>>>>>>>
>>>>>>>>> - How are dependencies between modules handled (just the manual
>>>>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>>>>>> introduced
>>>>>>>>> by
>>>>>>>>> gmake).
>>>>>>>>>
>>>>>>>>> See doc. on --from. Its done with <module>.done files
>>>>>>>>>
>>>>>>>>> - How is the output of the individual calls to dmake or GNU make
>>>>>>>> handled/made accessible. Ie. if there is a build error, how can I
>>>>>>>> look
>>>>> up
>>>>>>>>> the corresponding build output?
>>>>>>>>>
>>>>>>>>> see doc. script make_log
>>>>>>>>>
>>>>>>>>> - Are the gmake makefiles included (run in the same process) or
>>>>>>>> is
>>>>>>>>
>>>>>>> GNU
>>>>> make started for them it its own process?
>>>>>>>>> For a start they would be called (own process), but its something
>>>>>>>>> where
>>>>>>>>>
>>>>>>>>> I
>>>>>>>> have no strong opinions.
>>>>>>>>
>>>>>>>> Please (just to be sure), this proposal has nothing to do with the
>>>>>>>> students
>>>>>>>> work, its simply because I saw a positive discussion on removing
>>>>>>>> build.pl
>>>>>>>> ,
>>>>>>>> and spent a couple of hours looking at it. If there is a preference
>>>>>>>>
>>>>>>> not
>>>>> to
>>>>>>>> remove build.pl I will simply forget it.
>>>>>>>>
>>>>>>>> rgds
>>>>>>>> jan I.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>>> Andre
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It has been roughly tested it, thanks to a clever utility from
>>>>>>>>>
>>>>>>>> andre.
>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>>>>>> considered in a makefile.
>>>>>>>>>>
>>>>>>>>>> My suggestion is on
>>>>>>>>>> http://wiki.openoffice.org/********wiki/Build_System_Analysis:****<http://wiki.openoffice.org/******wiki/Build_System_Analysis:**>
>>>>>>>>>> <
>>>>>>>>>>
>>>>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>> **>
>>>>
>>>>> **<http://wiki.openoffice.org/******wiki/Build_System_**Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>>>>> <
>>>>>>>>>>
>>>>>>>>> http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>> build.pl_versus_makefile<http:******//wiki.openoffice.org/**wiki/****<http://wiki.openoffice.org/wiki/****>
>>>>>>>>>> <
>>>>>>>>>>
>>>>>>>>> http://wiki.openoffice.org/**wiki/**<http://wiki.openoffice.org/wiki/**>
>>>>> Build_System_Analysis:build.******pl_versus_makefile<http://**
>>>>>>>>>> wiki.openoffice.org/wiki/****Build_System_Analysis:build.**<http://wiki.openoffice.org/wiki/**Build_System_Analysis:build.**>
>>>>>>>>>> pl_versus_makefile<
>>>>>>>>>>
>>>>>>>>> http://wiki.openoffice.org/**wiki/Build_System_Analysis:**
>>>> build.pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>
>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>>>>>> options
>>>>>>>>>> a
>>>>>>>>>> lot, and some of them might be in use.
>>>>>>>>>>
>>>>>>>>>> thanks in advance for your comments.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------********------------------------*
>>>>>>>>>> *--**
>>>>>>>>>> --**
>>>>>>>>>>
>>>>>>>>>> --**---------
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.******a**
>>>>>>>>> pache.org<
>>>>>>>>> http://apache.org**>
>>>>>>>>> <dev-unsubscribe@**openoffice.****apache.org<
>>>>>>>>>
>>>>>>>> http://openoffice.apache.org>
>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>>>>>> <
>>>>>>>>>
>>>>>>>> dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------******--------------------------**
>>>>>>>>> --**
>>>>>>>>>
>>>>>>>> --**---------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>>>>
>>>>>> http://apache.org>
>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>>>> <
>>>>>>>
>>>>>> dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------****----------------------------**
>>>>> --**---------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>>>> <
>>>>>
>>>> dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>
>>>>>
>>>>>
>>>> --
>>>>
>>>> ------------------------------**------------------------------**
>>>> ------------------------------**-------
>>>> MzK
>>>>
>>>> "Truth is stranger than fiction, but it is because Fiction is obliged
>>>> to stick to possibilities. Truth isn't."
>>>> -- "Following the Equator", Mark Twain
>>>>
>>>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>>
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [proposal] replace build.pl with a central Makefile.
Posted by janI <ja...@apache.org>.
On 21 October 2013 10:58, Andre Fischer <aw...@gmail.com> wrote:
> On 20.10.2013 12:40, janI wrote:
>
>> On 19 October 2013 19:20, Kay Schenk <ka...@gmail.com> wrote:
>>
>> On Fri, Oct 18, 2013 at 7:52 AM, Andre Fischer <aw...@gmail.com>
>>> wrote:
>>>
>>> On 18.10.2013 15:58, janI wrote:
>>>>
>>>> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
>>>>>
>>>>> On 18.10.2013 14:02, janI wrote:
>>>>>
>>>>>> sd
>>>>>>
>>>>>>>
>>>>>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>>>>>>
>>>>>>> On 18.10.2013 11:32, janI wrote:
>>>>>>>
>>>>>>> Hi.
>>>>>>>>
>>>>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>>>>>> made a
>>>>>>>>> proposal for a central Makefile located in main.
>>>>>>>>>
>>>>>>>>> Hi Jan,
>>>>>>>>>
>>>>>>>>> it is great that you are going to improve this part of the build
>>>>>>>> system.
>>>>>>>> But I think that we need more details about how the proposed
>>>>>>>> build
>>>>>>>> system
>>>>>>>> works. Without them I can not really evaluate the proposal.
>>>>>>>>
>>>>>>>> First of all, I agree with juergens remarks that this should be
>>>>>>>>
>>>>>>>> discussed
>>>>>>> before implemented, hence the wiki page.
>>>>>>>
>>>>>>> Secondly this has nothing directly to do with the proposed build
>>>>>>>
>>>>>> system,
>>>
>>>> its a simple replacement of build.pl in the current system.
>>>>>>>
>>>>>>> Yes, that is how I understood it. I just did not know how to call
>>>>>>>
>>>>>> the
>>>
>>>> build.pl replacement.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I know that build.pl works, but having a Makefile in main, would
>>>>>> make
>>>>>>
>>>>>>> us
>>>>>>> one step closer on being compatible with the distros. To me this job
>>>>>>>
>>>>>> is
>>>
>>>> a
>>>>>>> simple cleanup, not something we deadly need, but nice to have.
>>>>>>>
>>>>>>>
>>>>>>> Some remarks regarding the missing options:
>>>>>>>
>>>>>>> --from <module>
>>>>>>>> This is one of the more important options and one that I use
>>>>>>>> frequently
>>>>>>>> (also in the form --all:<module>).
>>>>>>>> Note that if you are in <moduleA> and call 'make --from
>>>>>>>>
>>>>>>> <moduleB>'
>>>
>>>> then
>>>>>>>> all modules are built
>>>>>>>> a) which <moduleA> depends on
>>>>>>>> b) but not those that <moduleB> depends on
>>>>>>>> c) Both <moduleA> and <moduleB> are built.
>>>>>>>>
>>>>>>>> I have changed the documentation.
>>>>>>>>
>>>>>>>> I use the --all:<module> myself very often, and have changed the
>>>>>>> documentation, because it is of course supported.
>>>>>>>
>>>>>>> The difference is that you do the call in main, but that is a minor
>>>>>>> detail
>>>>>>> that can be easily corrected (have <module>/Makefile calling
>>>>>>> main/Makefile.
>>>>>>>
>>>>>>> I have also changed documentation on --html due to juergens comments.
>>>>>>>
>>>>>>> I am not sure that we understand --from and --since in the same way
>>>>>>>
>>>>>> so
>>>
>>>> I
>>>>>> will try to explain what I think they do.
>>>>>>
>>>>>> Let's imagine that we have a simple project with modules A, B, C, D
>>>>>> and
>>>>>> E.
>>>>>> where B depends on A, C on B, D on C, and E on D.
>>>>>> A ' make all' would mean 'make E'. The dependencies would then lead
>>>>>> to
>>>>>> building modules A, B, C, D, E in this order.
>>>>>> If I am in E and call 'make --from C' then only C, D, and E should be
>>>>>> built. A 'make --since C' would only build D and E.
>>>>>>
>>>>>> If I am in D and call 'make --from B' then modules B, C, and D are
>>>>>>
>>>>> built.
>>>
>>>> Call 'make --since B' to build only C and D.
>>>>>> Note that 'make --from' accepts more than one module name (while 'make
>>>>>> --all:<module>' does not).
>>>>>> Note also that in the above case (stand in D, call 'make --from B')
>>>>>> module
>>>>>> A is not built, regardless of whether there are changes in A or not.
>>>>>> Whereas a simple call to make (still standing in D) would build all
>>>>>> modules that D depends on, directly or indirectly. Thus the options
>>>>>> '--from' and '--since' exist to actively exclude modules from being
>>>>>> built.
>>>>>>
>>>>>> The whole thing becomes a little bit more complicated with multiple
>>>>>> options to '--from' (I never use '--since' and also don't know a valid
>>>>>> use
>>>>>> case so I will ignore it for now) and more complex dependencies then
>>>>>> in
>>>>>> the
>>>>>> simple example above. Let's say that if we stand in instsetoo_native
>>>>>>
>>>>> and
>>>
>>>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This
>>>>>>
>>>>> would
>>>
>>>> build svx, sfx2 and all modules that depend (directly or indirectly) on
>>>>>> svx
>>>>>> OR sfx2.
>>>>>>
>>>>>> got it, now I just have one problem, why would you not build the
>>>>>>
>>>>> dependent
>>>>> modules, if they needed to be built, thats a scenario I dont
>>>>> understand.
>>>>> With a central makefile, <module>/makefile will not be called so we do
>>>>>
>>>> not
>>>
>>>> waste cpu cycles.
>>>>>
>>>>> With the .done files, we know when a module was last built and all
>>>>>
>>>> modules
>>>
>>>> that depend it should be rebuilt which the rule
>>>>> <module>.done : <module_depend>.done
>>>>>
>>>>> will ensure, so If we have A -> B -> C -> D
>>>>>
>>>>> I go in B, and call make, then when I go in D and make, B,C,D will be
>>>>> made.
>>>>>
>>>>> If we have A -> B -> D C -> D
>>>>> and do the same then only D will be made.
>>>>>
>>>>> So --from is not really saving anything ?
>>>>>
>>>>> a) In your example you first go into B then, in a second step, into D.
>>>> The '--from' option lets you do the same (well, not really the same,
>>>> but
>>>> see below) just from D.
>>>>
>>>> b) You go first to B and call make. This makes A, if necessary, then B.
>>>> The making of A is exactly the thing that you want to prevent with the
>>>> '--from' option. Go into D and call 'make --from B'. A is not built.
>>>>
>>>> c) After the discussion with you I am not sure if we still need --from
>>>> because the two reasons I know for its existence my not be relevant with
>>>> the new approach.
>>>>
>>>> c1) With the '--from' option you can tweak the dependency rules at
>>>>
>>> runtime
>>>
>>>> (a bit). This allows you to exclude projects from being built when you
>>>> know that that is not necessary. But from experience I know that can
>>>>
>>> lead
>>>
>>>> to very subtle errors. Letting the system determine what to built is
>>>> usually more reliable.
>>>>
>>>> c2) With build.pl a 'build --all' still builds every module on which
>>>> the
>>>> one you are standing in depends on. When those modules have been built
>>>> previously, then no compilation takes place. But calling dmake for a
>>>> couple of directories for close to 200 modules (when you stand in
>>>> instsetoo_native) takes a lot of time (several minutes on Windows), even
>>>> when no file has to be compiled. This wasteful way of doing nothing can
>>>>
>>> be
>>>
>>>> prevented with the --from option.
>>>> However, with the new approach and its .done flag files you can
>>>> determine
>>>> which modules need to be built much faster. You don't have to call
>>>> dmake
>>>> on directories that where already built. Hm, but this again, does only
>>>> work if your .done files have dependencies on all relevant source files.
>>>> That is something that is missing at least from my script.
>>>>
>>>>
>>>> So, reasons for the existence of '--from' are a result of old/slow
>>>> computers, slow files systems (still valid on Windows), missing global
>>>> dependencies (which we now have for gbuild) and impatient developers.
>>>>
>>>
>>> Looking at what Jan has proposed and this discussion so far, I think we
>>> should make some attempt at moving in this direction -- using standard
>>> makefiles.
>>>
>>
> Yes, agreed.
>
>
> Given this discussion, I do understand the additional flexibility of
>>> build.pl, but I also wonder if the flexibility is worth it considering
>>> the
>>> IDE tools new developers are likely to be familiar with.
>>>
>>> A complete build will take a while no matter what is used, depending on
>>> your situation.
>>>
>>
> A complete build is the easiest of the tasks of a build system.
> The hard part is to handle the development stage where single file changes
> might require whole modules to be rebuilt.
this is for sure correct. A complete build would be a simple script.
>
>
> I think this proposal has a lot of merit.
>>>
>>
> I agree.
>
>
> Thx for the kind words.
>>
>> I have updated the wiki page, to reflect the specific make calls.
>>
>> and R1533869 shows my first version of the central Makefile
>>
>> https://svn.apache.org/repos/**asf/openoffice/branches/**
>> capstone2013/main/Makefile<https://svn.apache.org/repos/asf/openoffice/branches/capstone2013/main/Makefile>
>>
>> you will see it contains the intermodule dependencies and nothing more.
>> The
>> setup in every module is kept in a Makefile in the respective module, so
>> we
>> have a clear seperation of module and intermodule dependencies.
>>
>
> The gbuild modules already have files called 'Makefile' top-level. Are you
> going to reorganize this and the prj/ and util/ directories?
>
I have the following plan:
a) rename all Makefile to Makefile.gb, and correct prj/makefile.mk to the
new name
b) for each <module> add a new Makefile, that basically call the existing
makefile.mk, as per your script (and build.lst) and update build.lst
c) add deliver.lst to Makefile and update build.lst
once done, both build.pl and main/Makefile will work in the branch.
d) have all of you test, and critic the idea.
e) remove */prj and build.pl
f) integrate in trunk.
rgds
jan I.
>
> -Andre
>
>
>
>> hope you like it.
>> rgds
>> jan I.
>>
>>
>>
>>
>>>
>>>
>>>
>>>
>>>> While this is easy to do with eg Perl I am not sure how to handle
>>>>> this
>>>>>
>>>>>> with just a Makefile. The straightforward approach with handling
>>>>>> <module>.done files does not work. And that is one of the reasons why
>>>>>>
>>>>> I
>>>
>>>> don't think that (GNU) makefiles are a good solution for any problem.
>>>>>> Most
>>>>>> of us are used to program object oriented/imperative. Makefiles
>>>>>>
>>>>> require
>>>
>>>> a
>>>>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>>>>> Maybe
>>>>>> it would be better to reimplement build.pl with a lot fewer options
>>>>>>
>>>>> and
>>>
>>>> with better readable code.
>>>>>>
>>>>>> I agree that makefiles are nowhere near a good solution to many of
>>>>>>
>>>>> these
>>>
>>>> problems, but its like windows, I dont like it, but everybody uses it.
>>>>>
>>>>> We could easily write a new build.pl, that also took care of the local
>>>>> makefiles, but our build system would not be in the mainstream, and
>>>>> e.g.
>>>>> the distros would not like to integrate AOO.
>>>>>
>>>>> I have over the last years followed research in building systems, and
>>>>> there
>>>>> are (sadly enough) nobody that tries a real object oriented aproach.
>>>>>
>>>> Also
>>>
>>>> if you look at packages like visual studio, QT, eclipse they all use the
>>>>> principle of makefiles with declarative approach.
>>>>>
>>>>> So my simple question is, do we want to approach the main road
>>>>>
>>>> (makefiles
>>>
>>>> for unix, visual studio for windows/mac) or do we want to have better
>>>>>
>>>> but
>>>
>>>> non standard system.
>>>>>
>>>>> Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
>>>> ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
>>>> pragmatist says to use the make. It is good enough for others, it is
>>>>
>>> good
>>>
>>>> enough for use.
>>>> On the other hand, when I start a new project I usually start with the
>>>> question of what are the best tools for the problem at hand. And make
>>>>
>>> does
>>>
>>>> not seem to be the first or the best answer. Look at our dmake or
>>>> gbuild
>>>> system. Both don't use make in a standard way. gmake even defines its
>>>>
>>> own
>>>
>>>> language, object oriented and imperative, on top of the GNU make macros.
>>>> That is, for me, an act of desperation.
>>>> I have made experiments with an alternative approach, a domain specific
>>>> language somewhat similar to Java. I personally like this approach
>>>>
>>> because
>>>
>>>> a) it uses the paradigm that I already use when writing C++ code. That
>>>> means that I can apply my existing knowledge to the build process and
>>>>
>>> that
>>>
>>>> I don't have to remember all the tricks and pitfalls of makefiles.
>>>> b) as expected it was much easier to handle file dependencies and
>>>>
>>> parallel
>>>
>>>> processing of build jobs in Java than adding object orientation and
>>>> imperative control flow to makefiles.
>>>>
>>>> If I had the time and if I would be the one working on it then I would
>>>> prefer an non-Makefile approach. But maybe I am just suffering too much
>>>> from one of the 'three great virtues of a programmer': hubris.
>>>>
>>>> You are the one who leads the build project changes, so you have to
>>>>
>>> decide
>>>
>>>> which approach to use. I am not trying to make your life harder (than
>>>> necessary), I am only trying to point out some of the pitfalls that I
>>>>
>>> have
>>>
>>>> encountered in the past. And to prevent you from removing features that
>>>>
>>> I
>>>
>>>> use :-)
>>>>
>>>>
>>>>
>>>>
>>>> rgds
>>>>> jan I.
>>>>>
>>>>> Ps. its always refreshing to discuss with you, you often have an
>>>>> alternative approach, which is not just a dream but doable.
>>>>>
>>>>> Thanks. That makes two of us.
>>>>
>>>> Have a nice weekend,
>>>> Andre
>>>>
>>>>
>>>>
>>>> -Andre
>>>>>
>>>>>>
>>>>>> --prepare
>>>>>>
>>>>>>> Also one option that is important for our every day work. Use
>>>>>>>> case:
>>>>>>>> You make changes in <module> and are not sure if these changes are
>>>>>>>> compatible/incompatible. To be on the safe side you discard the
>>>>>>>>
>>>>>>> output
>>>
>>>> of
>>>>>>>> all depending modules. To save time you keep the output of all
>>>>>>>> other
>>>>>>>> modules.
>>>>>>>>
>>>>>>>> Often used together with '--from' like 'make --prepare --from
>>>>>>>> svx' to
>>>>>>>> prepare a build after making changes in svx.
>>>>>>>>
>>>>>>>> Documentation changed, funny thing is that svx does not clear
>>>>>>>> correctly
>>>>>>>>
>>>>>>>> on
>>>>>>> my ubuntu build.
>>>>>>>
>>>>>>>
>>>>>>> --since <module>
>>>>>>>
>>>>>>> A variant of '--from'. The only difference is that <module>
>>>>>>>> itself
>>>>>>>> is
>>>>>>>> not built.
>>>>>>>>
>>>>>>>> If your proposed approach is similar to what my script
>>>>>>>> produces
>>>>>>>> then
>>>>>>>> it
>>>>>>>> is not too difficult to support --from/--since. I made some
>>>>>>>> experiments
>>>>>>>> in
>>>>>>>> this direction but was to lazy to finish them.
>>>>>>>>
>>>>>>>> My approach is very similar, but I failed to see how --since is
>>>>>>>>
>>>>>>>> supported.
>>>>>>> And question is if its real important.
>>>>>>>
>>>>>>>
>>>>>>> --job
>>>>>>>
>>>>>>> --pre_job
>>>>>>>> --post_job
>>>>>>>> These are sometimes handy to run a non-standard command for all
>>>>>>>> modules.
>>>>>>>>
>>>>>>>> I have added them, they are by the way a good example why we
>>>>>>>> need a
>>>>>>>>
>>>>>>>> discussion I have never used them.
>>>>>>>
>>>>>>> However maybe the real discussion is "do we want to replace build and
>>>>>>> have
>>>>>>> a main/Makefile instead?"
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> - I have not used the rest of the unsupported options and would
>>>>>>> not
>>>>>>> miss
>>>>>>>
>>>>>>> them. Others may have other sets of options that are important to
>>>>>>>> them.
>>>>>>>>
>>>>>>>>
>>>>>>>> Some general remarks:
>>>>>>>>
>>>>>>>> - Why keep one makefile per module? Why not put all the
>>>>>>>> inter-module
>>>>>>>> dependencies into one file (like my script does)?
>>>>>>>>
>>>>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>>>>
>>>>>>>> "main/Makefile"
>>>>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today
>>>>>>>
>>>>>> just
>>>
>>>> will call the old makefiles as described in prj/build.lst
>>>>>>>
>>>>>>> - Why not use the oportunity to move (a part of) the build
>>>>>>> environment
>>>>>>> out
>>>>>>>
>>>>>>> of the way to, say, build/ ?
>>>>>>>
>>>>>>>> You have guessed my next step.
>>>>>>>>
>>>>>>>> - How are dependencies between modules handled (just the manual
>>>>>>>
>>>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>>>>> introduced
>>>>>>>> by
>>>>>>>> gmake).
>>>>>>>>
>>>>>>>> See doc. on --from. Its done with <module>.done files
>>>>>>>>
>>>>>>>> - How is the output of the individual calls to dmake or GNU make
>>>>>>>
>>>>>>> handled/made accessible. Ie. if there is a build error, how can I
>>>>>>>>
>>>>>>> look
>>>
>>>> up
>>>>>>>> the corresponding build output?
>>>>>>>>
>>>>>>>> see doc. script make_log
>>>>>>>>
>>>>>>>> - Are the gmake makefiles included (run in the same process) or
>>>>>>> is
>>>>>>>
>>>>>> GNU
>>>
>>>> make started for them it its own process?
>>>>>>>>
>>>>>>>> For a start they would be called (own process), but its something
>>>>>>>> where
>>>>>>>>
>>>>>>>> I
>>>>>>> have no strong opinions.
>>>>>>>
>>>>>>> Please (just to be sure), this proposal has nothing to do with the
>>>>>>> students
>>>>>>> work, its simply because I saw a positive discussion on removing
>>>>>>> build.pl
>>>>>>> ,
>>>>>>> and spent a couple of hours looking at it. If there is a preference
>>>>>>>
>>>>>> not
>>>
>>>> to
>>>>>>> remove build.pl I will simply forget it.
>>>>>>>
>>>>>>> rgds
>>>>>>> jan I.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>>> Andre
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> It has been roughly tested it, thanks to a clever utility from
>>>>>>>>
>>>>>>> andre.
>>>
>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>>>>> considered in a makefile.
>>>>>>>>>
>>>>>>>>> My suggestion is on
>>>>>>>>> http://wiki.openoffice.org/********wiki/Build_System_Analysis:****<http://wiki.openoffice.org/******wiki/Build_System_Analysis:**>
>>>>>>>>> <
>>>>>>>>>
>>>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>> **>
>>>
>>>> **<http://wiki.openoffice.org/******wiki/Build_System_**Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>>>> <
>>>>>>>>>
>>>>>>>> http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>> >
>>>
>>>> build.pl_versus_makefile<http:******//wiki.openoffice.org/**wiki/****<http://wiki.openoffice.org/wiki/****>
>>>>>>>>> <
>>>>>>>>>
>>>>>>>> http://wiki.openoffice.org/**wiki/**<http://wiki.openoffice.org/wiki/**>
>>> >
>>>
>>>> Build_System_Analysis:build.******pl_versus_makefile<http://**
>>>>>>>>> wiki.openoffice.org/wiki/****Build_System_Analysis:build.**<http://wiki.openoffice.org/wiki/**Build_System_Analysis:build.**>
>>>>>>>>> pl_versus_makefile<
>>>>>>>>>
>>>>>>>> http://wiki.openoffice.org/**wiki/Build_System_Analysis:**
>>> build.pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>
>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>>>>> options
>>>>>>>>> a
>>>>>>>>> lot, and some of them might be in use.
>>>>>>>>>
>>>>>>>>> thanks in advance for your comments.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------********------------------------*
>>>>>>>>> *--**
>>>>>>>>> --**
>>>>>>>>>
>>>>>>>>> --**---------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.******a**
>>>>>>>> pache.org<
>>>>>>>> http://apache.org**>
>>>>>>>> <dev-unsubscribe@**openoffice.****apache.org<
>>>>>>>>
>>>>>>> http://openoffice.apache.org>
>>>
>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>>>>> <
>>>>>>>>
>>>>>>> dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>>> >
>>>
>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------******--------------------------**
>>>>>>>> --**
>>>>>>>>
>>>>>>> --**---------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>>>
>>>>> http://apache.org>
>>>
>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>>> <
>>>>>>
>>>>> dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>>> >
>>>
>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------****----------------------------**
>>>> --**---------
>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>>> <
>>>>
>>> dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>>> >
>>>
>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>
>>>>
>>>>
>>> --
>>>
>>> ------------------------------**------------------------------**
>>> ------------------------------**-------
>>> MzK
>>>
>>> "Truth is stranger than fiction, but it is because Fiction is obliged
>>> to stick to possibilities. Truth isn't."
>>> -- "Following the Equator", Mark Twain
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: [proposal] replace build.pl with a central Makefile.
Posted by Andre Fischer <aw...@gmail.com>.
On 20.10.2013 12:40, janI wrote:
> On 19 October 2013 19:20, Kay Schenk <ka...@gmail.com> wrote:
>
>> On Fri, Oct 18, 2013 at 7:52 AM, Andre Fischer <aw...@gmail.com> wrote:
>>
>>> On 18.10.2013 15:58, janI wrote:
>>>
>>>> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
>>>>
>>>> On 18.10.2013 14:02, janI wrote:
>>>>> sd
>>>>>>
>>>>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>>>>>
>>>>>> On 18.10.2013 11:32, janI wrote:
>>>>>>
>>>>>>> Hi.
>>>>>>>
>>>>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>>>>> made a
>>>>>>>> proposal for a central Makefile located in main.
>>>>>>>>
>>>>>>>> Hi Jan,
>>>>>>>>
>>>>>>> it is great that you are going to improve this part of the build
>>>>>>> system.
>>>>>>> But I think that we need more details about how the proposed build
>>>>>>> system
>>>>>>> works. Without them I can not really evaluate the proposal.
>>>>>>>
>>>>>>> First of all, I agree with juergens remarks that this should be
>>>>>>>
>>>>>> discussed
>>>>>> before implemented, hence the wiki page.
>>>>>>
>>>>>> Secondly this has nothing directly to do with the proposed build
>> system,
>>>>>> its a simple replacement of build.pl in the current system.
>>>>>>
>>>>>> Yes, that is how I understood it. I just did not know how to call
>> the
>>>>> build.pl replacement.
>>>>>
>>>>>
>>>>>
>>>>> I know that build.pl works, but having a Makefile in main, would make
>>>>>> us
>>>>>> one step closer on being compatible with the distros. To me this job
>> is
>>>>>> a
>>>>>> simple cleanup, not something we deadly need, but nice to have.
>>>>>>
>>>>>>
>>>>>> Some remarks regarding the missing options:
>>>>>>
>>>>>>> --from <module>
>>>>>>> This is one of the more important options and one that I use
>>>>>>> frequently
>>>>>>> (also in the form --all:<module>).
>>>>>>> Note that if you are in <moduleA> and call 'make --from
>> <moduleB>'
>>>>>>> then
>>>>>>> all modules are built
>>>>>>> a) which <moduleA> depends on
>>>>>>> b) but not those that <moduleB> depends on
>>>>>>> c) Both <moduleA> and <moduleB> are built.
>>>>>>>
>>>>>>> I have changed the documentation.
>>>>>>>
>>>>>> I use the --all:<module> myself very often, and have changed the
>>>>>> documentation, because it is of course supported.
>>>>>>
>>>>>> The difference is that you do the call in main, but that is a minor
>>>>>> detail
>>>>>> that can be easily corrected (have <module>/Makefile calling
>>>>>> main/Makefile.
>>>>>>
>>>>>> I have also changed documentation on --html due to juergens comments.
>>>>>>
>>>>>> I am not sure that we understand --from and --since in the same way
>> so
>>>>> I
>>>>> will try to explain what I think they do.
>>>>>
>>>>> Let's imagine that we have a simple project with modules A, B, C, D and
>>>>> E.
>>>>> where B depends on A, C on B, D on C, and E on D.
>>>>> A ' make all' would mean 'make E'. The dependencies would then lead to
>>>>> building modules A, B, C, D, E in this order.
>>>>> If I am in E and call 'make --from C' then only C, D, and E should be
>>>>> built. A 'make --since C' would only build D and E.
>>>>>
>>>>> If I am in D and call 'make --from B' then modules B, C, and D are
>> built.
>>>>> Call 'make --since B' to build only C and D.
>>>>> Note that 'make --from' accepts more than one module name (while 'make
>>>>> --all:<module>' does not).
>>>>> Note also that in the above case (stand in D, call 'make --from B')
>>>>> module
>>>>> A is not built, regardless of whether there are changes in A or not.
>>>>> Whereas a simple call to make (still standing in D) would build all
>>>>> modules that D depends on, directly or indirectly. Thus the options
>>>>> '--from' and '--since' exist to actively exclude modules from being
>>>>> built.
>>>>>
>>>>> The whole thing becomes a little bit more complicated with multiple
>>>>> options to '--from' (I never use '--since' and also don't know a valid
>>>>> use
>>>>> case so I will ignore it for now) and more complex dependencies then in
>>>>> the
>>>>> simple example above. Let's say that if we stand in instsetoo_native
>> and
>>>>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This
>> would
>>>>> build svx, sfx2 and all modules that depend (directly or indirectly) on
>>>>> svx
>>>>> OR sfx2.
>>>>>
>>>>> got it, now I just have one problem, why would you not build the
>>>> dependent
>>>> modules, if they needed to be built, thats a scenario I dont understand.
>>>> With a central makefile, <module>/makefile will not be called so we do
>> not
>>>> waste cpu cycles.
>>>>
>>>> With the .done files, we know when a module was last built and all
>> modules
>>>> that depend it should be rebuilt which the rule
>>>> <module>.done : <module_depend>.done
>>>>
>>>> will ensure, so If we have A -> B -> C -> D
>>>>
>>>> I go in B, and call make, then when I go in D and make, B,C,D will be
>>>> made.
>>>>
>>>> If we have A -> B -> D C -> D
>>>> and do the same then only D will be made.
>>>>
>>>> So --from is not really saving anything ?
>>>>
>>> a) In your example you first go into B then, in a second step, into D.
>>> The '--from' option lets you do the same (well, not really the same, but
>>> see below) just from D.
>>>
>>> b) You go first to B and call make. This makes A, if necessary, then B.
>>> The making of A is exactly the thing that you want to prevent with the
>>> '--from' option. Go into D and call 'make --from B'. A is not built.
>>>
>>> c) After the discussion with you I am not sure if we still need --from
>>> because the two reasons I know for its existence my not be relevant with
>>> the new approach.
>>>
>>> c1) With the '--from' option you can tweak the dependency rules at
>> runtime
>>> (a bit). This allows you to exclude projects from being built when you
>>> know that that is not necessary. But from experience I know that can
>> lead
>>> to very subtle errors. Letting the system determine what to built is
>>> usually more reliable.
>>>
>>> c2) With build.pl a 'build --all' still builds every module on which the
>>> one you are standing in depends on. When those modules have been built
>>> previously, then no compilation takes place. But calling dmake for a
>>> couple of directories for close to 200 modules (when you stand in
>>> instsetoo_native) takes a lot of time (several minutes on Windows), even
>>> when no file has to be compiled. This wasteful way of doing nothing can
>> be
>>> prevented with the --from option.
>>> However, with the new approach and its .done flag files you can determine
>>> which modules need to be built much faster. You don't have to call dmake
>>> on directories that where already built. Hm, but this again, does only
>>> work if your .done files have dependencies on all relevant source files.
>>> That is something that is missing at least from my script.
>>>
>>>
>>> So, reasons for the existence of '--from' are a result of old/slow
>>> computers, slow files systems (still valid on Windows), missing global
>>> dependencies (which we now have for gbuild) and impatient developers.
>>
>> Looking at what Jan has proposed and this discussion so far, I think we
>> should make some attempt at moving in this direction -- using standard
>> makefiles.
Yes, agreed.
>> Given this discussion, I do understand the additional flexibility of
>> build.pl, but I also wonder if the flexibility is worth it considering the
>> IDE tools new developers are likely to be familiar with.
>>
>> A complete build will take a while no matter what is used, depending on
>> your situation.
A complete build is the easiest of the tasks of a build system.
The hard part is to handle the development stage where single file
changes might require whole modules to be rebuilt.
>> I think this proposal has a lot of merit.
I agree.
> Thx for the kind words.
>
> I have updated the wiki page, to reflect the specific make calls.
>
> and R1533869 shows my first version of the central Makefile
>
> https://svn.apache.org/repos/asf/openoffice/branches/capstone2013/main/Makefile
>
> you will see it contains the intermodule dependencies and nothing more. The
> setup in every module is kept in a Makefile in the respective module, so we
> have a clear seperation of module and intermodule dependencies.
The gbuild modules already have files called 'Makefile' top-level. Are
you going to reorganize this and the prj/ and util/ directories?
-Andre
>
> hope you like it.
> rgds
> jan I.
>
>
>
>>
>>
>>
>>
>>>
>>>> While this is easy to do with eg Perl I am not sure how to handle this
>>>>> with just a Makefile. The straightforward approach with handling
>>>>> <module>.done files does not work. And that is one of the reasons why
>> I
>>>>> don't think that (GNU) makefiles are a good solution for any problem.
>>>>> Most
>>>>> of us are used to program object oriented/imperative. Makefiles
>> require
>>>>> a
>>>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>>>> Maybe
>>>>> it would be better to reimplement build.pl with a lot fewer options
>> and
>>>>> with better readable code.
>>>>>
>>>>> I agree that makefiles are nowhere near a good solution to many of
>> these
>>>> problems, but its like windows, I dont like it, but everybody uses it.
>>>>
>>>> We could easily write a new build.pl, that also took care of the local
>>>> makefiles, but our build system would not be in the mainstream, and e.g.
>>>> the distros would not like to integrate AOO.
>>>>
>>>> I have over the last years followed research in building systems, and
>>>> there
>>>> are (sadly enough) nobody that tries a real object oriented aproach.
>> Also
>>>> if you look at packages like visual studio, QT, eclipse they all use the
>>>> principle of makefiles with declarative approach.
>>>>
>>>> So my simple question is, do we want to approach the main road
>> (makefiles
>>>> for unix, visual studio for windows/mac) or do we want to have better
>> but
>>>> non standard system.
>>>>
>>> Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
>>> ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
>>> pragmatist says to use the make. It is good enough for others, it is
>> good
>>> enough for use.
>>> On the other hand, when I start a new project I usually start with the
>>> question of what are the best tools for the problem at hand. And make
>> does
>>> not seem to be the first or the best answer. Look at our dmake or gbuild
>>> system. Both don't use make in a standard way. gmake even defines its
>> own
>>> language, object oriented and imperative, on top of the GNU make macros.
>>> That is, for me, an act of desperation.
>>> I have made experiments with an alternative approach, a domain specific
>>> language somewhat similar to Java. I personally like this approach
>> because
>>> a) it uses the paradigm that I already use when writing C++ code. That
>>> means that I can apply my existing knowledge to the build process and
>> that
>>> I don't have to remember all the tricks and pitfalls of makefiles.
>>> b) as expected it was much easier to handle file dependencies and
>> parallel
>>> processing of build jobs in Java than adding object orientation and
>>> imperative control flow to makefiles.
>>>
>>> If I had the time and if I would be the one working on it then I would
>>> prefer an non-Makefile approach. But maybe I am just suffering too much
>>> from one of the 'three great virtues of a programmer': hubris.
>>>
>>> You are the one who leads the build project changes, so you have to
>> decide
>>> which approach to use. I am not trying to make your life harder (than
>>> necessary), I am only trying to point out some of the pitfalls that I
>> have
>>> encountered in the past. And to prevent you from removing features that
>> I
>>> use :-)
>>>
>>>
>>>
>>>
>>>> rgds
>>>> jan I.
>>>>
>>>> Ps. its always refreshing to discuss with you, you often have an
>>>> alternative approach, which is not just a dream but doable.
>>>>
>>> Thanks. That makes two of us.
>>>
>>> Have a nice weekend,
>>> Andre
>>>
>>>
>>>
>>>> -Andre
>>>>>
>>>>> --prepare
>>>>>>> Also one option that is important for our every day work. Use
>>>>>>> case:
>>>>>>> You make changes in <module> and are not sure if these changes are
>>>>>>> compatible/incompatible. To be on the safe side you discard the
>> output
>>>>>>> of
>>>>>>> all depending modules. To save time you keep the output of all other
>>>>>>> modules.
>>>>>>>
>>>>>>> Often used together with '--from' like 'make --prepare --from
>>>>>>> svx' to
>>>>>>> prepare a build after making changes in svx.
>>>>>>>
>>>>>>> Documentation changed, funny thing is that svx does not clear
>>>>>>> correctly
>>>>>>>
>>>>>> on
>>>>>> my ubuntu build.
>>>>>>
>>>>>>
>>>>>> --since <module>
>>>>>>
>>>>>>> A variant of '--from'. The only difference is that <module>
>>>>>>> itself
>>>>>>> is
>>>>>>> not built.
>>>>>>>
>>>>>>> If your proposed approach is similar to what my script produces
>>>>>>> then
>>>>>>> it
>>>>>>> is not too difficult to support --from/--since. I made some
>>>>>>> experiments
>>>>>>> in
>>>>>>> this direction but was to lazy to finish them.
>>>>>>>
>>>>>>> My approach is very similar, but I failed to see how --since is
>>>>>>>
>>>>>> supported.
>>>>>> And question is if its real important.
>>>>>>
>>>>>>
>>>>>> --job
>>>>>>
>>>>>>> --pre_job
>>>>>>> --post_job
>>>>>>> These are sometimes handy to run a non-standard command for all
>>>>>>> modules.
>>>>>>>
>>>>>>> I have added them, they are by the way a good example why we need a
>>>>>>>
>>>>>> discussion I have never used them.
>>>>>>
>>>>>> However maybe the real discussion is "do we want to replace build and
>>>>>> have
>>>>>> a main/Makefile instead?"
>>>>>>
>>>>>>
>>>>>>
>>>>>> - I have not used the rest of the unsupported options and would not
>>>>>> miss
>>>>>>
>>>>>>> them. Others may have other sets of options that are important to
>>>>>>> them.
>>>>>>>
>>>>>>>
>>>>>>> Some general remarks:
>>>>>>>
>>>>>>> - Why keep one makefile per module? Why not put all the inter-module
>>>>>>> dependencies into one file (like my script does)?
>>>>>>>
>>>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>>>
>>>>>> "main/Makefile"
>>>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today
>> just
>>>>>> will call the old makefiles as described in prj/build.lst
>>>>>>
>>>>>> - Why not use the oportunity to move (a part of) the build environment
>>>>>> out
>>>>>>
>>>>>> of the way to, say, build/ ?
>>>>>>> You have guessed my next step.
>>>>>>>
>>>>>> - How are dependencies between modules handled (just the manual
>>>>>>
>>>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>>>> introduced
>>>>>>> by
>>>>>>> gmake).
>>>>>>>
>>>>>>> See doc. on --from. Its done with <module>.done files
>>>>>>>
>>>>>> - How is the output of the individual calls to dmake or GNU make
>>>>>>
>>>>>>> handled/made accessible. Ie. if there is a build error, how can I
>> look
>>>>>>> up
>>>>>>> the corresponding build output?
>>>>>>>
>>>>>>> see doc. script make_log
>>>>>>>
>>>>>> - Are the gmake makefiles included (run in the same process) or is
>> GNU
>>>>>>> make started for them it its own process?
>>>>>>>
>>>>>>> For a start they would be called (own process), but its something
>>>>>>> where
>>>>>>>
>>>>>> I
>>>>>> have no strong opinions.
>>>>>>
>>>>>> Please (just to be sure), this proposal has nothing to do with the
>>>>>> students
>>>>>> work, its simply because I saw a positive discussion on removing
>>>>>> build.pl
>>>>>> ,
>>>>>> and spent a couple of hours looking at it. If there is a preference
>> not
>>>>>> to
>>>>>> remove build.pl I will simply forget it.
>>>>>>
>>>>>> rgds
>>>>>> jan I.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>> Andre
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> It has been roughly tested it, thanks to a clever utility from
>> andre.
>>>>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>>>> considered in a makefile.
>>>>>>>>
>>>>>>>> My suggestion is on
>>>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<
>> http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<
>> http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****<
>> http://wiki.openoffice.org/wiki/**>
>>>>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://**
>>>>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.**
>>>>>>>> pl_versus_makefile<
>> http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile
>>>>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>>>> options
>>>>>>>> a
>>>>>>>> lot, and some of them might be in use.
>>>>>>>>
>>>>>>>> thanks in advance for your comments.
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------******--------------------------**
>>>>>>>> --**
>>>>>>>>
>>>>>>> --**---------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>>>> http://apache.org**>
>>>>>>> <dev-unsubscribe@**openoffice.**apache.org<
>> http://openoffice.apache.org>
>>>>>>> <dev-unsubscribe@**openoffice.apache.org<
>> dev-unsubscribe@openoffice.apache.org>
>>>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------****----------------------------**
>>>>> --**---------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<
>> http://apache.org>
>>>>> <dev-unsubscribe@**openoffice.apache.org<
>> dev-unsubscribe@openoffice.apache.org>
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>
>>>>>
>>>>>
>>> ------------------------------**------------------------------**---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<
>> dev-unsubscribe@openoffice.apache.org>
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>
>> --
>>
>> -------------------------------------------------------------------------------------------------
>> MzK
>>
>> "Truth is stranger than fiction, but it is because Fiction is obliged
>> to stick to possibilities. Truth isn't."
>> -- "Following the Equator", Mark Twain
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [proposal] replace build.pl with a central Makefile.
Posted by janI <ja...@apache.org>.
On 19 October 2013 19:20, Kay Schenk <ka...@gmail.com> wrote:
> On Fri, Oct 18, 2013 at 7:52 AM, Andre Fischer <aw...@gmail.com> wrote:
>
> > On 18.10.2013 15:58, janI wrote:
> >
> >> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
> >>
> >> On 18.10.2013 14:02, janI wrote:
> >>>
> >>> sd
> >>>>
> >>>>
> >>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
> >>>>
> >>>> On 18.10.2013 11:32, janI wrote:
> >>>>
> >>>>> Hi.
> >>>>>
> >>>>>> due to the discussion in thread "Mentor a new build system", I have
> >>>>>> made a
> >>>>>> proposal for a central Makefile located in main.
> >>>>>>
> >>>>>> Hi Jan,
> >>>>>>
> >>>>> it is great that you are going to improve this part of the build
> >>>>> system.
> >>>>> But I think that we need more details about how the proposed build
> >>>>> system
> >>>>> works. Without them I can not really evaluate the proposal.
> >>>>>
> >>>>> First of all, I agree with juergens remarks that this should be
> >>>>>
> >>>> discussed
> >>>> before implemented, hence the wiki page.
> >>>>
> >>>> Secondly this has nothing directly to do with the proposed build
> system,
> >>>> its a simple replacement of build.pl in the current system.
> >>>>
> >>>> Yes, that is how I understood it. I just did not know how to call
> the
> >>> build.pl replacement.
> >>>
> >>>
> >>>
> >>> I know that build.pl works, but having a Makefile in main, would make
> >>>> us
> >>>> one step closer on being compatible with the distros. To me this job
> is
> >>>> a
> >>>> simple cleanup, not something we deadly need, but nice to have.
> >>>>
> >>>>
> >>>> Some remarks regarding the missing options:
> >>>>
> >>>>> --from <module>
> >>>>> This is one of the more important options and one that I use
> >>>>> frequently
> >>>>> (also in the form --all:<module>).
> >>>>> Note that if you are in <moduleA> and call 'make --from
> <moduleB>'
> >>>>> then
> >>>>> all modules are built
> >>>>> a) which <moduleA> depends on
> >>>>> b) but not those that <moduleB> depends on
> >>>>> c) Both <moduleA> and <moduleB> are built.
> >>>>>
> >>>>> I have changed the documentation.
> >>>>>
> >>>> I use the --all:<module> myself very often, and have changed the
> >>>> documentation, because it is of course supported.
> >>>>
> >>>> The difference is that you do the call in main, but that is a minor
> >>>> detail
> >>>> that can be easily corrected (have <module>/Makefile calling
> >>>> main/Makefile.
> >>>>
> >>>> I have also changed documentation on --html due to juergens comments.
> >>>>
> >>>> I am not sure that we understand --from and --since in the same way
> so
> >>> I
> >>> will try to explain what I think they do.
> >>>
> >>> Let's imagine that we have a simple project with modules A, B, C, D and
> >>> E.
> >>> where B depends on A, C on B, D on C, and E on D.
> >>> A ' make all' would mean 'make E'. The dependencies would then lead to
> >>> building modules A, B, C, D, E in this order.
> >>> If I am in E and call 'make --from C' then only C, D, and E should be
> >>> built. A 'make --since C' would only build D and E.
> >>>
> >>> If I am in D and call 'make --from B' then modules B, C, and D are
> built.
> >>> Call 'make --since B' to build only C and D.
> >>> Note that 'make --from' accepts more than one module name (while 'make
> >>> --all:<module>' does not).
> >>> Note also that in the above case (stand in D, call 'make --from B')
> >>> module
> >>> A is not built, regardless of whether there are changes in A or not.
> >>> Whereas a simple call to make (still standing in D) would build all
> >>> modules that D depends on, directly or indirectly. Thus the options
> >>> '--from' and '--since' exist to actively exclude modules from being
> >>> built.
> >>>
> >>> The whole thing becomes a little bit more complicated with multiple
> >>> options to '--from' (I never use '--since' and also don't know a valid
> >>> use
> >>> case so I will ignore it for now) and more complex dependencies then in
> >>> the
> >>> simple example above. Let's say that if we stand in instsetoo_native
> and
> >>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This
> would
> >>> build svx, sfx2 and all modules that depend (directly or indirectly) on
> >>> svx
> >>> OR sfx2.
> >>>
> >>> got it, now I just have one problem, why would you not build the
> >> dependent
> >> modules, if they needed to be built, thats a scenario I dont understand.
> >> With a central makefile, <module>/makefile will not be called so we do
> not
> >> waste cpu cycles.
> >>
> >> With the .done files, we know when a module was last built and all
> modules
> >> that depend it should be rebuilt which the rule
> >> <module>.done : <module_depend>.done
> >>
> >> will ensure, so If we have A -> B -> C -> D
> >>
> >> I go in B, and call make, then when I go in D and make, B,C,D will be
> >> made.
> >>
> >> If we have A -> B -> D C -> D
> >> and do the same then only D will be made.
> >>
> >> So --from is not really saving anything ?
> >>
> >
> > a) In your example you first go into B then, in a second step, into D.
> > The '--from' option lets you do the same (well, not really the same, but
> > see below) just from D.
> >
> > b) You go first to B and call make. This makes A, if necessary, then B.
> > The making of A is exactly the thing that you want to prevent with the
> > '--from' option. Go into D and call 'make --from B'. A is not built.
> >
> > c) After the discussion with you I am not sure if we still need --from
> > because the two reasons I know for its existence my not be relevant with
> > the new approach.
> >
> > c1) With the '--from' option you can tweak the dependency rules at
> runtime
> > (a bit). This allows you to exclude projects from being built when you
> > know that that is not necessary. But from experience I know that can
> lead
> > to very subtle errors. Letting the system determine what to built is
> > usually more reliable.
> >
> > c2) With build.pl a 'build --all' still builds every module on which the
> > one you are standing in depends on. When those modules have been built
> > previously, then no compilation takes place. But calling dmake for a
> > couple of directories for close to 200 modules (when you stand in
> > instsetoo_native) takes a lot of time (several minutes on Windows), even
> > when no file has to be compiled. This wasteful way of doing nothing can
> be
> > prevented with the --from option.
> > However, with the new approach and its .done flag files you can determine
> > which modules need to be built much faster. You don't have to call dmake
> > on directories that where already built. Hm, but this again, does only
> > work if your .done files have dependencies on all relevant source files.
> > That is something that is missing at least from my script.
> >
> >
> > So, reasons for the existence of '--from' are a result of old/slow
> > computers, slow files systems (still valid on Windows), missing global
> > dependencies (which we now have for gbuild) and impatient developers.
>
>
> Looking at what Jan has proposed and this discussion so far, I think we
> should make some attempt at moving in this direction -- using standard
> makefiles.
>
> Given this discussion, I do understand the additional flexibility of
> build.pl, but I also wonder if the flexibility is worth it considering the
> IDE tools new developers are likely to be familiar with.
>
> A complete build will take a while no matter what is used, depending on
> your situation. I think this proposal has a lot of merit.
>
Thx for the kind words.
I have updated the wiki page, to reflect the specific make calls.
and R1533869 shows my first version of the central Makefile
https://svn.apache.org/repos/asf/openoffice/branches/capstone2013/main/Makefile
you will see it contains the intermodule dependencies and nothing more. The
setup in every module is kept in a Makefile in the respective module, so we
have a clear seperation of module and intermodule dependencies.
hope you like it.
rgds
jan I.
>
>
>
>
>
> >
> >
> >>
> >> While this is easy to do with eg Perl I am not sure how to handle this
> >>> with just a Makefile. The straightforward approach with handling
> >>> <module>.done files does not work. And that is one of the reasons why
> I
> >>> don't think that (GNU) makefiles are a good solution for any problem.
> >>> Most
> >>> of us are used to program object oriented/imperative. Makefiles
> require
> >>> a
> >>> declarative approach. Maybe the use of Perl is not such a bad idea.
> >>> Maybe
> >>> it would be better to reimplement build.pl with a lot fewer options
> and
> >>> with better readable code.
> >>>
> >>> I agree that makefiles are nowhere near a good solution to many of
> these
> >> problems, but its like windows, I dont like it, but everybody uses it.
> >>
> >> We could easily write a new build.pl, that also took care of the local
> >> makefiles, but our build system would not be in the mainstream, and e.g.
> >> the distros would not like to integrate AOO.
> >>
> >> I have over the last years followed research in building systems, and
> >> there
> >> are (sadly enough) nobody that tries a real object oriented aproach.
> Also
> >> if you look at packages like visual studio, QT, eclipse they all use the
> >> principle of makefiles with declarative approach.
> >>
> >> So my simple question is, do we want to approach the main road
> (makefiles
> >> for unix, visual studio for windows/mac) or do we want to have better
> but
> >> non standard system.
> >>
> >
> > Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
> > ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
> > pragmatist says to use the make. It is good enough for others, it is
> good
> > enough for use.
> > On the other hand, when I start a new project I usually start with the
> > question of what are the best tools for the problem at hand. And make
> does
> > not seem to be the first or the best answer. Look at our dmake or gbuild
> > system. Both don't use make in a standard way. gmake even defines its
> own
> > language, object oriented and imperative, on top of the GNU make macros.
> > That is, for me, an act of desperation.
> > I have made experiments with an alternative approach, a domain specific
> > language somewhat similar to Java. I personally like this approach
> because
> > a) it uses the paradigm that I already use when writing C++ code. That
> > means that I can apply my existing knowledge to the build process and
> that
> > I don't have to remember all the tricks and pitfalls of makefiles.
> > b) as expected it was much easier to handle file dependencies and
> parallel
> > processing of build jobs in Java than adding object orientation and
> > imperative control flow to makefiles.
> >
> > If I had the time and if I would be the one working on it then I would
> > prefer an non-Makefile approach. But maybe I am just suffering too much
> > from one of the 'three great virtues of a programmer': hubris.
> >
> > You are the one who leads the build project changes, so you have to
> decide
> > which approach to use. I am not trying to make your life harder (than
> > necessary), I am only trying to point out some of the pitfalls that I
> have
> > encountered in the past. And to prevent you from removing features that
> I
> > use :-)
> >
> >
> >
> >
> >> rgds
> >> jan I.
> >>
> >> Ps. its always refreshing to discuss with you, you often have an
> >> alternative approach, which is not just a dream but doable.
> >>
> >
> > Thanks. That makes two of us.
> >
> > Have a nice weekend,
> > Andre
> >
> >
> >
> >>
> >> -Andre
> >>>
> >>>
> >>> --prepare
> >>>>
> >>>>> Also one option that is important for our every day work. Use
> >>>>> case:
> >>>>> You make changes in <module> and are not sure if these changes are
> >>>>> compatible/incompatible. To be on the safe side you discard the
> output
> >>>>> of
> >>>>> all depending modules. To save time you keep the output of all other
> >>>>> modules.
> >>>>>
> >>>>> Often used together with '--from' like 'make --prepare --from
> >>>>> svx' to
> >>>>> prepare a build after making changes in svx.
> >>>>>
> >>>>> Documentation changed, funny thing is that svx does not clear
> >>>>> correctly
> >>>>>
> >>>> on
> >>>> my ubuntu build.
> >>>>
> >>>>
> >>>> --since <module>
> >>>>
> >>>>> A variant of '--from'. The only difference is that <module>
> >>>>> itself
> >>>>> is
> >>>>> not built.
> >>>>>
> >>>>> If your proposed approach is similar to what my script produces
> >>>>> then
> >>>>> it
> >>>>> is not too difficult to support --from/--since. I made some
> >>>>> experiments
> >>>>> in
> >>>>> this direction but was to lazy to finish them.
> >>>>>
> >>>>> My approach is very similar, but I failed to see how --since is
> >>>>>
> >>>> supported.
> >>>> And question is if its real important.
> >>>>
> >>>>
> >>>> --job
> >>>>
> >>>>> --pre_job
> >>>>> --post_job
> >>>>> These are sometimes handy to run a non-standard command for all
> >>>>> modules.
> >>>>>
> >>>>> I have added them, they are by the way a good example why we need a
> >>>>>
> >>>> discussion I have never used them.
> >>>>
> >>>> However maybe the real discussion is "do we want to replace build and
> >>>> have
> >>>> a main/Makefile instead?"
> >>>>
> >>>>
> >>>>
> >>>> - I have not used the rest of the unsupported options and would not
> >>>> miss
> >>>>
> >>>>> them. Others may have other sets of options that are important to
> >>>>> them.
> >>>>>
> >>>>>
> >>>>> Some general remarks:
> >>>>>
> >>>>> - Why keep one makefile per module? Why not put all the inter-module
> >>>>> dependencies into one file (like my script does)?
> >>>>>
> >>>>> Ups, I did not explain that correctly, I propose 1 Makefile
> >>>>>
> >>>> "main/Makefile"
> >>>> with all inter-module and 1 Makefile "<module>/Makefile" that today
> just
> >>>> will call the old makefiles as described in prj/build.lst
> >>>>
> >>>> - Why not use the oportunity to move (a part of) the build environment
> >>>> out
> >>>>
> >>>> of the way to, say, build/ ?
> >>>>>
> >>>>> You have guessed my next step.
> >>>>>
> >>>>
> >>>> - How are dependencies between modules handled (just the manual
> >>>>
> >>>>> dependencies from prj/build.lst or also the file dependencies
> >>>>> introduced
> >>>>> by
> >>>>> gmake).
> >>>>>
> >>>>> See doc. on --from. Its done with <module>.done files
> >>>>>
> >>>>
> >>>> - How is the output of the individual calls to dmake or GNU make
> >>>>
> >>>>> handled/made accessible. Ie. if there is a build error, how can I
> look
> >>>>> up
> >>>>> the corresponding build output?
> >>>>>
> >>>>> see doc. script make_log
> >>>>>
> >>>> - Are the gmake makefiles included (run in the same process) or is
> GNU
> >>>>
> >>>>> make started for them it its own process?
> >>>>>
> >>>>> For a start they would be called (own process), but its something
> >>>>> where
> >>>>>
> >>>> I
> >>>> have no strong opinions.
> >>>>
> >>>> Please (just to be sure), this proposal has nothing to do with the
> >>>> students
> >>>> work, its simply because I saw a positive discussion on removing
> >>>> build.pl
> >>>> ,
> >>>> and spent a couple of hours looking at it. If there is a preference
> not
> >>>> to
> >>>> remove build.pl I will simply forget it.
> >>>>
> >>>> rgds
> >>>> jan I.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Regards,
> >>>>> Andre
> >>>>>
> >>>>>
> >>>>>
> >>>>> It has been roughly tested it, thanks to a clever utility from
> andre.
> >>>>>
> >>>>>> As discussed build.pl contains a lot of options, which need to be
> >>>>>> considered in a makefile.
> >>>>>>
> >>>>>> My suggestion is on
> >>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<
> http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
> >>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<
> http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
> >>>>>> >
> >>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****<
> http://wiki.openoffice.org/wiki/**>
> >>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://**
> >>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.**
> >>>>>> pl_versus_makefile<
> http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile
> >
> >>>>>> >
> >>>>>>
> >>>>>> Please feel free to edit/comment on the page. I have reduced to
> >>>>>> options
> >>>>>> a
> >>>>>> lot, and some of them might be in use.
> >>>>>>
> >>>>>> thanks in advance for your comments.
> >>>>>>
> >>>>>>
> >>>>>> ------------------------------******--------------------------**
> >>>>>> --**
> >>>>>>
> >>>>> --**---------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
> >>>>> http://apache.org**>
> >>>>> <dev-unsubscribe@**openoffice.**apache.org<
> http://openoffice.apache.org>
> >>>>> <dev-unsubscribe@**openoffice.apache.org<
> dev-unsubscribe@openoffice.apache.org>
> >>>>> >
> >>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
> >>>>>
> >>>>>
> >>>>>
> >>>>> ------------------------------****----------------------------**
> >>> --**---------
> >>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<
> http://apache.org>
> >>> <dev-unsubscribe@**openoffice.apache.org<
> dev-unsubscribe@openoffice.apache.org>
> >>> >
> >>> For additional commands, e-mail: dev-help@openoffice.apache.org
> >>>
> >>>
> >>>
> >
> > ------------------------------**------------------------------**---------
> > To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<
> dev-unsubscribe@openoffice.apache.org>
> >
> > For additional commands, e-mail: dev-help@openoffice.apache.org
> >
> >
>
>
> --
>
> -------------------------------------------------------------------------------------------------
> MzK
>
> "Truth is stranger than fiction, but it is because Fiction is obliged
> to stick to possibilities. Truth isn't."
> -- "Following the Equator", Mark Twain
>
Re: [proposal] replace build.pl with a central Makefile.
Posted by Kay Schenk <ka...@gmail.com>.
On Fri, Oct 18, 2013 at 7:52 AM, Andre Fischer <aw...@gmail.com> wrote:
> On 18.10.2013 15:58, janI wrote:
>
>> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
>>
>> On 18.10.2013 14:02, janI wrote:
>>>
>>> sd
>>>>
>>>>
>>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>>>
>>>> On 18.10.2013 11:32, janI wrote:
>>>>
>>>>> Hi.
>>>>>
>>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>>> made a
>>>>>> proposal for a central Makefile located in main.
>>>>>>
>>>>>> Hi Jan,
>>>>>>
>>>>> it is great that you are going to improve this part of the build
>>>>> system.
>>>>> But I think that we need more details about how the proposed build
>>>>> system
>>>>> works. Without them I can not really evaluate the proposal.
>>>>>
>>>>> First of all, I agree with juergens remarks that this should be
>>>>>
>>>> discussed
>>>> before implemented, hence the wiki page.
>>>>
>>>> Secondly this has nothing directly to do with the proposed build system,
>>>> its a simple replacement of build.pl in the current system.
>>>>
>>>> Yes, that is how I understood it. I just did not know how to call the
>>> build.pl replacement.
>>>
>>>
>>>
>>> I know that build.pl works, but having a Makefile in main, would make
>>>> us
>>>> one step closer on being compatible with the distros. To me this job is
>>>> a
>>>> simple cleanup, not something we deadly need, but nice to have.
>>>>
>>>>
>>>> Some remarks regarding the missing options:
>>>>
>>>>> --from <module>
>>>>> This is one of the more important options and one that I use
>>>>> frequently
>>>>> (also in the form --all:<module>).
>>>>> Note that if you are in <moduleA> and call 'make --from <moduleB>'
>>>>> then
>>>>> all modules are built
>>>>> a) which <moduleA> depends on
>>>>> b) but not those that <moduleB> depends on
>>>>> c) Both <moduleA> and <moduleB> are built.
>>>>>
>>>>> I have changed the documentation.
>>>>>
>>>> I use the --all:<module> myself very often, and have changed the
>>>> documentation, because it is of course supported.
>>>>
>>>> The difference is that you do the call in main, but that is a minor
>>>> detail
>>>> that can be easily corrected (have <module>/Makefile calling
>>>> main/Makefile.
>>>>
>>>> I have also changed documentation on --html due to juergens comments.
>>>>
>>>> I am not sure that we understand --from and --since in the same way so
>>> I
>>> will try to explain what I think they do.
>>>
>>> Let's imagine that we have a simple project with modules A, B, C, D and
>>> E.
>>> where B depends on A, C on B, D on C, and E on D.
>>> A ' make all' would mean 'make E'. The dependencies would then lead to
>>> building modules A, B, C, D, E in this order.
>>> If I am in E and call 'make --from C' then only C, D, and E should be
>>> built. A 'make --since C' would only build D and E.
>>>
>>> If I am in D and call 'make --from B' then modules B, C, and D are built.
>>> Call 'make --since B' to build only C and D.
>>> Note that 'make --from' accepts more than one module name (while 'make
>>> --all:<module>' does not).
>>> Note also that in the above case (stand in D, call 'make --from B')
>>> module
>>> A is not built, regardless of whether there are changes in A or not.
>>> Whereas a simple call to make (still standing in D) would build all
>>> modules that D depends on, directly or indirectly. Thus the options
>>> '--from' and '--since' exist to actively exclude modules from being
>>> built.
>>>
>>> The whole thing becomes a little bit more complicated with multiple
>>> options to '--from' (I never use '--since' and also don't know a valid
>>> use
>>> case so I will ignore it for now) and more complex dependencies then in
>>> the
>>> simple example above. Let's say that if we stand in instsetoo_native and
>>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This would
>>> build svx, sfx2 and all modules that depend (directly or indirectly) on
>>> svx
>>> OR sfx2.
>>>
>>> got it, now I just have one problem, why would you not build the
>> dependent
>> modules, if they needed to be built, thats a scenario I dont understand.
>> With a central makefile, <module>/makefile will not be called so we do not
>> waste cpu cycles.
>>
>> With the .done files, we know when a module was last built and all modules
>> that depend it should be rebuilt which the rule
>> <module>.done : <module_depend>.done
>>
>> will ensure, so If we have A -> B -> C -> D
>>
>> I go in B, and call make, then when I go in D and make, B,C,D will be
>> made.
>>
>> If we have A -> B -> D C -> D
>> and do the same then only D will be made.
>>
>> So --from is not really saving anything ?
>>
>
> a) In your example you first go into B then, in a second step, into D.
> The '--from' option lets you do the same (well, not really the same, but
> see below) just from D.
>
> b) You go first to B and call make. This makes A, if necessary, then B.
> The making of A is exactly the thing that you want to prevent with the
> '--from' option. Go into D and call 'make --from B'. A is not built.
>
> c) After the discussion with you I am not sure if we still need --from
> because the two reasons I know for its existence my not be relevant with
> the new approach.
>
> c1) With the '--from' option you can tweak the dependency rules at runtime
> (a bit). This allows you to exclude projects from being built when you
> know that that is not necessary. But from experience I know that can lead
> to very subtle errors. Letting the system determine what to built is
> usually more reliable.
>
> c2) With build.pl a 'build --all' still builds every module on which the
> one you are standing in depends on. When those modules have been built
> previously, then no compilation takes place. But calling dmake for a
> couple of directories for close to 200 modules (when you stand in
> instsetoo_native) takes a lot of time (several minutes on Windows), even
> when no file has to be compiled. This wasteful way of doing nothing can be
> prevented with the --from option.
> However, with the new approach and its .done flag files you can determine
> which modules need to be built much faster. You don't have to call dmake
> on directories that where already built. Hm, but this again, does only
> work if your .done files have dependencies on all relevant source files.
> That is something that is missing at least from my script.
>
>
> So, reasons for the existence of '--from' are a result of old/slow
> computers, slow files systems (still valid on Windows), missing global
> dependencies (which we now have for gbuild) and impatient developers.
Looking at what Jan has proposed and this discussion so far, I think we
should make some attempt at moving in this direction -- using standard
makefiles.
Given this discussion, I do understand the additional flexibility of
build.pl, but I also wonder if the flexibility is worth it considering the
IDE tools new developers are likely to be familiar with.
A complete build will take a while no matter what is used, depending on
your situation. I think this proposal has a lot of merit.
>
>
>>
>> While this is easy to do with eg Perl I am not sure how to handle this
>>> with just a Makefile. The straightforward approach with handling
>>> <module>.done files does not work. And that is one of the reasons why I
>>> don't think that (GNU) makefiles are a good solution for any problem.
>>> Most
>>> of us are used to program object oriented/imperative. Makefiles require
>>> a
>>> declarative approach. Maybe the use of Perl is not such a bad idea.
>>> Maybe
>>> it would be better to reimplement build.pl with a lot fewer options and
>>> with better readable code.
>>>
>>> I agree that makefiles are nowhere near a good solution to many of these
>> problems, but its like windows, I dont like it, but everybody uses it.
>>
>> We could easily write a new build.pl, that also took care of the local
>> makefiles, but our build system would not be in the mainstream, and e.g.
>> the distros would not like to integrate AOO.
>>
>> I have over the last years followed research in building systems, and
>> there
>> are (sadly enough) nobody that tries a real object oriented aproach. Also
>> if you look at packages like visual studio, QT, eclipse they all use the
>> principle of makefiles with declarative approach.
>>
>> So my simple question is, do we want to approach the main road (makefiles
>> for unix, visual studio for windows/mac) or do we want to have better but
>> non standard system.
>>
>
> Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
> ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
> pragmatist says to use the make. It is good enough for others, it is good
> enough for use.
> On the other hand, when I start a new project I usually start with the
> question of what are the best tools for the problem at hand. And make does
> not seem to be the first or the best answer. Look at our dmake or gbuild
> system. Both don't use make in a standard way. gmake even defines its own
> language, object oriented and imperative, on top of the GNU make macros.
> That is, for me, an act of desperation.
> I have made experiments with an alternative approach, a domain specific
> language somewhat similar to Java. I personally like this approach because
> a) it uses the paradigm that I already use when writing C++ code. That
> means that I can apply my existing knowledge to the build process and that
> I don't have to remember all the tricks and pitfalls of makefiles.
> b) as expected it was much easier to handle file dependencies and parallel
> processing of build jobs in Java than adding object orientation and
> imperative control flow to makefiles.
>
> If I had the time and if I would be the one working on it then I would
> prefer an non-Makefile approach. But maybe I am just suffering too much
> from one of the 'three great virtues of a programmer': hubris.
>
> You are the one who leads the build project changes, so you have to decide
> which approach to use. I am not trying to make your life harder (than
> necessary), I am only trying to point out some of the pitfalls that I have
> encountered in the past. And to prevent you from removing features that I
> use :-)
>
>
>
>
>> rgds
>> jan I.
>>
>> Ps. its always refreshing to discuss with you, you often have an
>> alternative approach, which is not just a dream but doable.
>>
>
> Thanks. That makes two of us.
>
> Have a nice weekend,
> Andre
>
>
>
>>
>> -Andre
>>>
>>>
>>> --prepare
>>>>
>>>>> Also one option that is important for our every day work. Use
>>>>> case:
>>>>> You make changes in <module> and are not sure if these changes are
>>>>> compatible/incompatible. To be on the safe side you discard the output
>>>>> of
>>>>> all depending modules. To save time you keep the output of all other
>>>>> modules.
>>>>>
>>>>> Often used together with '--from' like 'make --prepare --from
>>>>> svx' to
>>>>> prepare a build after making changes in svx.
>>>>>
>>>>> Documentation changed, funny thing is that svx does not clear
>>>>> correctly
>>>>>
>>>> on
>>>> my ubuntu build.
>>>>
>>>>
>>>> --since <module>
>>>>
>>>>> A variant of '--from'. The only difference is that <module>
>>>>> itself
>>>>> is
>>>>> not built.
>>>>>
>>>>> If your proposed approach is similar to what my script produces
>>>>> then
>>>>> it
>>>>> is not too difficult to support --from/--since. I made some
>>>>> experiments
>>>>> in
>>>>> this direction but was to lazy to finish them.
>>>>>
>>>>> My approach is very similar, but I failed to see how --since is
>>>>>
>>>> supported.
>>>> And question is if its real important.
>>>>
>>>>
>>>> --job
>>>>
>>>>> --pre_job
>>>>> --post_job
>>>>> These are sometimes handy to run a non-standard command for all
>>>>> modules.
>>>>>
>>>>> I have added them, they are by the way a good example why we need a
>>>>>
>>>> discussion I have never used them.
>>>>
>>>> However maybe the real discussion is "do we want to replace build and
>>>> have
>>>> a main/Makefile instead?"
>>>>
>>>>
>>>>
>>>> - I have not used the rest of the unsupported options and would not
>>>> miss
>>>>
>>>>> them. Others may have other sets of options that are important to
>>>>> them.
>>>>>
>>>>>
>>>>> Some general remarks:
>>>>>
>>>>> - Why keep one makefile per module? Why not put all the inter-module
>>>>> dependencies into one file (like my script does)?
>>>>>
>>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>>>>
>>>> "main/Makefile"
>>>> with all inter-module and 1 Makefile "<module>/Makefile" that today just
>>>> will call the old makefiles as described in prj/build.lst
>>>>
>>>> - Why not use the oportunity to move (a part of) the build environment
>>>> out
>>>>
>>>> of the way to, say, build/ ?
>>>>>
>>>>> You have guessed my next step.
>>>>>
>>>>
>>>> - How are dependencies between modules handled (just the manual
>>>>
>>>>> dependencies from prj/build.lst or also the file dependencies
>>>>> introduced
>>>>> by
>>>>> gmake).
>>>>>
>>>>> See doc. on --from. Its done with <module>.done files
>>>>>
>>>>
>>>> - How is the output of the individual calls to dmake or GNU make
>>>>
>>>>> handled/made accessible. Ie. if there is a build error, how can I look
>>>>> up
>>>>> the corresponding build output?
>>>>>
>>>>> see doc. script make_log
>>>>>
>>>> - Are the gmake makefiles included (run in the same process) or is GNU
>>>>
>>>>> make started for them it its own process?
>>>>>
>>>>> For a start they would be called (own process), but its something
>>>>> where
>>>>>
>>>> I
>>>> have no strong opinions.
>>>>
>>>> Please (just to be sure), this proposal has nothing to do with the
>>>> students
>>>> work, its simply because I saw a positive discussion on removing
>>>> build.pl
>>>> ,
>>>> and spent a couple of hours looking at it. If there is a preference not
>>>> to
>>>> remove build.pl I will simply forget it.
>>>>
>>>> rgds
>>>> jan I.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Regards,
>>>>> Andre
>>>>>
>>>>>
>>>>>
>>>>> It has been roughly tested it, thanks to a clever utility from andre.
>>>>>
>>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>>> considered in a makefile.
>>>>>>
>>>>>> My suggestion is on
>>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**>
>>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>>> >
>>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****<http://wiki.openoffice.org/wiki/**>
>>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://**
>>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.**
>>>>>> pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>>> >
>>>>>>
>>>>>> Please feel free to edit/comment on the page. I have reduced to
>>>>>> options
>>>>>> a
>>>>>> lot, and some of them might be in use.
>>>>>>
>>>>>> thanks in advance for your comments.
>>>>>>
>>>>>>
>>>>>> ------------------------------******--------------------------**
>>>>>> --**
>>>>>>
>>>>> --**---------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org<
>>>>> http://apache.org**>
>>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org>
>>>>> <de...@openoffice.apache.org>
>>>>> >
>>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------****----------------------------**
>>> --**---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>> <de...@openoffice.apache.org>
>>> >
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
--
-------------------------------------------------------------------------------------------------
MzK
"Truth is stranger than fiction, but it is because Fiction is obliged
to stick to possibilities. Truth isn't."
-- "Following the Equator", Mark Twain
Re: [proposal] replace build.pl with a central Makefile.
Posted by Andre Fischer <aw...@gmail.com>.
On 18.10.2013 15:58, janI wrote:
> On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
>
>> On 18.10.2013 14:02, janI wrote:
>>
>>> sd
>>>
>>>
>>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>>
>>> On 18.10.2013 11:32, janI wrote:
>>>> Hi.
>>>>> due to the discussion in thread "Mentor a new build system", I have
>>>>> made a
>>>>> proposal for a central Makefile located in main.
>>>>>
>>>>> Hi Jan,
>>>> it is great that you are going to improve this part of the build system.
>>>> But I think that we need more details about how the proposed build
>>>> system
>>>> works. Without them I can not really evaluate the proposal.
>>>>
>>>> First of all, I agree with juergens remarks that this should be
>>> discussed
>>> before implemented, hence the wiki page.
>>>
>>> Secondly this has nothing directly to do with the proposed build system,
>>> its a simple replacement of build.pl in the current system.
>>>
>> Yes, that is how I understood it. I just did not know how to call the
>> build.pl replacement.
>>
>>
>>
>>> I know that build.pl works, but having a Makefile in main, would make us
>>> one step closer on being compatible with the distros. To me this job is a
>>> simple cleanup, not something we deadly need, but nice to have.
>>>
>>>
>>> Some remarks regarding the missing options:
>>>> --from <module>
>>>> This is one of the more important options and one that I use
>>>> frequently
>>>> (also in the form --all:<module>).
>>>> Note that if you are in <moduleA> and call 'make --from <moduleB>'
>>>> then
>>>> all modules are built
>>>> a) which <moduleA> depends on
>>>> b) but not those that <moduleB> depends on
>>>> c) Both <moduleA> and <moduleB> are built.
>>>>
>>>> I have changed the documentation.
>>> I use the --all:<module> myself very often, and have changed the
>>> documentation, because it is of course supported.
>>>
>>> The difference is that you do the call in main, but that is a minor detail
>>> that can be easily corrected (have <module>/Makefile calling
>>> main/Makefile.
>>>
>>> I have also changed documentation on --html due to juergens comments.
>>>
>> I am not sure that we understand --from and --since in the same way so I
>> will try to explain what I think they do.
>>
>> Let's imagine that we have a simple project with modules A, B, C, D and E.
>> where B depends on A, C on B, D on C, and E on D.
>> A ' make all' would mean 'make E'. The dependencies would then lead to
>> building modules A, B, C, D, E in this order.
>> If I am in E and call 'make --from C' then only C, D, and E should be
>> built. A 'make --since C' would only build D and E.
>>
>> If I am in D and call 'make --from B' then modules B, C, and D are built.
>> Call 'make --since B' to build only C and D.
>> Note that 'make --from' accepts more than one module name (while 'make
>> --all:<module>' does not).
>> Note also that in the above case (stand in D, call 'make --from B') module
>> A is not built, regardless of whether there are changes in A or not.
>> Whereas a simple call to make (still standing in D) would build all
>> modules that D depends on, directly or indirectly. Thus the options
>> '--from' and '--since' exist to actively exclude modules from being built.
>>
>> The whole thing becomes a little bit more complicated with multiple
>> options to '--from' (I never use '--since' and also don't know a valid use
>> case so I will ignore it for now) and more complex dependencies then in the
>> simple example above. Let's say that if we stand in instsetoo_native and
>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This would
>> build svx, sfx2 and all modules that depend (directly or indirectly) on svx
>> OR sfx2.
>>
> got it, now I just have one problem, why would you not build the dependent
> modules, if they needed to be built, thats a scenario I dont understand.
> With a central makefile, <module>/makefile will not be called so we do not
> waste cpu cycles.
>
> With the .done files, we know when a module was last built and all modules
> that depend it should be rebuilt which the rule
> <module>.done : <module_depend>.done
>
> will ensure, so If we have A -> B -> C -> D
>
> I go in B, and call make, then when I go in D and make, B,C,D will be made.
>
> If we have A -> B -> D C -> D
> and do the same then only D will be made.
>
> So --from is not really saving anything ?
a) In your example you first go into B then, in a second step, into D.
The '--from' option lets you do the same (well, not really the same, but
see below) just from D.
b) You go first to B and call make. This makes A, if necessary, then
B. The making of A is exactly the thing that you want to prevent with
the '--from' option. Go into D and call 'make --from B'. A is not built.
c) After the discussion with you I am not sure if we still need --from
because the two reasons I know for its existence my not be relevant with
the new approach.
c1) With the '--from' option you can tweak the dependency rules at
runtime (a bit). This allows you to exclude projects from being built
when you know that that is not necessary. But from experience I know
that can lead to very subtle errors. Letting the system determine what
to built is usually more reliable.
c2) With build.pl a 'build --all' still builds every module on which the
one you are standing in depends on. When those modules have been built
previously, then no compilation takes place. But calling dmake for a
couple of directories for close to 200 modules (when you stand in
instsetoo_native) takes a lot of time (several minutes on Windows), even
when no file has to be compiled. This wasteful way of doing nothing can
be prevented with the --from option.
However, with the new approach and its .done flag files you can
determine which modules need to be built much faster. You don't have to
call dmake on directories that where already built. Hm, but this again,
does only work if your .done files have dependencies on all relevant
source files. That is something that is missing at least from my script.
So, reasons for the existence of '--from' are a result of old/slow
computers, slow files systems (still valid on Windows), missing global
dependencies (which we now have for gbuild) and impatient developers.
>
>
>> While this is easy to do with eg Perl I am not sure how to handle this
>> with just a Makefile. The straightforward approach with handling
>> <module>.done files does not work. And that is one of the reasons why I
>> don't think that (GNU) makefiles are a good solution for any problem. Most
>> of us are used to program object oriented/imperative. Makefiles require a
>> declarative approach. Maybe the use of Perl is not such a bad idea. Maybe
>> it would be better to reimplement build.pl with a lot fewer options and
>> with better readable code.
>>
> I agree that makefiles are nowhere near a good solution to many of these
> problems, but its like windows, I dont like it, but everybody uses it.
>
> We could easily write a new build.pl, that also took care of the local
> makefiles, but our build system would not be in the mainstream, and e.g.
> the distros would not like to integrate AOO.
>
> I have over the last years followed research in building systems, and there
> are (sadly enough) nobody that tries a real object oriented aproach. Also
> if you look at packages like visual studio, QT, eclipse they all use the
> principle of makefiles with declarative approach.
>
> So my simple question is, do we want to approach the main road (makefiles
> for unix, visual studio for windows/mac) or do we want to have better but
> non standard system.
Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen,
ach! in meiner Brust" (two souls alas! are dwelling in my breast). The
pragmatist says to use the make. It is good enough for others, it is
good enough for use.
On the other hand, when I start a new project I usually start with the
question of what are the best tools for the problem at hand. And make
does not seem to be the first or the best answer. Look at our dmake or
gbuild system. Both don't use make in a standard way. gmake even
defines its own language, object oriented and imperative, on top of the
GNU make macros. That is, for me, an act of desperation.
I have made experiments with an alternative approach, a domain specific
language somewhat similar to Java. I personally like this approach because
a) it uses the paradigm that I already use when writing C++ code. That
means that I can apply my existing knowledge to the build process and
that I don't have to remember all the tricks and pitfalls of makefiles.
b) as expected it was much easier to handle file dependencies and
parallel processing of build jobs in Java than adding object orientation
and imperative control flow to makefiles.
If I had the time and if I would be the one working on it then I would
prefer an non-Makefile approach. But maybe I am just suffering too much
from one of the 'three great virtues of a programmer': hubris.
You are the one who leads the build project changes, so you have to
decide which approach to use. I am not trying to make your life harder
(than necessary), I am only trying to point out some of the pitfalls
that I have encountered in the past. And to prevent you from removing
features that I use :-)
>
> rgds
> jan I.
>
> Ps. its always refreshing to discuss with you, you often have an
> alternative approach, which is not just a dream but doable.
Thanks. That makes two of us.
Have a nice weekend,
Andre
>
>
>> -Andre
>>
>>
>>> --prepare
>>>> Also one option that is important for our every day work. Use case:
>>>> You make changes in <module> and are not sure if these changes are
>>>> compatible/incompatible. To be on the safe side you discard the output
>>>> of
>>>> all depending modules. To save time you keep the output of all other
>>>> modules.
>>>>
>>>> Often used together with '--from' like 'make --prepare --from svx' to
>>>> prepare a build after making changes in svx.
>>>>
>>>> Documentation changed, funny thing is that svx does not clear correctly
>>> on
>>> my ubuntu build.
>>>
>>>
>>> --since <module>
>>>> A variant of '--from'. The only difference is that <module> itself
>>>> is
>>>> not built.
>>>>
>>>> If your proposed approach is similar to what my script produces then
>>>> it
>>>> is not too difficult to support --from/--since. I made some experiments
>>>> in
>>>> this direction but was to lazy to finish them.
>>>>
>>>> My approach is very similar, but I failed to see how --since is
>>> supported.
>>> And question is if its real important.
>>>
>>>
>>> --job
>>>> --pre_job
>>>> --post_job
>>>> These are sometimes handy to run a non-standard command for all
>>>> modules.
>>>>
>>>> I have added them, they are by the way a good example why we need a
>>> discussion I have never used them.
>>>
>>> However maybe the real discussion is "do we want to replace build and have
>>> a main/Makefile instead?"
>>>
>>>
>>>
>>> - I have not used the rest of the unsupported options and would not miss
>>>> them. Others may have other sets of options that are important to them.
>>>>
>>>>
>>>> Some general remarks:
>>>>
>>>> - Why keep one makefile per module? Why not put all the inter-module
>>>> dependencies into one file (like my script does)?
>>>>
>>>> Ups, I did not explain that correctly, I propose 1 Makefile
>>> "main/Makefile"
>>> with all inter-module and 1 Makefile "<module>/Makefile" that today just
>>> will call the old makefiles as described in prj/build.lst
>>>
>>> - Why not use the oportunity to move (a part of) the build environment out
>>>
>>>> of the way to, say, build/ ?
>>>>
>>>> You have guessed my next step.
>>>
>>> - How are dependencies between modules handled (just the manual
>>>> dependencies from prj/build.lst or also the file dependencies introduced
>>>> by
>>>> gmake).
>>>>
>>>> See doc. on --from. Its done with <module>.done files
>>>
>>> - How is the output of the individual calls to dmake or GNU make
>>>> handled/made accessible. Ie. if there is a build error, how can I look
>>>> up
>>>> the corresponding build output?
>>>>
>>>> see doc. script make_log
>>> - Are the gmake makefiles included (run in the same process) or is GNU
>>>> make started for them it its own process?
>>>>
>>>> For a start they would be called (own process), but its something where
>>> I
>>> have no strong opinions.
>>>
>>> Please (just to be sure), this proposal has nothing to do with the
>>> students
>>> work, its simply because I saw a positive discussion on removing build.pl
>>> ,
>>> and spent a couple of hours looking at it. If there is a preference not to
>>> remove build.pl I will simply forget it.
>>>
>>> rgds
>>> jan I.
>>>
>>>
>>>
>>>
>>>
>>>> Regards,
>>>> Andre
>>>>
>>>>
>>>>
>>>> It has been roughly tested it, thanks to a clever utility from andre.
>>>>> As discussed build.pl contains a lot of options, which need to be
>>>>> considered in a makefile.
>>>>>
>>>>> My suggestion is on
>>>>> http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>>> build.pl_versus_makefile<http:**//wiki.openoffice.org/wiki/**
>>>>> Build_System_Analysis:build.**pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>>>
>>>>> Please feel free to edit/comment on the page. I have reduced to options
>>>>> a
>>>>> lot, and some of them might be in use.
>>>>>
>>>>> thanks in advance for your comments.
>>>>>
>>>>>
>>>>> ------------------------------****----------------------------**
>>>> --**---------
>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>>> <de...@openoffice.apache.org>
>>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>>
>>>>
>>>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [proposal] replace build.pl with a central Makefile.
Posted by janI <ja...@apache.org>.
On 18 October 2013 15:00, Andre Fischer <aw...@gmail.com> wrote:
> On 18.10.2013 14:02, janI wrote:
>
>> sd
>>
>>
>> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>>
>> On 18.10.2013 11:32, janI wrote:
>>>
>>> Hi.
>>>>
>>>> due to the discussion in thread "Mentor a new build system", I have
>>>> made a
>>>> proposal for a central Makefile located in main.
>>>>
>>>> Hi Jan,
>>>
>>> it is great that you are going to improve this part of the build system.
>>> But I think that we need more details about how the proposed build
>>> system
>>> works. Without them I can not really evaluate the proposal.
>>>
>>> First of all, I agree with juergens remarks that this should be
>> discussed
>> before implemented, hence the wiki page.
>>
>> Secondly this has nothing directly to do with the proposed build system,
>> its a simple replacement of build.pl in the current system.
>>
>
> Yes, that is how I understood it. I just did not know how to call the
> build.pl replacement.
>
>
>
>> I know that build.pl works, but having a Makefile in main, would make us
>> one step closer on being compatible with the distros. To me this job is a
>> simple cleanup, not something we deadly need, but nice to have.
>>
>>
>> Some remarks regarding the missing options:
>>>
>>> --from <module>
>>> This is one of the more important options and one that I use
>>> frequently
>>> (also in the form --all:<module>).
>>> Note that if you are in <moduleA> and call 'make --from <moduleB>'
>>> then
>>> all modules are built
>>> a) which <moduleA> depends on
>>> b) but not those that <moduleB> depends on
>>> c) Both <moduleA> and <moduleB> are built.
>>>
>>> I have changed the documentation.
>>
>> I use the --all:<module> myself very often, and have changed the
>> documentation, because it is of course supported.
>>
>> The difference is that you do the call in main, but that is a minor detail
>> that can be easily corrected (have <module>/Makefile calling
>> main/Makefile.
>>
>> I have also changed documentation on --html due to juergens comments.
>>
>
> I am not sure that we understand --from and --since in the same way so I
> will try to explain what I think they do.
>
> Let's imagine that we have a simple project with modules A, B, C, D and E.
> where B depends on A, C on B, D on C, and E on D.
> A ' make all' would mean 'make E'. The dependencies would then lead to
> building modules A, B, C, D, E in this order.
> If I am in E and call 'make --from C' then only C, D, and E should be
> built. A 'make --since C' would only build D and E.
>
> If I am in D and call 'make --from B' then modules B, C, and D are built.
> Call 'make --since B' to build only C and D.
> Note that 'make --from' accepts more than one module name (while 'make
> --all:<module>' does not).
> Note also that in the above case (stand in D, call 'make --from B') module
> A is not built, regardless of whether there are changes in A or not.
> Whereas a simple call to make (still standing in D) would build all
> modules that D depends on, directly or indirectly. Thus the options
> '--from' and '--since' exist to actively exclude modules from being built.
>
> The whole thing becomes a little bit more complicated with multiple
> options to '--from' (I never use '--since' and also don't know a valid use
> case so I will ignore it for now) and more complex dependencies then in the
> simple example above. Let's say that if we stand in instsetoo_native and
> call 'make --from svx sfx2'. Note that svx depends on sfx2. This would
> build svx, sfx2 and all modules that depend (directly or indirectly) on svx
> OR sfx2.
>
got it, now I just have one problem, why would you not build the dependent
modules, if they needed to be built, thats a scenario I dont understand.
With a central makefile, <module>/makefile will not be called so we do not
waste cpu cycles.
With the .done files, we know when a module was last built and all modules
that depend it should be rebuilt which the rule
<module>.done : <module_depend>.done
will ensure, so If we have A -> B -> C -> D
I go in B, and call make, then when I go in D and make, B,C,D will be made.
If we have A -> B -> D C -> D
and do the same then only D will be made.
So --from is not really saving anything ?
>
> While this is easy to do with eg Perl I am not sure how to handle this
> with just a Makefile. The straightforward approach with handling
> <module>.done files does not work. And that is one of the reasons why I
> don't think that (GNU) makefiles are a good solution for any problem. Most
> of us are used to program object oriented/imperative. Makefiles require a
> declarative approach. Maybe the use of Perl is not such a bad idea. Maybe
> it would be better to reimplement build.pl with a lot fewer options and
> with better readable code.
>
I agree that makefiles are nowhere near a good solution to many of these
problems, but its like windows, I dont like it, but everybody uses it.
We could easily write a new build.pl, that also took care of the local
makefiles, but our build system would not be in the mainstream, and e.g.
the distros would not like to integrate AOO.
I have over the last years followed research in building systems, and there
are (sadly enough) nobody that tries a real object oriented aproach. Also
if you look at packages like visual studio, QT, eclipse they all use the
principle of makefiles with declarative approach.
So my simple question is, do we want to approach the main road (makefiles
for unix, visual studio for windows/mac) or do we want to have better but
non standard system.
rgds
jan I.
Ps. its always refreshing to discuss with you, you often have an
alternative approach, which is not just a dream but doable.
> -Andre
>
>
>>
>> --prepare
>>> Also one option that is important for our every day work. Use case:
>>> You make changes in <module> and are not sure if these changes are
>>> compatible/incompatible. To be on the safe side you discard the output
>>> of
>>> all depending modules. To save time you keep the output of all other
>>> modules.
>>>
>>> Often used together with '--from' like 'make --prepare --from svx' to
>>> prepare a build after making changes in svx.
>>>
>>> Documentation changed, funny thing is that svx does not clear correctly
>> on
>> my ubuntu build.
>>
>>
>> --since <module>
>>> A variant of '--from'. The only difference is that <module> itself
>>> is
>>> not built.
>>>
>>> If your proposed approach is similar to what my script produces then
>>> it
>>> is not too difficult to support --from/--since. I made some experiments
>>> in
>>> this direction but was to lazy to finish them.
>>>
>>> My approach is very similar, but I failed to see how --since is
>> supported.
>> And question is if its real important.
>>
>>
>> --job
>>> --pre_job
>>> --post_job
>>> These are sometimes handy to run a non-standard command for all
>>> modules.
>>>
>>> I have added them, they are by the way a good example why we need a
>> discussion I have never used them.
>>
>> However maybe the real discussion is "do we want to replace build and have
>> a main/Makefile instead?"
>>
>>
>>
>> - I have not used the rest of the unsupported options and would not miss
>>> them. Others may have other sets of options that are important to them.
>>>
>>>
>>> Some general remarks:
>>>
>>> - Why keep one makefile per module? Why not put all the inter-module
>>> dependencies into one file (like my script does)?
>>>
>>> Ups, I did not explain that correctly, I propose 1 Makefile
>> "main/Makefile"
>> with all inter-module and 1 Makefile "<module>/Makefile" that today just
>> will call the old makefiles as described in prj/build.lst
>>
>> - Why not use the oportunity to move (a part of) the build environment out
>>
>>> of the way to, say, build/ ?
>>>
>>> You have guessed my next step.
>>
>>
>> - How are dependencies between modules handled (just the manual
>>> dependencies from prj/build.lst or also the file dependencies introduced
>>> by
>>> gmake).
>>>
>>> See doc. on --from. Its done with <module>.done files
>>
>>
>> - How is the output of the individual calls to dmake or GNU make
>>> handled/made accessible. Ie. if there is a build error, how can I look
>>> up
>>> the corresponding build output?
>>>
>>> see doc. script make_log
>>
>> - Are the gmake makefiles included (run in the same process) or is GNU
>>> make started for them it its own process?
>>>
>>> For a start they would be called (own process), but its something where
>> I
>> have no strong opinions.
>>
>> Please (just to be sure), this proposal has nothing to do with the
>> students
>> work, its simply because I saw a positive discussion on removing build.pl
>> ,
>> and spent a couple of hours looking at it. If there is a preference not to
>> remove build.pl I will simply forget it.
>>
>> rgds
>> jan I.
>>
>>
>>
>>
>>
>>> Regards,
>>> Andre
>>>
>>>
>>>
>>> It has been roughly tested it, thanks to a clever utility from andre.
>>>>
>>>> As discussed build.pl contains a lot of options, which need to be
>>>> considered in a makefile.
>>>>
>>>> My suggestion is on
>>>> http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**>
>>>> build.pl_versus_makefile<http:**//wiki.openoffice.org/wiki/**
>>>> Build_System_Analysis:build.**pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>> >
>>>>
>>>>
>>>> Please feel free to edit/comment on the page. I have reduced to options
>>>> a
>>>> lot, and some of them might be in use.
>>>>
>>>> thanks in advance for your comments.
>>>>
>>>>
>>>> ------------------------------****----------------------------**
>>> --**---------
>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org<http://apache.org>
>>> <de...@openoffice.apache.org>
>>> >
>>>
>>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>>
>>>
>>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: [proposal] replace build.pl with a central Makefile.
Posted by Andre Fischer <aw...@gmail.com>.
On 18.10.2013 14:02, janI wrote:
> sd
>
>
> On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
>
>> On 18.10.2013 11:32, janI wrote:
>>
>>> Hi.
>>>
>>> due to the discussion in thread "Mentor a new build system", I have made a
>>> proposal for a central Makefile located in main.
>>>
>> Hi Jan,
>>
>> it is great that you are going to improve this part of the build system.
>> But I think that we need more details about how the proposed build system
>> works. Without them I can not really evaluate the proposal.
>>
> First of all, I agree with juergens remarks that this should be discussed
> before implemented, hence the wiki page.
>
> Secondly this has nothing directly to do with the proposed build system,
> its a simple replacement of build.pl in the current system.
Yes, that is how I understood it. I just did not know how to call the
build.pl replacement.
>
> I know that build.pl works, but having a Makefile in main, would make us
> one step closer on being compatible with the distros. To me this job is a
> simple cleanup, not something we deadly need, but nice to have.
>
>
>> Some remarks regarding the missing options:
>>
>> --from <module>
>> This is one of the more important options and one that I use frequently
>> (also in the form --all:<module>).
>> Note that if you are in <moduleA> and call 'make --from <moduleB>' then
>> all modules are built
>> a) which <moduleA> depends on
>> b) but not those that <moduleB> depends on
>> c) Both <moduleA> and <moduleB> are built.
>>
> I have changed the documentation.
>
> I use the --all:<module> myself very often, and have changed the
> documentation, because it is of course supported.
>
> The difference is that you do the call in main, but that is a minor detail
> that can be easily corrected (have <module>/Makefile calling main/Makefile.
>
> I have also changed documentation on --html due to juergens comments.
I am not sure that we understand --from and --since in the same way so I
will try to explain what I think they do.
Let's imagine that we have a simple project with modules A, B, C, D and
E. where B depends on A, C on B, D on C, and E on D.
A ' make all' would mean 'make E'. The dependencies would then lead to
building modules A, B, C, D, E in this order.
If I am in E and call 'make --from C' then only C, D, and E should be
built. A 'make --since C' would only build D and E.
If I am in D and call 'make --from B' then modules B, C, and D are
built. Call 'make --since B' to build only C and D.
Note that 'make --from' accepts more than one module name (while 'make
--all:<module>' does not).
Note also that in the above case (stand in D, call 'make --from B')
module A is not built, regardless of whether there are changes in A or
not. Whereas a simple call to make (still standing in D) would build
all modules that D depends on, directly or indirectly. Thus the options
'--from' and '--since' exist to actively exclude modules from being built.
The whole thing becomes a little bit more complicated with multiple
options to '--from' (I never use '--since' and also don't know a valid
use case so I will ignore it for now) and more complex dependencies then
in the simple example above. Let's say that if we stand in
instsetoo_native and call 'make --from svx sfx2'. Note that svx depends
on sfx2. This would build svx, sfx2 and all modules that depend
(directly or indirectly) on svx OR sfx2.
While this is easy to do with eg Perl I am not sure how to handle this
with just a Makefile. The straightforward approach with handling
<module>.done files does not work. And that is one of the reasons why I
don't think that (GNU) makefiles are a good solution for any problem.
Most of us are used to program object oriented/imperative. Makefiles
require a declarative approach. Maybe the use of Perl is not such a bad
idea. Maybe it would be better to reimplement build.pl with a lot fewer
options and with better readable code.
-Andre
>
>
>> --prepare
>> Also one option that is important for our every day work. Use case:
>> You make changes in <module> and are not sure if these changes are
>> compatible/incompatible. To be on the safe side you discard the output of
>> all depending modules. To save time you keep the output of all other
>> modules.
>>
>> Often used together with '--from' like 'make --prepare --from svx' to
>> prepare a build after making changes in svx.
>>
> Documentation changed, funny thing is that svx does not clear correctly on
> my ubuntu build.
>
>
>> --since <module>
>> A variant of '--from'. The only difference is that <module> itself is
>> not built.
>>
>> If your proposed approach is similar to what my script produces then it
>> is not too difficult to support --from/--since. I made some experiments in
>> this direction but was to lazy to finish them.
>>
> My approach is very similar, but I failed to see how --since is supported.
> And question is if its real important.
>
>
>> --job
>> --pre_job
>> --post_job
>> These are sometimes handy to run a non-standard command for all modules.
>>
> I have added them, they are by the way a good example why we need a
> discussion I have never used them.
>
> However maybe the real discussion is "do we want to replace build and have
> a main/Makefile instead?"
>
>
>
>> - I have not used the rest of the unsupported options and would not miss
>> them. Others may have other sets of options that are important to them.
>>
>>
>> Some general remarks:
>>
>> - Why keep one makefile per module? Why not put all the inter-module
>> dependencies into one file (like my script does)?
>>
> Ups, I did not explain that correctly, I propose 1 Makefile "main/Makefile"
> with all inter-module and 1 Makefile "<module>/Makefile" that today just
> will call the old makefiles as described in prj/build.lst
>
> - Why not use the oportunity to move (a part of) the build environment out
>> of the way to, say, build/ ?
>>
> You have guessed my next step.
>
>
>> - How are dependencies between modules handled (just the manual
>> dependencies from prj/build.lst or also the file dependencies introduced by
>> gmake).
>>
> See doc. on --from. Its done with <module>.done files
>
>
>> - How is the output of the individual calls to dmake or GNU make
>> handled/made accessible. Ie. if there is a build error, how can I look up
>> the corresponding build output?
>>
> see doc. script make_log
>
>> - Are the gmake makefiles included (run in the same process) or is GNU
>> make started for them it its own process?
>>
> For a start they would be called (own process), but its something where I
> have no strong opinions.
>
> Please (just to be sure), this proposal has nothing to do with the students
> work, its simply because I saw a positive discussion on removing build.pl,
> and spent a couple of hours looking at it. If there is a preference not to
> remove build.pl I will simply forget it.
>
> rgds
> jan I.
>
>
>
>
>>
>> Regards,
>> Andre
>>
>>
>>
>>> It has been roughly tested it, thanks to a clever utility from andre.
>>>
>>> As discussed build.pl contains a lot of options, which need to be
>>> considered in a makefile.
>>>
>>> My suggestion is on
>>> http://wiki.openoffice.org/**wiki/Build_System_Analysis:**
>>> build.pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>>
>>> Please feel free to edit/comment on the page. I have reduced to options a
>>> lot, and some of them might be in use.
>>>
>>> thanks in advance for your comments.
>>>
>>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
>> For additional commands, e-mail: dev-help@openoffice.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [proposal] replace build.pl with a central Makefile.
Posted by janI <ja...@apache.org>.
sd
On 18 October 2013 13:36, Andre Fischer <aw...@gmail.com> wrote:
> On 18.10.2013 11:32, janI wrote:
>
>> Hi.
>>
>> due to the discussion in thread "Mentor a new build system", I have made a
>> proposal for a central Makefile located in main.
>>
> Hi Jan,
>
> it is great that you are going to improve this part of the build system.
> But I think that we need more details about how the proposed build system
> works. Without them I can not really evaluate the proposal.
>
First of all, I agree with juergens remarks that this should be discussed
before implemented, hence the wiki page.
Secondly this has nothing directly to do with the proposed build system,
its a simple replacement of build.pl in the current system.
I know that build.pl works, but having a Makefile in main, would make us
one step closer on being compatible with the distros. To me this job is a
simple cleanup, not something we deadly need, but nice to have.
>
> Some remarks regarding the missing options:
>
> --from <module>
> This is one of the more important options and one that I use frequently
> (also in the form --all:<module>).
> Note that if you are in <moduleA> and call 'make --from <moduleB>' then
> all modules are built
> a) which <moduleA> depends on
> b) but not those that <moduleB> depends on
> c) Both <moduleA> and <moduleB> are built.
>
I have changed the documentation.
I use the --all:<module> myself very often, and have changed the
documentation, because it is of course supported.
The difference is that you do the call in main, but that is a minor detail
that can be easily corrected (have <module>/Makefile calling main/Makefile.
I have also changed documentation on --html due to juergens comments.
> --prepare
> Also one option that is important for our every day work. Use case:
> You make changes in <module> and are not sure if these changes are
> compatible/incompatible. To be on the safe side you discard the output of
> all depending modules. To save time you keep the output of all other
> modules.
>
> Often used together with '--from' like 'make --prepare --from svx' to
> prepare a build after making changes in svx.
>
Documentation changed, funny thing is that svx does not clear correctly on
my ubuntu build.
>
> --since <module>
> A variant of '--from'. The only difference is that <module> itself is
> not built.
>
> If your proposed approach is similar to what my script produces then it
> is not too difficult to support --from/--since. I made some experiments in
> this direction but was to lazy to finish them.
>
My approach is very similar, but I failed to see how --since is supported.
And question is if its real important.
>
> --job
> --pre_job
> --post_job
> These are sometimes handy to run a non-standard command for all modules.
>
I have added them, they are by the way a good example why we need a
discussion I have never used them.
However maybe the real discussion is "do we want to replace build and have
a main/Makefile instead?"
>
> - I have not used the rest of the unsupported options and would not miss
> them. Others may have other sets of options that are important to them.
>
>
> Some general remarks:
>
> - Why keep one makefile per module? Why not put all the inter-module
> dependencies into one file (like my script does)?
>
Ups, I did not explain that correctly, I propose 1 Makefile "main/Makefile"
with all inter-module and 1 Makefile "<module>/Makefile" that today just
will call the old makefiles as described in prj/build.lst
- Why not use the oportunity to move (a part of) the build environment out
> of the way to, say, build/ ?
>
You have guessed my next step.
>
> - How are dependencies between modules handled (just the manual
> dependencies from prj/build.lst or also the file dependencies introduced by
> gmake).
>
See doc. on --from. Its done with <module>.done files
> - How is the output of the individual calls to dmake or GNU make
> handled/made accessible. Ie. if there is a build error, how can I look up
> the corresponding build output?
>
see doc. script make_log
>
> - Are the gmake makefiles included (run in the same process) or is GNU
> make started for them it its own process?
>
For a start they would be called (own process), but its something where I
have no strong opinions.
Please (just to be sure), this proposal has nothing to do with the students
work, its simply because I saw a positive discussion on removing build.pl,
and spent a couple of hours looking at it. If there is a preference not to
remove build.pl I will simply forget it.
rgds
jan I.
>
>
> Regards,
> Andre
>
>
>
>> It has been roughly tested it, thanks to a clever utility from andre.
>>
>> As discussed build.pl contains a lot of options, which need to be
>> considered in a makefile.
>>
>> My suggestion is on
>> http://wiki.openoffice.org/**wiki/Build_System_Analysis:**
>> build.pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile>
>>
>> Please feel free to edit/comment on the page. I have reduced to options a
>> lot, and some of them might be in use.
>>
>> thanks in advance for your comments.
>>
>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org<de...@openoffice.apache.org>
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: [proposal] replace build.pl with a central Makefile.
Posted by Andre Fischer <aw...@gmail.com>.
On 18.10.2013 11:32, janI wrote:
> Hi.
>
> due to the discussion in thread "Mentor a new build system", I have made a
> proposal for a central Makefile located in main.
Hi Jan,
it is great that you are going to improve this part of the build
system. But I think that we need more details about how the proposed
build system works. Without them I can not really evaluate the proposal.
Some remarks regarding the missing options:
--from <module>
This is one of the more important options and one that I use
frequently (also in the form --all:<module>).
Note that if you are in <moduleA> and call 'make --from <moduleB>'
then all modules are built
a) which <moduleA> depends on
b) but not those that <moduleB> depends on
c) Both <moduleA> and <moduleB> are built.
--prepare
Also one option that is important for our every day work. Use case:
You make changes in <module> and are not sure if these changes are
compatible/incompatible. To be on the safe side you discard the output
of all depending modules. To save time you keep the output of all other
modules.
Often used together with '--from' like 'make --prepare --from svx'
to prepare a build after making changes in svx.
--since <module>
A variant of '--from'. The only difference is that <module> itself
is not built.
If your proposed approach is similar to what my script produces then
it is not too difficult to support --from/--since. I made some
experiments in this direction but was to lazy to finish them.
--job
--pre_job
--post_job
These are sometimes handy to run a non-standard command for all modules.
- I have not used the rest of the unsupported options and would not miss
them. Others may have other sets of options that are important to them.
Some general remarks:
- Why keep one makefile per module? Why not put all the inter-module
dependencies into one file (like my script does)?
- Why not use the oportunity to move (a part of) the build environment
out of the way to, say, build/ ?
- How are dependencies between modules handled (just the manual
dependencies from prj/build.lst or also the file dependencies introduced
by gmake).
- How is the output of the individual calls to dmake or GNU make
handled/made accessible. Ie. if there is a build error, how can I look
up the corresponding build output?
- Are the gmake makefiles included (run in the same process) or is GNU
make started for them it its own process?
Regards,
Andre
>
> It has been roughly tested it, thanks to a clever utility from andre.
>
> As discussed build.pl contains a lot of options, which need to be
> considered in a makefile.
>
> My suggestion is on
> http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile
>
> Please feel free to edit/comment on the page. I have reduced to options a
> lot, and some of them might be in use.
>
> thanks in advance for your comments.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [proposal] replace build.pl with a central Makefile.
Posted by Andre Fischer <aw...@gmail.com>.
On 18.10.2013 11:32, janI wrote:
> Hi.
>
> due to the discussion in thread "Mentor a new build system", I have made a
> proposal for a central Makefile located in main.
One thing that I forgot earlier:
Can you state the problems with build.pl that you are addressing and how
the new approach does better?
-Andre
>
> It has been roughly tested it, thanks to a clever utility from andre.
>
> As discussed build.pl contains a lot of options, which need to be
> considered in a makefile.
>
> My suggestion is on
> http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile
>
> Please feel free to edit/comment on the page. I have reduced to options a
> lot, and some of them might be in use.
>
> thanks in advance for your comments.
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [proposal] replace build.pl with a central Makefile.
Posted by Jürgen Schmidt <jo...@gmail.com>.
On 10/18/13 11:32 AM, janI wrote:
> Hi.
>
> due to the discussion in thread "Mentor a new build system", I have made a
> proposal for a central Makefile located in main.
>
> It has been roughly tested it, thanks to a clever utility from andre.
>
> As discussed build.pl contains a lot of options, which need to be
> considered in a makefile.
>
> My suggestion is on
> http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile
>
> Please feel free to edit/comment on the page. I have reduced to options a
> lot, and some of them might be in use.
>
> thanks in advance for your comments.
first of all thanks for your interest to move things forward here. But
keep in mind that it is a very important area and we have to double
check everything before we drop an option ...
For example build --from is an often used option and as long as we have
no "real"dependencies that work properly we need probably something like
this. The html switches should have at least a replacement, we need
something that can be used to for the build bots for example. We have at
least to show an alternative approach to use log files or whatever.
Juergen
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org