You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2021/04/06 14:44:18 UTC
[GitHub] [incubator-nuttx] v01d opened a new issue #3455: Use CMake
v01d opened a new issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455
I'm opening this issue to discuss whether there would be interest in eventually migrating the build system to CMake. The motivation is in part a recent message by @dagar where he asked about this. But furthermore, the build system is sometimes quite a pain to maintain, and is extremely complex to fully understand.
The benefits of migrating towards CMake could be the following:
- Improved cross platform developer experience (Windows native build likely feasible)
- Significantly improved build times, both incremental and clean (supports not only Makefile generation, but ninja which is highly efficient for large parallel builds)
- Solves dependency handling on its own
- Better integration with 3rd party projects (consuming NuttX) and libs (eg LLVM libc++ cmake). Can also interact with pure Make based projects easily.
- Proper IDE support (VSCode, CLion, etc), nice things like Intellisense will "just work"
- Potential for tighter integration between configuration and build
Furthermore, it can help alleviate issues that we face:
- As CMake is not just a language but includes higher-level functionality (functions handling build targets, etc.) which structures the build in known ways, it can be more easily understood and maintained. Makefiles use hand-written rules which require explicit documentation (which does not really exist right now) for "the right way" to do things (how to structure a directory, which variables to define, in what order to add includes, etc) and is thus easier to break.
- Our dependency handling is brittle: it has been broken and fixed many times and requires much manual work (it still is not right, particularly for external projects)
- Multiplatform support requires separate code paths and approaches which need to be kept in sync
- Our build requires slow traversal of all sub-directories for various operations
- Object files dispersed over source directories. CMake builds in separate directory and opens the potential to having multiple builds of the same code base simultaneously.
- We have to deal with collision between object file names in different directories (currently handled by embedding path to file as part of object file name)
From the technical standpoint, migration would be indeed a lot of work but not necessarily hard. @dagar has already done prior work on this (on older version of NuttX). See here:
- https://github.com/PX4/NuttX/commits/pr-cmake
- https://github.com/PX4/NuttX-apps/tree/pr-cmake
In general, the approach is more or less like this:
1. Define a set of CMake utility functions, to define libraries and applications
2. Replace every Make.defs with a CMakeLists.txt, which defines the source files to be added from that directory
3. Define custom targets for things like menuconfig, etc.
4. Define custom commands for things like creating symlinks, etc.
5. Provide a top-level Makefile that simplifies the "mkdir build; cd build; cmake ..; make" procedure
The idea would be to do this work in a separate branch and initially reach a minimal POC (support one board for example, and most of the complexity such as different platforms, kernel build, etc.). This can then be iterated until the user experience is right and then finally extended to all boards/arch.
@dagar has offered to help and I would also like to do so. At the moment, we would like to assess whether the community at least feels this is worthy of an attempt so we can slowly start heading this direction (we could aim for a major release of NuttX, so this is not rushed either).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] davids5 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
davids5 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-817305510
+1
It is a good move. Build speed will increase and we should be able to get
CI times down to allow a review-merge cycle that does not span hours or
days. We may also benefit from reducing the circular dependencies to a
minimum.
*From:* Anthony Merlino ***@***.***
*Sent:* Saturday, April 10, 2021 7:06 PM
*To:* apache/incubator-nuttx
*Cc:* David Sidrane; Mention
*Subject:* Re: [apache/incubator-nuttx] Use CMake (#3455)
I'm onboard with this. All I've wanted for years now was to be able to
consume NuttX and NuttX-apps in a larger project directly from cmake. I
mean, I do it now - but it's not clean by any means - mostly because of the
in-source building. Out of tree building is probably the #1
<https://github.com/apache/incubator-nuttx/pull/1> benefit for doing this,
IMO. But I think the benefits list is actually pretty long.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-817234698>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAO3BXIK4INH3PCCOCBWGRLTID7X7ANCNFSM42O4HLAA>
.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-816389564
Ok, I will send an email in this weekend.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] v01d commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
v01d commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-816363312
@xiaoxiang781216 do you want to send an email linking to this issue?
I see some positive feedback already but it would be great to know others got the chance to read this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] dagar edited a comment on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
dagar edited a comment on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814246623
> From CI standpoint here are 73 build of NuttX (1.5M SLOC )including PX4 (6M SLOC) in 12m 13s done as work flows
> https://github.com/PX4/PX4-Autopilot/actions/runs/722727878
Current PX4/NuttX is a bit more complex because the underlying NuttX build is still with Make, but PX4 parallelizes it per library (from cmake).
The build history from my previous attempt (https://github.com/PX4/NuttX/pull/46) has been lost, but David captured some of the details in his presentation. https://nuttx.events/wp-content/uploads/2019/11/DSidrane_nx2019.pdf
Those build machines were not very fast, but were still configuring + compiling NuttX configurations in 12-20 seconds. I vaguely recall the existing Makefile build system building the same configurations on that hardware was closer to 1-2 minutes per build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] dagar edited a comment on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
dagar edited a comment on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814246623
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814292900
@dagar I saw you already done a huge work((PX4/NuttX#46), I am postive to integrate your work into mainline to simplify the daily work in PX4 and NuttX community. @v01d should we come up a detailed propose and send to dev@ for discussion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814208024
@davids5 do you have some performance number from PX4 CMake compared with NuttX Makefile?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] v01d commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
v01d commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814230895
Those are impressive times!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] v01d commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
v01d commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-819677105
Given the positive feedback so far I decided to start building the POC in the hopes that this helps move the decision forward.
I'm basing my work on @dagar 's `pr-cmake_master` branch, which still is somewhat old w.r.t. current master. I already managed to succesfully build for a specific board (stm32f103-minimum) but lots of things still need to be done. It works quite well so far though.
I'll try to continue a bit more and maybe open a draft PR soon, mostly for visbility, as it will take quite some time to ensure this is mergeable (and even so, I would pospone merging until other big changes are done). I mostly added new files and not touched existing makefiles, which will allow for easy rebasing as time goes on. I also would like some input in some points I haven't yet addressed correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-820020555
Nice! I expect that many known issues will be resolved by migrating to CMake:
1. Support more host
2. Out of tree build
3. Fast build speed
4. Remove the duplicated setting in board's Make.defs
5. Various hack in Makefile(ar lock, long command line, file name mangle...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814288082
The number is great. The out of tree is also very useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] davids5 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
davids5 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814213935
@xiaoxiang781216 I have the slides from the 2019 Trip to meet with you. Shall I post them here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] davids5 commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
davids5 commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814221868
From CI standpoint here are 73 build of NuttX (1.5M SLOC )including PX4 (6M SLOC) in 12m 13s done as work flows
https://github.com/PX4/PX4-Autopilot/actions/runs/722727878
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] dagar commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
dagar commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814246623
> From CI standpoint here are 73 build of NuttX (1.5M SLOC )including PX4 (6M SLOC) in 12m 13s done as work flows
> https://github.com/PX4/PX4-Autopilot/actions/runs/722727878
Current PX4/NuttX is a bit more complex because the underlying NuttX build is still with Make, but PX4 parallelisms it per library (from cmake).
The build history from my previous attempt (https://github.com/PX4/NuttX/pull/46) has been lost, but David captured some of the details in his presentation. https://nuttx.events/wp-content/uploads/2019/11/DSidrane_nx2019.pdf
Those build machines were not very fast, but were still configuring + compiling NuttX configurations in 12-20 seconds. I vaguely recall the existing Makefile build system building the same configurations on that hardware was closer to 1-2 minutes per build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] v01d commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
v01d commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814297182
If you want you can send email to mailing list linking here, so that we do not split the discussion.
As for the detailed proposal, I briefly discussed it with @dagar and the result is in the description of this issue above. I wouldn't know what else to add. I think the preliminary work by @dagar already gives a good starting point (I would create a branch and start porting the work and adapting/improving as necessary).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] dagar edited a comment on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
dagar edited a comment on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814246623
> From CI standpoint here are 73 build of NuttX (1.5M SLOC )including PX4 (6M SLOC) in 12m 13s done as work flows
> https://github.com/PX4/PX4-Autopilot/actions/runs/722727878
Current PX4/NuttX is a bit more complex because the underlying NuttX build is still with Make, but PX4 parallelizes it per NuttX library (via cmake).
The build history from my previous attempt (https://github.com/PX4/NuttX/pull/46) has been lost, but it was drastically faster per build (multiple times faster).
EDIT: I will verify the numbers for current comparison, but recursive Makefile usage seems to be a huge contributor to the difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] antmerlino commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
antmerlino commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-817234698
I'm onboard with this. All I've wanted for years now was to be able to consume NuttX and NuttX-apps in a larger project directly from cmake. I mean, I do it now - but it's not clean by any means - mostly because of the in-source building. Out of tree building is probably the #1 benefit for doing this, IMO. But I think the benefits list is actually pretty long.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] niurenyige commented on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
niurenyige commented on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-861094439
really looking forward to that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] dagar edited a comment on issue #3455: Use CMake
Posted by GitBox <gi...@apache.org>.
dagar edited a comment on issue #3455:
URL: https://github.com/apache/incubator-nuttx/issues/3455#issuecomment-814246623
> From CI standpoint here are 73 build of NuttX (1.5M SLOC )including PX4 (6M SLOC) in 12m 13s done as work flows
> https://github.com/PX4/PX4-Autopilot/actions/runs/722727878
Current PX4/NuttX is a bit more complex because the underlying NuttX build is still with Make, but PX4 parallelized it per library (from cmake).
The build history from my previous attempt (https://github.com/PX4/NuttX/pull/46) has been lost, but it was drastically faster per build.
EDIT: I will verify the numbers for current comparison, but recursive Makefile usage seems to be a huge contributor to the difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org