You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2020/09/04 00:18:22 UTC

[GitHub] [incubator-nuttx-testing] btashton opened a new pull request #61: CI: Test enabling ccache for MacOS builds

btashton opened a new pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61


   ## Summary
   The MacOS builds are significantly slower than Linux.  Let's see if we can improve the performance with ccache.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] xiaoxiang781216 commented on pull request #61: CI: Enable ccache for builds

Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#issuecomment-687574797


   The change looks good, does it ready for merge? @btashton.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] xiaoxiang781216 merged pull request #61: CI: Enable ccache for builds

Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 merged pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on pull request #61: CI: Enable ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#issuecomment-687640677


   > The change looks good, does it ready for merge? @btashton.
   
   Just removed duplicate ccache links from the workflow file since they are now in the docker image and will be used once this is merged.  It is ready to merge.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on a change in pull request #61: CI: Test enabling ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#discussion_r483412824



##########
File path: cibuild.sh
##########
@@ -36,12 +37,12 @@ EXTRA_PATH=
 
 case $os in
   Darwin)
-    install="python-tools u-boot-tools discoteq-flock elf-toolchain gen-romfs kconfig-frontends bloaty arm-gcc-toolchain riscv-gcc-toolchain xtensa-esp32-gcc-toolchain avr-gcc-toolchain"

Review comment:
       We are not yet using the bloaty feature so no reason to bother paying the installation cost yet.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on a change in pull request #61: CI: Test enabling ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#discussion_r483413273



##########
File path: cibuild.sh
##########
@@ -324,9 +342,8 @@ function usage {
 
 function enable_ccache {
   export USE_CCACHE=1;
-  export CCACHE_DIR=$prebuilt/ccache/.ccache;
   ccache -c
-  ccache -M 5G;
+  ccache -M 200M;

Review comment:
       Reduce this to keep to a size that we can likely cache all the parallel builds under the github limits.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on pull request #61: CI: Enable ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#issuecomment-687411082


   > I suspect we might be able to improve the MacOS builds by using a ramdisk but it is hard to know what the bottleneck is besides just being slower machines.
   
   After testing this it made almost no difference running the build on a ramdisk or not, I suspect we are CPU bound here.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on pull request #61: CI: Enable ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#issuecomment-687291479


   @xiaoxiang781216 Any thoughts on this? Note the one change that needs to be backed our prior to merging.  I suspect we might be able to improve the MacOS builds by using a ramdisk but it is hard to know what the bottleneck is besides just being slower machines.
   
   Also we should consider consolidating some of the download logic used by external packages so we can cache that (and control how we spew progress bars in the build logs)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on a change in pull request #61: CI: Test enabling ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#discussion_r483412336



##########
File path: .github/workflows/build.yml
##########
@@ -63,16 +64,41 @@ jobs:
         timeout_minutes: 10
         max_attempts: 3
         retry_wait_seconds: 10
-        command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
+        command: docker pull $IMAGE_TAG
 
+    - name: Restore ccache
+      id: ccache
+      uses: actions/cache@v2
+      with:
+        path: ccache
+        key: ccache-${{ runner.os }}-${{matrix.boards}}-
+        restore-keys: ccache-${{ runner.os }}-${{matrix.boards}}-${{ github.run_id }}
     - name: Run builds
       uses: ./testing/.github/actions/ci-container
       env:
         BLOBDIR: /tools/blobs
       with:
         run: |
+          apt-get update -qq && apt-get install -y -qq --no-install-recommends ccache

Review comment:
       Note the installation of ccache and the symbolic links are only here because the run will not make use of the changes in the Docker image.  Once this is ready they will be dropped from here and only exist in the image.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on pull request #61: CI: Enable ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#issuecomment-687646183


   @v01d it should fail, the last push removed the temporary ccache install in the GitHub workflow. It will pass once it can use the new Docker container


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] v01d commented on pull request #61: CI: Enable ccache for builds

Posted by GitBox <gi...@apache.org>.
v01d commented on pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#issuecomment-687645229


   The CI appears to be failing partially. Besides that, the macOS build appears to be twice as fast!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on a change in pull request #61: CI: Test enabling ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#discussion_r483761503



##########
File path: .github/workflows/build.yml
##########
@@ -107,16 +138,29 @@ jobs:
         repository: apache/incubator-nuttx-testing
         path: testing
 
-    - name: Restore cache
+    - name: Restore tools cache
       id: cache-tools
-      uses: actions/cache@v1
+      uses: actions/cache@v2

Review comment:
       New version of cache makes use of zstd and is much faster




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on a change in pull request #61: CI: Test enabling ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#discussion_r483413438



##########
File path: cibuild.sh
##########
@@ -324,9 +342,8 @@ function usage {
 
 function enable_ccache {
   export USE_CCACHE=1;
-  export CCACHE_DIR=$prebuilt/ccache/.ccache;

Review comment:
       Needs to be able to be set by the environment.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-nuttx-testing] btashton commented on a change in pull request #61: CI: Test enabling ccache for builds

Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #61:
URL: https://github.com/apache/incubator-nuttx-testing/pull/61#discussion_r483413273



##########
File path: cibuild.sh
##########
@@ -324,9 +342,8 @@ function usage {
 
 function enable_ccache {
   export USE_CCACHE=1;
-  export CCACHE_DIR=$prebuilt/ccache/.ccache;
   ccache -c
-  ccache -M 5G;
+  ccache -M 200M;

Review comment:
       Reduce this to keep to a size that we can likely cache all the parallel builds under the github limits.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org