You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by GitBox <gi...@apache.org> on 2020/04/23 07:47:43 UTC
[GitHub] [incubator-nuttx] btashton opened a new pull request #850: CI: Add logic for determining which branches of repos should be used
btashton opened a new pull request #850:
URL: https://github.com/apache/incubator-nuttx/pull/850
The primary goal of this change is to control how we match the repos that should be used in the CI run. There is now an additional build job that creates a source bundle that will be used for the remainder of the build job. It will also try to cache this to improve performance and fall back on an artifact if needed. This is **NOT** the release tarball this is purely used to translate state between the jobs.
The basic logic is this:
1) Is the PR against a release branch? Yes, use the release branch from the corresponding app or os repo.
2) Is build triggered by a tag? Yes, use the same tag on the corresponding app or os repo
3) Always use master for the testing repo
4) Default to using master of the corresponding app or os repo if no other cases are found.
Later I would like to expand this to be able to target a PR branch in the app/os branch that it is connected with, but that will have to come later.
This also now runs builds on pushes of tags and release branches.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#issuecomment-618571329
LGTM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on a change in pull request #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on a change in pull request #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#discussion_r413665129
##########
File path: .github/workflows/build.yml
##########
@@ -14,109 +14,201 @@ name: Build
on:
pull_request:
+ push:
+ branches:
+ - master
+ - 'releases/*'
+ tags:
jobs:
+ Fetch-Source:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Determine Target Branches
+ id: gittargets
+ shell: bash
+ run: |
+ TESTING_REF="master" # Always use master for testing
+ OS_REF=""
+ APPS_REF=""
+
+ REF=$GITHUB_REF
+
+ # If a base ref is set this is a PR and we will want to use
+ # the base ref instead of the ref that triggered the event
+ if [ ${GITHUB_BASE_REF} ]; then
+ REF=refs/heads/$GITHUB_BASE_REF
+ fi
+
+ echo "Working with ref: $REF"
+
+ # We modify for all tags and release branches
+ if [[ $REF =~ refs/heads/releases/*|refs/tags/* ]]; then
+ if [[ $REF =~ refs/heads/* ]]
+ then
+ REF_NAME=${REF##refs/heads/}
+ echo "Working with a branch: $REF_NAME"
+ else
+ REF_NAME=${REF##refs/tags/}
+ echo "Working with a tag: $REF_NAME"
+ fi
+
+ # Determine the repo and leave that unset to use the normal checkout behavior
+ # of using the merge commit instead of HEAD
+ case $GITHUB_REPOSITORY in
+ "apache/incubator-nuttx")
+ # OS
+ echo "Triggered by change in OS"
+ APPS_REF=$REF_NAME
+ ;;
+
+ "apache/incubator-nuttx-apps" )
+ # APPS
+ OS_REF=$REF_NAME
+ echo "Triggered by change in APPS"
+ ;;
+
+ *)
+ echo "Trigger by change on $GITHUB_REPOSITORY. This is unexpected."
+ ;;
+ esac
+ fi
+
+ echo ::set-output name=os_ref::$OS_REF
+ echo ::set-output name=apps_ref::$APPS_REF
+ echo ::set-output name=testing_ref::$TESTING_REF
+
+ - name: Checkout nuttx repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx
+ ref: ${{ steps.gittargets.outputs.os_ref }}
+ path: sources/nuttx
+ fetch-depth: 1
+
+ - name: Checkout apps repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-apps
+ ref: ${{ steps.gittargets.outputs.apps_ref }}
+ path: sources/apps
+ fetch-depth: 1
+
+ - name: Checkout testing repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-testing
+ ref: ${{ steps.gittargets.outputs.testing_ref }}
+ path: sources/testing
+ fetch-depth: 1
+
+ - name: Create Source Bundle
+ run: tar -czf sources.tar.gz sources
+ - name: Archive Source Bundle
+ uses: actions/upload-artifact@v1
+ with:
+ name: source-bundle
+ path: sources.tar.gz
+
+ - name: Cache Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources
+ key: build-sources-${{ github.run_id }}
+
Linux:
- runs-on: ubuntu-18.04
+ needs: Fetch-Source
+ runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: 1
strategy:
matrix:
- boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
+ boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Docker Login
- uses: azure/docker-login@v1
- with:
- login-server: docker.pkg.github.com
- username: ${GITHUB_ACTOR}
- password: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Docker Pull
- uses: nick-invision/retry@v1
- with:
- timeout_minutes: 10
- max_attempts: 3
- retry_wait_seconds: 10
- command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
-
- - name: Run builds
- uses: ./testing/.github/actions/ci-container
- env:
- BLOBDIR: /tools/blobs
- with:
- run: |
- cd testing
- ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
+ steps:
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Docker Login
+ uses: azure/docker-login@v1
+ with:
+ login-server: docker.pkg.github.com
+ username: ${GITHUB_ACTOR}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Docker Pull
+ uses: nick-invision/retry@v1
+ with:
+ timeout_minutes: 10
+ max_attempts: 3
+ retry_wait_seconds: 10
+ command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
+
+ - name: Run builds
+ uses: ./sources/testing/.github/actions/ci-container
+ env:
+ BLOBDIR: /tools/blobs
+ with:
+ run: |
+ git -C sources/nuttx fetch --tags
+ cd sources/testing
+ ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
macOS:
runs-on: macos-10.15
-
+ needs: Fetch-Source
strategy:
matrix:
boards: [arm-12, mips-riscv-x86-xtensa, sim]
-
steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
-
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Restore cache
- id: cache-tools
- uses: actions/cache@v1
- env:
- cache-name: ${{ runner.os }}-cache-tools
- with:
- path: prebuilt
- key: ${{ runner.os }}-tools-${{ hashFiles('./testing/cibuild.sh') }}
-
- - name: Run builds
- run: |
- cd testing
- ./cibuild.sh -i -x -G testlist/${{matrix.boards}}.dat
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Restore Tools Cache
+ id: cache-tools
+ uses: actions/cache@v1
+ env:
+ cache-name: ${{ runner.os }}-cache-tools
+ with:
+ path: prebuilt
+ key: ${{ runner.os }}-tools-${{ hashFiles('./sources/testing/cibuild.sh') }}
+
+ - name: Run Builds
+ run: |
+ git -C sources/nuttx fetch --tags
+ cd sources/testing
+ ./cibuild.sh -i -x -G testlist/${{matrix.boards}}.dat
Review comment:
add newline
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 edited a comment on issue #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 edited a comment on issue #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#issuecomment-618300245
> The basic logic is this:
>
> 1. Is the PR against a release branch? Yes, use the release branch from the corresponding app or os repo.
> 2. Is build triggered by a tag? Yes, use the same tag on the corresponding app or os repo
I am curious when we will use this case?
> 3. Always use master for the testing repo
> 4. Default to using master of the corresponding app or os repo if no other cases are found.
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] btashton commented on a change in pull request #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#discussion_r413998814
##########
File path: .github/workflows/build.yml
##########
@@ -14,109 +14,201 @@ name: Build
on:
pull_request:
+ push:
+ branches:
+ - master
+ - 'releases/*'
+ tags:
jobs:
+ Fetch-Source:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Determine Target Branches
+ id: gittargets
+ shell: bash
+ run: |
+ TESTING_REF="master" # Always use master for testing
+ OS_REF=""
+ APPS_REF=""
+
+ REF=$GITHUB_REF
+
+ # If a base ref is set this is a PR and we will want to use
+ # the base ref instead of the ref that triggered the event
+ if [ ${GITHUB_BASE_REF} ]; then
+ REF=refs/heads/$GITHUB_BASE_REF
+ fi
+
+ echo "Working with ref: $REF"
+
+ # We modify for all tags and release branches
+ if [[ $REF =~ refs/heads/releases/*|refs/tags/* ]]; then
+ if [[ $REF =~ refs/heads/* ]]
+ then
+ REF_NAME=${REF##refs/heads/}
+ echo "Working with a branch: $REF_NAME"
+ else
+ REF_NAME=${REF##refs/tags/}
+ echo "Working with a tag: $REF_NAME"
+ fi
+
+ # Determine the repo and leave that unset to use the normal checkout behavior
+ # of using the merge commit instead of HEAD
+ case $GITHUB_REPOSITORY in
+ "apache/incubator-nuttx")
+ # OS
+ echo "Triggered by change in OS"
+ APPS_REF=$REF_NAME
+ ;;
+
+ "apache/incubator-nuttx-apps" )
+ # APPS
+ OS_REF=$REF_NAME
+ echo "Triggered by change in APPS"
+ ;;
+
+ *)
+ echo "Trigger by change on $GITHUB_REPOSITORY. This is unexpected."
+ ;;
+ esac
+ fi
+
+ echo ::set-output name=os_ref::$OS_REF
+ echo ::set-output name=apps_ref::$APPS_REF
+ echo ::set-output name=testing_ref::$TESTING_REF
+
+ - name: Checkout nuttx repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx
+ ref: ${{ steps.gittargets.outputs.os_ref }}
+ path: sources/nuttx
+ fetch-depth: 1
+
+ - name: Checkout apps repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-apps
+ ref: ${{ steps.gittargets.outputs.apps_ref }}
+ path: sources/apps
+ fetch-depth: 1
+
+ - name: Checkout testing repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-testing
+ ref: ${{ steps.gittargets.outputs.testing_ref }}
+ path: sources/testing
+ fetch-depth: 1
+
+ - name: Create Source Bundle
+ run: tar -czf sources.tar.gz sources
+ - name: Archive Source Bundle
+ uses: actions/upload-artifact@v1
+ with:
+ name: source-bundle
+ path: sources.tar.gz
+
+ - name: Cache Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources
+ key: build-sources-${{ github.run_id }}
+
Linux:
- runs-on: ubuntu-18.04
+ needs: Fetch-Source
+ runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: 1
strategy:
matrix:
- boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
+ boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Docker Login
- uses: azure/docker-login@v1
- with:
- login-server: docker.pkg.github.com
- username: ${GITHUB_ACTOR}
- password: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Docker Pull
- uses: nick-invision/retry@v1
- with:
- timeout_minutes: 10
- max_attempts: 3
- retry_wait_seconds: 10
- command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
-
- - name: Run builds
- uses: ./testing/.github/actions/ci-container
- env:
- BLOBDIR: /tools/blobs
- with:
- run: |
- cd testing
- ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
+ steps:
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Docker Login
+ uses: azure/docker-login@v1
+ with:
+ login-server: docker.pkg.github.com
+ username: ${GITHUB_ACTOR}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Docker Pull
+ uses: nick-invision/retry@v1
+ with:
+ timeout_minutes: 10
+ max_attempts: 3
+ retry_wait_seconds: 10
+ command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
+
+ - name: Run builds
+ uses: ./sources/testing/.github/actions/ci-container
+ env:
+ BLOBDIR: /tools/blobs
+ with:
+ run: |
+ git -C sources/nuttx fetch --tags
Review comment:
I intentionally moved this out of the source fetching because it over tripled the size of the cache/artifact because it pulls down even more from git which slows things down quite a bit. It is only used to generate the `.version` file and I think we need to refine this more later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] Ouss4 commented on issue #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
Ouss4 commented on issue #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#issuecomment-618426615
Thanks, Brennan.
Is there going to be an equivalent change for apps/?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] btashton commented on a change in pull request #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
btashton commented on a change in pull request #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#discussion_r413999459
##########
File path: .github/workflows/build.yml
##########
@@ -14,109 +14,201 @@ name: Build
on:
pull_request:
+ push:
+ branches:
+ - master
+ - 'releases/*'
+ tags:
jobs:
+ Fetch-Source:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Determine Target Branches
+ id: gittargets
+ shell: bash
+ run: |
+ TESTING_REF="master" # Always use master for testing
+ OS_REF=""
+ APPS_REF=""
+
+ REF=$GITHUB_REF
+
+ # If a base ref is set this is a PR and we will want to use
+ # the base ref instead of the ref that triggered the event
+ if [ ${GITHUB_BASE_REF} ]; then
+ REF=refs/heads/$GITHUB_BASE_REF
+ fi
+
+ echo "Working with ref: $REF"
+
+ # We modify for all tags and release branches
+ if [[ $REF =~ refs/heads/releases/*|refs/tags/* ]]; then
+ if [[ $REF =~ refs/heads/* ]]
+ then
+ REF_NAME=${REF##refs/heads/}
+ echo "Working with a branch: $REF_NAME"
+ else
+ REF_NAME=${REF##refs/tags/}
+ echo "Working with a tag: $REF_NAME"
+ fi
+
+ # Determine the repo and leave that unset to use the normal checkout behavior
+ # of using the merge commit instead of HEAD
+ case $GITHUB_REPOSITORY in
+ "apache/incubator-nuttx")
+ # OS
+ echo "Triggered by change in OS"
+ APPS_REF=$REF_NAME
+ ;;
+
+ "apache/incubator-nuttx-apps" )
+ # APPS
+ OS_REF=$REF_NAME
+ echo "Triggered by change in APPS"
+ ;;
+
+ *)
+ echo "Trigger by change on $GITHUB_REPOSITORY. This is unexpected."
+ ;;
+ esac
+ fi
+
+ echo ::set-output name=os_ref::$OS_REF
+ echo ::set-output name=apps_ref::$APPS_REF
+ echo ::set-output name=testing_ref::$TESTING_REF
+
+ - name: Checkout nuttx repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx
+ ref: ${{ steps.gittargets.outputs.os_ref }}
+ path: sources/nuttx
+ fetch-depth: 1
+
+ - name: Checkout apps repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-apps
+ ref: ${{ steps.gittargets.outputs.apps_ref }}
+ path: sources/apps
+ fetch-depth: 1
+
+ - name: Checkout testing repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-testing
+ ref: ${{ steps.gittargets.outputs.testing_ref }}
+ path: sources/testing
+ fetch-depth: 1
+
+ - name: Create Source Bundle
+ run: tar -czf sources.tar.gz sources
+ - name: Archive Source Bundle
+ uses: actions/upload-artifact@v1
+ with:
+ name: source-bundle
+ path: sources.tar.gz
+
+ - name: Cache Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources
+ key: build-sources-${{ github.run_id }}
+
Linux:
- runs-on: ubuntu-18.04
+ needs: Fetch-Source
+ runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: 1
strategy:
matrix:
- boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
+ boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Docker Login
- uses: azure/docker-login@v1
- with:
- login-server: docker.pkg.github.com
- username: ${GITHUB_ACTOR}
- password: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Docker Pull
- uses: nick-invision/retry@v1
- with:
- timeout_minutes: 10
- max_attempts: 3
- retry_wait_seconds: 10
- command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
-
- - name: Run builds
- uses: ./testing/.github/actions/ci-container
- env:
- BLOBDIR: /tools/blobs
- with:
- run: |
- cd testing
- ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
+ steps:
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Docker Login
+ uses: azure/docker-login@v1
+ with:
+ login-server: docker.pkg.github.com
+ username: ${GITHUB_ACTOR}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Docker Pull
+ uses: nick-invision/retry@v1
+ with:
+ timeout_minutes: 10
+ max_attempts: 3
+ retry_wait_seconds: 10
+ command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
+
+ - name: Run builds
+ uses: ./sources/testing/.github/actions/ci-container
+ env:
+ BLOBDIR: /tools/blobs
+ with:
+ run: |
+ git -C sources/nuttx fetch --tags
+ cd sources/testing
+ ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
macOS:
runs-on: macos-10.15
-
+ needs: Fetch-Source
strategy:
matrix:
boards: [arm-12, mips-riscv-x86-xtensa, sim]
-
steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
-
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Restore cache
- id: cache-tools
- uses: actions/cache@v1
- env:
- cache-name: ${{ runner.os }}-cache-tools
- with:
- path: prebuilt
- key: ${{ runner.os }}-tools-${{ hashFiles('./testing/cibuild.sh') }}
-
- - name: Run builds
- run: |
- cd testing
- ./cibuild.sh -i -x -G testlist/${{matrix.boards}}.dat
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Restore Tools Cache
+ id: cache-tools
+ uses: actions/cache@v1
+ env:
+ cache-name: ${{ runner.os }}-cache-tools
+ with:
+ path: prebuilt
+ key: ${{ runner.os }}-tools-${{ hashFiles('./sources/testing/cibuild.sh') }}
+
+ - name: Run Builds
+ run: |
+ git -C sources/nuttx fetch --tags
Review comment:
> Thanks, Brennan.
> Is there going to be an equivalent change for apps/?
This should just work on apps and testing if we sync it across.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] liuguo09 commented on issue #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
liuguo09 commented on issue #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#issuecomment-618283298
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] btashton commented on issue #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
btashton commented on issue #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#issuecomment-618546891
> > 1. Is the PR against a release branch? Yes, use the release branch from the corresponding app or os repo.
> > 2. Is build triggered by a tag? Yes, use the same tag on the corresponding app or os repo
>
> I am curious when we will use this case?
Sure, we have `releases/9.0` which will have tags under it for multiple patch versions. We want to make sure for a tag in the os on this branch we use the same tag in the apps branch which is what we will ship, not whatever the latest is in the apps branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on a change in pull request #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on a change in pull request #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#discussion_r413700082
##########
File path: .github/workflows/build.yml
##########
@@ -14,109 +14,201 @@ name: Build
on:
pull_request:
+ push:
+ branches:
+ - master
+ - 'releases/*'
+ tags:
jobs:
+ Fetch-Source:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Determine Target Branches
+ id: gittargets
+ shell: bash
+ run: |
+ TESTING_REF="master" # Always use master for testing
+ OS_REF=""
+ APPS_REF=""
+
+ REF=$GITHUB_REF
+
+ # If a base ref is set this is a PR and we will want to use
+ # the base ref instead of the ref that triggered the event
+ if [ ${GITHUB_BASE_REF} ]; then
+ REF=refs/heads/$GITHUB_BASE_REF
+ fi
+
+ echo "Working with ref: $REF"
+
+ # We modify for all tags and release branches
+ if [[ $REF =~ refs/heads/releases/*|refs/tags/* ]]; then
+ if [[ $REF =~ refs/heads/* ]]
+ then
+ REF_NAME=${REF##refs/heads/}
+ echo "Working with a branch: $REF_NAME"
+ else
+ REF_NAME=${REF##refs/tags/}
+ echo "Working with a tag: $REF_NAME"
+ fi
+
+ # Determine the repo and leave that unset to use the normal checkout behavior
+ # of using the merge commit instead of HEAD
+ case $GITHUB_REPOSITORY in
+ "apache/incubator-nuttx")
+ # OS
+ echo "Triggered by change in OS"
+ APPS_REF=$REF_NAME
+ ;;
+
+ "apache/incubator-nuttx-apps" )
+ # APPS
+ OS_REF=$REF_NAME
+ echo "Triggered by change in APPS"
+ ;;
+
+ *)
+ echo "Trigger by change on $GITHUB_REPOSITORY. This is unexpected."
+ ;;
+ esac
+ fi
+
+ echo ::set-output name=os_ref::$OS_REF
+ echo ::set-output name=apps_ref::$APPS_REF
+ echo ::set-output name=testing_ref::$TESTING_REF
+
+ - name: Checkout nuttx repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx
+ ref: ${{ steps.gittargets.outputs.os_ref }}
+ path: sources/nuttx
+ fetch-depth: 1
+
+ - name: Checkout apps repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-apps
+ ref: ${{ steps.gittargets.outputs.apps_ref }}
+ path: sources/apps
+ fetch-depth: 1
+
+ - name: Checkout testing repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-testing
+ ref: ${{ steps.gittargets.outputs.testing_ref }}
+ path: sources/testing
+ fetch-depth: 1
+
+ - name: Create Source Bundle
+ run: tar -czf sources.tar.gz sources
+ - name: Archive Source Bundle
+ uses: actions/upload-artifact@v1
+ with:
+ name: source-bundle
+ path: sources.tar.gz
+
+ - name: Cache Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources
+ key: build-sources-${{ github.run_id }}
+
Linux:
- runs-on: ubuntu-18.04
+ needs: Fetch-Source
+ runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: 1
strategy:
matrix:
- boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
+ boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Docker Login
- uses: azure/docker-login@v1
- with:
- login-server: docker.pkg.github.com
- username: ${GITHUB_ACTOR}
- password: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Docker Pull
- uses: nick-invision/retry@v1
- with:
- timeout_minutes: 10
- max_attempts: 3
- retry_wait_seconds: 10
- command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
-
- - name: Run builds
- uses: ./testing/.github/actions/ci-container
- env:
- BLOBDIR: /tools/blobs
- with:
- run: |
- cd testing
- ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
+ steps:
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Docker Login
+ uses: azure/docker-login@v1
+ with:
+ login-server: docker.pkg.github.com
+ username: ${GITHUB_ACTOR}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Docker Pull
+ uses: nick-invision/retry@v1
+ with:
+ timeout_minutes: 10
+ max_attempts: 3
+ retry_wait_seconds: 10
+ command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
+
+ - name: Run builds
+ uses: ./sources/testing/.github/actions/ci-container
+ env:
+ BLOBDIR: /tools/blobs
+ with:
+ run: |
+ git -C sources/nuttx fetch --tags
+ cd sources/testing
+ ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
macOS:
runs-on: macos-10.15
-
+ needs: Fetch-Source
strategy:
matrix:
boards: [arm-12, mips-riscv-x86-xtensa, sim]
-
steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
-
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Restore cache
- id: cache-tools
- uses: actions/cache@v1
- env:
- cache-name: ${{ runner.os }}-cache-tools
- with:
- path: prebuilt
- key: ${{ runner.os }}-tools-${{ hashFiles('./testing/cibuild.sh') }}
-
- - name: Run builds
- run: |
- cd testing
- ./cibuild.sh -i -x -G testlist/${{matrix.boards}}.dat
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Restore Tools Cache
+ id: cache-tools
+ uses: actions/cache@v1
+ env:
+ cache-name: ${{ runner.os }}-cache-tools
+ with:
+ path: prebuilt
+ key: ${{ runner.os }}-tools-${{ hashFiles('./sources/testing/cibuild.sh') }}
+
+ - name: Run Builds
+ run: |
+ git -C sources/nuttx fetch --tags
Review comment:
Should we move this step to "Create Source Bundle" line 105
##########
File path: .github/workflows/build.yml
##########
@@ -14,109 +14,201 @@ name: Build
on:
pull_request:
+ push:
+ branches:
+ - master
+ - 'releases/*'
+ tags:
jobs:
+ Fetch-Source:
+ runs-on: ubuntu-latest
+ steps:
+ - name: Determine Target Branches
+ id: gittargets
+ shell: bash
+ run: |
+ TESTING_REF="master" # Always use master for testing
+ OS_REF=""
+ APPS_REF=""
+
+ REF=$GITHUB_REF
+
+ # If a base ref is set this is a PR and we will want to use
+ # the base ref instead of the ref that triggered the event
+ if [ ${GITHUB_BASE_REF} ]; then
+ REF=refs/heads/$GITHUB_BASE_REF
+ fi
+
+ echo "Working with ref: $REF"
+
+ # We modify for all tags and release branches
+ if [[ $REF =~ refs/heads/releases/*|refs/tags/* ]]; then
+ if [[ $REF =~ refs/heads/* ]]
+ then
+ REF_NAME=${REF##refs/heads/}
+ echo "Working with a branch: $REF_NAME"
+ else
+ REF_NAME=${REF##refs/tags/}
+ echo "Working with a tag: $REF_NAME"
+ fi
+
+ # Determine the repo and leave that unset to use the normal checkout behavior
+ # of using the merge commit instead of HEAD
+ case $GITHUB_REPOSITORY in
+ "apache/incubator-nuttx")
+ # OS
+ echo "Triggered by change in OS"
+ APPS_REF=$REF_NAME
+ ;;
+
+ "apache/incubator-nuttx-apps" )
+ # APPS
+ OS_REF=$REF_NAME
+ echo "Triggered by change in APPS"
+ ;;
+
+ *)
+ echo "Trigger by change on $GITHUB_REPOSITORY. This is unexpected."
+ ;;
+ esac
+ fi
+
+ echo ::set-output name=os_ref::$OS_REF
+ echo ::set-output name=apps_ref::$APPS_REF
+ echo ::set-output name=testing_ref::$TESTING_REF
+
+ - name: Checkout nuttx repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx
+ ref: ${{ steps.gittargets.outputs.os_ref }}
+ path: sources/nuttx
+ fetch-depth: 1
+
+ - name: Checkout apps repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-apps
+ ref: ${{ steps.gittargets.outputs.apps_ref }}
+ path: sources/apps
+ fetch-depth: 1
+
+ - name: Checkout testing repo
+ uses: actions/checkout@v2
+ with:
+ repository: apache/incubator-nuttx-testing
+ ref: ${{ steps.gittargets.outputs.testing_ref }}
+ path: sources/testing
+ fetch-depth: 1
+
+ - name: Create Source Bundle
+ run: tar -czf sources.tar.gz sources
+ - name: Archive Source Bundle
+ uses: actions/upload-artifact@v1
+ with:
+ name: source-bundle
+ path: sources.tar.gz
+
+ - name: Cache Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources
+ key: build-sources-${{ github.run_id }}
+
Linux:
- runs-on: ubuntu-18.04
+ needs: Fetch-Source
+ runs-on: ubuntu-latest
env:
DOCKER_BUILDKIT: 1
strategy:
matrix:
- boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- steps:
- - name: Checkout nuttx repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx
- path: nuttx
- fetch-depth: 0
-
- - name: Fetch nuttx tags
- run: |
- cd nuttx
- git fetch --tags
+ boards: [arm-01, arm-02, arm-03, arm-04, arm-05, arm-06, arm-07, arm-08, arm-09, arm-10, arm-11, arm-12, arm-13, mips-riscv-x86-xtensa, sim]
- - name: Checkout apps repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-apps
- path: apps
- fetch-depth: 0
-
- - name: Checkout testing repo
- uses: actions/checkout@v2
- with:
- repository: apache/incubator-nuttx-testing
- path: testing
-
- - name: Docker Login
- uses: azure/docker-login@v1
- with:
- login-server: docker.pkg.github.com
- username: ${GITHUB_ACTOR}
- password: ${{ secrets.GITHUB_TOKEN }}
-
- - name: Docker Pull
- uses: nick-invision/retry@v1
- with:
- timeout_minutes: 10
- max_attempts: 3
- retry_wait_seconds: 10
- command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
-
- - name: Run builds
- uses: ./testing/.github/actions/ci-container
- env:
- BLOBDIR: /tools/blobs
- with:
- run: |
- cd testing
- ./cibuild.sh -x -G testlist/${{matrix.boards}}.dat
+ steps:
+ - name: Fetch Cached Source
+ id: cache-source
+ uses: actions/cache@v1
+ with:
+ path: sources-cache
+ key: build-sources-${{ github.run_id }}
+ - name: Prevent Updating Source Cache
+ if: steps.cache-source.outputs.cache-hit == 'true'
+ run: mv sources-cache sources
+ - name: Download Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ uses: actions/download-artifact@v1
+ with:
+ name: source-bundle
+ path: ./
+ - name: Extract Source Artifact
+ if: steps.cache-source.outputs.cache-hit != 'true'
+ run: tar -xf sources.tar.gz
+
+ - name: Docker Login
+ uses: azure/docker-login@v1
+ with:
+ login-server: docker.pkg.github.com
+ username: ${GITHUB_ACTOR}
+ password: ${{ secrets.GITHUB_TOKEN }}
+
+ - name: Docker Pull
+ uses: nick-invision/retry@v1
+ with:
+ timeout_minutes: 10
+ max_attempts: 3
+ retry_wait_seconds: 10
+ command: docker pull docker.pkg.github.com/apache/incubator-nuttx-testing/nuttx-ci-linux
+
+ - name: Run builds
+ uses: ./sources/testing/.github/actions/ci-container
+ env:
+ BLOBDIR: /tools/blobs
+ with:
+ run: |
+ git -C sources/nuttx fetch --tags
Review comment:
Should we move this step to "Create Source Bundle" line 105
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nuttx] xiaoxiang781216 commented on issue #850: CI: Add logic for determining which branches of repos should be used
Posted by GitBox <gi...@apache.org>.
xiaoxiang781216 commented on issue #850:
URL: https://github.com/apache/incubator-nuttx/pull/850#issuecomment-618300245
> The primary goal of this change is to control how we match the repos that should be used in the CI run. There is now an additional build job that creates a source bundle that will be used for the remainder of the build job. It will also try to cache this to improve performance and fall back on an artifact if needed. This is **NOT** the release tarball this is purely used to translate state between the jobs.
>
> The basic logic is this:
>
> 1. Is the PR against a release branch? Yes, use the release branch from the corresponding app or os repo.
> 2. Is build triggered by a tag? Yes, use the same tag on the corresponding app or os repo
I am curious when we will use this case?
> 3. Always use master for the testing repo
> 4. Default to using master of the corresponding app or os repo if no other cases are found.
>
> Later I would like to expand this to be able to target a PR branch in the app/os branch that it is connected with, but that will have to come later.
>
> This also now runs builds on pushes of tags and release branches.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org