You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by "stbenn (via GitHub)" <gi...@apache.org> on 2024/11/01 22:04:01 UTC
[I] [HELP] Running github actions locally with act [nuttx]
stbenn opened a new issue, #14601:
URL: https://github.com/apache/nuttx/issues/14601
### Description
I am looking for ways to better verify my work before doing a pull request, to prevent unsuccessful use of the runners. I came across act, which has an extension for the github cli that claims to run workflows locally.
Has anyone had tried or had success with this before? I have it recognizing the workflows but am having failures from access tokens, password requirements, etc.
Trying to run it on Ubuntu-24.04 in WSL2
### Verification
- [X] I have verified before submitting the report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "lupyuen (via GitHub)" <gi...@apache.org>.
lupyuen commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2453620323
Yep that's because GCC Compiler helpfully colours the messages with ANSI Escape Codes :-) Try this to remove the Special Characters:
```bash
## Strip the control chars
function clean_log {
local tmp_file=/tmp/release-tmp.log
cat $log_file \
| tr -d '\r' \
| tr -d '\r' \
| sed 's/\x08/ /g' \
| sed 's/\x1B(B//g' \
| sed 's/\x1B\[K//g' \
| sed 's/\x1B[<=>]//g' \
| sed 's/\x1B\[[0-9:;<=>?]*[!]*[A-Za-z]//g' \
| sed 's/\x1B[@A-Z\\\]^_]\|\x1B\[[0-9:;<=>?]*[-!"#$%&'"'"'()*+,.\/]*[][\\@A-Z^_`a-z{|}~]//g' \
>$tmp_file
mv $tmp_file $log_file
echo ----- "Done! $log_file"
}
```
https://github.com/lupyuen/nuttx-release/blob/main/run-ci.sh#L30-L47
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "xiaoxiang781216 (via GitHub)" <gi...@apache.org>.
xiaoxiang781216 commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2452850027
@lupyuen is our ci guru, could you answer this problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "lupyuen (via GitHub)" <gi...@apache.org>.
lupyuen commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2453596806
@stbenn Thank you so much! I'll update the instructions :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "lupyuen (via GitHub)" <gi...@apache.org>.
lupyuen commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2452875114
@stbenn Thank you so much for asking! Yep it's super important to test our PRs before submitting. But our CI Workflow is going through plenty of changes (due to cost cutting), lemme know if anything below isn't quite correct thanks :-)
GitHub ACT is probably too complicated? I recommend 2 ways of testing our PRs:
1. Run the CI Builds with Docker
2. Or run the CI Builds with GitHub Actions
(1) might be slower, depending on your PC. With (2) we don't need to worry about Wasting GitHub Runners, so long as the CI Workflow runs entirely in your repo, before submitting to NuttX Repo. (I'll explain)
### Option 1: Run the CI Builds with Docker
Suppose our PR is at `github.com/USER/nuttx/tree/BRANCH`. This is how we compile it with Docker:
```bash
## Build a NuttX Target Group with Docker
## Parameter is the Target Group, like "arm-01"
job=arm-01
## TODO: Install Docker Engine
## https://docs.docker.com/engine/install/ubuntu/
## Download the Docker Image for NuttX
sudo docker pull \
ghcr.io/apache/nuttx/apache-nuttx-ci-linux:latest
## Inside the Docker Container:
## Build the Target Group
sudo docker run -it \
ghcr.io/apache/nuttx/apache-nuttx-ci-linux:latest \
/bin/bash -c "
cd ;
pwd ;
git clone github.com/USER/nuttx --branch BRANCH
git clone https://github.com/apache/nuttx-apps apps ;
pushd nuttx ; echo NuttX Source: https://github.com/apache/nuttx/tree/\$(git rev-parse HEAD) ; popd ;
pushd apps ; echo NuttX Apps: https://github.com/apache/nuttx-apps/tree/\$(git rev-parse HEAD) ; popd ;
cd nuttx/tools/ci ;
(./cibuild.sh -c -A -N -R testlist/$job.dat || echo '***** BUILD FAILED') ;
"
```
[(Based on this)](https://lupyuen.codeberg.page/articles/ci2.html#build-nuttx-for-one-target-group)
It the CI Build fails, we will see "BUILD FAILED". Remember I said it's slow? That's because we need to repeat the above for `job` = arm-01 ... arm-04, risc-v-01 ... risc-v-06, xtensa, arm64, other, etc. [(Full List Here)](https://lupyuen.codeberg.page/articles/ci2.html#target-groups-in-nuttx)
And it won't run the macOS and Windows Jobs. Quicker Way is probably to use GitHub Actions...
### Option 2: Run the CI Builds with GitHub Actions
Suppose our PR is at `github.com/USER/nuttx/tree/BRANCH`. We need to patch the CI Workflow in the `master` branch:
1. Edit `master` branch [.github/workflows/build.yml](https://github.com/apache/nuttx/blob/master/.github/workflows/build.yml). Modify the 4 references to __arch.yml__, point them to our own repo:
```yaml
Linux-Arch:
uses: USER/nuttx/.github/workflows/arch.yml@master
macOS-Arch:
uses: USER/nuttx/.github/workflows/arch.yml@master
msys2-Arch:
uses: USER/nuttx/.github/workflows/arch.yml@master
msvc-Arch:
uses: USER/nuttx/.github/workflows/arch.yml@master
```
[(Similar to this)](https://github.com/apache/nuttx/issues/14407)
2. Then edit [.github/workflows/arch.yml](https://github.com/apache/nuttx/blob/master/.github/workflows/arch.yml). Delete this section:
```yaml
# Skip all macOS and Windows Builds
if [[ "${{ inputs.os }}" != "Linux" ]]; then
echo "Skipping all macOS and Windows Builds"
echo "skip_all_builds=1" | tee -a $GITHUB_OUTPUT
exit
fi
```
3. Delete this section:
```yaml
# If PR was Created or Modified: Exclude some boards
pr=${{github.event.pull_request.number}}
if [[ "$pr" != "" ]]; then
echo "Excluding arm-0[1249], arm-1[124-9], risc-v-04..06, sim-03, xtensa-02"
boards=$(
echo '${{ inputs.boards }}' |
jq --compact-output \
'map(
select(
test("arm-0[1249]") == false and test("arm-1[124-9]") == false and
test("risc-v-0[4-9]") == false and
test("sim-0[3-9]") == false and
test("xtensa-0[2-9]") == false
)
)'
)
fi
```
1. In our repo, create a PR that will merge BRANCH to our own `master`. (But don't merge the PR)
The CI Workflow will build everything, including macOS and Windows Builds. It should complete in 2.5 hours.
Hope this works OK :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "stbenn (via GitHub)" <gi...@apache.org>.
stbenn commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2453024962
@lupyuen I will try the docker version this weekend on a fresh install and take notes if theres any special steps I needed for it to work. In case others want to do this in the future as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "stbenn (via GitHub)" <gi...@apache.org>.
stbenn commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2453608922
The only oddity I encountered is when sending the output of the test to a .txt file (with `source runci.sh > ci_out.txt` ), the encoding appears to be weird:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "stbenn (via GitHub)" <gi...@apache.org>.
stbenn commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2453595402
@lupyuen The running locally with docker was exactly what I was looking for, thank you! :) I am not sure if it is strictly necessary for this, but to get docker working properly for act on WSL2, I needed to do these first:
1. Install docker desktop on the windows side (not sure why/if this is really necessary)
2. In docker desktop settings -> Resources Enable integration with the specific WSL2 distro I was using
3. Install docker engine on the distro with the instructions you linked here: https://docs.docker.com/engine/install/ubuntu/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on issue #14601:
URL: https://github.com/apache/nuttx/issues/14601#issuecomment-2453010100
@lupyuen maybe this explanation should be included in our Contributing documentation, like a recommended step before creating the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [HELP] Running github actions locally with act [nuttx]
Posted by "stbenn (via GitHub)" <gi...@apache.org>.
stbenn closed issue #14601: [HELP] Running github actions locally with act
URL: https://github.com/apache/nuttx/issues/14601
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org