You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nuttx.apache.org by "btashton (via GitHub)" <gi...@apache.org> on 2023/05/31 07:19:31 UTC
[GitHub] [nuttx] btashton opened a new pull request, #9438: Generate junit xml test report in CI tooling.
btashton opened a new pull request, #9438:
URL: https://github.com/apache/nuttx/pull/9438
## Summary
Generates junit XML test reports for CI jobs that can be visualized in the job summary. See https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/ for what this might look like.
## Impact
Increased visibility into CI failures by creating a formal test report as well as integrating into the Github Action UI.
## Testing
WIP. Primarily though the CI system.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nuttx] acassis commented on a diff in pull request #9438: Generate junit xml test report in CI tooling.
Posted by "acassis (via GitHub)" <gi...@apache.org>.
acassis commented on code in PR #9438:
URL: https://github.com/apache/nuttx/pull/9438#discussion_r1221581288
##########
tools/testbuild.sh:
##########
@@ -354,33 +405,53 @@ function refresh {
if [ -d $nuttx/.git ] || [ -d $APPSDIR/.git ]; then
if [[ -n $(git -C $nuttx status -s) ]]; then
git -C $nuttx status
+ report_test_failure "OS repo not clean."
fail=1
fi
if [[ -n $(git -C $APPSDIR status -s) ]]; then
git -C $APPSDIR status
+ report_test_failure "Apps repo not clean."
fail=1
fi
fi
fi
+ if [ ${fail} -eq 0 ]; then
+ report_test_pass
+ # Restore fail flag.
+ fail=$tmp_fail
+ fi
+ end_test_case
+
return $fail
}
function run {
+ start_test_case run
+
if [ ${RUN} -ne 0 ]; then
run_script="$path/run"
if [ -x $run_script ]; then
echo " Running NuttX..."
if ! $run_script; then
fail=1
+ report_test_failure "Runtime tests failed see logs."
Review Comment:
Is there some way to have some kind of link to move user to fail error log?
##########
tools/testbuild.sh:
##########
@@ -343,8 +391,11 @@ function build {
function refresh {
# Ensure defconfig in the canonical form
+ start_test_case refresh
+ local tmp_fail=$fail
if ! ./tools/refresh.sh --silent $config; then
+ report_test_failure "Could not refresh config."
Review Comment:
Please include the config name that failed
##########
tools/testbuild.sh:
##########
@@ -283,25 +301,39 @@ function distclean {
if [ -d $nuttx/.git ] || [ -d $APPSDIR/.git ]; then
if [[ -n $(git -C $nuttx status --ignored -s) ]]; then
git -C $nuttx status --ignored
+ report_test_failure "OS repo not clean."
fail=1
fi
if [[ -n $(git -C $APPSDIR status --ignored -s) ]]; then
git -C $APPSDIR status --ignored
+ report_test_failure "Apps repo not clean."
fail=1
fi
fi
fi
fi
fi
+ if [ ${fail} -eq 0 ]; then
+ report_test_pass
+ # Restore fail flag.
+ fail=$tmp_fail
+ fi
+ end_test_case
+
return $fail
}
# Configure for the next build
function configure {
+ start_test_case configure
+ local tmp_fail=$fail
+ fail=0
+
echo " Configuring..."
if ! ./tools/configure.sh ${HOPTION} $config ${JOPTION} 1>/dev/null; then
+ report_test_failure "Failed to configure."
Review Comment:
Please include the config name/option: $config ${JOPTION} in the failure message
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [nuttx] btashton commented on a diff in pull request #9438: Generate junit xml test report in CI tooling.
Posted by "btashton (via GitHub)" <gi...@apache.org>.
btashton commented on code in PR #9438:
URL: https://github.com/apache/nuttx/pull/9438#discussion_r1221700807
##########
tools/testbuild.sh:
##########
@@ -283,25 +301,39 @@ function distclean {
if [ -d $nuttx/.git ] || [ -d $APPSDIR/.git ]; then
if [[ -n $(git -C $nuttx status --ignored -s) ]]; then
git -C $nuttx status --ignored
+ report_test_failure "OS repo not clean."
fail=1
fi
if [[ -n $(git -C $APPSDIR status --ignored -s) ]]; then
git -C $APPSDIR status --ignored
+ report_test_failure "Apps repo not clean."
fail=1
fi
fi
fi
fi
fi
+ if [ ${fail} -eq 0 ]; then
+ report_test_pass
+ # Restore fail flag.
+ fail=$tmp_fail
+ fi
+ end_test_case
+
return $fail
}
# Configure for the next build
function configure {
+ start_test_case configure
+ local tmp_fail=$fail
+ fail=0
+
echo " Configuring..."
if ! ./tools/configure.sh ${HOPTION} $config ${JOPTION} 1>/dev/null; then
+ report_test_failure "Failed to configure."
Review Comment:
This is already part of the metadata for the job. I will provide an example once I have the code to render the results. If you want to see an early preview you can download the xml report from the report.
Also if we really want to have detailed test visibility we should convert the CI running script to use pytest... But my energy for having that argument is really really low.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@nuttx.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org