You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nuttx.apache.org by Gregory Nutt <sp...@gmail.com> on 2020/01/02 17:36:46 UTC
Build testing
I have some scripts that I use to perform build testing. That is, just
build as many configurations. This only verifies that no existing
builds are broken. It does not prove that a change is correct or that a
change will not break a build that actually uses the change. It would
take a much smarter test to do that then such a brute force test (when,
I don't advocate such dumb build testing).
But it would be nice to have it running as a 'cron' somewhere. It
currently builds 419 ARM configurations! plus several dozen sim, AVR,
and MIPS (PIC32 configurations). It would be nice if there were
available online some so that if someone wants to make a change to many
files like this in the future, we can at least be assured it won't break
builds.
The build test is based primarily on nuttx/tools/testbuild.sh which will
build a list of configurations. The configuration lists are elsewhere
at https://bitbucket.org/nuttx/tools/src/master/buildtest/ There are
also some bonehead scripts at the location that only work on my
machine. These basically just setup paths, and manage running
nuttx/tools/testbuild.sh. Those bonehead scripts are setup to work only
in my environment and would not work anywhere else without modification.
But if we could make this runnable online (perhaps via Jenkins?), then
we could avoid such extensive code breakage in the future.
Greg
Re: Build testing
Posted by Alin Jerpelea <je...@gmail.com>.
I am using the same scripts on my test machine and if anyone would like I
can dump the daily logs on a github for refference
Alin
On Thu, Jan 2, 2020 at 6:50 PM Disruptive Solutions <
disruptivesolutionsnl@gmail.com> wrote:
> Very interesting... I would like to use build scripts in the Jenkins
> solution I have running to test my STM book configs...
>
> Verstuurd vanaf mijn iPhone
>
> > Op 2 jan. 2020 om 18:40 heeft Gregory Nutt <sp...@gmail.com> het
> volgende geschreven:
> >
> >
> >> I have some scripts that I use to perform build testing. That is, just
> build as many configurations. This only verifies that no existing builds
> are broken. It does not prove that a change is correct or that a change
> will not break a build that actually uses the change. It would take a much
> smarter test to do that then such a brute force test (when, I don't
> advocate such dumb build testing).
> >>
> >> But it would be nice to have it running as a 'cron' somewhere. It
> currently builds 419 ARM configurations! plus several dozen sim, AVR, and
> MIPS (PIC32 configurations). It would be nice if there were available
> online some so that if someone wants to make a change to many files like
> this in the future, we can at least be assured it won't break builds.
> >>
> >> The build test is based primarily on nuttx/tools/testbuild.sh which
> will build a list of configurations. The configuration lists are elsewhere
> at https://bitbucket.org/nuttx/tools/src/master/buildtest/ There are
> also some bonehead scripts at the location that only work on my machine.
> These basically just setup paths, and manage running
> nuttx/tools/testbuild.sh. Those bonehead scripts are setup to work only in
> my environment and would not work anywhere else without modification.
> >>
> >> But if we could make this runnable online (perhaps via Jenkins?), then
> we could avoid such extensive code breakage in the future.
> >
> > I should also mention how I use this.
> >
> > * I keep the last "successful" build test output in, for example,
> > armtest.log.last
> > * I run the test generating a new armtest.log
> > * I then 'diff -u armtest.log.last armtest.log | vim -' and I can see
> > if any new warnings or errors occur.
> >
> > There is another bonehead script, monitor.sh, that just shows me the
> running status of the test. I am sure that the people on this list with
> more devops skills than I could make this all work smoother.
> >
> >
> >
>
Re: Build testing
Posted by Disruptive Solutions <di...@gmail.com>.
Very interesting... I would like to use build scripts in the Jenkins solution I have running to test my STM book configs...
Verstuurd vanaf mijn iPhone
> Op 2 jan. 2020 om 18:40 heeft Gregory Nutt <sp...@gmail.com> het volgende geschreven:
>
>
>> I have some scripts that I use to perform build testing. That is, just build as many configurations. This only verifies that no existing builds are broken. It does not prove that a change is correct or that a change will not break a build that actually uses the change. It would take a much smarter test to do that then such a brute force test (when, I don't advocate such dumb build testing).
>>
>> But it would be nice to have it running as a 'cron' somewhere. It currently builds 419 ARM configurations! plus several dozen sim, AVR, and MIPS (PIC32 configurations). It would be nice if there were available online some so that if someone wants to make a change to many files like this in the future, we can at least be assured it won't break builds.
>>
>> The build test is based primarily on nuttx/tools/testbuild.sh which will build a list of configurations. The configuration lists are elsewhere at https://bitbucket.org/nuttx/tools/src/master/buildtest/ There are also some bonehead scripts at the location that only work on my machine. These basically just setup paths, and manage running nuttx/tools/testbuild.sh. Those bonehead scripts are setup to work only in my environment and would not work anywhere else without modification.
>>
>> But if we could make this runnable online (perhaps via Jenkins?), then we could avoid such extensive code breakage in the future.
>
> I should also mention how I use this.
>
> * I keep the last "successful" build test output in, for example,
> armtest.log.last
> * I run the test generating a new armtest.log
> * I then 'diff -u armtest.log.last armtest.log | vim -' and I can see
> if any new warnings or errors occur.
>
> There is another bonehead script, monitor.sh, that just shows me the running status of the test. I am sure that the people on this list with more devops skills than I could make this all work smoother.
>
>
>
Re: Build testing
Posted by Gregory Nutt <sp...@gmail.com>.
> I have some scripts that I use to perform build testing. That is,
> just build as many configurations. This only verifies that no
> existing builds are broken. It does not prove that a change is
> correct or that a change will not break a build that actually uses the
> change. It would take a much smarter test to do that then such a
> brute force test (when, I don't advocate such dumb build testing).
>
> But it would be nice to have it running as a 'cron' somewhere. It
> currently builds 419 ARM configurations! plus several dozen sim, AVR,
> and MIPS (PIC32 configurations). It would be nice if there were
> available online some so that if someone wants to make a change to
> many files like this in the future, we can at least be assured it
> won't break builds.
>
> The build test is based primarily on nuttx/tools/testbuild.sh which
> will build a list of configurations. The configuration lists are
> elsewhere at https://bitbucket.org/nuttx/tools/src/master/buildtest/
> There are also some bonehead scripts at the location that only work on
> my machine. These basically just setup paths, and manage running
> nuttx/tools/testbuild.sh. Those bonehead scripts are setup to work
> only in my environment and would not work anywhere else without
> modification.
>
> But if we could make this runnable online (perhaps via Jenkins?), then
> we could avoid such extensive code breakage in the future.
I should also mention how I use this.
* I keep the last "successful" build test output in, for example,
armtest.log.last
* I run the test generating a new armtest.log
* I then 'diff -u armtest.log.last armtest.log | vim -' and I can see
if any new warnings or errors occur.
There is another bonehead script, monitor.sh, that just shows me the
running status of the test. I am sure that the people on this list with
more devops skills than I could make this all work smoother.