You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nuttx.apache.org by "Alan C. Assis" <ac...@gmail.com> on 2024/07/18 19:14:20 UTC
Help us to create a distributed board testing farm
Hi Everyone,
We are creating a testing farm for NuttX to be integrated on our CI.
As you know, NuttX supports more than 300 boards and we need to test all
boards everytime someone modifies our code base.
So, let's to map each person have each board, please include your name in
the front of each board you have (case there is already some else name
there, put your name in the next column)
https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
Thank you very much!
BR,
Alan
Re: Help us to create a distributed board testing farm
Posted by "Alan C. Assis" <ac...@gmail.com>.
Hi Gábor,
Amazing, let us know your idea and let us join forces to make it happen!
BR,
Alan
On Thu, Jul 18, 2024 at 5:29 PM Gábor Kiss-Vámosi <ki...@gmail.com>
wrote:
> Oh, I see. Nice! I guess making it fully automated would be too good to be
> true.
>
> We test the supported boards manually, but we support and maintain much
> less boards, and test them only before releases.
>
> We started to work on something which can be related and seems like a good
> collaboration opportunity. I will write about it tomorrow.
>
> Regards,
> Gabor
>
> On Thu, Jul 18, 2024, 22:12 Alan C. Assis <ac...@gmail.com> wrote:
>
> > Hi Gábor,
> >
> > Our idea is to do it automatically: the CI will build the binary and
> notify
> > the users to start the flashing process, test it and return the result.
> >
> > We will need a board to do the flashing process (i.e. an ESP32 with
> > SWD/JTAG flashing emulation).
> >
> > Does LVGL have farm board testing?
> >
> > BR,
> >
> > Alan
> >
> > On Thu, Jul 18, 2024 at 5:03 PM Gábor Kiss-Vámosi <ki...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > In practice will it be manual testing by humans, or can you run some
> > tests
> > > automatically and remotely?
> > >
> > > On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com> wrote:
> > >
> > > > Actually I shared it publicly with modify permissions (danger mode,
> > maybe
> > > > it is better to use Comment option, so people can ask for invitation)
> > > >
> > > > BR,
> > > >
> > > > Alan
> > > >
> > > > On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pe...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Alan,
> > > > >
> > > > > I guess you need to be invited to view that Google Doc?
> > > > >
> > > > > Ken
> > > > >
> > > > > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > > > > Hi Everyone,
> > > > > >
> > > > > > We are creating a testing farm for NuttX to be integrated on our
> > CI.
> > > > > >
> > > > > > As you know, NuttX supports more than 300 boards and we need to
> > test
> > > > all
> > > > > > boards everytime someone modifies our code base.
> > > > > >
> > > > > > So, let's to map each person have each board, please include your
> > > name
> > > > in
> > > > > > the front of each board you have (case there is already some else
> > > name
> > > > > > there, put your name in the next column)
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > > > > >
> > > > > > Thank you very much!
> > > > > >
> > > > > > BR,
> > > > > >
> > > > > > Alan
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help us to create a distributed board testing farm
Posted by Gábor Kiss-Vámosi <ki...@gmail.com>.
Oh, I see. Nice! I guess making it fully automated would be too good to be
true.
We test the supported boards manually, but we support and maintain much
less boards, and test them only before releases.
We started to work on something which can be related and seems like a good
collaboration opportunity. I will write about it tomorrow.
Regards,
Gabor
On Thu, Jul 18, 2024, 22:12 Alan C. Assis <ac...@gmail.com> wrote:
> Hi Gábor,
>
> Our idea is to do it automatically: the CI will build the binary and notify
> the users to start the flashing process, test it and return the result.
>
> We will need a board to do the flashing process (i.e. an ESP32 with
> SWD/JTAG flashing emulation).
>
> Does LVGL have farm board testing?
>
> BR,
>
> Alan
>
> On Thu, Jul 18, 2024 at 5:03 PM Gábor Kiss-Vámosi <ki...@gmail.com>
> wrote:
>
> > Hi,
> >
> > In practice will it be manual testing by humans, or can you run some
> tests
> > automatically and remotely?
> >
> > On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com> wrote:
> >
> > > Actually I shared it publicly with modify permissions (danger mode,
> maybe
> > > it is better to use Comment option, so people can ask for invitation)
> > >
> > > BR,
> > >
> > > Alan
> > >
> > > On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pe...@gmail.com>
> > wrote:
> > >
> > > > Hi Alan,
> > > >
> > > > I guess you need to be invited to view that Google Doc?
> > > >
> > > > Ken
> > > >
> > > > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > > > Hi Everyone,
> > > > >
> > > > > We are creating a testing farm for NuttX to be integrated on our
> CI.
> > > > >
> > > > > As you know, NuttX supports more than 300 boards and we need to
> test
> > > all
> > > > > boards everytime someone modifies our code base.
> > > > >
> > > > > So, let's to map each person have each board, please include your
> > name
> > > in
> > > > > the front of each board you have (case there is already some else
> > name
> > > > > there, put your name in the next column)
> > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > > > >
> > > > > Thank you very much!
> > > > >
> > > > > BR,
> > > > >
> > > > > Alan
> > > > >
> > > >
> > >
> >
>
Re: Help us to create a distributed board testing farm
Posted by "Alan C. Assis" <ac...@gmail.com>.
Hi Stuart,
Thank you very much for these ideas!
Really interesting this FPGA HW Test that EnjoyDigital created, I didn't
know about it.
This is the kind of link that should be included on our page.
I think a FPGA could be also used for that idea of the base board. It could
simulate many devices, so instead of communicating with a physical sensor,
it could be a simulated one in FPGA.
BR,
Alan
On Thursday, July 18, 2024, Stuart Ianna <st...@motec.com.au> wrote:
> This would be fantastic. enjoy-digital, the authors of Litex, already have
> a HIL configuration for the arty_a7 and NuttX. Perhaps that could be used
> with a little bit of collaboration.
>
> What is the expected test frequency? Expecting a result back from every
> runner on a PR may be problematic if there are hardware issues outside of
> the PR scope.
>
> Possibly a nightly build of the master branch which could be used to report
> a pass fail status? This could then be presented on the NuttX github
> landing page, or somewhere similar.
>
> The litex project with NuttX can be seen here:
> https://github.com/enjoy-digital/litex_hw_ci
>
> On Fri, 19 Jul 2024 at 07:52, Alan C. Assis <ac...@gmail.com> wrote:
>
> > Hi Maciej,
> >
> > Yes, using a computer to compile is also in our radar because it will
> > recreate real usage scenarios (if we only build the binary in the Github
> CI
> > we are limited to its toolchain version and libraries).
> >
> > I think your suggestions are really good and it is aligned with what I
> have
> > in mind in the long run.
> >
> > My suggestion to start using an ESP32 is because it is low cost, low
> power
> > and easy to scale.
> >
> > For the future I think we need to create some SOMs (System On Modules)
> > similar to Raspberry Pi SOM for different MCUs/CPUs and a Base board with
> > peripherals to exercise NuttX functionalities.
> >
> > That base board could have just some connectors like MikroBUS, Grove,
> Pmod
> > and Qwiic and each person will connect his own modules to test, but it
> will
> > require more customization.
> >
> > Let's start with the basics, for now we just need to guarantee that the
> > boards are starting correctly to avoid the frustration that David Barrass
> > faced when he discovered that LPC23xx is broken!
> >
> > BR,
> >
> > Alan
> >
> > On Thu, Jul 18, 2024 at 5:29 PM Maciej Wójcik <w8...@gmail.com> wrote:
> >
> > > Most of us have some spare PC around, such PC can have fresh Linux
> with a
> > > gitlab-runner service and this gitlab-runner can build, flash and run
> > > tests, a pipeline that gives positive or negative results.
> > >
> > > With binaries already built it could just download instead.
> > >
> > > I know that with GitLab this is more or less trivial. With GitHub
> > probably
> > > also, although I have no experience with it. Configuration of such a
> > > solution is not very difficult, but requires some expertise.
> > >
> > > We could create a repository with description and some configuration
> > files
> > > allowing easy setup of such a solution.
> > >
> > > Even if NuttX has only a limited amount of automated tests, automated
> > tests
> > > can be expanded and developed separately from such automated
> > > testing solution.
> > >
> > > Finally there could be a very simple website that board maintainers
> could
> > > automatically upload results of tests using some previously received
> > > tokens.
> > >
> > > Am Do., 18. Juli 2024 um 22:12 Uhr schrieb Alan C. Assis <
> > > acassis@gmail.com
> > > >:
> > >
> > > > Hi Gábor,
> > > >
> > > > Our idea is to do it automatically: the CI will build the binary and
> > > notify
> > > > the users to start the flashing process, test it and return the
> result.
> > > >
> > > > We will need a board to do the flashing process (i.e. an ESP32 with
> > > > SWD/JTAG flashing emulation).
> > > >
> > > > Does LVGL have farm board testing?
> > > >
> > > > BR,
> > > >
> > > > Alan
> > > >
> > > > On Thu, Jul 18, 2024 at 5:03 PM Gábor Kiss-Vámosi <
> > kisvegabor@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > In practice will it be manual testing by humans, or can you run
> some
> > > > tests
> > > > > automatically and remotely?
> > > > >
> > > > > On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com>
> wrote:
> > > > >
> > > > > > Actually I shared it publicly with modify permissions (danger
> mode,
> > > > maybe
> > > > > > it is better to use Comment option, so people can ask for
> > invitation)
> > > > > >
> > > > > > BR,
> > > > > >
> > > > > > Alan
> > > > > >
> > > > > > On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <
> pettitkd@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Hi Alan,
> > > > > > >
> > > > > > > I guess you need to be invited to view that Google Doc?
> > > > > > >
> > > > > > > Ken
> > > > > > >
> > > > > > > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > > > > > > Hi Everyone,
> > > > > > > >
> > > > > > > > We are creating a testing farm for NuttX to be integrated on
> > our
> > > > CI.
> > > > > > > >
> > > > > > > > As you know, NuttX supports more than 300 boards and we need
> to
> > > > test
> > > > > > all
> > > > > > > > boards everytime someone modifies our code base.
> > > > > > > >
> > > > > > > > So, let's to map each person have each board, please include
> > your
> > > > > name
> > > > > > in
> > > > > > > > the front of each board you have (case there is already some
> > else
> > > > > name
> > > > > > > > there, put your name in the next column)
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_
> HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > > > > > > >
> > > > > > > > Thank you very much!
> > > > > > > >
> > > > > > > > BR,
> > > > > > > >
> > > > > > > > Alan
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
> --
> *Kind regards,*
>
> *Stuart Ianna*
> *Firmware Engineer*
>
> --
> *MoTeC Pty Ltd*
>
> 121 Merrindale Drive
> Croydon South 3136
> Victoria Australia
> *T: *61 3 9761 5050
> *W: *www.motec.com.au <https://www.motec.com.au/>
>
>
> --
> <http://www.facebook.com/motec.global>
> <http://www.youtube.com/user/MoTeCAustralia>
> <https://www.instagram.com/motec_global/>
> <https://www.linkedin.com/company/motec-global>
>
>
> --
> <https://www.motec.com.au/site/careers>
>
> --
>
>
> Disclaimer Notice: This message, including any attachments, contains
> confidential information intended for a specific individual and purpose
> and
> is protected by law. If you are not the intended recipient you should
> delete this message. Any disclosure, copying, or distribution of this
> message or the taking of any action based on it is strictly prohibited.
>
Re: Help us to create a distributed board testing farm
Posted by Stuart Ianna <st...@motec.com.au>.
This would be fantastic. enjoy-digital, the authors of Litex, already have
a HIL configuration for the arty_a7 and NuttX. Perhaps that could be used
with a little bit of collaboration.
What is the expected test frequency? Expecting a result back from every
runner on a PR may be problematic if there are hardware issues outside of
the PR scope.
Possibly a nightly build of the master branch which could be used to report
a pass fail status? This could then be presented on the NuttX github
landing page, or somewhere similar.
The litex project with NuttX can be seen here:
https://github.com/enjoy-digital/litex_hw_ci
On Fri, 19 Jul 2024 at 07:52, Alan C. Assis <ac...@gmail.com> wrote:
> Hi Maciej,
>
> Yes, using a computer to compile is also in our radar because it will
> recreate real usage scenarios (if we only build the binary in the Github CI
> we are limited to its toolchain version and libraries).
>
> I think your suggestions are really good and it is aligned with what I have
> in mind in the long run.
>
> My suggestion to start using an ESP32 is because it is low cost, low power
> and easy to scale.
>
> For the future I think we need to create some SOMs (System On Modules)
> similar to Raspberry Pi SOM for different MCUs/CPUs and a Base board with
> peripherals to exercise NuttX functionalities.
>
> That base board could have just some connectors like MikroBUS, Grove, Pmod
> and Qwiic and each person will connect his own modules to test, but it will
> require more customization.
>
> Let's start with the basics, for now we just need to guarantee that the
> boards are starting correctly to avoid the frustration that David Barrass
> faced when he discovered that LPC23xx is broken!
>
> BR,
>
> Alan
>
> On Thu, Jul 18, 2024 at 5:29 PM Maciej Wójcik <w8...@gmail.com> wrote:
>
> > Most of us have some spare PC around, such PC can have fresh Linux with a
> > gitlab-runner service and this gitlab-runner can build, flash and run
> > tests, a pipeline that gives positive or negative results.
> >
> > With binaries already built it could just download instead.
> >
> > I know that with GitLab this is more or less trivial. With GitHub
> probably
> > also, although I have no experience with it. Configuration of such a
> > solution is not very difficult, but requires some expertise.
> >
> > We could create a repository with description and some configuration
> files
> > allowing easy setup of such a solution.
> >
> > Even if NuttX has only a limited amount of automated tests, automated
> tests
> > can be expanded and developed separately from such automated
> > testing solution.
> >
> > Finally there could be a very simple website that board maintainers could
> > automatically upload results of tests using some previously received
> > tokens.
> >
> > Am Do., 18. Juli 2024 um 22:12 Uhr schrieb Alan C. Assis <
> > acassis@gmail.com
> > >:
> >
> > > Hi Gábor,
> > >
> > > Our idea is to do it automatically: the CI will build the binary and
> > notify
> > > the users to start the flashing process, test it and return the result.
> > >
> > > We will need a board to do the flashing process (i.e. an ESP32 with
> > > SWD/JTAG flashing emulation).
> > >
> > > Does LVGL have farm board testing?
> > >
> > > BR,
> > >
> > > Alan
> > >
> > > On Thu, Jul 18, 2024 at 5:03 PM Gábor Kiss-Vámosi <
> kisvegabor@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > In practice will it be manual testing by humans, or can you run some
> > > tests
> > > > automatically and remotely?
> > > >
> > > > On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com> wrote:
> > > >
> > > > > Actually I shared it publicly with modify permissions (danger mode,
> > > maybe
> > > > > it is better to use Comment option, so people can ask for
> invitation)
> > > > >
> > > > > BR,
> > > > >
> > > > > Alan
> > > > >
> > > > > On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pettitkd@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > Hi Alan,
> > > > > >
> > > > > > I guess you need to be invited to view that Google Doc?
> > > > > >
> > > > > > Ken
> > > > > >
> > > > > > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > > > > > Hi Everyone,
> > > > > > >
> > > > > > > We are creating a testing farm for NuttX to be integrated on
> our
> > > CI.
> > > > > > >
> > > > > > > As you know, NuttX supports more than 300 boards and we need to
> > > test
> > > > > all
> > > > > > > boards everytime someone modifies our code base.
> > > > > > >
> > > > > > > So, let's to map each person have each board, please include
> your
> > > > name
> > > > > in
> > > > > > > the front of each board you have (case there is already some
> else
> > > > name
> > > > > > > there, put your name in the next column)
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > > > > > >
> > > > > > > Thank you very much!
> > > > > > >
> > > > > > > BR,
> > > > > > >
> > > > > > > Alan
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
*Kind regards,*
*Stuart Ianna*
*Firmware Engineer*
--
*MoTeC Pty Ltd*
121 Merrindale Drive
Croydon South 3136
Victoria Australia
*T: *61 3 9761 5050
*W: *www.motec.com.au <https://www.motec.com.au/>
--
<http://www.facebook.com/motec.global>
<http://www.youtube.com/user/MoTeCAustralia>
<https://www.instagram.com/motec_global/>
<https://www.linkedin.com/company/motec-global>
--
<https://www.motec.com.au/site/careers>
--
Disclaimer Notice: This message, including any attachments, contains
confidential information intended for a specific individual and purpose and
is protected by law. If you are not the intended recipient you should
delete this message. Any disclosure, copying, or distribution of this
message or the taking of any action based on it is strictly prohibited.
Re: Help us to create a distributed board testing farm
Posted by "Alan C. Assis" <ac...@gmail.com>.
Hi Maciej,
Yes, using a computer to compile is also in our radar because it will
recreate real usage scenarios (if we only build the binary in the Github CI
we are limited to its toolchain version and libraries).
I think your suggestions are really good and it is aligned with what I have
in mind in the long run.
My suggestion to start using an ESP32 is because it is low cost, low power
and easy to scale.
For the future I think we need to create some SOMs (System On Modules)
similar to Raspberry Pi SOM for different MCUs/CPUs and a Base board with
peripherals to exercise NuttX functionalities.
That base board could have just some connectors like MikroBUS, Grove, Pmod
and Qwiic and each person will connect his own modules to test, but it will
require more customization.
Let's start with the basics, for now we just need to guarantee that the
boards are starting correctly to avoid the frustration that David Barrass
faced when he discovered that LPC23xx is broken!
BR,
Alan
On Thu, Jul 18, 2024 at 5:29 PM Maciej Wójcik <w8...@gmail.com> wrote:
> Most of us have some spare PC around, such PC can have fresh Linux with a
> gitlab-runner service and this gitlab-runner can build, flash and run
> tests, a pipeline that gives positive or negative results.
>
> With binaries already built it could just download instead.
>
> I know that with GitLab this is more or less trivial. With GitHub probably
> also, although I have no experience with it. Configuration of such a
> solution is not very difficult, but requires some expertise.
>
> We could create a repository with description and some configuration files
> allowing easy setup of such a solution.
>
> Even if NuttX has only a limited amount of automated tests, automated tests
> can be expanded and developed separately from such automated
> testing solution.
>
> Finally there could be a very simple website that board maintainers could
> automatically upload results of tests using some previously received
> tokens.
>
> Am Do., 18. Juli 2024 um 22:12 Uhr schrieb Alan C. Assis <
> acassis@gmail.com
> >:
>
> > Hi Gábor,
> >
> > Our idea is to do it automatically: the CI will build the binary and
> notify
> > the users to start the flashing process, test it and return the result.
> >
> > We will need a board to do the flashing process (i.e. an ESP32 with
> > SWD/JTAG flashing emulation).
> >
> > Does LVGL have farm board testing?
> >
> > BR,
> >
> > Alan
> >
> > On Thu, Jul 18, 2024 at 5:03 PM Gábor Kiss-Vámosi <ki...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > In practice will it be manual testing by humans, or can you run some
> > tests
> > > automatically and remotely?
> > >
> > > On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com> wrote:
> > >
> > > > Actually I shared it publicly with modify permissions (danger mode,
> > maybe
> > > > it is better to use Comment option, so people can ask for invitation)
> > > >
> > > > BR,
> > > >
> > > > Alan
> > > >
> > > > On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pe...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Alan,
> > > > >
> > > > > I guess you need to be invited to view that Google Doc?
> > > > >
> > > > > Ken
> > > > >
> > > > > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > > > > Hi Everyone,
> > > > > >
> > > > > > We are creating a testing farm for NuttX to be integrated on our
> > CI.
> > > > > >
> > > > > > As you know, NuttX supports more than 300 boards and we need to
> > test
> > > > all
> > > > > > boards everytime someone modifies our code base.
> > > > > >
> > > > > > So, let's to map each person have each board, please include your
> > > name
> > > > in
> > > > > > the front of each board you have (case there is already some else
> > > name
> > > > > > there, put your name in the next column)
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > > > > >
> > > > > > Thank you very much!
> > > > > >
> > > > > > BR,
> > > > > >
> > > > > > Alan
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help us to create a distributed board testing farm
Posted by Maciej Wójcik <w8...@gmail.com>.
Most of us have some spare PC around, such PC can have fresh Linux with a
gitlab-runner service and this gitlab-runner can build, flash and run
tests, a pipeline that gives positive or negative results.
With binaries already built it could just download instead.
I know that with GitLab this is more or less trivial. With GitHub probably
also, although I have no experience with it. Configuration of such a
solution is not very difficult, but requires some expertise.
We could create a repository with description and some configuration files
allowing easy setup of such a solution.
Even if NuttX has only a limited amount of automated tests, automated tests
can be expanded and developed separately from such automated
testing solution.
Finally there could be a very simple website that board maintainers could
automatically upload results of tests using some previously received tokens.
Am Do., 18. Juli 2024 um 22:12 Uhr schrieb Alan C. Assis <acassis@gmail.com
>:
> Hi Gábor,
>
> Our idea is to do it automatically: the CI will build the binary and notify
> the users to start the flashing process, test it and return the result.
>
> We will need a board to do the flashing process (i.e. an ESP32 with
> SWD/JTAG flashing emulation).
>
> Does LVGL have farm board testing?
>
> BR,
>
> Alan
>
> On Thu, Jul 18, 2024 at 5:03 PM Gábor Kiss-Vámosi <ki...@gmail.com>
> wrote:
>
> > Hi,
> >
> > In practice will it be manual testing by humans, or can you run some
> tests
> > automatically and remotely?
> >
> > On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com> wrote:
> >
> > > Actually I shared it publicly with modify permissions (danger mode,
> maybe
> > > it is better to use Comment option, so people can ask for invitation)
> > >
> > > BR,
> > >
> > > Alan
> > >
> > > On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pe...@gmail.com>
> > wrote:
> > >
> > > > Hi Alan,
> > > >
> > > > I guess you need to be invited to view that Google Doc?
> > > >
> > > > Ken
> > > >
> > > > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > > > Hi Everyone,
> > > > >
> > > > > We are creating a testing farm for NuttX to be integrated on our
> CI.
> > > > >
> > > > > As you know, NuttX supports more than 300 boards and we need to
> test
> > > all
> > > > > boards everytime someone modifies our code base.
> > > > >
> > > > > So, let's to map each person have each board, please include your
> > name
> > > in
> > > > > the front of each board you have (case there is already some else
> > name
> > > > > there, put your name in the next column)
> > > > >
> > > > >
> > > >
> > >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > > > >
> > > > > Thank you very much!
> > > > >
> > > > > BR,
> > > > >
> > > > > Alan
> > > > >
> > > >
> > >
> >
>
Re: Help us to create a distributed board testing farm
Posted by "Alan C. Assis" <ac...@gmail.com>.
Hi Gábor,
Our idea is to do it automatically: the CI will build the binary and notify
the users to start the flashing process, test it and return the result.
We will need a board to do the flashing process (i.e. an ESP32 with
SWD/JTAG flashing emulation).
Does LVGL have farm board testing?
BR,
Alan
On Thu, Jul 18, 2024 at 5:03 PM Gábor Kiss-Vámosi <ki...@gmail.com>
wrote:
> Hi,
>
> In practice will it be manual testing by humans, or can you run some tests
> automatically and remotely?
>
> On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com> wrote:
>
> > Actually I shared it publicly with modify permissions (danger mode, maybe
> > it is better to use Comment option, so people can ask for invitation)
> >
> > BR,
> >
> > Alan
> >
> > On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pe...@gmail.com>
> wrote:
> >
> > > Hi Alan,
> > >
> > > I guess you need to be invited to view that Google Doc?
> > >
> > > Ken
> > >
> > > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > > Hi Everyone,
> > > >
> > > > We are creating a testing farm for NuttX to be integrated on our CI.
> > > >
> > > > As you know, NuttX supports more than 300 boards and we need to test
> > all
> > > > boards everytime someone modifies our code base.
> > > >
> > > > So, let's to map each person have each board, please include your
> name
> > in
> > > > the front of each board you have (case there is already some else
> name
> > > > there, put your name in the next column)
> > > >
> > > >
> > >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > > >
> > > > Thank you very much!
> > > >
> > > > BR,
> > > >
> > > > Alan
> > > >
> > >
> >
>
Re: Help us to create a distributed board testing farm
Posted by Gábor Kiss-Vámosi <ki...@gmail.com>.
Hi,
In practice will it be manual testing by humans, or can you run some tests
automatically and remotely?
On Thu, Jul 18, 2024, 21:50 Alan C. Assis <ac...@gmail.com> wrote:
> Actually I shared it publicly with modify permissions (danger mode, maybe
> it is better to use Comment option, so people can ask for invitation)
>
> BR,
>
> Alan
>
> On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pe...@gmail.com> wrote:
>
> > Hi Alan,
> >
> > I guess you need to be invited to view that Google Doc?
> >
> > Ken
> >
> > On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > > Hi Everyone,
> > >
> > > We are creating a testing farm for NuttX to be integrated on our CI.
> > >
> > > As you know, NuttX supports more than 300 boards and we need to test
> all
> > > boards everytime someone modifies our code base.
> > >
> > > So, let's to map each person have each board, please include your name
> in
> > > the front of each board you have (case there is already some else name
> > > there, put your name in the next column)
> > >
> > >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> > >
> > > Thank you very much!
> > >
> > > BR,
> > >
> > > Alan
> > >
> >
>
Re: Help us to create a distributed board testing farm
Posted by "Alan C. Assis" <ac...@gmail.com>.
Actually I shared it publicly with modify permissions (danger mode, maybe
it is better to use Comment option, so people can ask for invitation)
BR,
Alan
On Thu, Jul 18, 2024 at 4:39 PM Kenneth Pettit <pe...@gmail.com> wrote:
> Hi Alan,
>
> I guess you need to be invited to view that Google Doc?
>
> Ken
>
> On 7/18/24 12:14 PM, Alan C. Assis wrote:
> > Hi Everyone,
> >
> > We are creating a testing farm for NuttX to be integrated on our CI.
> >
> > As you know, NuttX supports more than 300 boards and we need to test all
> > boards everytime someone modifies our code base.
> >
> > So, let's to map each person have each board, please include your name in
> > the front of each board you have (case there is already some else name
> > there, put your name in the next column)
> >
> >
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
> >
> > Thank you very much!
> >
> > BR,
> >
> > Alan
> >
>
Re: Help us to create a distributed board testing farm
Posted by Kenneth Pettit <pe...@gmail.com>.
Hi Alan,
I guess you need to be invited to view that Google Doc?
Ken
On 7/18/24 12:14 PM, Alan C. Assis wrote:
> Hi Everyone,
>
> We are creating a testing farm for NuttX to be integrated on our CI.
>
> As you know, NuttX supports more than 300 boards and we need to test all
> boards everytime someone modifies our code base.
>
> So, let's to map each person have each board, please include your name in
> the front of each board you have (case there is already some else name
> there, put your name in the next column)
>
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
>
> Thank you very much!
>
> BR,
>
> Alan
>
Re: Help us to create a distributed board testing farm
Posted by "Alan C. Assis" <ac...@gmail.com>.
Hi Michal,
I agree with you about the security side.
In fact my initial idea was using Raspberry Pi to do it using uhubctl see:
https://github.com/mvp/uhubctl (at bottom of page there are many projects
using it) with a 16 port USB HUB.
This way we could turn ON/OFF up to 16 boards and flash it over USB (it
will require a board with usb programming/flash support like
stm32f4discovery or raspberrypi-pico).
But then I think we could "eat our dog food", we can use ESP32 with NuttX
and flash turn ON/OFF and flash other boards.
Something similar what this project does:
https://vcon.io/automated-firmware-tests/
BR,
Alan
On Thu, Jul 18, 2024 at 5:25 PM <mi...@bofc.pl> wrote:
> Hi
> On 2024-07-18 16:14:20, Alan C. Assis wrote:
> > We are creating a testing farm for NuttX to be integrated on our CI.
> How is this going to work? Since, this seems like, tests will be conducted
> on real hardware, I assume I would have to install some daemon at my home
> so
> that CI could send commands to it to perform tests?
>
> This poses serious security risks. The only way I see it working is to have
> raspberry pi (or any other mini computer) in isolated network and vlan.
>
> What CI software are you going to use? I remember using Buildbot in the
> past,
> which may be good candidate, since I remember it being build just for
> cases like
> that. That is you host Master, and anyone that wants to participate in
> testing
> their arch/platform would set up a slave, which would connect to master and
> listen for build commands. Might be worth investigating it.
>
> I think it would be beneficial if anyone could just setup slave on his
> own, and
> then just connect to the master without any real authorization to
> participate
> in testing. I think there is a risk of such slave feeding false
> positives/negatives tho and someone who does not like nuttx could sabotage
> tests ;)
>
> --
>
> .-----------------.-------------------.----------------------.-----------------.
> | Michal Lyszczek | Embedded C, Linux | Company Address | .-.
> opensource |
> | +48 727 564 419 | Software Engineer | Akacjowa 10a; 55-330 | oo|
> supporter |
> | https://bofc.pl `----.--------------: Brzezinka Sredzka PL | /`'\
> & |
> | GPG FF1EBFE7E3A974B1 | Bits of Code | NIP: 813 349 58 78 |(\_;/)
> programer |
>
> `----------------------^--------------^----------------------^-----------------'
>
Re: Help us to create a distributed board testing farm
Posted by mi...@bofc.pl.
Hi
On 2024-07-18 16:14:20, Alan C. Assis wrote:
> We are creating a testing farm for NuttX to be integrated on our CI.
How is this going to work? Since, this seems like, tests will be conducted
on real hardware, I assume I would have to install some daemon at my home so
that CI could send commands to it to perform tests?
This poses serious security risks. The only way I see it working is to have
raspberry pi (or any other mini computer) in isolated network and vlan.
What CI software are you going to use? I remember using Buildbot in the past,
which may be good candidate, since I remember it being build just for cases like
that. That is you host Master, and anyone that wants to participate in testing
their arch/platform would set up a slave, which would connect to master and
listen for build commands. Might be worth investigating it.
I think it would be beneficial if anyone could just setup slave on his own, and
then just connect to the master without any real authorization to participate
in testing. I think there is a risk of such slave feeding false
positives/negatives tho and someone who does not like nuttx could sabotage
tests ;)
--
.-----------------.-------------------.----------------------.-----------------.
| Michal Lyszczek | Embedded C, Linux | Company Address | .-. opensource |
| +48 727 564 419 | Software Engineer | Akacjowa 10a; 55-330 | oo| supporter |
| https://bofc.pl `----.--------------: Brzezinka Sredzka PL | /`'\ & |
| GPG FF1EBFE7E3A974B1 | Bits of Code | NIP: 813 349 58 78 |(\_;/) programer |
`----------------------^--------------^----------------------^-----------------'
Re: Help us to create a distributed board testing farm
Posted by "Alan C. Assis" <ac...@gmail.com>.
Only my bad English, sometimes I try to do some direct translation, without
paying attention to the grammar rules.
"So let's map out which people have which development board".
BR,
Alan
On Thu, Jul 18, 2024 at 4:14 PM Alan C. Assis <ac...@gmail.com> wrote:
> Hi Everyone,
>
> We are creating a testing farm for NuttX to be integrated on our CI.
>
> As you know, NuttX supports more than 300 boards and we need to test all
> boards everytime someone modifies our code base.
>
> So, let's to map each person have each board, please include your name in
> the front of each board you have (case there is already some else name
> there, put your name in the next column)
>
>
> https://docs.google.com/spreadsheets/d/1nDMquG-gjfQ0c9jIfv_HztGeq1ysfCeJzOEts3X3Eks/edit?gid=0#gid=0
>
> Thank you very much!
>
> BR,
>
> Alan
>