You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nuttx.apache.org by Gregory Nutt <sp...@gmail.com> on 2019/12/21 00:28:40 UTC
Test Repository
Looking at https://gitbox.apache.org/repos/, I see several projects,
perhaps most projects, that have a special repository just to contain
testing logic. Often this is named <project>-testing and often is has
other names (like integration and others).
So there is a precedence for a nuttx-testing repository to hold testing
data... Things like yml files and the like. I would much prefer to a
have a separate repository to hold test files than to contaminate the
incubator-nuttx and incubator-nuttx-apps repositories with test data.
Another thing that I note by googling ".gitmodules testing
site:github.com/apache" is that many of these testing directories also
contain submodules to coordinate testing. I have gotten to rather like
the idea of three repositories: incubator-nuttx, incubator-nuttx-apps,
and incubator-nuttx-testing that holds all of the test logic with we
really don't want to contaminate the user facing repositories with.
Best to put testing logic in one place and keep the end-user
repositories as clean as possible to support the user needs.
Greg
RE: Test Repository
Posted by David Sidrane <Da...@nscdg.com>.
+1
-----Original Message-----
From: Gregory Nutt [mailto:spudaneco@gmail.com]
Sent: Friday, December 20, 2019 4:29 PM
To: dev@nuttx.apache.org
Subject: Test Repository
Looking at https://gitbox.apache.org/repos/, I see several projects,
perhaps most projects, that have a special repository just to contain
testing logic. Often this is named <project>-testing and often is has
other names (like integration and others).
So there is a precedence for a nuttx-testing repository to hold testing
data... Things like yml files and the like. I would much prefer to a
have a separate repository to hold test files than to contaminate the
incubator-nuttx and incubator-nuttx-apps repositories with test data.
Another thing that I note by googling ".gitmodules testing
site:github.com/apache" is that many of these testing directories also
contain submodules to coordinate testing. I have gotten to rather like
the idea of three repositories: incubator-nuttx, incubator-nuttx-apps,
and incubator-nuttx-testing that holds all of the test logic with we
really don't want to contaminate the user facing repositories with.
Best to put testing logic in one place and keep the end-user
repositories as clean as possible to support the user needs.
Greg
Re: Test Repository
Posted by David Sidrane <da...@apache.org>.
Hi Justin,
Great points. See below
On 2019/12/21 00:42:58, Justin Mclean <ju...@classsoftware.com> wrote:
> Hi,
>
> You may need to think how this fits into releases. some things to consider:
> - Do you want the release to contain test information or not? A lot of Apache project do include that but some don’t - it may be size dependant.
> - Do you wan users to be able to easy test releases? (Not providing test infrastructure in the same releases may make that harder).
> - This is a common scenario, a user is on an old version and needs to test it, the new testing repo has been updated to work with teh latest release, how do they test it?
>
> Thanks,
> Justin
>- This is a common scenario, a user is on an old version and needs to test it, the new testing repo has been updated to work with teh latest release, how do they test it?
The test repo has submudules to the nuttx and apps.
The master branch of testing is tied to master on apps and nuttx
The release branch(s) of testing is tied to the release branch(s) on apps and nuttx
Then it always works.
However take nxtyle as an example.
We ask submitter to run nxtyle on code to make sure the submission abides by the NuttX coding standard.
If on day 94 the nxtyle finds an issue better then on day 12 of its life. A user may contribute a submission from a release that does not abides by the NuttX coding standard.
This can be mitigated by keeping the style tools in a separate repo and asking dev to always stay on master of that repo. It also enables CI to always use the best nxtyle we have. Thus keeping master properly formatted.
David
Re: Test Repository
Posted by Gregory Nutt <sp...@gmail.com>.
> You may need to think how this fits into releases. some things to consider:
> - Do you want the release to contain test information or not? A lot of Apache project do include that but some don’t - it may be size dependant.
> - Do you wan users to be able to easy test releases? (Not providing test infrastructure in the same releases may make that harder).
> - This is a common scenario, a user is on an old version and needs to test it, the new testing repo has been updated to work with teh latest release, how do they test it?
I don't know if such a repository would contain user usable testing of
not. Its primary purpose would be to integrate testing on github and
support the change workflow.
This is nothing set in concrete and just a thought experiment. There is
no workflow or testing plan in place.
Re: Test Repository
Posted by Justin Mclean <ju...@classsoftware.com>.
Hi,
You may need to think how this fits into releases. some things to consider:
- Do you want the release to contain test information or not? A lot of Apache project do include that but some don’t - it may be size dependant.
- Do you wan users to be able to easy test releases? (Not providing test infrastructure in the same releases may make that harder).
- This is a common scenario, a user is on an old version and needs to test it, the new testing repo has been updated to work with teh latest release, how do they test it?
Thanks,
Justin
RE: Test Repository
Posted by David Sidrane <Da...@nscdg.com>.
Greg,
-----Original Message-----
From: Gregory Nutt [mailto:spudaneco@gmail.com]
Sent: Friday, December 20, 2019 4:32 PM
To: dev@nuttx.apache.org
Subject: Re: Test Repository
> Looking at https://gitbox.apache.org/repos/, I see several projects,
> perhaps most projects, that have a special repository just to contain
> testing logic. Often this is named <project>-testing and often is has
> other names (like integration and others).
>
> So there is a precedence for a nuttx-testing repository to hold
> testing data... Things like yml files and the like. I would much
> prefer to a have a separate repository to hold test files than to
> contaminate the incubator-nuttx and incubator-nuttx-apps repositories
> with test data.
>
> Another thing that I note by googling ".gitmodules testing
> site:github.com/apache" is that many of these testing directories also
> contain submodules to coordinate testing. I have gotten to rather
> like the idea of three repositories: incubator-nuttx,
> incubator-nuttx-apps, and incubator-nuttx-testing that holds all of
> the test logic with we really don't want to contaminate the user
> facing repositories with. Best to put testing logic in one place and
> keep the end-user repositories as clean as possible to support the
> user needs.
>
I know, I am violating my own principles. We should not really be
discussing testing repository organization UNTIL we have established,
agreed-upon workflow requirements. We don't have any such requirements
so just think of this as idle talk. For me it clarifies my position on
this. I am not in favor of any mechanization that keeps the end-user
repositories and clean as possible.
Greg
>I am not in favor of any mechanization that keeps the end-user
repositories and clean as possible.
Is it?
s/not/now
Re: Test Repository
Posted by Gregory Nutt <sp...@gmail.com>.
> Looking at https://gitbox.apache.org/repos/, I see several projects,
> perhaps most projects, that have a special repository just to contain
> testing logic. Often this is named <project>-testing and often is has
> other names (like integration and others).
>
> So there is a precedence for a nuttx-testing repository to hold
> testing data... Things like yml files and the like. I would much
> prefer to a have a separate repository to hold test files than to
> contaminate the incubator-nuttx and incubator-nuttx-apps repositories
> with test data.
>
> Another thing that I note by googling ".gitmodules testing
> site:github.com/apache" is that many of these testing directories also
> contain submodules to coordinate testing. I have gotten to rather
> like the idea of three repositories: incubator-nuttx,
> incubator-nuttx-apps, and incubator-nuttx-testing that holds all of
> the test logic with we really don't want to contaminate the user
> facing repositories with. Best to put testing logic in one place and
> keep the end-user repositories as clean as possible to support the
> user needs.
>
I know, I am violating my own principles. We should not really be
discussing testing repository organization UNTIL we have established,
agreed-upon workflow requirements. We don't have any such requirements
so just think of this as idle talk. For me it clarifies my position on
this. I am not in favor of any mechanization that keeps the end-user
repositories and clean as possible.
Greg