Re: Working Effectively (was Point of Order)

[Alin Jerpelea <je...@gmail.com>]

I think that we should update all code with nxstyle then we start with a
clean code on which patches will be checked

Alin

On Tue, Dec 31, 2019 at 3:42 PM Haitao Liu <li...@gmail.com> wrote:

> Greg,  I'll finish the nxstyle & check script these days firstly and then
> take time to handle the two issues you mentioned if allowed. I wish I could
> be competent to take over full responsibility for nxstyle.c.
>
> Gregory Nutt <sp...@gmail.com> 于2019年12月30日周一 下午11:31写道：
>
> > [moved from private@nuttx.apache.org]
> >
> > Hi, Xiao Xiang and Haitao Liu
> >
> > > Haitao is preparing a script for style check, the feature include:
> > > 1.Auto build nxstyle
> > > 2.Improve nxstyle to check the partial file for supporting patch like
> > file
> > > 3.Input can be the source files, patch file or commit id
> >
> > I mentioned that I had two things I planned to do for nxstyle.c too.
> > Perhaps Haitao would like to take on these changes too?
> >
> > 1. Derive maximum line width from block comments.  Ref:
> >
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#fileorganization
> >
> >   * The Maximum line width should be the same as the width of the file
> >     section block comment, by default the last asterisk '*' lies on
> >     column 78, but wider lines are okay too, as long as things are
> >     scaled corrected.  Instead of the line width being a hardcoded 78 or
> >     specified the command line, nxstyle would parse the entire file,
> >     look at the block comments, find the column of the last '*'  The
> >     line width is that colument + 1;
> >   * Then also verify that every block comment in the file is the same
> width
> >   * Perhaps the -m argument could change to a amount that you will
> >     tolerate extending into the right margin.  So the block comment with
> >     is 78, then -m 8 would allow lines to extend to 86.  That is better
> >     than a hard-coded 86 because the line width will vary with file
> >     content.  For some register definition files, the line width may be
> >     120 or so.  In that case -m 8 would allow the line to extend to 128.
> >
> > 2. Verify that file section block comments are present.
> >
> >   * Add an enumeration value that indicates every block comment type.
> >     See
> >
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#cfilestructure
> >     and
> >
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#hfilestructure
> >   * Currently logic detects only the presence of "* Private Functions"
> >     and "* Private Functions".  Add logic to detect all other the block
> >     comment types as well.
> >   * Add a global variable to indicate which code block we are currently
> >     in.  Each time a file section block comment is found, set the global
> >     variable.
> >   * The if '#   define" is encountered outside of the "* Pre-processor
> >     Definitions" file section, emit an error
> >   * If '#  include" is encountered outside of the "* Included Files"
> >     section, emit an error.
> >   * This logic could eventually be extended to assure that all static
> >     functions are defined in the "* Private Functions" section, all
> >     global functions are defined in the "* Public Functions section",
> >     all static data definitions appear in the "* Private Data" section,
> >     and all global data definitions appear in the "* Public data
> >     definitions."
> >   * We could also enable special parsing depending on the section.  That
> >     is already done for the Public/Private Functions.  But we could, for
> >     example also verify that variables defined at global scope start
> >     with "g-"
> >
> > Let me know.  I was planning to do these things, but if Haitao Liu is
> > interested, he can take over full responsibility for nxstyle.c.
> >
> > Greg
> >
> >
> >
> >
> >
>

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

> In configure.sh, there are four wenv options for windows as below:
>      -c selects the Windows host and Cygwin (c) environment.
>      -u selects the Windows host and Ubuntu under Windows 10 (u) environment.
>      -g selects the Windows host and MinGW/MSYS environment.
>      -n selects the Windows host and Windows native (n) environment.
> Should we cover all of them in CI builds? I think  we could cover all of
> them
> if the infra could provides all the four windows enviroment.

There have been a lot of changes to the build system lately and I am 
sure that the Windows native build is not ready to use.  It will take 
some effort to use Windows native and there is, apparently, no users 
using the native build now.  There have been in the past and there will 
be in the future.  But I think we can wait on it.

You could omit Ubuntu under windows for now too.  It is just Ubuntu and 
really differs only in that you could be using a Windows native 
toolchain with Ubuntu.  Otherwise, it is the same as Ubuntu Linux so 
there is not very much to be gained in testing.

There is an issues of "return on investment" and "low hanging fruit."  
Return on investment refers to how much benefit the testing provides 
based on the effort required to the testing. Some platforms should be 
tested but have a low return on investment.  But other platforms oare 
"low hanging fruit", that is, they are very easy to test and give us a 
great testing benefit.  Linux is low hanging fruit.

Cygwin and MSYS2 are the primary windows platforms.  They are 
essentially the same.  In fact, MSYS2 derives from Cygwin but with 
change to make it work better in Windows.  For example, it understands 
Windows paths as well as POSIX paths.  The other big difference is that 
it does not support any kind of symbolic links and it is especially 
intolerant of spaces in paths.

>> It would be better to substitute FreeBSD for macOS if MacOS not available
> finally.
> I'll raise an Infra jira later to see if FreeBSD available in Apache Infra.
You might need to keep the initial implementation simple and add more 
platforms later.
> ... And we could figure out how long the full build time.
> According to the time, we could adjust build configs for PR check build if
> necessary. Daily build should cover all the target builds any way.

I have an old, early generation I5 running Mint Linux that I do build 
testing on.  It takes many hours to build 419 configurations, almost 6 
hours!  I should buy something faster!

It used to be a lot faster, but I upgraded the OS the toolchain recently 
and lost all of the performance.  No idea why.

Greg

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Gregory Nutt <sp...@gmail.com> 于2020年1月9日周四 上午11:55写道：

>
> > Thanks, I think we can firstly get start in Linux and Windows. Then
> > consider how to setup Mac later, donate slave or other.
> > Two other question, nuttx uses some cross compile toolchains etc. In
> > addition, Cygwin or MinGW is in need once nuttx built under Windows
> slaves.
> > Should we ask for Apache Infra to help install in slave machines?
>
> MinGW is MSYS2, right?  I use both Cygwin and MSYS2, they are a little
> different.
>
> You are right, just google MSYS2 is more flexible.Since I mainly built
under Ubuntu Linux, it may take more
effort to handle CI under Windows.
In configure.sh, there are four wenv options for windows as below:
    -c selects the Windows host and Cygwin (c) environment.
    -u selects the Windows host and Ubuntu under Windows 10 (u) environment.
    -g selects the Windows host and MinGW/MSYS environment.
    -n selects the Windows host and Windows native (n) environment.
Should we cover all of them in CI builds? I think  we could cover all of
them
if the infra could provides all the four windows enviroment.

There is usually a small number of people using some form of BSD, NetBSD
> or FreeBSD.  Most macOS build issues are due to the fact that macOS uses
> the BSD coreutils which are not 100% compatible with GNU coreutils.  I
> suspect you could substitute FreeBSD for macOS and also catch most if
> not all macOS build issues.
>
> In the past, there have been users on Solaris and HP-UX, but I don't
> think we need to go there now.
>
> It would be better to substitute FreeBSD for macOS if MacOS not available
finally.
I'll raise an Infra jira later to see if FreeBSD available in Apache Infra.

Which targets do you plan to build?  I routinely build all ARM targets
> (419 configurations and that omits several that are more difficult to
> automate), but we should also consider PIC32 and and Simulator.  PIC32
> would require a MIPS toolchain.  There are many other architectures but
> most do not warrant investing a lot of test effort.  Renesas RX65N,
> RISC-V, ESP32, AVR might be feasible because they also use GCC
> toolchains and do get used.  Some of the others do not use GCC or have
> much less use.
>
Our aim is to build as much as possible.  As you point out, all ARM
targets, PIC32 and Simulator
could be covered at first stage. Renesas RX65N,
RISC-V, ESP32, AVR could also be covered. And we could figure out how long
the full build time.
According to the time, we could adjust build configs for PR check build if
necessary. Daily build should
cover all the target builds any way.

>
> There have been IAR tools used with ARM in the past.  There have been
> requests for Mikroelektronika tool support in the past, but that was
> never implemented.  SDCC and ZiLOG tools are used for some
> architectures.  I think we can ignore all those for now.
>
> Greg
>
>
>
>
>

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Justin Mclean <ju...@classsoftware.com> 于2020年1月17日周五 上午5:19写道：

> Hi,
>
> I think working on a CI document together it a great idea.
>
>
I also agree with Miguel's idea  to working on a CI  document. So save and
inspire more best practices in CI.

> - What architectures we want to include: all of them? (list them) some of
> > them can be deferred until "Perfect CI"? do Apache Jenkins have the
> machine
> > for that architectures? if not ... are there emulators? and the
> toolchains?
> > options, pros/cons
>
> Personally I suggest getting it working with the most common platform and
> go from there. Having any CI is better than no CI.
>

That's right. Now we are starting from the common used linux platform. IMO,
The nightly build CI may be ready soon.


> > - What artifacts will be used for the Releases?
>
> None of the CI artefacts will be used for releases as per ASF requirements.
>
> > And sorry for the long email :)
>
> Thanks for you the long email.
>
> Justin

Re: Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

I think working on a CI document together it a great idea.

> - What architectures we want to include: all of them? (list them) some of
> them can be deferred until "Perfect CI"? do Apache Jenkins have the machine
> for that architectures? if not ... are there emulators? and the toolchains?
> options, pros/cons

Personally I suggest getting it working with the most common platform and go from there. Having any CI is better than no CI.

> - What artifacts will be used for the Releases?

None of the CI artefacts will be used for releases as per ASF requirements.

> And sorry for the long email :)

Thanks for you the long email.

Justin

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

TLDR: I think we need to make a "CI requirements and design" Draft Proposal
(https://cwiki.apache.org/confluence/display/NUTTX/Draft+Proposals)
---

Hi,

I think that in order to get a working CI as soon as possible we need to
please define a plan in a coordinated way.

I think "perfection" is the enemy of "good", so I propose to have two
stages:

  1st stage: Defining a "Good CI" to be ready ASAP, so we could include it
as a requirement for the also-to-be-defined workflow (CI should go in the
Criteria for acceptance of workflow
https://cwiki.apache.org/confluence/display/NUTTX/Code+Contribution+Workflow+--+Brennan+Ashton#CodeContributionWorkflow--BrennanAshton-CriteriaForAcceptance
).

  2nd stage: Defining a "Perfect CI" to be agree to future
nice-to-have-but-not-essential parts of the CI, that can be extended later
upon the basis of "Good CI".

So, from now on I will talk about "Good CI" exclusively.

I appreciate Haitao is preparing the scripts, but as being said, it should
not be the only one defining it. I don't know Haitao and probably nobody
knows me, so every decision is better done by the merits of the proposal
itself and not the person.

I think that for a "Good CI" we need to understand the needs for NuttX:
some feedback from the people that have worked all these years with NuttX
about how to test it and what to test to run to conclude if something has
quality or not.

So I would suggest to create a "Draft Proposal" (
https://cwiki.apache.org/confluence/display/NUTTX/Draft+Proposals) called
something like "CI requirements and design" where to develop that proposal.

In the proposal we should address what parts of the features of a CI we
want for the "Good CI" among the ones one could expect in a CI (for
example, from the ones described in
https://en.wikipedia.org/wiki/Continuous_integration#Common_practices) .
Despite being commong for CIs all over the world, we need to agree on a
minimum.

But more important even, as I said, is to know what is needed
_specifically_ for NuttX. For example:

 - What sanity checks are mandatory? what tools are we going to use to
enforce them?

 - What architectures we want to include: all of them? (list them) some of
them can be deferred until "Perfect CI"? do Apache Jenkins have the machine
for that architectures? if not ... are there emulators? and the toolchains?
options, pros/cons

 - What are the stages of a NuttX build? What are the artifacts generated?
can these artifacts be reused in different tests? how does that relate to
the number of jobs? is the job pipeline a line, or some artifacts are fed
to parallel jobs with different configurations to accelerate the testing?

 - Are we going to test the interface with the applications? Are the
applications being tested in the CI pipeline using end-to-end tests?

 - What is the general CI "pipeline/job graph"? What are the triggers? What
are the outputs?

 - What are the minimum level of logs to help debugging problems?

 - What artifacts will be used for the Releases?

I have little experience with NuttX still, so more experienced people need
to address these things.

This document would include little or code at all. Having this document
would be of great use, since we all could agree on a CI structure, even non
programmers, and then, Haitao, myself or any other could help on the
nitty-gritty details of translating that to specific CI (Apache infra
Jenkins by now).

Also, of course Haitao and others can continue to develop the
implementations in parallel. The more stones he remove from the path the
better.

And sorry for the long email :)

Miguel


On Thu, Jan 16, 2020 at 3:26 PM Gregory Nutt <sp...@gmail.com> wrote:

>
> > In addition, I notice http://www.nuttx.org/Documentation/NuttX.html say
> gnu
> > toolchains based on buildroot
> > for many architectures.
>
> There are configurations at
> https://bitbucket.org/nuttx/buildroot/src/master/configs/ for:
>
> avr-defconfig-4.3.3
> avr-defconfig-4.5.2
>
> I am not sure of the state of those, however.
>
> > So for AVR gcc toolchains, I just found
> >
> https://www.microchip.com/mplab/avr-support/avr-and-arm-toolchains-c-compilers
> > ,
> > but not right for nuttx. Do I need to build based on builtroot?
>
> I don't know about this, but the toolchain from AtmelStudio is quite nice.
>
> Notice that there is a build test for AVR (and MIPS) here:
> https://bitbucket.org/nuttx/tools/src/master/buildtest/ . If you look in
> doavr.sh, you an see that the test is configured to use the AtmelStudio
> toolchain.
>
> I am not certain if AVR will build without AtmelStudio.  I think it
> expects some resource from AtmelStudio, but I might be wrong.
>
> Greg
>
>
>
>
>

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

> Hi, Greg, may I ask where you find the right mips gcc toolchain?
> I haven't found an official prebuilt mips gcc toolchain yet(mips.com
> connect failed here). Or build it from source code.

I was talking with Abdelatif about this a few days ago.  He is the 
MIPS/PIC32 expert on the team.  We are both using the Pinguino toolchain 
but that is getting old and does not appear to be supported any more.  
So I don't think there  a future with those tools.

I think that Abdelatif and I both agreed that using the NuttX buildroot 
MIPS toolchain is the best option. 
https://bitbucket.org/nuttx/buildroot/src/master/  it does support MIPs 
tool builds with current GCC but would probably need some verification.  
I don't see any existing MIPs configurations.

Greg

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

On Fri, Jan 31, 2020 at 4:30 PM Gregory Nutt <sp...@gmail.com> wrote:

>
> Perhaps Miguel could provide the review of PR3.  I will be happy to
> finish the merge after some review is provided.
>
>
I can review the script, but maybe mostly generic (not too specific to
NuttX though).


> Once it has been merged, both you and Miguel can continue the
> development with PRs.  But we need to find someone competent to provide
> review.
>
> Greg
>

Maybe @yamt (see
https://github.com/apache/incubator-nuttx/pull/183/commits/bc07eb680e09db3d07ddf0ab1da655661c066bef)
can take a look also, as he created that config for CircleCI (and maybe
integrate their changes later in Jenkins too).

I will try to give a review asap.

Cheers,
Miguel

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

Thanks for your summary. Some comments:

On Thu, Feb 13, 2020 at 5:16 AM Haitao Liu <li...@gmail.com> wrote:

>                                      Apache Jenkins
>                            |    Github action
>
> ============================================================================================================================
> ...
> Restrict accounts access to update jenkins configs
>        |     Free to Make PRs to update githubaction workflow
>

That is not exactly true: accounts access is restricted to create the
_Jenkins jobs_. The logic executed in these jobs, once created, is stored
in a Jenkinsfile or in Groovy pipeline script, so the workflow can be
updated by a PR as in Github. Moreover, if using shared libraries, the
logic can reside in another repository, as Greg requested. I don't know if
that is possible with Github.


> So, I think we could consider to use apache jenkins ci as nightly build
> and use github action ci as PR check build accordingly.
>

I agree with that, but mostly due to apparently saturation of AFS CI
service.

1. Apache Jenkins CI for Nightly Build
>
> There is an existing freestyle NuttX Nightly Build do cron job at
> midnight each day. However, it's very simple. And there is still flock
> issue to resolve to make parallel build all pass.
>
>
> https://builds.apache.org/view/Incubator%20Projects/job/NuttX-Nightly-Build/
>
>  Thanks to @Maht, he made multibranch jenkins pipeline PR which would make
> the Nightly build more robust. But it may still need some time to switch to
> it because of Apache Jenkins access control. The PRs as below:
>
> https://github.com/apache/incubator-nuttx-testing/pull/2
>
> https://github.com/apache/incubator-nuttx/pull/174
>
>
Well, my multibranch proposal is originally intended to manage PR triggers
(each PR is a branch). For nightly, as long there is only one branch now
(master), there is less added value. Still, the pipeline logic could be
modified to be more "parallel" (to be done, though).

Also, there is still travis CI pull request form @yamt, we could integrate
> it into incubator-nuttx-testing as maht do.
>
> https://github.com/apache/incubator-nuttx/pull/183
>
> <
> https://builds.apache.org/view/Incubator%20Projects/job/NuttX-Nightly-Build/
> >
>

Yes, I think the idea of having the CI/testing stuff in a separate repo is
good, as Greg requested, despite maybe some minimal file info has to be
kept in the main repo. But I won't consider it a strong requirement for the
short term if achieving it would take a lot of effort to implement outside
of the typical usage of Github Actions/CircleCI/TravisCI/etc.

2. Github action CI for Check Build
>
> I have just make two PRs to incubator-nuttx and inucbator-nuttx-apps
> accordingly as below:
>
> https://github.com/apache/incubator-nuttx/pull/261
>
> https://github.com/apache/incubator-nuttx-apps/pull/65
>
> The two PRs are not intended to be merged right now. They still need align
> to check testlist update under incubator-nuttx-testing.
>
> For example, Consider have a minimal check testlist set that always run.
>
> As for github action check build CI, there are still more space to
> optimize:
>
> 1. Matrix builds to allow concurrent jobs and even multi-platorms
>
> To speed up the check build time, use multiple jobs in github action
> workflow to do parallel builds as @btashton and @davids suggest.
>

That would be a very nice addition, but I am not sure how to do it Github
Actions, and have no much time this week to explore.


> 2. Use docker images with tools preinstalled
>
> Now it takes several minitues to install all essential tools in ubuntu vm.
>
> we may build nuttx with docker images which tools preinstalled in the
> future.
>

 @davids suggested to use PX4 Docker images, and I think they are very good
to use/reference: https://hub.docker.com/r/px4io/px4-dev-nuttx


> Anyway, These are just  some thoughts from our previous discussions,
> welcome more reviews and feedbacks from community.
>  Thanks!
>

Thanks for the effort and count on me if you need some feedback.

Cheers,
Miguel

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

After practise with Apache Jenkins CI and Github action CI for a time and
do some CI problems search in Apache maillist and google, let's sum up as
below:

                                     Apache Jenkins
                                                   |    Github action

============================================================================================================================

Provide Linux/Windows real Machines, but MacOS not available
                   |     Provide Linux/MacOS/Windows VM

 Apache jenkins slaves are limited and shared with other ASF projects
                    |     The computing power can be accessed for each user
at first time

 Github PRs may wait nearly hours to be scheduled, for nightly build it
works well         |     Integrate with the github workflow smoothly

Restrict accounts access to update jenkins configs
                               |     Free to Make PRs to update github
action workflow

More info on Jenkins and Github action for reference:

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/about-github-actions#usage-limits

https://knapsackpro.com/ci_comparisons/jenkins/vs/github-actions // sth
maybe obsolete for github action

So, I think we could consider to use apache jenkins ci as nightly build
and use github action ci as PR check build accordingly.

1. Apache Jenkins CI for Nightly Build

There is an existing freestyle NuttX Nightly Build do cron job at
midnight each day. However, it's very simple. And there is still flock
issue to resolve to make parallel build all pass.

https://builds.apache.org/view/Incubator%20Projects/job/NuttX-Nightly-Build/

 Thanks to @Maht, he made multibranch jenkins pipeline PR which would make
the Nightly build more robust. But it may still need some time to switch to
it because of Apache Jenkins access control. The PRs as below:

https://github.com/apache/incubator-nuttx-testing/pull/2

https://github.com/apache/incubator-nuttx/pull/174

Also, there is still travis CI pull request form @yamt, we could integrate
it into incubator-nuttx-testing as maht do.

https://github.com/apache/incubator-nuttx/pull/183

<https://builds.apache.org/view/Incubator%20Projects/job/NuttX-Nightly-Build/>

2. Github action CI for Check Build

I have just make two PRs to incubator-nuttx and inucbator-nuttx-apps
accordingly as below:

https://github.com/apache/incubator-nuttx/pull/261

https://github.com/apache/incubator-nuttx-apps/pull/65

The two PRs are not intended to be merged right now. They still need align
to check testlist update under incubator-nuttx-testing.

For example, Consider have a minimal check testlist set that always run.

As for github action check build CI, there are still more space to optimize:

1. Matrix builds to allow concurrent jobs and even multi-platorms

To speed up the check build time, use multiple jobs in github action
workflow to do parallel builds as @btashton and @davids suggest.

2. Use docker images with tools preinstalled

Now it takes several minitues to install all essential tools in ubuntu vm.

we may build nuttx with docker images which tools preinstalled in the
future.

Anyway, These are just  some thoughts from our previous discussions,
welcome more reviews and feedbacks from community.
 Thanks!


Miguel Ángel Herranz <mi...@midokura.com.invalid> 于2020年2月4日周二 下午7:23写道：

> Hi Haitao,
>
> I will take a look.
>
> By the way, do Apache Jenkins slave includes Docker? Or alternatively, does
> Apache Jenkins have Docker Slave [1] plugin available?
>
> If we could use either launch Docker containers in the slave or launch the
> jobs in a Docker container we could use an image with all the tools pre
> installed.
>
> Please someone with read privileges on Jenkins check that whenever possible
> (Xiang? Justin?).
>
> Thanks,
> Miguel
>
> [1] https://plugins.jenkins.io/docker-slaves
>
> On Tue, Feb 4, 2020 at 12:04 PM Haitao Liu <li...@gmail.com> wrote:
>
> > Thanks Xiang!
> > Another PR https://github.com/apache/incubator-nuttx-testing/pull/4
> since
> > Apache jenkins slave haven't gperf preinstalled,
> > so switch to use our owned prebuilt gperf. Xiang and Miguel please help
> to
> > review.
> > Or in another way, may I ask Justin Apache Infra team to apt-get install
> > gperf instead?
> >
> > Xiang Xiao <xi...@gmail.com> 于2020年2月4日周二 下午2:36写道：
> >
> > > I will merge it.
> > >
> > > On Tue, Feb 4, 2020 at 12:50 PM Haitao Liu <li...@gmail.com> wrote:
> > > >
> > > > Hi, Greg,
> > > > Thanks to Miguel's careful review, could you please help to finish
> the
> > PR
> > > > merge?
> > > > Then I and Miguel can continue the development with more PRs and
> > > refinement.
> > > >
> > > > Gregory Nutt <sp...@gmail.com> 于2020年1月31日周五 下午11:30写道：
> > > >
> > > > >
> > > > > > I have just added the CI scripts and testlist files in
> > > > > > https://github.com/apache/incubator-nuttx-testing/pull/3
> > > > > > And asked XiaoXiang help to create Nuttx Nightly build job in
> > > > > > builds.apache.org before. But it's just a freestyle CI job
> > > > > > which call cibuild.sh simply. Once CI script and testlist file
> > verify
> > > > > well.
> > > > > > It would be better if it could be integrated into the
> > > > > > pipeline definition you mentioned. You could help to use the
> > pipeline
> > > > > > way
> > > > > > if access granted. Let's work together on it :)
> > > > >
> > > > > Perhaps Miguel could provide the review of PR3.  I will be happy to
> > > > > finish the merge after some review is provided.
> > > > >
> > > > > Once it has been merged, both you and Miguel can continue the
> > > > > development with PRs.  But we need to find someone competent to
> > provide
> > > > > review.
> > > > >
> > > > > Greg
> > > > >
> > > > >
> > > > >
> > >
> >
>

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

Hi Haitao,

I will take a look.

By the way, do Apache Jenkins slave includes Docker? Or alternatively, does
Apache Jenkins have Docker Slave [1] plugin available?

If we could use either launch Docker containers in the slave or launch the
jobs in a Docker container we could use an image with all the tools pre
installed.

Please someone with read privileges on Jenkins check that whenever possible
(Xiang? Justin?).

Thanks,
Miguel

[1] https://plugins.jenkins.io/docker-slaves

On Tue, Feb 4, 2020 at 12:04 PM Haitao Liu <li...@gmail.com> wrote:

> Thanks Xiang!
> Another PR https://github.com/apache/incubator-nuttx-testing/pull/4 since
> Apache jenkins slave haven't gperf preinstalled,
> so switch to use our owned prebuilt gperf. Xiang and Miguel please help to
> review.
> Or in another way, may I ask Justin Apache Infra team to apt-get install
> gperf instead?
>
> Xiang Xiao <xi...@gmail.com> 于2020年2月4日周二 下午2:36写道：
>
> > I will merge it.
> >
> > On Tue, Feb 4, 2020 at 12:50 PM Haitao Liu <li...@gmail.com> wrote:
> > >
> > > Hi, Greg,
> > > Thanks to Miguel's careful review, could you please help to finish the
> PR
> > > merge?
> > > Then I and Miguel can continue the development with more PRs and
> > refinement.
> > >
> > > Gregory Nutt <sp...@gmail.com> 于2020年1月31日周五 下午11:30写道：
> > >
> > > >
> > > > > I have just added the CI scripts and testlist files in
> > > > > https://github.com/apache/incubator-nuttx-testing/pull/3
> > > > > And asked XiaoXiang help to create Nuttx Nightly build job in
> > > > > builds.apache.org before. But it's just a freestyle CI job
> > > > > which call cibuild.sh simply. Once CI script and testlist file
> verify
> > > > well.
> > > > > It would be better if it could be integrated into the
> > > > > pipeline definition you mentioned. You could help to use the
> pipeline
> > > > > way
> > > > > if access granted. Let's work together on it :)
> > > >
> > > > Perhaps Miguel could provide the review of PR3.  I will be happy to
> > > > finish the merge after some review is provided.
> > > >
> > > > Once it has been merged, both you and Miguel can continue the
> > > > development with PRs.  But we need to find someone competent to
> provide
> > > > review.
> > > >
> > > > Greg
> > > >
> > > >
> > > >
> >
>

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Thanks Xiang!
Another PR https://github.com/apache/incubator-nuttx-testing/pull/4 since
Apache jenkins slave haven't gperf preinstalled,
so switch to use our owned prebuilt gperf. Xiang and Miguel please help to
review.
Or in another way, may I ask Justin Apache Infra team to apt-get install
gperf instead?

Xiang Xiao <xi...@gmail.com> 于2020年2月4日周二 下午2:36写道：

> I will merge it.
>
> On Tue, Feb 4, 2020 at 12:50 PM Haitao Liu <li...@gmail.com> wrote:
> >
> > Hi, Greg,
> > Thanks to Miguel's careful review, could you please help to finish the PR
> > merge?
> > Then I and Miguel can continue the development with more PRs and
> refinement.
> >
> > Gregory Nutt <sp...@gmail.com> 于2020年1月31日周五 下午11:30写道：
> >
> > >
> > > > I have just added the CI scripts and testlist files in
> > > > https://github.com/apache/incubator-nuttx-testing/pull/3
> > > > And asked XiaoXiang help to create Nuttx Nightly build job in
> > > > builds.apache.org before. But it's just a freestyle CI job
> > > > which call cibuild.sh simply. Once CI script and testlist file verify
> > > well.
> > > > It would be better if it could be integrated into the
> > > > pipeline definition you mentioned. You could help to use the pipeline
> > > > way
> > > > if access granted. Let's work together on it :)
> > >
> > > Perhaps Miguel could provide the review of PR3.  I will be happy to
> > > finish the merge after some review is provided.
> > >
> > > Once it has been merged, both you and Miguel can continue the
> > > development with PRs.  But we need to find someone competent to provide
> > > review.
> > >
> > > Greg
> > >
> > >
> > >
>

Re: Jenkins CI

[Xiang Xiao <xi...@gmail.com>]

I will merge it.

On Tue, Feb 4, 2020 at 12:50 PM Haitao Liu <li...@gmail.com> wrote:
>
> Hi, Greg,
> Thanks to Miguel's careful review, could you please help to finish the PR
> merge?
> Then I and Miguel can continue the development with more PRs and refinement.
>
> Gregory Nutt <sp...@gmail.com> 于2020年1月31日周五 下午11:30写道：
>
> >
> > > I have just added the CI scripts and testlist files in
> > > https://github.com/apache/incubator-nuttx-testing/pull/3
> > > And asked XiaoXiang help to create Nuttx Nightly build job in
> > > builds.apache.org before. But it's just a freestyle CI job
> > > which call cibuild.sh simply. Once CI script and testlist file verify
> > well.
> > > It would be better if it could be integrated into the
> > > pipeline definition you mentioned. You could help to use the pipeline
> > > way
> > > if access granted. Let's work together on it :)
> >
> > Perhaps Miguel could provide the review of PR3.  I will be happy to
> > finish the merge after some review is provided.
> >
> > Once it has been merged, both you and Miguel can continue the
> > development with PRs.  But we need to find someone competent to provide
> > review.
> >
> > Greg
> >
> >
> >

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Hi, Greg,
Thanks to Miguel's careful review, could you please help to finish the PR
merge?
Then I and Miguel can continue the development with more PRs and refinement.

Gregory Nutt <sp...@gmail.com> 于2020年1月31日周五 下午11:30写道：

>
> > I have just added the CI scripts and testlist files in
> > https://github.com/apache/incubator-nuttx-testing/pull/3
> > And asked XiaoXiang help to create Nuttx Nightly build job in
> > builds.apache.org before. But it's just a freestyle CI job
> > which call cibuild.sh simply. Once CI script and testlist file verify
> well.
> > It would be better if it could be integrated into the
> > pipeline definition you mentioned. You could help to use the pipeline way
> > if access granted. Let's work together on it :)
>
> Perhaps Miguel could provide the review of PR3.  I will be happy to
> finish the merge after some review is provided.
>
> Once it has been merged, both you and Miguel can continue the
> development with PRs.  But we need to find someone competent to provide
> review.
>
> Greg
>
>
>

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

> I have just added the CI scripts and testlist files in
> https://github.com/apache/incubator-nuttx-testing/pull/3
> And asked XiaoXiang help to create Nuttx Nightly build job in
> builds.apache.org before. But it's just a freestyle CI job
> which call cibuild.sh simply. Once CI script and testlist file verify well.
> It would be better if it could be integrated into the
> pipeline definition you mentioned. You could help to use the pipeline way
> if access granted. Let's work together on it :)

Perhaps Miguel could provide the review of PR3.  I will be happy to 
finish the merge after some review is provided.

Once it has been merged, both you and Miguel can continue the 
development with PRs.  But we need to find someone competent to provide 
review.

Greg

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

On Fri, Jan 31, 2020 at 4:17 PM Haitao Liu <li...@gmail.com> wrote:

> Hi, Miguel,
>
> I have just added the CI scripts and testlist files in
> https://github.com/apache/incubator-nuttx-testing/pull/3
> And asked XiaoXiang help to create Nuttx Nightly build job in
> builds.apache.org before. But it's just a freestyle CI job
> which call cibuild.sh simply. Once CI script and testlist file verify well.
> It would be better if it could be integrated into the
> pipeline definition you mentioned. You could help to use the pipeline way
> if access granted.
>

Hi Haitao,

yes, I just show it I would like to take a deeper look but I can't right
now, maybe late today.

I will review it and modify the pipeline job definition to call your script
(basically call it from vars/runContinuousIntegrationPipeline.groovy
<https://github.com/apache/incubator-nuttx-testing/pull/2/files#diff-7e5fc8ac62e6e1af7759777be1ca7a60>
).

Once we check it works ok XiaoXiang can manually create the multibranch
pipeline or using the scripts and XML job definition provided in the same
pull request.

| Let's work together on it :)

Apache way :)

Cheers,
/Miguel



>
>
> Miguel Ángel Herranz <mi...@midokura.com.invalid> 于2020年1月29日周三
> 上午12:56写道：
>
> > Hi,
> >
> > after some research about new releases of Jenkins, it seems that my
> > proposal can be improved towards a simpler setup, so I modified it.
> >
> > I have included some code as pull requests that could be useful to
> discuss
> > upon: https://github.com/apache/incubator-nuttx-testing/pull/2 and
> > https://github.com/apache/incubator-nuttx/pull/174
> >
> > The actual logic from Haitao scripts can be used inside the pipeline
> > definition, so we could have a mostly functional "Github pull request
> based
> > workflow", once a comitter with enough rights could create the jobs in
> > builds.apache.org (I am still waiting confirmation for the CCLA, sorry,
> > and
> > anyway the access is not granted).
> >
> > Cheers,
> > Miguel
> >
> >
> >
> >
> > On Tue, Jan 21, 2020 at 1:11 PM Miguel Ángel Herranz <
> miguel@midokura.com>
> > wrote:
> >
> > >
> > > > We probably can get infra to add you, submitting an ICLA might help
> as
> > > that could be used to create an account.
> > >
> > > Thanks, good to know. I am asking my employer about including me in a
> > > corporate one (CCLA [1]), but if it is too much paperwork I guess I can
> > > just submit the ICLA. Let's see.
> > >
> > > [1] https://www.apache.org/licenses/contributor-agreements.html
> > >
> > > On Tue, Jan 21, 2020 at 2:01 AM Justin Mclean
> > <ju...@me.com.invalid>
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >> > Oh, I did check that wiki page but I was not aware that it was a
> > >> > requirement to be a comitter or PPMC member to have an LDAP account,
> > >> but it
> > >> > makes sense, thanks anyway.
> > >>
> > >> We probably can get infra to add you, submitting an ICLA might help as
> > >> that could be used to create an account.
> > >>
> > >> Thanks,
> > >> Justin
> > >>
> > >>
> > >>
> >
>

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Hi, Miguel,

I have just added the CI scripts and testlist files in
https://github.com/apache/incubator-nuttx-testing/pull/3
And asked XiaoXiang help to create Nuttx Nightly build job in
builds.apache.org before. But it's just a freestyle CI job
which call cibuild.sh simply. Once CI script and testlist file verify well.
It would be better if it could be integrated into the
pipeline definition you mentioned. You could help to use the pipeline way
if access granted. Let's work together on it :)


Miguel Ángel Herranz <mi...@midokura.com.invalid> 于2020年1月29日周三 上午12:56写道：

> Hi,
>
> after some research about new releases of Jenkins, it seems that my
> proposal can be improved towards a simpler setup, so I modified it.
>
> I have included some code as pull requests that could be useful to discuss
> upon: https://github.com/apache/incubator-nuttx-testing/pull/2 and
> https://github.com/apache/incubator-nuttx/pull/174
>
> The actual logic from Haitao scripts can be used inside the pipeline
> definition, so we could have a mostly functional "Github pull request based
> workflow", once a comitter with enough rights could create the jobs in
> builds.apache.org (I am still waiting confirmation for the CCLA, sorry,
> and
> anyway the access is not granted).
>
> Cheers,
> Miguel
>
>
>
>
> On Tue, Jan 21, 2020 at 1:11 PM Miguel Ángel Herranz <mi...@midokura.com>
> wrote:
>
> >
> > > We probably can get infra to add you, submitting an ICLA might help as
> > that could be used to create an account.
> >
> > Thanks, good to know. I am asking my employer about including me in a
> > corporate one (CCLA [1]), but if it is too much paperwork I guess I can
> > just submit the ICLA. Let's see.
> >
> > [1] https://www.apache.org/licenses/contributor-agreements.html
> >
> > On Tue, Jan 21, 2020 at 2:01 AM Justin Mclean
> <ju...@me.com.invalid>
> > wrote:
> >
> >> Hi,
> >>
> >> > Oh, I did check that wiki page but I was not aware that it was a
> >> > requirement to be a comitter or PPMC member to have an LDAP account,
> >> but it
> >> > makes sense, thanks anyway.
> >>
> >> We probably can get infra to add you, submitting an ICLA might help as
> >> that could be used to create an account.
> >>
> >> Thanks,
> >> Justin
> >>
> >>
> >>
>

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

Hi,

after some research about new releases of Jenkins, it seems that my
proposal can be improved towards a simpler setup, so I modified it.

I have included some code as pull requests that could be useful to discuss
upon: https://github.com/apache/incubator-nuttx-testing/pull/2 and
https://github.com/apache/incubator-nuttx/pull/174

The actual logic from Haitao scripts can be used inside the pipeline
definition, so we could have a mostly functional "Github pull request based
workflow", once a comitter with enough rights could create the jobs in
builds.apache.org (I am still waiting confirmation for the CCLA, sorry, and
anyway the access is not granted).

Cheers,
Miguel




On Tue, Jan 21, 2020 at 1:11 PM Miguel Ángel Herranz <mi...@midokura.com>
wrote:

>
> > We probably can get infra to add you, submitting an ICLA might help as
> that could be used to create an account.
>
> Thanks, good to know. I am asking my employer about including me in a
> corporate one (CCLA [1]), but if it is too much paperwork I guess I can
> just submit the ICLA. Let's see.
>
> [1] https://www.apache.org/licenses/contributor-agreements.html
>
> On Tue, Jan 21, 2020 at 2:01 AM Justin Mclean <ju...@me.com.invalid>
> wrote:
>
>> Hi,
>>
>> > Oh, I did check that wiki page but I was not aware that it was a
>> > requirement to be a comitter or PPMC member to have an LDAP account,
>> but it
>> > makes sense, thanks anyway.
>>
>> We probably can get infra to add you, submitting an ICLA might help as
>> that could be used to create an account.
>>
>> Thanks,
>> Justin
>>
>>
>>

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

On Tue, Jan 28, 2020 at 8:58 PM Justin Mclean <ju...@classsoftware.com>
wrote:
...

> You would need to submit an ICLA. The CCLA is for more for your employer
> benefit not ours.


Hi Justin, sorry for the delay in answering.

In that case there is no need to delay I guess. I have just sent the ICLA
and waiting for confirmation about it.

Thanks,
/Miguel

Re: Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

> Thanks, good to know. I am asking my employer about including me in a
> corporate one (CCLA [1]), but if it is too much paperwork I guess I can
> just submit the ICLA. 

You would need to submit an ICLA. The CCLA is for more for your employer benefit not ours.

Thanks,
Justin

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

> We probably can get infra to add you, submitting an ICLA might help as
that could be used to create an account.

Thanks, good to know. I am asking my employer about including me in a
corporate one (CCLA [1]), but if it is too much paperwork I guess I can
just submit the ICLA. Let's see.

[1] https://www.apache.org/licenses/contributor-agreements.html

On Tue, Jan 21, 2020 at 2:01 AM Justin Mclean <ju...@me.com.invalid>
wrote:

> Hi,
>
> > Oh, I did check that wiki page but I was not aware that it was a
> > requirement to be a comitter or PPMC member to have an LDAP account, but
> it
> > makes sense, thanks anyway.
>
> We probably can get infra to add you, submitting an ICLA might help as
> that could be used to create an account.
>
> Thanks,
> Justin
>
>
>

Re: Jenkins CI

[Justin Mclean <ju...@me.com.INVALID>]

Hi,

> Oh, I did check that wiki page but I was not aware that it was a
> requirement to be a comitter or PPMC member to have an LDAP account, but it
> makes sense, thanks anyway.

We probably can get infra to add you, submitting an ICLA might help as that could be used to create an account.

Thanks,
Justin

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

> I can give people Jenkin admin access but they need to be in LDAP. [1]
that means you need to be a committer or PPMC member.

Oh, I did check that wiki page but I was not aware that it was a
requirement to be a comitter or PPMC member to have an LDAP account, but it
makes sense, thanks anyway.

In that case I will just try to set up a local Jenkins server for testing,
using plugins usually available by default and doing a pull request against
incubator-nuttx-testing so other people can review or even adapt it and try
in the real server.

Re: Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

> Also, a request: maybe it would be good to test if Jenkins in
> builds.apache.org support pipelines or any Jenkins plugin is missing, so if
> I could get the rights to access create a proof-of-concept pipeline to
> check everything is available.

I can give people Jenkin admin access but they need to be in LDAP. [1] that means you need to be a committer or PPMC member.

Thanks,
Justin

1. https://cwiki.apache.org/confluence/display/INFRA/Jenkins#Jenkins-HowdoIgetanaccount?

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

> You should have permission now.

Thanks Justin.

I have created a draft proposal (postfixing my name like the other one by
Brennan Ashton, in case others want to add another proposal from scratch,
but contributions are welcome):

https://cwiki.apache.org/confluence/display/NUTTX/Continuous+integration+--+Miguel+Herranz

It is incomplete still, of course, expecting some feedback.

Also, a request: maybe it would be good to test if Jenkins in
builds.apache.org support pipelines or any Jenkins plugin is missing, so if
I could get the rights to access create a proof-of-concept pipeline to
check everything is available.

Cheers,
Miguel

Re: Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

> so good you liked the idea. I tried to create the page in Confluence, but
> it seems I don't have enough rights.

You should have permission now.

Kind Regards,
Justin

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

Hi Justin and Haitao,

so good you liked the idea. I tried to create the page in Confluence, but
it seems I don't have enough rights.

Can someone else give me the needed rights to add it or create and give
edit permissions? My Confluence user in cwiki.apache.org is 'maht'

Cheers,
Miguel

On Fri, Jan 17, 2020 at 4:40 AM Haitao Liu <li...@gmail.com> wrote:

> Thanks Greg and  Abdelatif. I am going to use the Pinguino toolchain at
> first to let it work, and try the buildroot way later with free time.
>
> Gregory Nutt <sp...@gmail.com> 于2020年1月16日周四 下午10:26写道：
>
> >
> > > In addition, I notice http://www.nuttx.org/Documentation/NuttX.html
> say
> > gnu
> > > toolchains based on buildroot
> > > for many architectures.
> >
> > There are configurations at
> > https://bitbucket.org/nuttx/buildroot/src/master/configs/ for:
> >
> > avr-defconfig-4.3.3
> > avr-defconfig-4.5.2
> >
> > I am not sure of the state of those, however.
> >
> > > So for AVR gcc toolchains, I just found
> > >
> >
> https://www.microchip.com/mplab/avr-support/avr-and-arm-toolchains-c-compilers
> > > ,
> > > but not right for nuttx. Do I need to build based on builtroot?
> >
> > I don't know about this, but the toolchain from AtmelStudio is quite
> nice.
> >
> > Notice that there is a build test for AVR (and MIPS) here:
> > https://bitbucket.org/nuttx/tools/src/master/buildtest/ . If you look in
> > doavr.sh, you an see that the test is configured to use the AtmelStudio
> > toolchain.
> >
> > I am not certain if AVR will build without AtmelStudio.  I think it
> > expects some resource from AtmelStudio, but I might be wrong.
> >
> > Greg
> >
> >
> >
> >
> >
>

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Thanks Greg and  Abdelatif. I am going to use the Pinguino toolchain at
first to let it work, and try the buildroot way later with free time.

Gregory Nutt <sp...@gmail.com> 于2020年1月16日周四 下午10:26写道：

>
> > In addition, I notice http://www.nuttx.org/Documentation/NuttX.html say
> gnu
> > toolchains based on buildroot
> > for many architectures.
>
> There are configurations at
> https://bitbucket.org/nuttx/buildroot/src/master/configs/ for:
>
> avr-defconfig-4.3.3
> avr-defconfig-4.5.2
>
> I am not sure of the state of those, however.
>
> > So for AVR gcc toolchains, I just found
> >
> https://www.microchip.com/mplab/avr-support/avr-and-arm-toolchains-c-compilers
> > ,
> > but not right for nuttx. Do I need to build based on builtroot?
>
> I don't know about this, but the toolchain from AtmelStudio is quite nice.
>
> Notice that there is a build test for AVR (and MIPS) here:
> https://bitbucket.org/nuttx/tools/src/master/buildtest/ . If you look in
> doavr.sh, you an see that the test is configured to use the AtmelStudio
> toolchain.
>
> I am not certain if AVR will build without AtmelStudio.  I think it
> expects some resource from AtmelStudio, but I might be wrong.
>
> Greg
>
>
>
>
>

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

> In addition, I notice http://www.nuttx.org/Documentation/NuttX.html say gnu
> toolchains based on buildroot
> for many architectures.

There are configurations at 
https://bitbucket.org/nuttx/buildroot/src/master/configs/ for:

avr-defconfig-4.3.3
avr-defconfig-4.5.2

I am not sure of the state of those, however.

> So for AVR gcc toolchains, I just found
> https://www.microchip.com/mplab/avr-support/avr-and-arm-toolchains-c-compilers
> ,
> but not right for nuttx. Do I need to build based on builtroot?

I don't know about this, but the toolchain from AtmelStudio is quite nice.

Notice that there is a build test for AVR (and MIPS) here: 
https://bitbucket.org/nuttx/tools/src/master/buildtest/ . If you look in 
doavr.sh, you an see that the test is configured to use the AtmelStudio 
toolchain.

I am not certain if AVR will build without AtmelStudio.  I think it 
expects some resource from AtmelStudio, but I might be wrong.

Greg

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Thanks very much, It really works after correct some mips-elf-gcc/ld etc
symlinks.
In addition, I notice http://www.nuttx.org/Documentation/NuttX.html say gnu
toolchains based on buildroot
for many architectures. So for AVR gcc toolchains, I just found
https://www.microchip.com/mplab/avr-support/avr-and-arm-toolchains-c-compilers
,
but not right for nuttx. Do I need to build based on builtroot?


Abdelatif Guettouche <ab...@gmail.com> 于2020年1月16日周四
下午8:57写道：

> Hi Haitao,
>
> You can use this https://github.com/PinguinoIDE/pinguino-compilers
> Even though it hasn't been updated for years, it does the job.
>
> On Thu, Jan 16, 2020 at 1:46 PM Haitao Liu <li...@gmail.com> wrote:
> >
> > Hi, Greg, may I ask where you find the right mips gcc toolchain?
> > I haven't found an official prebuilt mips gcc toolchain yet(mips.com
> > connect failed here). Or build it from source code.
> >
> > Gregory Nutt <sp...@gmail.com> 于2020年1月9日周四 上午11:55写道：
> >
> > >
> > > > Thanks, I think we can firstly get start in Linux and Windows. Then
> > > > consider how to setup Mac later, donate slave or other.
> > > > Two other question, nuttx uses some cross compile toolchains etc. In
> > > > addition, Cygwin or MinGW is in need once nuttx built under Windows
> > > slaves.
> > > > Should we ask for Apache Infra to help install in slave machines?
> > >
> > > MinGW is MSYS2, right?  I use both Cygwin and MSYS2, they are a little
> > > different.
> > >
> > > There is usually a small number of people using some form of BSD,
> NetBSD
> > > or FreeBSD.  Most macOS build issues are due to the fact that macOS
> uses
> > > the BSD coreutils which are not 100% compatible with GNU coreutils.  I
> > > suspect you could substitute FreeBSD for macOS and also catch most if
> > > not all macOS build issues.
> > >
> > > In the past, there have been users on Solaris and HP-UX, but I don't
> > > think we need to go there now.
> > >
> > > Which targets do you plan to build?  I routinely build all ARM targets
> > > (419 configurations and that omits several that are more difficult to
> > > automate), but we should also consider PIC32 and and Simulator.  PIC32
> > > would require a MIPS toolchain.  There are many other architectures but
> > > most do not warrant investing a lot of test effort.  Renesas RX65N,
> > > RISC-V, ESP32, AVR might be feasible because they also use GCC
> > > toolchains and do get used.  Some of the others do not use GCC or have
> > > much less use.
> > >
> > > There have been IAR tools used with ARM in the past.  There have been
> > > requests for Mikroelektronika tool support in the past, but that was
> > > never implemented.  SDCC and ZiLOG tools are used for some
> > > architectures.  I think we can ignore all those for now.
> > >
> > > Greg
> > >
> > >
> > >
> > >
> > >
>

Re: Jenkins CI

[Abdelatif Guettouche <ab...@gmail.com>]

Hi Haitao,

You can use this https://github.com/PinguinoIDE/pinguino-compilers
Even though it hasn't been updated for years, it does the job.

On Thu, Jan 16, 2020 at 1:46 PM Haitao Liu <li...@gmail.com> wrote:
>
> Hi, Greg, may I ask where you find the right mips gcc toolchain?
> I haven't found an official prebuilt mips gcc toolchain yet(mips.com
> connect failed here). Or build it from source code.
>
> Gregory Nutt <sp...@gmail.com> 于2020年1月9日周四 上午11:55写道：
>
> >
> > > Thanks, I think we can firstly get start in Linux and Windows. Then
> > > consider how to setup Mac later, donate slave or other.
> > > Two other question, nuttx uses some cross compile toolchains etc. In
> > > addition, Cygwin or MinGW is in need once nuttx built under Windows
> > slaves.
> > > Should we ask for Apache Infra to help install in slave machines?
> >
> > MinGW is MSYS2, right?  I use both Cygwin and MSYS2, they are a little
> > different.
> >
> > There is usually a small number of people using some form of BSD, NetBSD
> > or FreeBSD.  Most macOS build issues are due to the fact that macOS uses
> > the BSD coreutils which are not 100% compatible with GNU coreutils.  I
> > suspect you could substitute FreeBSD for macOS and also catch most if
> > not all macOS build issues.
> >
> > In the past, there have been users on Solaris and HP-UX, but I don't
> > think we need to go there now.
> >
> > Which targets do you plan to build?  I routinely build all ARM targets
> > (419 configurations and that omits several that are more difficult to
> > automate), but we should also consider PIC32 and and Simulator.  PIC32
> > would require a MIPS toolchain.  There are many other architectures but
> > most do not warrant investing a lot of test effort.  Renesas RX65N,
> > RISC-V, ESP32, AVR might be feasible because they also use GCC
> > toolchains and do get used.  Some of the others do not use GCC or have
> > much less use.
> >
> > There have been IAR tools used with ARM in the past.  There have been
> > requests for Mikroelektronika tool support in the past, but that was
> > never implemented.  SDCC and ZiLOG tools are used for some
> > architectures.  I think we can ignore all those for now.
> >
> > Greg
> >
> >
> >
> >
> >

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Hi, Greg, may I ask where you find the right mips gcc toolchain?
I haven't found an official prebuilt mips gcc toolchain yet(mips.com
connect failed here). Or build it from source code.

Gregory Nutt <sp...@gmail.com> 于2020年1月9日周四 上午11:55写道：

>
> > Thanks, I think we can firstly get start in Linux and Windows. Then
> > consider how to setup Mac later, donate slave or other.
> > Two other question, nuttx uses some cross compile toolchains etc. In
> > addition, Cygwin or MinGW is in need once nuttx built under Windows
> slaves.
> > Should we ask for Apache Infra to help install in slave machines?
>
> MinGW is MSYS2, right?  I use both Cygwin and MSYS2, they are a little
> different.
>
> There is usually a small number of people using some form of BSD, NetBSD
> or FreeBSD.  Most macOS build issues are due to the fact that macOS uses
> the BSD coreutils which are not 100% compatible with GNU coreutils.  I
> suspect you could substitute FreeBSD for macOS and also catch most if
> not all macOS build issues.
>
> In the past, there have been users on Solaris and HP-UX, but I don't
> think we need to go there now.
>
> Which targets do you plan to build?  I routinely build all ARM targets
> (419 configurations and that omits several that are more difficult to
> automate), but we should also consider PIC32 and and Simulator.  PIC32
> would require a MIPS toolchain.  There are many other architectures but
> most do not warrant investing a lot of test effort.  Renesas RX65N,
> RISC-V, ESP32, AVR might be feasible because they also use GCC
> toolchains and do get used.  Some of the others do not use GCC or have
> much less use.
>
> There have been IAR tools used with ARM in the past.  There have been
> requests for Mikroelektronika tool support in the past, but that was
> never implemented.  SDCC and ZiLOG tools are used for some
> architectures.  I think we can ignore all those for now.
>
> Greg
>
>
>
>
>

Re: Jenkins CI

[Adam Feuer <ad...@starcat.io>]

On Wed, Jan 8, 2020 at 7:55 PM Gregory Nutt <sp...@gmail.com> wrote:

> I suspect you could substitute FreeBSD for macOS and also catch most if
> not all macOS build issues.


I think you could also require installing coreutils via brew or macports,
they both have it. On macOS I can't imagine trying to build NuttX without
having access to one of these package managers.

cheers
adam
--
Adam Feuer
adam@starcat.io

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

> Thanks, I think we can firstly get start in Linux and Windows. Then
> consider how to setup Mac later, donate slave or other.
> Two other question, nuttx uses some cross compile toolchains etc. In
> addition, Cygwin or MinGW is in need once nuttx built under Windows slaves.
> Should we ask for Apache Infra to help install in slave machines?

MinGW is MSYS2, right?  I use both Cygwin and MSYS2, they are a little 
different.

There is usually a small number of people using some form of BSD, NetBSD 
or FreeBSD.  Most macOS build issues are due to the fact that macOS uses 
the BSD coreutils which are not 100% compatible with GNU coreutils.  I 
suspect you could substitute FreeBSD for macOS and also catch most if 
not all macOS build issues.

In the past, there have been users on Solaris and HP-UX, but I don't 
think we need to go there now.

Which targets do you plan to build?  I routinely build all ARM targets 
(419 configurations and that omits several that are more difficult to 
automate), but we should also consider PIC32 and and Simulator.  PIC32 
would require a MIPS toolchain.  There are many other architectures but 
most do not warrant investing a lot of test effort.  Renesas RX65N, 
RISC-V, ESP32, AVR might be feasible because they also use GCC 
toolchains and do get used.  Some of the others do not use GCC or have 
much less use.

There have been IAR tools used with ARM in the past.  There have been 
requests for Mikroelektronika tool support in the past, but that was 
never implemented.  SDCC and ZiLOG tools are used for some 
architectures.  I think we can ignore all those for now.

Greg

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

That's great nuttx-testing project is available. I think empty
nuttx-testing project is ok now.

We'll put the CI scripts and testing related stuff here later.


So sum up, in my opinion, there would be four CI jobs for NuttX projects:


1  One incubator-nuttx-website  CI Job:

Once commit merged into master, do CI and then publish to ASF-Site


2 One incubator-nuttx CI Job

3 One incubator-nuttx-apps CI Job

The two CI jobs do nuttx/ PR check build & apps/ PR check build accordingly,
however they would share the same build script.


4 One nightly CI job for NuttX build testing

Sync the latest nuttx&apps merged code, and then do full build.


Start from simple, first nightly CI build testing under Linux platform.

Then nuttx/ and apps/ PR check CI builds, do PR code check, build verify
and feedback.

Finally support Windows and MacOS/FreeBSD platform CI Job builds.


Any suggestions are welcome.

Gregory Nutt <sp...@gmail.com> 于2020年1月10日周五 上午5:07写道：

> Haitao,
>
> Would you like me to initialize the repository?  I can do that for you
> if you like with apps/ and nuttx/ as submodules but nothing else.
>
> Greg
>
>
>

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

Haitao,

Would you like me to initialize the repository?  I can do that for you 
if you like with apps/ and nuttx/ as submodules but nothing else.

Greg

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

> Or https://github.com/NuttX/tools would be better to hold the jenkins CI
> scripts in this period?

Let's please not use nuttx/tools.  Let's not let any testing framework 
develop inside of the RTOS.

I have created this for you 
https://github.com/apache/incubator-nuttx-testing so that you will not 
have to do that.

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

>
>> In my opinion, Jenkins CI scripts could be version controlled firstly
>> locally or my github.
>> Once the whole process is functional, I think the best place is to 
>> put them
>> into nuttx testing project if available.
>
> We need to request the creation of the NuttX testing repository. I 
> created the new repository using the Self-Server here: 
> https://gitbox.apache.org/setup/newrepo.html
>
> You can see it here: https://github.com/apache/incubator-nuttx-testing

There are now four NuttX repositories:

https://gitbox.apache.org/repos/asf?s=incubator-nuttx

  * incubator-nuttx
  * incubator-nuttx-apps
  * incubator-nuttx-website
  * incubator-nuttx-testing

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

> In my opinion, Jenkins CI scripts could be version controlled firstly
> locally or my github.
> Once the whole process is functional, I think the best place is to put them
> into nuttx testing project if available.

We need to request the creation of the NuttX testing repository. I 
created the new repository using the Self-Server here: 
https://gitbox.apache.org/setup/newrepo.html

You can see it here: https://github.com/apache/incubator-nuttx-testing

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

On Fri, Jan 10, 2020 at 2:12 PM Justin Mclean <ju...@classsoftware.com>
wrote:

> Hi,
>
> > One request: I checked the Jenkins and it seems that the job
> configuration
> > is not accessible without having an account, and you have to be a
> committer
> > and must be a member of the hudson-jobadmin group
>
> I can add people to that as needed. I created jobs and given access o
> people in the past.
>

Ok, I would like to be added in order to collaborate, if other people
involved agree. Not urgent by now.

> Could it be possible to make the configuration read-only, even without
> > account or with a restricted one?
>
> I'm not 100% sure that can be done, but you have a history of changes in
> Jenkins so it’s not hard to manage.
>

Ok, if not possible, I guess is ok to give a normal account, and just rely
on accountability of changes.

Another request: is there any document describing the things we are
expecting to check in the CI for each commit/PR and for each release? I
guess that at the very least we want to test the tests in
https://github.com/apache/incubator-nuttx-apps/tree/master/testing and in
the examples, for the most important architectures.

Thanks,
Miguel

Re: Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

> One request: I checked the Jenkins and it seems that the job configuration
> is not accessible without having an account, and you have to be a committer
> and must be a member of the hudson-jobadmin group

I can add people to that as needed. I created jobs and given access o people in the past.

> Could it be possible to make the configuration read-only, even without
> account or with a restricted one?

I'm not 100% sure that can be done, but you have a history of changes in Jenkins so it’s not hard to manage.

Thanks,
Justin

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

On Fri, Jan 10, 2020 at 4:24 AM Haitao Liu <li...@gmail.com> wrote:

> Thanks Miguel share your approach.  We'll look into it,  the aim for NuttX
> is keep nuttx/ and apps/ user-end projects
> clean and do not mess up with CI stuffs.


Thanks for the feedback. And yes, separate repo for CI stuff is a
reasonable approach.

And now prepare to use Apache
> Infra Jenkins CI firstly.
>

Yes, probably the best approach is to prepare CI jobs in a traditional way,
and once it is almost stable and you and people involved have less work
with it start to translating it to something versionable as the Jenkins Job
Builder definitions. This way the transition  curve is easier and avoid
interfering with the priority goal of having a working CI.

One request: I checked the Jenkins and it seems that the job configuration
is not accessible without having an account, and you have to be a committer
and must be a member of the hudson-jobadmin group (according to
https://cwiki.apache.org/confluence/display/INFRA/Jenkins#Jenkins-HowdoIgetanaccount?).
Could it be possible to make the configuration read-only, even without
account or with a restricted one? This way it would be easier for me and
other people to help and review job definitions and I could help to create
job builder definition templates in the meanwhile, based on the jobs the
committer is creating manually with the UI.

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Thanks Miguel share your approach.  We'll look into it,  the aim for NuttX
is keep nuttx/ and apps/ user-end projects
clean and do not mess up with CI stuffs. And now prepare to use Apache
Infra Jenkins CI firstly.


Miguel Ángel Herranz <mi...@midokura.com.invalid> 于2020年1月9日周四 下午7:45写道：

> Hi,
>
> I have configured Jenkins CI pipelines in the past and aside the scripts
> running in the jobs, managing the jobs definition in the UI can be a pain,
> since is easy to apply untrackable changes that impact the pipeline.
>
> So I am wondering if you are going to use any kind of solution like Jenkins
> Job Builder (https://docs.openstack.org/infra/jenkins-job-builder/). It
> was
> created by OpenStack to manage Jenkins jobs. Basically, aside of
> bash/python/etc scripts to be run by the jobs, you define your
> pipeline/jobs using YAML files that could be easily versioned in a public
> repo, and put your Jenkins user/pass in a private file. Then invoking the
> tool that will contact the Jenkins service and setup everything as declared
> in the YAML files. This way to replicate all the pipeline and jobs to
> another Jenkins instance should be easy.
>
> I successfully used that approach in previous projects and I think is
> really useful to managing complex situations. If there is interest I can
> help, since I have previous experience with it.
>
> Cheers,
> Miguel
>
>
>
>
>
>
>
> On Thu, Jan 9, 2020 at 10:32 AM Haitao Liu <li...@gmail.com> wrote:
>
> > Or https://github.com/NuttX/tools would be better to hold the jenkins CI
> > scripts in this period?
> >
> > Haitao Liu <li...@gmail.com> 于2020年1月9日周四 下午4:08写道：
> >
> > > In my opinion, Jenkins CI scripts could be version controlled firstly
> > > locally or my github.
> > > Once the whole process is functional, I think the best place is to put
> > > them into nuttx testing project if available.
> > >
> > > Adam Feuer <ad...@starcat.io> 于2020年1月9日周四 下午12:51写道：
> > >
> > >> Haitao,
> > >>
> > >> Will the Jenkins CI scripts live in version control somewhere?
> > >>
> > >> cheers
> > >> adam
> > >>
> > >> On Wed, Jan 8, 2020 at 7:44 PM Haitao Liu <li...@gmail.com> wrote:
> > >>
> > >> > Ok, get  it. Thanks!
> > >> >
> > >> > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午11:07写道：
> > >> >
> > >> > > Hi,
> > >> > >
> > >> > > > Thanks, I think we can firstly get start in Linux and Windows.
> > Then
> > >> > > > consider how to setup Mac later, donate slave or other.
> > >> > > > Two other question, nuttx uses some cross compile toolchains
> etc.
> > In
> > >> > > > addition, Cygwin or MinGW is in need once nuttx built under
> > Windows
> > >> > > slaves.
> > >> > > > Should we ask for Apache Infra to help install in slave
> machines?
> > >> > >
> > >> > > If it's not already installed yet you need to ask for it to be
> > >> installed.
> > >> > > Best way to do is to raise an Infra JIRA.
> > >> > >
> > >> > > Thanks,
> > >> > > Justin
> > >> >
> > >>
> > >>
> > >> --
> > >> Adam Feuer <ad...@starcat.io>
> > >>
> > >
> >
>

Re: Jenkins CI

[Miguel Ángel Herranz <mi...@midokura.com.INVALID>]

Hi,

I have configured Jenkins CI pipelines in the past and aside the scripts
running in the jobs, managing the jobs definition in the UI can be a pain,
since is easy to apply untrackable changes that impact the pipeline.

So I am wondering if you are going to use any kind of solution like Jenkins
Job Builder (https://docs.openstack.org/infra/jenkins-job-builder/). It was
created by OpenStack to manage Jenkins jobs. Basically, aside of
bash/python/etc scripts to be run by the jobs, you define your
pipeline/jobs using YAML files that could be easily versioned in a public
repo, and put your Jenkins user/pass in a private file. Then invoking the
tool that will contact the Jenkins service and setup everything as declared
in the YAML files. This way to replicate all the pipeline and jobs to
another Jenkins instance should be easy.

I successfully used that approach in previous projects and I think is
really useful to managing complex situations. If there is interest I can
help, since I have previous experience with it.

Cheers,
Miguel







On Thu, Jan 9, 2020 at 10:32 AM Haitao Liu <li...@gmail.com> wrote:

> Or https://github.com/NuttX/tools would be better to hold the jenkins CI
> scripts in this period?
>
> Haitao Liu <li...@gmail.com> 于2020年1月9日周四 下午4:08写道：
>
> > In my opinion, Jenkins CI scripts could be version controlled firstly
> > locally or my github.
> > Once the whole process is functional, I think the best place is to put
> > them into nuttx testing project if available.
> >
> > Adam Feuer <ad...@starcat.io> 于2020年1月9日周四 下午12:51写道：
> >
> >> Haitao,
> >>
> >> Will the Jenkins CI scripts live in version control somewhere?
> >>
> >> cheers
> >> adam
> >>
> >> On Wed, Jan 8, 2020 at 7:44 PM Haitao Liu <li...@gmail.com> wrote:
> >>
> >> > Ok, get  it. Thanks!
> >> >
> >> > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午11:07写道：
> >> >
> >> > > Hi,
> >> > >
> >> > > > Thanks, I think we can firstly get start in Linux and Windows.
> Then
> >> > > > consider how to setup Mac later, donate slave or other.
> >> > > > Two other question, nuttx uses some cross compile toolchains etc.
> In
> >> > > > addition, Cygwin or MinGW is in need once nuttx built under
> Windows
> >> > > slaves.
> >> > > > Should we ask for Apache Infra to help install in slave machines?
> >> > >
> >> > > If it's not already installed yet you need to ask for it to be
> >> installed.
> >> > > Best way to do is to raise an Infra JIRA.
> >> > >
> >> > > Thanks,
> >> > > Justin
> >> >
> >>
> >>
> >> --
> >> Adam Feuer <ad...@starcat.io>
> >>
> >
>

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Or https://github.com/NuttX/tools would be better to hold the jenkins CI
scripts in this period?

Haitao Liu <li...@gmail.com> 于2020年1月9日周四 下午4:08写道：

> In my opinion, Jenkins CI scripts could be version controlled firstly
> locally or my github.
> Once the whole process is functional, I think the best place is to put
> them into nuttx testing project if available.
>
> Adam Feuer <ad...@starcat.io> 于2020年1月9日周四 下午12:51写道：
>
>> Haitao,
>>
>> Will the Jenkins CI scripts live in version control somewhere?
>>
>> cheers
>> adam
>>
>> On Wed, Jan 8, 2020 at 7:44 PM Haitao Liu <li...@gmail.com> wrote:
>>
>> > Ok, get  it. Thanks!
>> >
>> > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午11:07写道：
>> >
>> > > Hi,
>> > >
>> > > > Thanks, I think we can firstly get start in Linux and Windows. Then
>> > > > consider how to setup Mac later, donate slave or other.
>> > > > Two other question, nuttx uses some cross compile toolchains etc. In
>> > > > addition, Cygwin or MinGW is in need once nuttx built under Windows
>> > > slaves.
>> > > > Should we ask for Apache Infra to help install in slave machines?
>> > >
>> > > If it's not already installed yet you need to ask for it to be
>> installed.
>> > > Best way to do is to raise an Infra JIRA.
>> > >
>> > > Thanks,
>> > > Justin
>> >
>>
>>
>> --
>> Adam Feuer <ad...@starcat.io>
>>
>

Re: Jenkins CI

[Adam Feuer <ad...@starcat.io>]

Haitao,

Cool! Anywhere in a public git repo is great; nuttx testing project also
sounds great.

cheers
adam


On Thu, Jan 9, 2020 at 12:08 AM Haitao Liu <li...@gmail.com> wrote:

> In my opinion, Jenkins CI scripts could be version controlled firstly
> locally or my github.
> Once the whole process is functional, I think the best place is to put them
> into nuttx testing project if available.
>
> Adam Feuer <ad...@starcat.io> 于2020年1月9日周四 下午12:51写道：
>
> > Haitao,
> >
> > Will the Jenkins CI scripts live in version control somewhere?
> >
> > cheers
> > adam
> >
> > On Wed, Jan 8, 2020 at 7:44 PM Haitao Liu <li...@gmail.com> wrote:
> >
> > > Ok, get  it. Thanks!
> > >
> > > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午11:07写道：
> > >
> > > > Hi,
> > > >
> > > > > Thanks, I think we can firstly get start in Linux and Windows. Then
> > > > > consider how to setup Mac later, donate slave or other.
> > > > > Two other question, nuttx uses some cross compile toolchains etc.
> In
> > > > > addition, Cygwin or MinGW is in need once nuttx built under Windows
> > > > slaves.
> > > > > Should we ask for Apache Infra to help install in slave machines?
> > > >
> > > > If it's not already installed yet you need to ask for it to be
> > installed.
> > > > Best way to do is to raise an Infra JIRA.
> > > >
> > > > Thanks,
> > > > Justin
> > >
> >
> >
> > --
> > Adam Feuer <ad...@starcat.io>
> >
>


-- 
Adam Feuer <ad...@starcat.io>

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

In my opinion, Jenkins CI scripts could be version controlled firstly
locally or my github.
Once the whole process is functional, I think the best place is to put them
into nuttx testing project if available.

Adam Feuer <ad...@starcat.io> 于2020年1月9日周四 下午12:51写道：

> Haitao,
>
> Will the Jenkins CI scripts live in version control somewhere?
>
> cheers
> adam
>
> On Wed, Jan 8, 2020 at 7:44 PM Haitao Liu <li...@gmail.com> wrote:
>
> > Ok, get  it. Thanks!
> >
> > Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午11:07写道：
> >
> > > Hi,
> > >
> > > > Thanks, I think we can firstly get start in Linux and Windows. Then
> > > > consider how to setup Mac later, donate slave or other.
> > > > Two other question, nuttx uses some cross compile toolchains etc. In
> > > > addition, Cygwin or MinGW is in need once nuttx built under Windows
> > > slaves.
> > > > Should we ask for Apache Infra to help install in slave machines?
> > >
> > > If it's not already installed yet you need to ask for it to be
> installed.
> > > Best way to do is to raise an Infra JIRA.
> > >
> > > Thanks,
> > > Justin
> >
>
>
> --
> Adam Feuer <ad...@starcat.io>
>

Re: Jenkins CI

[Adam Feuer <ad...@starcat.io>]

Haitao,

Will the Jenkins CI scripts live in version control somewhere?

cheers
adam

On Wed, Jan 8, 2020 at 7:44 PM Haitao Liu <li...@gmail.com> wrote:

> Ok, get  it. Thanks!
>
> Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午11:07写道：
>
> > Hi,
> >
> > > Thanks, I think we can firstly get start in Linux and Windows. Then
> > > consider how to setup Mac later, donate slave or other.
> > > Two other question, nuttx uses some cross compile toolchains etc. In
> > > addition, Cygwin or MinGW is in need once nuttx built under Windows
> > slaves.
> > > Should we ask for Apache Infra to help install in slave machines?
> >
> > If it's not already installed yet you need to ask for it to be installed.
> > Best way to do is to raise an Infra JIRA.
> >
> > Thanks,
> > Justin
>


-- 
Adam Feuer <ad...@starcat.io>

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Ok, get  it. Thanks!

Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午11:07写道：

> Hi,
>
> > Thanks, I think we can firstly get start in Linux and Windows. Then
> > consider how to setup Mac later, donate slave or other.
> > Two other question, nuttx uses some cross compile toolchains etc. In
> > addition, Cygwin or MinGW is in need once nuttx built under Windows
> slaves.
> > Should we ask for Apache Infra to help install in slave machines?
>
> If it's not already installed yet you need to ask for it to be installed.
> Best way to do is to raise an Infra JIRA.
>
> Thanks,
> Justin

Re: Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

> Thanks, I think we can firstly get start in Linux and Windows. Then
> consider how to setup Mac later, donate slave or other.
> Two other question, nuttx uses some cross compile toolchains etc. In
> addition, Cygwin or MinGW is in need once nuttx built under Windows slaves.
> Should we ask for Apache Infra to help install in slave machines?

If it's not already installed yet you need to ask for it to be installed. Best way to do is to raise an Infra JIRA.

Thanks,
Justin

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Thanks, I think we can firstly get start in Linux and Windows. Then
consider how to setup Mac later, donate slave or other.
Two other question, nuttx uses some cross compile toolchains etc. In
addition, Cygwin or MinGW is in need once nuttx built under Windows slaves.
Should we ask for Apache Infra to help install in slave machines?

Justin Mclean <ju...@classsoftware.com> 于2020年1月9日周四 上午10:14写道：

> Hi,
>
> > I have some questions about Apache Infra. Since NuttX would be built
> under
> > Linux/Mac/Windows environment, does Apache Infra Jenkins CI provides Mac
> > and Windows slave machines to ease CI build?
>
> There was a mac machine, but I think it was removed a while back. There
> are windows machines (e.g [1]) and arm machines.
>
> I believe you can donate slaves to be added if you have large workload or
> require special setup.
>
> Thanks,
> Justin
>
> 1. https://builds.apache.org/job/netbeans-windows/

Re: Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

> I have some questions about Apache Infra. Since NuttX would be built under
> Linux/Mac/Windows environment, does Apache Infra Jenkins CI provides Mac
> and Windows slave machines to ease CI build?

There was a mac machine, but I think it was removed a while back. There are windows machines (e.g [1]) and arm machines. 

I believe you can donate slaves to be added if you have large workload or require special setup.

Thanks,
Justin

1. https://builds.apache.org/job/netbeans-windows/

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Hi, Justin,

I have some questions about Apache Infra. Since NuttX would be built under
Linux/Mac/Windows environment, does Apache Infra Jenkins CI provides Mac
and Windows slave machines to ease CI build?

Justin Mclean <ju...@classsoftware.com> 于2020年1月8日周三 下午12:00写道：

> Hi,
>
> Changing subjects.
>
> > Since code check part is available now, could I and Duo prepare to setup
> > Jenkins CI job firstly?
>
> Where are you intending to set up this job? Here [1] or elsewhere?
>
> Thanks,
> Justin
>
> 1. https://builds.apache.org
>

Re: Jenkins CI

[Xiang Xiao <xi...@gmail.com>]

If you prefer IDE(Eclipse) environment, here is instruction done by David:
https://github.com/apache/incubator-nuttx/pull/12
Just need change "nxstyle -m 86" to "checkpatch.sh -f"

On Wed, Jan 8, 2020 at 4:46 PM Haitao Liu <li...@gmail.com> wrote:
>
> Now there is checkpatch.sh in nuttx/tools which call nxstyle to support
> patch/file/commit check.
> It could be used as a git pre-commit hook.
>
> Disruptive Solutions <di...@gmail.com> 于2020年1月8日周三
> 下午3:14写道：
>
> > Is there a “code style” formatter possible for say Eclipse for Apache
> > Nuttx? This can be custom made and gives the quality already while coding?
> >
> > Or is this not preferred adding it to a backend styling in git?
> >
> > Just thinking....
>
>
> > Verstuurd vanaf mijn iPhone
> >
> > > Op 8 jan. 2020 om 05:03 heeft Gregory Nutt <sp...@gmail.com> het
> > volgende geschreven:
> > >
> > > 
> > >>
> > >>> Since code check part is available now, could I and Duo prepare to
> > setup
> > >>> Jenkins CI job firstly?
> > >> Where are you intending to set up this job? Here [1] or elsewhere?
> > >
> > > Hopefully not in the end-user directories :-\
> > >
> > > Greg
> > >
> >

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Now there is checkpatch.sh in nuttx/tools which call nxstyle to support
patch/file/commit check.
It could be used as a git pre-commit hook.

Disruptive Solutions <di...@gmail.com> 于2020年1月8日周三
下午3:14写道：

> Is there a “code style” formatter possible for say Eclipse for Apache
> Nuttx? This can be custom made and gives the quality already while coding?
>
> Or is this not preferred adding it to a backend styling in git?
>
> Just thinking....


> Verstuurd vanaf mijn iPhone
>
> > Op 8 jan. 2020 om 05:03 heeft Gregory Nutt <sp...@gmail.com> het
> volgende geschreven:
> >
> > 
> >>
> >>> Since code check part is available now, could I and Duo prepare to
> setup
> >>> Jenkins CI job firstly?
> >> Where are you intending to set up this job? Here [1] or elsewhere?
> >
> > Hopefully not in the end-user directories :-\
> >
> > Greg
> >
>

Re: Jenkins CI

[Disruptive Solutions <di...@gmail.com>]

Is there a “code style” formatter possible for say Eclipse for Apache Nuttx? This can be custom made and gives the quality already while coding?

Or is this not preferred adding it to a backend styling in git?

Just thinking.....

Verstuurd vanaf mijn iPhone

> Op 8 jan. 2020 om 05:03 heeft Gregory Nutt <sp...@gmail.com> het volgende geschreven:
> 
> 
>> 
>>> Since code check part is available now, could I and Duo prepare to setup
>>> Jenkins CI job firstly?
>> Where are you intending to set up this job? Here [1] or elsewhere?
> 
> Hopefully not in the end-user directories :-\
> 
> Greg
> 

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

Greg, all the CI specific scripts would only be in Jenkins Job Server. They
won't come into nuttx/ and apps/ end-user directories.

Gregory Nutt <sp...@gmail.com> 于2020年1月8日周三 下午12:03写道：

> >> Since code check part is available now, could I and Duo prepare to setup
> >> Jenkins CI job firstly?
> > Where are you intending to set up this job? Here [1] or elsewhere?
>
> Hopefully not in the end-user directories :-\
>
> Greg
>
>

Re: Jenkins CI

[Gregory Nutt <sp...@gmail.com>]

>> Since code check part is available now, could I and Duo prepare to setup
>> Jenkins CI job firstly?
> Where are you intending to set up this job? Here [1] or elsewhere?

Hopefully not in the end-user directories :-\

Greg

Re: Jenkins CI

[Haitao Liu <li...@gmail.com>]

In my opionion,  it's best setup job here in https://builds.apache.org

Justin Mclean <ju...@classsoftware.com> 于2020年1月8日周三 下午12:00写道：

> Hi,
>
> Changing subjects.
>
> > Since code check part is available now, could I and Duo prepare to setup
> > Jenkins CI job firstly?
>
> Where are you intending to set up this job? Here [1] or elsewhere?
>
> Thanks,
> Justin
>
> 1. https://builds.apache.org
>

Jenkins CI

[Justin Mclean <ju...@classsoftware.com>]

Hi,

Changing subjects.

> Since code check part is available now, could I and Duo prepare to setup
> Jenkins CI job firstly?

Where are you intending to set up this job? Here [1] or elsewhere?

Thanks,
Justin

1. https://builds.apache.org

Re: Working Effectively (was Point of Order)

[Brennan Ashton <ba...@brennanashton.com>]

On Wed, Jan 8, 2020, 11:05 AM Nathan Hartman <ha...@gmail.com>
wrote:

> On Wed, Jan 8, 2020 at 3:05 AM Xiang Xiao <xi...@gmail.com>
> wrote:
>
> > Yes, all Jenkins related files should put on the Jenkins server as
> > much as possible.
>
>
> What if something happens to the Jenkins server? Will the files exist
> somewhere else too?
>

Also how are we going to test Mac and Windows using that build setup. This
is one reason I asked about using GitHub Actions or something that easily
provides that for us.

>

Re: Working Effectively (was Point of Order)

[Nathan Hartman <ha...@gmail.com>]

On Wed, Jan 8, 2020 at 3:05 AM Xiang Xiao <xi...@gmail.com> wrote:

> Yes, all Jenkins related files should put on the Jenkins server as
> much as possible.


What if something happens to the Jenkins server? Will the files exist
somewhere else too?

Thanks,
Nathan

Re: Working Effectively (was Point of Order)

[Xiang Xiao <xi...@gmail.com>]

On Wed, Jan 8, 2020 at 12:01 PM Gregory Nutt <sp...@gmail.com> wrote:
>
>
> >> Since code check part is available now, could I and Duo prepare to setup
> >> Jenkins CI job firstly?
> >> At fist, start to do the PR check stage according to the workflow option
> >> chosen, then go to the build stage  in the next couple of days.
> >> We could optimize all the stages step by step once we start working
> >> on it :
> >> )
> >
> > He should talk about how this would be implemented.
> >
> > In the past, we talk about trying to keep as much of the test support
> > files out of the nuttx/ and apps/ repositories as possible.  I would
> > prefer that we try to keep those repositories as clean as possible
> > with little or not testing files.
> >

Yes, all Jenkins related files should put on the Jenkins server as
much as possible.

> > We talking about a third testing/ repository that would be an
> > environment for testing.  In that repository, you could support the
> > nuttx/ and apps/ directories as submodules and have an environment
> > that could support unlimited testing without contaminating the nuttx/
> > and apps/ end-user directories.
> >
> > Can we please consider this?  Let's not put any Jenkins CI in those
> > end-user directories.
>

Let's see whether the submodule is needed during the development. But
the principle is that: submodule should be used only for
building/testing verification, the user/contributor don't need to know
anything about submodule.

> DavidS also suggested moving some of the common tools, such as nxstyle,
> out of nuttx/tools to testing/tools.  The coding standard check is not
> specific to the nuttx/ repository.
>

But, most user also need these tools to ensure that they don't break
the rule before submit PR.
Basically, we have two catagory tools:
1.The workflow independent scripts, like checkpatch.sh and testbuild.sh
    These tools can run standalone in the developer machine, and very
usefull for their daily work. The best location is nuttx/tools folder.
2.The workflow dependent scripts
   The best location for these tools is the Jenkins server.

> Greg
>
>

Re: Working Effectively (was Point of Order)

[Gregory Nutt <sp...@gmail.com>]

>> Since code check part is available now, could I and Duo prepare to setup
>> Jenkins CI job firstly?
>> At fist, start to do the PR check stage according to the workflow option
>> chosen, then go to the build stage  in the next couple of days.
>> We could optimize all the stages step by step once we start working 
>> on it :
>> )
>
> He should talk about how this would be implemented.
>
> In the past, we talk about trying to keep as much of the test support 
> files out of the nuttx/ and apps/ repositories as possible.  I would 
> prefer that we try to keep those repositories as clean as possible 
> with little or not testing files.
>
> We talking about a third testing/ repository that would be an 
> environment for testing.  In that repository, you could support the 
> nuttx/ and apps/ directories as submodules and have an environment 
> that could support unlimited testing without contaminating the nuttx/ 
> and apps/ end-user directories.
>
> Can we please consider this?  Let's not put any Jenkins CI in those 
> end-user directories.

DavidS also suggested moving some of the common tools, such as nxstyle, 
out of nuttx/tools to testing/tools.  The coding standard check is not 
specific to the nuttx/ repository.

Greg

Re: Working Effectively (was Point of Order)

[Haitao Liu <li...@gmail.com>]

Greg, Jenkins CI would not touch nuttx/ and apps/ end-user directories.

Gregory Nutt <sp...@gmail.com> 于2020年1月8日周三 上午11:58写道：

> > Since code check part is available now, could I and Duo prepare to setup
> > Jenkins CI job firstly?
> > At fist, start to do the PR check stage according to the workflow option
> > chosen, then go to the build stage  in the next couple of days.
> > We could optimize all the stages step by step once we start working on
> it :
> > )
>
> He should talk about how this would be implemented.
>
> In the past, we talk about trying to keep as much of the test support
> files out of the nuttx/ and apps/ repositories as possible.  I would
> prefer that we try to keep those repositories as clean as possible with
> little or not testing files.
>
> We talking about a third testing/ repository that would be an
> environment for testing.  In that repository, you could support the
> nuttx/ and apps/ directories as submodules and have an environment that
> could support unlimited testing without contaminating the nuttx/ and
> apps/ end-user directories.
>
> Can we please consider this?  Let's not put any Jenkins CI in those
> end-user directories.
>
> Greg
>
>
>

Re: Working Effectively (was Point of Order)

[Gregory Nutt <sp...@gmail.com>]

> Since code check part is available now, could I and Duo prepare to setup
> Jenkins CI job firstly?
> At fist, start to do the PR check stage according to the workflow option
> chosen, then go to the build stage  in the next couple of days.
> We could optimize all the stages step by step once we start working on it :
> )

He should talk about how this would be implemented.

In the past, we talk about trying to keep as much of the test support 
files out of the nuttx/ and apps/ repositories as possible.  I would 
prefer that we try to keep those repositories as clean as possible with 
little or not testing files.

We talking about a third testing/ repository that would be an 
environment for testing.  In that repository, you could support the 
nuttx/ and apps/ directories as submodules and have an environment that 
could support unlimited testing without contaminating the nuttx/ and 
apps/ end-user directories.

Can we please consider this?  Let's not put any Jenkins CI in those 
end-user directories.

Greg

Re: Working Effectively (was Point of Order)

[Haitao Liu <li...@gmail.com>]

Greg, Xiang and All,

Since code check part is available now, could I and Duo prepare to setup
Jenkins CI job firstly?
At fist, start to do the PR check stage according to the workflow option
chosen, then go to the build stage  in the next couple of days.
We could optimize all the stages step by step once we start working on it :
)

Xiang Xiao <xi...@gmail.com> 于2020年1月8日周三 上午10:30写道：

> David, why you think this tool promote the violation?
> With -p or -c option, we can ensure no new code violate the coding style.
> With -f option, we can review and correct the current code base.
> The jenkins job can select one option base on the workflow definition,
> the tool self hasn't any preference.
> Base on your measurement:
> https://github.com/apache/incubator-nuttx/pull/12
> 4043 files doesn't confirm the coding standard, how about each
> committer(candidate) take several folders to fix the style issue?
>
> On Wed, Jan 8, 2020 at 12:40 AM David Sidrane <Da...@nscdg.com>
> wrote:
> >
> > +1
> >
> > Adding a tool that promotes violating the INVIOLABLES.txt on several
> aspect
> > (expedience and non conformant code into the repo) feels like a very bad
> > choice.
> >
> > How is this being justified?
> >
> > David
> >
> >
> > -----Original Message-----
> > From: Alin Jerpelea [mailto:jerpelea@gmail.com]
> > Sent: Tuesday, January 07, 2020 4:18 AM
> > To: dev@nuttx.apache.org
> > Subject: Re: Working Effectively (was Point of Order)
> >
> > I think that we should update all code with nxstyle then we start with a
> > clean code on which patches will be checked
> >
> > Alin
> >
> > On Tue, Dec 31, 2019 at 3:42 PM Haitao Liu <li...@gmail.com> wrote:
> >
> > > Greg,  I'll finish the nxstyle & check script these days firstly and
> then
> > > take time to handle the two issues you mentioned if allowed. I wish I
> > > could
> > > be competent to take over full responsibility for nxstyle.c.
> > >
> > > Gregory Nutt <sp...@gmail.com> 于2019年12月30日周一 下午11:31写道：
> > >
> > > > [moved from private@nuttx.apache.org]
> > > >
> > > > Hi, Xiao Xiang and Haitao Liu
> > > >
> > > > > Haitao is preparing a script for style check, the feature include:
> > > > > 1.Auto build nxstyle
> > > > > 2.Improve nxstyle to check the partial file for supporting patch
> like
> > > > file
> > > > > 3.Input can be the source files, patch file or commit id
> > > >
> > > > I mentioned that I had two things I planned to do for nxstyle.c too.
> > > > Perhaps Haitao would like to take on these changes too?
> > > >
> > > > 1. Derive maximum line width from block comments.  Ref:
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#fileorganization
> > > >
> > > >   * The Maximum line width should be the same as the width of the
> file
> > > >     section block comment, by default the last asterisk '*' lies on
> > > >     column 78, but wider lines are okay too, as long as things are
> > > >     scaled corrected.  Instead of the line width being a hardcoded
> 78 or
> > > >     specified the command line, nxstyle would parse the entire file,
> > > >     look at the block comments, find the column of the last '*'  The
> > > >     line width is that colument + 1;
> > > >   * Then also verify that every block comment in the file is the same
> > > width
> > > >   * Perhaps the -m argument could change to a amount that you will
> > > >     tolerate extending into the right margin.  So the block comment
> with
> > > >     is 78, then -m 8 would allow lines to extend to 86.  That is
> better
> > > >     than a hard-coded 86 because the line width will vary with file
> > > >     content.  For some register definition files, the line width may
> be
> > > >     120 or so.  In that case -m 8 would allow the line to extend to
> 128.
> > > >
> > > > 2. Verify that file section block comments are present.
> > > >
> > > >   * Add an enumeration value that indicates every block comment type.
> > > >     See
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#cfilestructure
> > > >     and
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#hfilestructure
> > > >   * Currently logic detects only the presence of "* Private
> Functions"
> > > >     and "* Private Functions".  Add logic to detect all other the
> block
> > > >     comment types as well.
> > > >   * Add a global variable to indicate which code block we are
> currently
> > > >     in.  Each time a file section block comment is found, set the
> global
> > > >     variable.
> > > >   * The if '#   define" is encountered outside of the "*
> Pre-processor
> > > >     Definitions" file section, emit an error
> > > >   * If '#  include" is encountered outside of the "* Included Files"
> > > >     section, emit an error.
> > > >   * This logic could eventually be extended to assure that all static
> > > >     functions are defined in the "* Private Functions" section, all
> > > >     global functions are defined in the "* Public Functions section",
> > > >     all static data definitions appear in the "* Private Data"
> section,
> > > >     and all global data definitions appear in the "* Public data
> > > >     definitions."
> > > >   * We could also enable special parsing depending on the section.
> That
> > > >     is already done for the Public/Private Functions.  But we could,
> for
> > > >     example also verify that variables defined at global scope start
> > > >     with "g-"
> > > >
> > > > Let me know.  I was planning to do these things, but if Haitao Liu is
> > > > interested, he can take over full responsibility for nxstyle.c.
> > > >
> > > > Greg
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
>

RE: Working Effectively (was Point of Order)

[David Sidrane <Da...@nscdg.com>]

Hi Xiang,

-----Original Message-----
From: Xiang Xiao [mailto:xiaoxiang781216@gmail.com]
Sent: Tuesday, January 07, 2020 6:30 PM
To: dev@nuttx.apache.org
Subject: Re: Working Effectively (was Point of Order)

David, why you think this tool promote the violation?

[DBS]
May be I do not understand what the -p and -c options do?

"no new code" All lines in the file or the lines that were changed in the
file?

If it is the latter, then refer the this:

o Strict conformance to the NuttX coding style.  No "revolutionary"
    changes to the coding standard (but perhaps some "evolutionary"
    changes).
  o Personal or organizational preference is not a justification for a
    coding style change.
  o Nothing can come into NuttX that does not follow the coding standard.
  o Expediency is not a justification for violating the coding standard.

With -p or -c option, we can ensure no new code violate the coding style.
With -f option, we can review and correct the current code base.
The jenkins job can select one option base on the workflow definition,
the tool self hasn't any preference.
Base on your measurement:
https://github.com/apache/incubator-nuttx/pull/12
4043 files doesn't confirm the coding standard, how about each
committer(candidate) take several folders to fix the style issue?

 [DBS] Yes when there is a documented set of work instructions (The commands
and steps and repos to be used)

David

On Wed, Jan 8, 2020 at 12:40 AM David Sidrane <Da...@nscdg.com>
wrote:
>
> +1
>
> Adding a tool that promotes violating the INVIOLABLES.txt on several
> aspect
> (expedience and non conformant code into the repo) feels like a very bad
> choice.
>
> How is this being justified?
>
> David
>
>
> -----Original Message-----
> From: Alin Jerpelea [mailto:jerpelea@gmail.com]
> Sent: Tuesday, January 07, 2020 4:18 AM
> To: dev@nuttx.apache.org
> Subject: Re: Working Effectively (was Point of Order)
>
> I think that we should update all code with nxstyle then we start with a
> clean code on which patches will be checked
>
> Alin
>
> On Tue, Dec 31, 2019 at 3:42 PM Haitao Liu <li...@gmail.com> wrote:
>
> > Greg,  I'll finish the nxstyle & check script these days firstly and
> > then
> > take time to handle the two issues you mentioned if allowed. I wish I
> > could
> > be competent to take over full responsibility for nxstyle.c.
> >
> > Gregory Nutt <sp...@gmail.com> 于2019年12月30日周一 下午11:31写道：
> >
> > > [moved from private@nuttx.apache.org]
> > >
> > > Hi, Xiao Xiang and Haitao Liu
> > >
> > > > Haitao is preparing a script for style check, the feature include:
> > > > 1.Auto build nxstyle
> > > > 2.Improve nxstyle to check the partial file for supporting patch
> > > > like
> > > file
> > > > 3.Input can be the source files, patch file or commit id
> > >
> > > I mentioned that I had two things I planned to do for nxstyle.c too.
> > > Perhaps Haitao would like to take on these changes too?
> > >
> > > 1. Derive maximum line width from block comments.  Ref:
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#fileorganization
> > >
> > >   * The Maximum line width should be the same as the width of the file
> > >     section block comment, by default the last asterisk '*' lies on
> > >     column 78, but wider lines are okay too, as long as things are
> > >     scaled corrected.  Instead of the line width being a hardcoded 78
> > > or
> > >     specified the command line, nxstyle would parse the entire file,
> > >     look at the block comments, find the column of the last '*'  The
> > >     line width is that colument + 1;
> > >   * Then also verify that every block comment in the file is the same
> > width
> > >   * Perhaps the -m argument could change to a amount that you will
> > >     tolerate extending into the right margin.  So the block comment
> > > with
> > >     is 78, then -m 8 would allow lines to extend to 86.  That is
> > > better
> > >     than a hard-coded 86 because the line width will vary with file
> > >     content.  For some register definition files, the line width may
> > > be
> > >     120 or so.  In that case -m 8 would allow the line to extend to
> > > 128.
> > >
> > > 2. Verify that file section block comments are present.
> > >
> > >   * Add an enumeration value that indicates every block comment type.
> > >     See
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#cfilestructure
> > >     and
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#hfilestructure
> > >   * Currently logic detects only the presence of "* Private Functions"
> > >     and "* Private Functions".  Add logic to detect all other the
> > > block
> > >     comment types as well.
> > >   * Add a global variable to indicate which code block we are
> > > currently
> > >     in.  Each time a file section block comment is found, set the
> > > global
> > >     variable.
> > >   * The if '#   define" is encountered outside of the "* Pre-processor
> > >     Definitions" file section, emit an error
> > >   * If '#  include" is encountered outside of the "* Included Files"
> > >     section, emit an error.
> > >   * This logic could eventually be extended to assure that all static
> > >     functions are defined in the "* Private Functions" section, all
> > >     global functions are defined in the "* Public Functions section",
> > >     all static data definitions appear in the "* Private Data"
> > > section,
> > >     and all global data definitions appear in the "* Public data
> > >     definitions."
> > >   * We could also enable special parsing depending on the section.
> > > That
> > >     is already done for the Public/Private Functions.  But we could,
> > > for
> > >     example also verify that variables defined at global scope start
> > >     with "g-"
> > >
> > > Let me know.  I was planning to do these things, but if Haitao Liu is
> > > interested, he can take over full responsibility for nxstyle.c.
> > >
> > > Greg
> > >
> > >
> > >
> > >
> > >
> >

Re: Working Effectively (was Point of Order)

[Xiang Xiao <xi...@gmail.com>]

David, why you think this tool promote the violation?
With -p or -c option, we can ensure no new code violate the coding style.
With -f option, we can review and correct the current code base.
The jenkins job can select one option base on the workflow definition,
the tool self hasn't any preference.
Base on your measurement:
https://github.com/apache/incubator-nuttx/pull/12
4043 files doesn't confirm the coding standard, how about each
committer(candidate) take several folders to fix the style issue?

On Wed, Jan 8, 2020 at 12:40 AM David Sidrane <Da...@nscdg.com> wrote:
>
> +1
>
> Adding a tool that promotes violating the INVIOLABLES.txt on several aspect
> (expedience and non conformant code into the repo) feels like a very bad
> choice.
>
> How is this being justified?
>
> David
>
>
> -----Original Message-----
> From: Alin Jerpelea [mailto:jerpelea@gmail.com]
> Sent: Tuesday, January 07, 2020 4:18 AM
> To: dev@nuttx.apache.org
> Subject: Re: Working Effectively (was Point of Order)
>
> I think that we should update all code with nxstyle then we start with a
> clean code on which patches will be checked
>
> Alin
>
> On Tue, Dec 31, 2019 at 3:42 PM Haitao Liu <li...@gmail.com> wrote:
>
> > Greg,  I'll finish the nxstyle & check script these days firstly and then
> > take time to handle the two issues you mentioned if allowed. I wish I
> > could
> > be competent to take over full responsibility for nxstyle.c.
> >
> > Gregory Nutt <sp...@gmail.com> 于2019年12月30日周一 下午11:31写道：
> >
> > > [moved from private@nuttx.apache.org]
> > >
> > > Hi, Xiao Xiang and Haitao Liu
> > >
> > > > Haitao is preparing a script for style check, the feature include:
> > > > 1.Auto build nxstyle
> > > > 2.Improve nxstyle to check the partial file for supporting patch like
> > > file
> > > > 3.Input can be the source files, patch file or commit id
> > >
> > > I mentioned that I had two things I planned to do for nxstyle.c too.
> > > Perhaps Haitao would like to take on these changes too?
> > >
> > > 1. Derive maximum line width from block comments.  Ref:
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#fileorganization
> > >
> > >   * The Maximum line width should be the same as the width of the file
> > >     section block comment, by default the last asterisk '*' lies on
> > >     column 78, but wider lines are okay too, as long as things are
> > >     scaled corrected.  Instead of the line width being a hardcoded 78 or
> > >     specified the command line, nxstyle would parse the entire file,
> > >     look at the block comments, find the column of the last '*'  The
> > >     line width is that colument + 1;
> > >   * Then also verify that every block comment in the file is the same
> > width
> > >   * Perhaps the -m argument could change to a amount that you will
> > >     tolerate extending into the right margin.  So the block comment with
> > >     is 78, then -m 8 would allow lines to extend to 86.  That is better
> > >     than a hard-coded 86 because the line width will vary with file
> > >     content.  For some register definition files, the line width may be
> > >     120 or so.  In that case -m 8 would allow the line to extend to 128.
> > >
> > > 2. Verify that file section block comments are present.
> > >
> > >   * Add an enumeration value that indicates every block comment type.
> > >     See
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#cfilestructure
> > >     and
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#hfilestructure
> > >   * Currently logic detects only the presence of "* Private Functions"
> > >     and "* Private Functions".  Add logic to detect all other the block
> > >     comment types as well.
> > >   * Add a global variable to indicate which code block we are currently
> > >     in.  Each time a file section block comment is found, set the global
> > >     variable.
> > >   * The if '#   define" is encountered outside of the "* Pre-processor
> > >     Definitions" file section, emit an error
> > >   * If '#  include" is encountered outside of the "* Included Files"
> > >     section, emit an error.
> > >   * This logic could eventually be extended to assure that all static
> > >     functions are defined in the "* Private Functions" section, all
> > >     global functions are defined in the "* Public Functions section",
> > >     all static data definitions appear in the "* Private Data" section,
> > >     and all global data definitions appear in the "* Public data
> > >     definitions."
> > >   * We could also enable special parsing depending on the section.  That
> > >     is already done for the Public/Private Functions.  But we could, for
> > >     example also verify that variables defined at global scope start
> > >     with "g-"
> > >
> > > Let me know.  I was planning to do these things, but if Haitao Liu is
> > > interested, he can take over full responsibility for nxstyle.c.
> > >
> > > Greg
> > >
> > >
> > >
> > >
> > >
> >

RE: Working Effectively (was Point of Order)

[David Sidrane <Da...@nscdg.com>]

+1

Adding a tool that promotes violating the INVIOLABLES.txt on several aspect
(expedience and non conformant code into the repo) feels like a very bad
choice.

How is this being justified?

David


-----Original Message-----
From: Alin Jerpelea [mailto:jerpelea@gmail.com]
Sent: Tuesday, January 07, 2020 4:18 AM
To: dev@nuttx.apache.org
Subject: Re: Working Effectively (was Point of Order)

I think that we should update all code with nxstyle then we start with a
clean code on which patches will be checked

Alin

On Tue, Dec 31, 2019 at 3:42 PM Haitao Liu <li...@gmail.com> wrote:

> Greg,  I'll finish the nxstyle & check script these days firstly and then
> take time to handle the two issues you mentioned if allowed. I wish I
> could
> be competent to take over full responsibility for nxstyle.c.
>
> Gregory Nutt <sp...@gmail.com> 于2019年12月30日周一 下午11:31写道：
>
> > [moved from private@nuttx.apache.org]
> >
> > Hi, Xiao Xiang and Haitao Liu
> >
> > > Haitao is preparing a script for style check, the feature include:
> > > 1.Auto build nxstyle
> > > 2.Improve nxstyle to check the partial file for supporting patch like
> > file
> > > 3.Input can be the source files, patch file or commit id
> >
> > I mentioned that I had two things I planned to do for nxstyle.c too.
> > Perhaps Haitao would like to take on these changes too?
> >
> > 1. Derive maximum line width from block comments.  Ref:
> >
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#fileorganization
> >
> >   * The Maximum line width should be the same as the width of the file
> >     section block comment, by default the last asterisk '*' lies on
> >     column 78, but wider lines are okay too, as long as things are
> >     scaled corrected.  Instead of the line width being a hardcoded 78 or
> >     specified the command line, nxstyle would parse the entire file,
> >     look at the block comments, find the column of the last '*'  The
> >     line width is that colument + 1;
> >   * Then also verify that every block comment in the file is the same
> width
> >   * Perhaps the -m argument could change to a amount that you will
> >     tolerate extending into the right margin.  So the block comment with
> >     is 78, then -m 8 would allow lines to extend to 86.  That is better
> >     than a hard-coded 86 because the line width will vary with file
> >     content.  For some register definition files, the line width may be
> >     120 or so.  In that case -m 8 would allow the line to extend to 128.
> >
> > 2. Verify that file section block comments are present.
> >
> >   * Add an enumeration value that indicates every block comment type.
> >     See
> >
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#cfilestructure
> >     and
> >
> >
> https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#hfilestructure
> >   * Currently logic detects only the presence of "* Private Functions"
> >     and "* Private Functions".  Add logic to detect all other the block
> >     comment types as well.
> >   * Add a global variable to indicate which code block we are currently
> >     in.  Each time a file section block comment is found, set the global
> >     variable.
> >   * The if '#   define" is encountered outside of the "* Pre-processor
> >     Definitions" file section, emit an error
> >   * If '#  include" is encountered outside of the "* Included Files"
> >     section, emit an error.
> >   * This logic could eventually be extended to assure that all static
> >     functions are defined in the "* Private Functions" section, all
> >     global functions are defined in the "* Public Functions section",
> >     all static data definitions appear in the "* Private Data" section,
> >     and all global data definitions appear in the "* Public data
> >     definitions."
> >   * We could also enable special parsing depending on the section.  That
> >     is already done for the Public/Private Functions.  But we could, for
> >     example also verify that variables defined at global scope start
> >     with "g-"
> >
> > Let me know.  I was planning to do these things, but if Haitao Liu is
> > interested, he can take over full responsibility for nxstyle.c.
> >
> > Greg
> >
> >
> >
> >
> >
>

Re: Working Effectively (was Point of Order)

[Xiang Xiao <xi...@gmail.com>]

https://github.com/apache/incubator-nuttx/pull/47
checkpatch.sh is ready, you can invoke it like this:
tools/checkpatch.sh -p patch-list
tools/checkpatch.sh -c commit-list
tools/checkpatch.sh -f file-list
The first two just check the modified portion, the last one check the
whole content.
And with the below commands:
echo "git diff --cached | ./tools/checkpatch.sh -" > .git/hooks/pre-commit
chmod a+x .git/hooks/pre-commit
git will do the check each new commit automatically.
Haitao will incorporate it into the github in the next couple day.

On Tue, Jan 7, 2020 at 8:18 PM Alin Jerpelea <je...@gmail.com> wrote:
>
> I think that we should update all code with nxstyle then we start with a
> clean code on which patches will be checked
>

Should we distribute the job to committer or interesting people?

> Alin
>
> On Tue, Dec 31, 2019 at 3:42 PM Haitao Liu <li...@gmail.com> wrote:
>
> > Greg,  I'll finish the nxstyle & check script these days firstly and then
> > take time to handle the two issues you mentioned if allowed. I wish I could
> > be competent to take over full responsibility for nxstyle.c.
> >
> > Gregory Nutt <sp...@gmail.com> 于2019年12月30日周一 下午11:31写道：
> >
> > > [moved from private@nuttx.apache.org]
> > >
> > > Hi, Xiao Xiang and Haitao Liu
> > >
> > > > Haitao is preparing a script for style check, the feature include:
> > > > 1.Auto build nxstyle
> > > > 2.Improve nxstyle to check the partial file for supporting patch like
> > > file
> > > > 3.Input can be the source files, patch file or commit id
> > >
> > > I mentioned that I had two things I planned to do for nxstyle.c too.
> > > Perhaps Haitao would like to take on these changes too?
> > >
> > > 1. Derive maximum line width from block comments.  Ref:
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#fileorganization
> > >
> > >   * The Maximum line width should be the same as the width of the file
> > >     section block comment, by default the last asterisk '*' lies on
> > >     column 78, but wider lines are okay too, as long as things are
> > >     scaled corrected.  Instead of the line width being a hardcoded 78 or
> > >     specified the command line, nxstyle would parse the entire file,
> > >     look at the block comments, find the column of the last '*'  The
> > >     line width is that colument + 1;
> > >   * Then also verify that every block comment in the file is the same
> > width
> > >   * Perhaps the -m argument could change to a amount that you will
> > >     tolerate extending into the right margin.  So the block comment with
> > >     is 78, then -m 8 would allow lines to extend to 86.  That is better
> > >     than a hard-coded 86 because the line width will vary with file
> > >     content.  For some register definition files, the line width may be
> > >     120 or so.  In that case -m 8 would allow the line to extend to 128.
> > >
> > > 2. Verify that file section block comments are present.
> > >
> > >   * Add an enumeration value that indicates every block comment type.
> > >     See
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#cfilestructure
> > >     and
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NUTTX/Coding+Standard#hfilestructure
> > >   * Currently logic detects only the presence of "* Private Functions"
> > >     and "* Private Functions".  Add logic to detect all other the block
> > >     comment types as well.
> > >   * Add a global variable to indicate which code block we are currently
> > >     in.  Each time a file section block comment is found, set the global
> > >     variable.
> > >   * The if '#   define" is encountered outside of the "* Pre-processor
> > >     Definitions" file section, emit an error
> > >   * If '#  include" is encountered outside of the "* Included Files"
> > >     section, emit an error.
> > >   * This logic could eventually be extended to assure that all static
> > >     functions are defined in the "* Private Functions" section, all
> > >     global functions are defined in the "* Public Functions section",
> > >     all static data definitions appear in the "* Private Data" section,
> > >     and all global data definitions appear in the "* Public data
> > >     definitions."
> > >   * We could also enable special parsing depending on the section.  That
> > >     is already done for the Public/Private Functions.  But we could, for
> > >     example also verify that variables defined at global scope start
> > >     with "g-"
> > >
> > > Let me know.  I was planning to do these things, but if Haitao Liu is
> > > interested, he can take over full responsibility for nxstyle.c.
> > >
> > > Greg
> > >
> > >
> > >
> > >
> > >
> >