You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@netbeans.apache.org by Eric Bresie <eb...@gmail.com> on 2021/04/11 20:42:53 UTC

Time it takes for Test PHP Cluster

I was curious...any reason during the build checks that Test PHP Cluster takes so long? It seems like most of the checks are about 1 to 10 minutes but this seems to take closer to an hour.

I see a lot of things like the below.

[junit] ------------- Standard Error -----------------
9931 (https://github.com/apache/netbeans/runs/2318602490#step:7:9931) [junit] Apr 11, 2021 8:36:45 PM org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform

9932 (https://github.com/apache/netbeans/runs/2318602490#step:7:9932) [junit] INFO: Native file watcher is disabled

9933 (https://github.com/apache/netbeans/runs/2318602490#step:7:9933) [junit] Apr 11, 2021 8:36:46 PM org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>

9934 (https://github.com/apache/netbeans/runs/2318602490#step:7:9934) [junit] WARNING: Not listening on module system

9935 (https://github.com/apache/netbeans/runs/2318602490#step:7:9935) [junit] Apr 11, 2021 8:36:47 PM org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>

9936 (https://github.com/apache/netbeans/runs/2318602490#step:7:9936) [junit] INFO: Parsing of big PHP files enabled: false (max size: 5,000,000)

Eric Bresie
Ebresie@gmail.com (mailto:Ebresie@gmail.com)


Re: Using test distribution & Apache's Jenkins pipelines

Posted by Michael Bien <mb...@gmail.com>.
On 07.10.21 15:34, Neil C Smith wrote:
> On Thu, 7 Oct 2021 at 10:56, Eric Barboni <sk...@apache.org> wrote:
>> I would prefer to go to Apache Jenkins,
>>
>> The aim here is not  for PR validation it's more a "nightly" verification ?
> ...
>> My "release manager" POV on why jenkins for that particular aspect.
>>
>> (I'm not against travis (I prefer it greeeen :D)  or github action. GA may have some limitation, AFAIK Airflow own private repo on github to make their workflow.)
> Yes, think we'd all prefer those be greeeen! :-)
>
> I think we have to be careful with our CI usage, on whatever platform.
> And it might be time to consider dropping Travis - interesting stuff
> from Airflow around all this, eg. at
> https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub
>
> GitHub Actions also gives us the widest OS coverage?
>
> My thought would be -
>
> * Migrate important and stable tests to GitHub Actions for PR validation.
> * Drop Travis CI.
> * Switch on option for tests to pass before PR merge is enabled.
> * Look at a nightly verification step on Jenkins for wider testing,
> making sure we have good reporting from it.
> * Be more proactive about reverting problem PRs (eg. revert, fix,
> re-review rather than fix in place in most cases).

github actions make it also easy to selectively trigger certain test 
jobs. E.g it could trigger via PR labels, slash commands or parameters 
entered in the PR text. There is no need for nodes testing the php 
cluster for almost 2x 2h when something in the java editor changes.

i feel like every time i push to a PR someone has to cut down a tree 
somewhere and feed it to the testing cluster :)

whatever the new setup will be, a basic requirement for the usefulness 
of a CI server is to be able to actually display what failed. Having to 
go through a few meters of logs (in a browser) should not be acceptable 
for a CI server - it wastes simply too much time over the long term (and 
is no fun). I have a PR somewhere with a test failing on travis and 
there is no stack trace in the log of that failed test - its certainly 
somewhere in a xml but the current travis setup does not display those.

-michael

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Using test distribution & Apache's Jenkins pipelines

Posted by Neil C Smith <ne...@apache.org>.
On Thu, 7 Oct 2021 at 10:56, Eric Barboni <sk...@apache.org> wrote:
> I would prefer to go to Apache Jenkins,
>
> The aim here is not  for PR validation it's more a "nightly" verification ?
...
> My "release manager" POV on why jenkins for that particular aspect.
>
> (I'm not against travis (I prefer it greeeen :D)  or github action. GA may have some limitation, AFAIK Airflow own private repo on github to make their workflow.)

Yes, think we'd all prefer those be greeeen! :-)

I think we have to be careful with our CI usage, on whatever platform.
And it might be time to consider dropping Travis - interesting stuff
from Airflow around all this, eg. at
https://cwiki.apache.org/confluence/display/INFRA/Github+Actions+to+DockerHub

GitHub Actions also gives us the widest OS coverage?

My thought would be -

* Migrate important and stable tests to GitHub Actions for PR validation.
* Drop Travis CI.
* Switch on option for tests to pass before PR merge is enabled.
* Look at a nightly verification step on Jenkins for wider testing,
making sure we have good reporting from it.
* Be more proactive about reverting problem PRs (eg. revert, fix,
re-review rather than fix in place in most cases).

2c

Best wishes,

Neil

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




RE: Using test distribution & Apache's Jenkins pipelines

Posted by Eric Barboni <sk...@apache.org>.
I would prefer to go to Apache Jenkins,

The aim here is not  for PR validation it's more a "nightly" verification ?

Will try to prototype but not sure on how to use the test distribution.

--
My "release manager" POV on why jenkins for that particular aspect.

(I'm not against travis (I prefer it greeeen :D)  or github action. GA may have some limitation, AFAIK Airflow own private repo on github to make their workflow.)

It's just that for release we bake the bits on Apache Jenkins with no "test-platform". We assume travis is ok and netbeans-linux almost ok.
But on netbeans-linux, netbeans-windows on jenkins we have issue we do not have  on travis.
Example: (more on windows build)
On travis , my own wsl box: the following issue doesn't occurs [1] , https://github.com/apache/netbeans/pull/3205. Seems related on how the xml is generated on the jenkins node.


Best Regards
Eric

[1]
https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/lastCompletedBuild/testReport/org.openide.filesystems.annotations/LayerBuilderTest/testSourcePath/

-----Message d'origine-----
De : Jaroslav Tulach <ja...@gmail.com> 
Envoyé : jeudi 7 octobre 2021 08:30
À : Netbeans Developer List <de...@netbeans.apache.org>
Cc : Zajac Petr <pe...@gmail.com>
Objet : Re: Using test distribution & Apache's Jenkins pipelines

> So, what specifically needs to be done?

#1 from my perspective: We need support for build pipelines. In particular

```
var ide = buildIDEZip();
var testDist = buildTestDistributionZip();

var moduleSet = [ "platform", "ide", "hints1", "hints2" ]; var jdkSet = [ "jdk8", "jdk11", "jdk17", "jdk-latest" ];

new ForkJoinPool("many parallel executors") (module, jdk) {
   setup jdk

   unzip ide
   unzip testDist

   teste module
}
```

Can that be implemented (easily) in Travis? Not likely. Can that be implemented easily in GitHub actions (that'd be great, but I doubt). Can that be implemented in `Jenkinsfile`? Probably.

Can somebody prototype it? That'd be great.
-jt


Dne neděle 26. září 2021 19:01:03 CEST, Eric Bresie napsal(a):
> I know there have been discussions in the past on CI builds issues 
> like [1], [2]. [3], [4], [5], "Build Strategies" [6], and time on the 
> builds [7] but...
> 
> It seems some build jobs are done on GitHub commits, Travis CI, 
> Jenkins, etc.  Assume some of this is dependent on the type of jobs 
> and tests (OS or capability focus), and the infrastructure and 
> available resources involved (i.e. Apache CI-Jenkins across Apache 
> projects, GitHub build [across apache projects], Travis CI [assume constrained in some way]).
> 
> So, what specifically needs to be done?
> 
> Do the types of jobs need to be grouped and run by intent/needs in an 
> applicable CI environment?
> Do the types of jobs need to be migrated into a single place (i.e. 
> Github, Travis, Jenkins, etc.)?
> 
> Eric Bresie
> ebresie@gmail.com
> 
> References
> [1]
> https://lists.apache.org/thread.html/r944d5e9e79b6abdf9337293f417c3b51
> 229e8b 7d7989833028328359%40%3Cdev.netbeans.apache.org%3E [2]
> https://lists.apache.org/thread.html/rcddd3063fb087fb894eeea38b6c555c8
> 02c441 2e15a27ada1b4dccbe%40%3Cdev.netbeans.apache.org%3E [3] 
> https://lists.apache.org/thread.html/r3d55f95ee5dc6383a4330ee2cf4cf987
> 85149d f461063330bd1cb535%40%3Cdev.netbeans.apache.org%3E [4] 
> https://lists.apache.org/thread.html/r7937df039aea7254d6d6f0847e47a1bb
> 3f14ee 603d97bed58e8f7ccc%40%3Cdev.netbeans.apache.org%3E [5] 
> https://lists.apache.org/thread.html/r22e89737af4b1c0808d6f06385e5fd00
> c37a6b 435e7a81cf06619501%40%3Cdev.netbeans.apache.org%3E [6]
> https://lists.apache.org/thread.html/rcdcb7a718c01fec2796bdb472e979a89
> 00c346 efb9aebcc6024e31b9%40%3Cdev.netbeans.apache.org%3E [7]
> https://lists.apache.org/thread.html/r8694f9ca65746542f1ee1c73c219f17a
> cea5a5 27f5809d5c68394de6%40%3Cdev.netbeans.apache.org%3E
> 
> On Sat, Sep 25, 2021 at 1:52 AM Jaroslav Tulach 
> <ja...@gmail.com>
> 
> wrote:
> > Dne pátek 24. září 2021 17:25:39 CEST, Eric Barboni napsal(a):
> > > Hi,
> > > 
> > > I juste retrieved an old mail for Jaroslav.
> > 
> > ;-)
> > 
> > > I'm a bit "irritated" because of
> > > CI. I'm haunted  by the "restart job button".
> > 
> > I can live with the "restart button", but I agree: Such frequent 
> > failures aren't professional.
> > 
> > > Is a only Apache jenkins build + PR review something we can do ?
> > > or would we have some limitation and should rely also on GA or 
> > > travis
> > 
> > too?
> > 
> > > Hector started something 
> > > https://github.com/apache/netbeans/pull/2443
> > 
> > to do
> > 
> > > jenkins PR test.
> > > 
> > > BUT on jenkins
> > > This build trigger test error:
> > > https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/
> > > ant build test-platform build-nbms generate-uc-catalog 
> > > build-source-zips => 1 issue on 
> > > org.openide.filesystems.annotations.LayerBuilderTest.testSourcePat
> > > h The same ant call on my ubuntu => 0 issue
> > > 
> > > Complicated to trust jenkins too.
> > > But I would like to see the CI issues tackled so Release
> > 
> > Manager,commiter,
> > 
> > > reviewers may have better life :D. ------
> > 
> > +1
> > 
> > My original may was however about "test distribution" - e.g. build 
> > the IDE and the test distribution once, and then use it for running 
> > the tests across different nodes.
> > 
> > Yes, Jenkins pipelines (on Apache own infra) would be ideal for that 
> > I think.
> > -jt
> > 
> > > -----Message d'origine-----
> > > De : Jaroslav Tulach <ja...@gmail.com> Envoyé : lundi 26 
> > > avril 2021 06:16 À : dev@netbeans.apache.org Cc : Petr Zajac 
> > > <pe...@gmail.com> Objet : Using test distribution & Apache's 
> > > Jenkins pipelines was: Time it takes for Test PHP Cluster
> > > 
> > > Dne pátek 16. dubna 2021 18:53:31 CEST, Tomáš Procházka napsal(a):
> > > > I stareted conversion of other jobs
> > > > (https://github.com/apache/netbeans/pull/2708) but stopped given 
> > > > current status of Github Actions 
> > > > (https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actio
> > > > ns+sta
> > > > tus)
> > > 
> > > Hello guys,
> > > maybe there is a time to improve the structure of build jobs and 
> > > to
> > 
> > increase
> > 
> > > the throughput by eliminating duplicated tasks.
> > > 
> > > These days every job builds the IDE, builds the tests and then it
> > 
> > executes
> > 
> > > some tests. We can do better than that! There has been a "test 
> > > distribution" in NetBeans for years and according to its creator 
> > > Petr Zajac, it is still working!
> > > 
> > > It would be good to have one job that builds the ZIP of the IDE 
> > > and
> > 
> > builds
> > 
> > > the ZIP for the test distribution. Only then other "testing" jobs 
> > > are started, download the two ZIP files and use them to perform 
> > > the tests
> > 
> > only.
> > 
> > > Btw. When at it, shouldn't we consider to use
> > 
> > https://ci-builds.apache.org/
> > 
> > > ? It is an infrastructure that fully runs on Apache's hardware - 
> > > e.g. we aren't going to be influenced deals with external 
> > > providers like Travis
> > 
> > or
> > 
> > > Azure. Moreover Jenkins supports pipelines to pass the ZIPs from 
> > > one job
> > 
> > to
> > 
> > > another...
> > > 
> > > Together we can save the planet!
> > > -jt
> > > 
> > > 
> > > 
> > > 
> > > ------------------------------------------------------------------
> > > --- To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > > For additional commands, e-mail: dev-help@netbeans.apache.org
> > > 
> > > For further information about the NetBeans mailing lists, visit:
> > > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> > > 
> > > 
> > > 
> > > 
> > > 
> > > ------------------------------------------------------------------
> > > --- To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > > For additional commands, e-mail: dev-help@netbeans.apache.org
> > > 
> > > For further information about the NetBeans mailing lists, visit:
> > > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> > 
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > For additional commands, e-mail: dev-help@netbeans.apache.org
> > 
> > For further information about the NetBeans mailing lists, visit:
> > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists





---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists





---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Using test distribution & Apache's Jenkins pipelines

Posted by Eric Bresie <eb...@gmail.com>.
Would having some sort of "test matrix" to show where different parts of
build testing is currently (i.e. some current done in GA, is travis, in
jenkins) be worth wild?

Eric Bresie
ebresie@gmail.com


On Tue, Oct 12, 2021 at 11:41 PM Jaroslav Tulach <ja...@gmail.com>
wrote:

> Dne čtvrtek 7. října 2021 18:36:29 CEST, Michael Bien napsal(a):
> > On 07.10.21 08:30, Jaroslav Tulach wrote:
> > >> So, what specifically needs to be done?
> > >
> > > #1 from my perspective: We need support for build pipelines. In
> particular
> > >
> > > ```
> > > var ide = buildIDEZip();
> > > var testDist = buildTestDistributionZip();
> > >
> > > var moduleSet = [ "platform", "ide", "hints1", "hints2" ];
> > > var jdkSet = [ "jdk8", "jdk11", "jdk17", "jdk-latest" ];
> > >
> > > new ForkJoinPool("many parallel executors") (module, jdk) {
> > >
> > >     setup jdk
> > >
> > >     unzip ide
> > >     unzip testDist
> > >
> > >     teste module
> > >
> > > }
> > > ```
> > >
> > > Can that be implemented (easily) in Travis? Not likely. Can that be
> > > implemented easily in GitHub actions (that'd be great, but I doubt).
> >
> > i think one way of doing this with gh actions is to zip the build and
> > upload it as artifact. Subsequent jobs in the pipeline would use that
> > artifact and continue from there.
> >
> >
> https://docs.github.com/en/actions/learn-github-actions/essential-features-o
> > f-github-actions#sharing-data-between-jobs
> >
> > having artifacts would also mitigate the issue of being able to access
> > test results, if they don't end up in the console for whatever reason.
>
> It would be great to reconfigure the GitHub actions to do the build of IDE
> ZIP
> and test distribution ZIP only once.
> -jt
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
>

Re: Using test distribution & Apache's Jenkins pipelines

Posted by Jaroslav Tulach <ja...@gmail.com>.
Dne čtvrtek 7. října 2021 18:36:29 CEST, Michael Bien napsal(a):
> On 07.10.21 08:30, Jaroslav Tulach wrote:
> >> So, what specifically needs to be done?
> > 
> > #1 from my perspective: We need support for build pipelines. In particular
> > 
> > ```
> > var ide = buildIDEZip();
> > var testDist = buildTestDistributionZip();
> > 
> > var moduleSet = [ "platform", "ide", "hints1", "hints2" ];
> > var jdkSet = [ "jdk8", "jdk11", "jdk17", "jdk-latest" ];
> > 
> > new ForkJoinPool("many parallel executors") (module, jdk) {
> > 
> >     setup jdk
> >     
> >     unzip ide
> >     unzip testDist
> >     
> >     teste module
> > 
> > }
> > ```
> > 
> > Can that be implemented (easily) in Travis? Not likely. Can that be
> > implemented easily in GitHub actions (that'd be great, but I doubt).
> 
> i think one way of doing this with gh actions is to zip the build and
> upload it as artifact. Subsequent jobs in the pipeline would use that
> artifact and continue from there.
> 
> https://docs.github.com/en/actions/learn-github-actions/essential-features-o
> f-github-actions#sharing-data-between-jobs
> 
> having artifacts would also mitigate the issue of being able to access
> test results, if they don't end up in the console for whatever reason.

It would be great to reconfigure the GitHub actions to do the build of IDE ZIP 
and test distribution ZIP only once.
-jt




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Using test distribution & Apache's Jenkins pipelines

Posted by Michael Bien <mb...@gmail.com>.
On 07.10.21 08:30, Jaroslav Tulach wrote:
>> So, what specifically needs to be done?
> #1 from my perspective: We need support for build pipelines. In particular
>
> ```
> var ide = buildIDEZip();
> var testDist = buildTestDistributionZip();
>
> var moduleSet = [ "platform", "ide", "hints1", "hints2" ];
> var jdkSet = [ "jdk8", "jdk11", "jdk17", "jdk-latest" ];
>
> new ForkJoinPool("many parallel executors") (module, jdk) {
>     setup jdk
>
>     unzip ide
>     unzip testDist
>
>     teste module
> }
> ```
>
> Can that be implemented (easily) in Travis? Not likely. Can that be
> implemented easily in GitHub actions (that'd be great, but I doubt).

i think one way of doing this with gh actions is to zip the build and 
upload it as artifact. Subsequent jobs in the pipeline would use that 
artifact and continue from there.

https://docs.github.com/en/actions/learn-github-actions/essential-features-of-github-actions#sharing-data-between-jobs

having artifacts would also mitigate the issue of being able to access 
test results, if they don't end up in the console for whatever reason.

-mbien

>   Can that
> be implemented in `Jenkinsfile`? Probably.
>
> Can somebody prototype it? That'd be great.
> -jt
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Using test distribution & Apache's Jenkins pipelines

Posted by Jaroslav Tulach <ja...@gmail.com>.
> So, what specifically needs to be done?

#1 from my perspective: We need support for build pipelines. In particular

```
var ide = buildIDEZip();
var testDist = buildTestDistributionZip();

var moduleSet = [ "platform", "ide", "hints1", "hints2" ];
var jdkSet = [ "jdk8", "jdk11", "jdk17", "jdk-latest" ];

new ForkJoinPool("many parallel executors") (module, jdk) {
   setup jdk

   unzip ide
   unzip testDist

   teste module  
}
```

Can that be implemented (easily) in Travis? Not likely. Can that be 
implemented easily in GitHub actions (that'd be great, but I doubt). Can that 
be implemented in `Jenkinsfile`? Probably.

Can somebody prototype it? That'd be great.
-jt


Dne neděle 26. září 2021 19:01:03 CEST, Eric Bresie napsal(a):
> I know there have been discussions in the past on CI builds issues like
> [1], [2]. [3], [4], [5], "Build Strategies" [6], and time on the builds [7]
> but...
> 
> It seems some build jobs are done on GitHub commits, Travis CI, Jenkins,
> etc.  Assume some of this is dependent on the type of jobs and tests (OS or
> capability focus), and the infrastructure and available resources involved
> (i.e. Apache CI-Jenkins across Apache projects, GitHub build [across apache
> projects], Travis CI [assume constrained in some way]).
> 
> So, what specifically needs to be done?
> 
> Do the types of jobs need to be grouped and run by intent/needs in an
> applicable CI environment?
> Do the types of jobs need to be migrated into a single place (i.e. Github,
> Travis, Jenkins, etc.)?
> 
> Eric Bresie
> ebresie@gmail.com
> 
> References
> [1]
> https://lists.apache.org/thread.html/r944d5e9e79b6abdf9337293f417c3b51229e8b
> 7d7989833028328359%40%3Cdev.netbeans.apache.org%3E [2]
> https://lists.apache.org/thread.html/rcddd3063fb087fb894eeea38b6c555c802c441
> 2e15a27ada1b4dccbe%40%3Cdev.netbeans.apache.org%3E [3]
> https://lists.apache.org/thread.html/r3d55f95ee5dc6383a4330ee2cf4cf98785149d
> f461063330bd1cb535%40%3Cdev.netbeans.apache.org%3E [4]
> https://lists.apache.org/thread.html/r7937df039aea7254d6d6f0847e47a1bb3f14ee
> 603d97bed58e8f7ccc%40%3Cdev.netbeans.apache.org%3E [5]
> https://lists.apache.org/thread.html/r22e89737af4b1c0808d6f06385e5fd00c37a6b
> 435e7a81cf06619501%40%3Cdev.netbeans.apache.org%3E [6]
> https://lists.apache.org/thread.html/rcdcb7a718c01fec2796bdb472e979a8900c346
> efb9aebcc6024e31b9%40%3Cdev.netbeans.apache.org%3E [7]
> https://lists.apache.org/thread.html/r8694f9ca65746542f1ee1c73c219f17acea5a5
> 27f5809d5c68394de6%40%3Cdev.netbeans.apache.org%3E
> 
> On Sat, Sep 25, 2021 at 1:52 AM Jaroslav Tulach <ja...@gmail.com>
> 
> wrote:
> > Dne pátek 24. září 2021 17:25:39 CEST, Eric Barboni napsal(a):
> > > Hi,
> > > 
> > > I juste retrieved an old mail for Jaroslav.
> > 
> > ;-)
> > 
> > > I'm a bit "irritated" because of
> > > CI. I'm haunted  by the "restart job button".
> > 
> > I can live with the "restart button", but I agree: Such frequent failures
> > aren't professional.
> > 
> > > Is a only Apache jenkins build + PR review something we can do ?
> > > or would we have some limitation and should rely also on GA or travis
> > 
> > too?
> > 
> > > Hector started something https://github.com/apache/netbeans/pull/2443
> > 
> > to do
> > 
> > > jenkins PR test.
> > > 
> > > BUT on jenkins
> > > This build trigger test error:
> > > https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/
> > > ant build test-platform build-nbms generate-uc-catalog build-source-zips
> > > => 1 issue on
> > > org.openide.filesystems.annotations.LayerBuilderTest.testSourcePath The
> > > same ant call on my ubuntu
> > > => 0 issue
> > > 
> > > Complicated to trust jenkins too.
> > > But I would like to see the CI issues tackled so Release
> > 
> > Manager,commiter,
> > 
> > > reviewers may have better life :D. ------
> > 
> > +1
> > 
> > My original may was however about "test distribution" - e.g. build the IDE
> > and
> > the test distribution once, and then use it for running the tests across
> > different nodes.
> > 
> > Yes, Jenkins pipelines (on Apache own infra) would be ideal for that I
> > think.
> > -jt
> > 
> > > -----Message d'origine-----
> > > De : Jaroslav Tulach <ja...@gmail.com>
> > > Envoyé : lundi 26 avril 2021 06:16
> > > À : dev@netbeans.apache.org
> > > Cc : Petr Zajac <pe...@gmail.com>
> > > Objet : Using test distribution & Apache's Jenkins pipelines was: Time
> > > it
> > > takes for Test PHP Cluster
> > > 
> > > Dne pátek 16. dubna 2021 18:53:31 CEST, Tomáš Procházka napsal(a):
> > > > I stareted conversion of other jobs
> > > > (https://github.com/apache/netbeans/pull/2708) but stopped given
> > > > current status of Github Actions
> > > > (https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+sta
> > > > tus)
> > > 
> > > Hello guys,
> > > maybe there is a time to improve the structure of build jobs and to
> > 
> > increase
> > 
> > > the throughput by eliminating duplicated tasks.
> > > 
> > > These days every job builds the IDE, builds the tests and then it
> > 
> > executes
> > 
> > > some tests. We can do better than that! There has been a "test
> > > distribution" in NetBeans for years and according to its creator Petr
> > > Zajac, it is still working!
> > > 
> > > It would be good to have one job that builds the ZIP of the IDE and
> > 
> > builds
> > 
> > > the ZIP for the test distribution. Only then other "testing" jobs are
> > > started, download the two ZIP files and use them to perform the tests
> > 
> > only.
> > 
> > > Btw. When at it, shouldn't we consider to use
> > 
> > https://ci-builds.apache.org/
> > 
> > > ? It is an infrastructure that fully runs on Apache's hardware - e.g. we
> > > aren't going to be influenced deals with external providers like Travis
> > 
> > or
> > 
> > > Azure. Moreover Jenkins supports pipelines to pass the ZIPs from one job
> > 
> > to
> > 
> > > another...
> > > 
> > > Together we can save the planet!
> > > -jt
> > > 
> > > 
> > > 
> > > 
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > > For additional commands, e-mail: dev-help@netbeans.apache.org
> > > 
> > > For further information about the NetBeans mailing lists, visit:
> > > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> > > 
> > > 
> > > 
> > > 
> > > 
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > > For additional commands, e-mail: dev-help@netbeans.apache.org
> > > 
> > > For further information about the NetBeans mailing lists, visit:
> > > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > For additional commands, e-mail: dev-help@netbeans.apache.org
> > 
> > For further information about the NetBeans mailing lists, visit:
> > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists





---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Using test distribution & Apache's Jenkins pipelines was: Time it takes for Test PHP Cluster

Posted by Eric Bresie <eb...@gmail.com>.
I know there have been discussions in the past on CI builds issues like
[1], [2]. [3], [4], [5], "Build Strategies" [6], and time on the builds [7]
but...

It seems some build jobs are done on GitHub commits, Travis CI, Jenkins,
etc.  Assume some of this is dependent on the type of jobs and tests (OS or
capability focus), and the infrastructure and available resources involved
(i.e. Apache CI-Jenkins across Apache projects, GitHub build [across apache
projects], Travis CI [assume constrained in some way]).

So, what specifically needs to be done?

Do the types of jobs need to be grouped and run by intent/needs in an
applicable CI environment?
Do the types of jobs need to be migrated into a single place (i.e. Github,
Travis, Jenkins, etc.)?

Eric Bresie
ebresie@gmail.com

References
[1]
https://lists.apache.org/thread.html/r944d5e9e79b6abdf9337293f417c3b51229e8b7d7989833028328359%40%3Cdev.netbeans.apache.org%3E
[2]
https://lists.apache.org/thread.html/rcddd3063fb087fb894eeea38b6c555c802c4412e15a27ada1b4dccbe%40%3Cdev.netbeans.apache.org%3E
[3]
https://lists.apache.org/thread.html/r3d55f95ee5dc6383a4330ee2cf4cf98785149df461063330bd1cb535%40%3Cdev.netbeans.apache.org%3E
[4]
https://lists.apache.org/thread.html/r7937df039aea7254d6d6f0847e47a1bb3f14ee603d97bed58e8f7ccc%40%3Cdev.netbeans.apache.org%3E
[5]
https://lists.apache.org/thread.html/r22e89737af4b1c0808d6f06385e5fd00c37a6b435e7a81cf06619501%40%3Cdev.netbeans.apache.org%3E
[6]
https://lists.apache.org/thread.html/rcdcb7a718c01fec2796bdb472e979a8900c346efb9aebcc6024e31b9%40%3Cdev.netbeans.apache.org%3E
[7]
https://lists.apache.org/thread.html/r8694f9ca65746542f1ee1c73c219f17acea5a527f5809d5c68394de6%40%3Cdev.netbeans.apache.org%3E

On Sat, Sep 25, 2021 at 1:52 AM Jaroslav Tulach <ja...@gmail.com>
wrote:

> Dne pátek 24. září 2021 17:25:39 CEST, Eric Barboni napsal(a):
> > Hi,
> >
> > I juste retrieved an old mail for Jaroslav.
>
> ;-)
>
> > I'm a bit "irritated" because of
> > CI. I'm haunted  by the "restart job button".
>
> I can live with the "restart button", but I agree: Such frequent failures
> aren't professional.
>
> > Is a only Apache jenkins build + PR review something we can do ?
> > or would we have some limitation and should rely also on GA or travis
> too?
> >
> >
> > Hector started something https://github.com/apache/netbeans/pull/2443
> to do
> > jenkins PR test.
> >
> > BUT on jenkins
> > This build trigger test error:
> > https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/
> > ant build test-platform build-nbms generate-uc-catalog build-source-zips
> > => 1 issue on
> > org.openide.filesystems.annotations.LayerBuilderTest.testSourcePath The
> > same ant call on my ubuntu
> > => 0 issue
> >
> > Complicated to trust jenkins too.
> > But I would like to see the CI issues tackled so Release
> Manager,commiter,
> > reviewers may have better life :D. ------
>
> +1
>
> My original may was however about "test distribution" - e.g. build the IDE
> and
> the test distribution once, and then use it for running the tests across
> different nodes.
>
> Yes, Jenkins pipelines (on Apache own infra) would be ideal for that I
> think.
> -jt
>
>
>
> > -----Message d'origine-----
> > De : Jaroslav Tulach <ja...@gmail.com>
> > Envoyé : lundi 26 avril 2021 06:16
> > À : dev@netbeans.apache.org
> > Cc : Petr Zajac <pe...@gmail.com>
> > Objet : Using test distribution & Apache's Jenkins pipelines was: Time it
> > takes for Test PHP Cluster
> > Dne pátek 16. dubna 2021 18:53:31 CEST, Tomáš Procházka napsal(a):
> > > I stareted conversion of other jobs
> > > (https://github.com/apache/netbeans/pull/2708) but stopped given
> > > current status of Github Actions
> > > (https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+sta
> > > tus)
> >
> > Hello guys,
> > maybe there is a time to improve the structure of build jobs and to
> increase
> > the throughput by eliminating duplicated tasks.
> >
> > These days every job builds the IDE, builds the tests and then it
> executes
> > some tests. We can do better than that! There has been a "test
> > distribution" in NetBeans for years and according to its creator Petr
> > Zajac, it is still working!
> >
> > It would be good to have one job that builds the ZIP of the IDE and
> builds
> > the ZIP for the test distribution. Only then other "testing" jobs are
> > started, download the two ZIP files and use them to perform the tests
> only.
> >
> > Btw. When at it, shouldn't we consider to use
> https://ci-builds.apache.org/
> > ? It is an infrastructure that fully runs on Apache's hardware - e.g. we
> > aren't going to be influenced deals with external providers like Travis
> or
> > Azure. Moreover Jenkins supports pipelines to pass the ZIPs from one job
> to
> > another...
> >
> > Together we can save the planet!
> > -jt
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > For additional commands, e-mail: dev-help@netbeans.apache.org
> >
> > For further information about the NetBeans mailing lists, visit:
> > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> > For additional commands, e-mail: dev-help@netbeans.apache.org
> >
> > For further information about the NetBeans mailing lists, visit:
> > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
>

Re: Using test distribution & Apache's Jenkins pipelines was: Time it takes for Test PHP Cluster

Posted by Jaroslav Tulach <ja...@gmail.com>.
Dne pátek 24. září 2021 17:25:39 CEST, Eric Barboni napsal(a):
> Hi,
> 
> I juste retrieved an old mail for Jaroslav. 

;-)

> I'm a bit "irritated" because of
> CI. I'm haunted  by the "restart job button".

I can live with the "restart button", but I agree: Such frequent failures 
aren't professional.

> Is a only Apache jenkins build + PR review something we can do ?
> or would we have some limitation and should rely also on GA or travis too?
> 
> 
> Hector started something https://github.com/apache/netbeans/pull/2443 to do
> jenkins PR test.
> 
> BUT on jenkins
> This build trigger test error:
> https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/
> ant build test-platform build-nbms generate-uc-catalog build-source-zips
> => 1 issue on
> org.openide.filesystems.annotations.LayerBuilderTest.testSourcePath The
> same ant call on my ubuntu
> => 0 issue
> 
> Complicated to trust jenkins too.
> But I would like to see the CI issues tackled so Release Manager,commiter,
> reviewers may have better life :D. ------

+1

My original may was however about "test distribution" - e.g. build the IDE and 
the test distribution once, and then use it for running the tests across 
different nodes.

Yes, Jenkins pipelines (on Apache own infra) would be ideal for that I think.
-jt



> -----Message d'origine-----
> De : Jaroslav Tulach <ja...@gmail.com>
> Envoyé : lundi 26 avril 2021 06:16
> À : dev@netbeans.apache.org
> Cc : Petr Zajac <pe...@gmail.com>
> Objet : Using test distribution & Apache's Jenkins pipelines was: Time it
> takes for Test PHP Cluster
> Dne pátek 16. dubna 2021 18:53:31 CEST, Tomáš Procházka napsal(a):
> > I stareted conversion of other jobs
> > (https://github.com/apache/netbeans/pull/2708) but stopped given
> > current status of Github Actions
> > (https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+sta
> > tus)
> 
> Hello guys,
> maybe there is a time to improve the structure of build jobs and to increase
> the throughput by eliminating duplicated tasks.
> 
> These days every job builds the IDE, builds the tests and then it executes
> some tests. We can do better than that! There has been a "test
> distribution" in NetBeans for years and according to its creator Petr
> Zajac, it is still working!
> 
> It would be good to have one job that builds the ZIP of the IDE and builds
> the ZIP for the test distribution. Only then other "testing" jobs are
> started, download the two ZIP files and use them to perform the tests only.
> 
> Btw. When at it, shouldn't we consider to use https://ci-builds.apache.org/
> ? It is an infrastructure that fully runs on Apache's hardware - e.g. we
> aren't going to be influenced deals with external providers like Travis or
> Azure. Moreover Jenkins supports pipelines to pass the ZIPs from one job to
> another...
> 
> Together we can save the planet!
> -jt
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
> 
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> 
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
> 
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists





---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




RE: Using test distribution & Apache's Jenkins pipelines was: Time it takes for Test PHP Cluster

Posted by Eric Barboni <sk...@apache.org>.
Hi,

I juste retrieved an old mail for Jaroslav. I'm a bit "irritated" because of CI. I'm haunted  by the "restart job button".

Is a only Apache jenkins build + PR review something we can do ?
or would we have some limitation and should rely also on GA or travis too?


Hector started something https://github.com/apache/netbeans/pull/2443 to do jenkins PR test.

BUT on jenkins
This build trigger test error:
https://ci-builds.apache.org/job/Netbeans/job/netbeans-linux/
ant build test-platform build-nbms generate-uc-catalog build-source-zips 
=> 1 issue on org.openide.filesystems.annotations.LayerBuilderTest.testSourcePath
The same ant call on my ubuntu
=> 0 issue

Complicated to trust jenkins too. 
But I would like to see the CI issues tackled so Release Manager,commiter, reviewers may have better life :D.
------

Best Regards
Eric


-----Message d'origine-----
De : Jaroslav Tulach <ja...@gmail.com> 
Envoyé : lundi 26 avril 2021 06:16
À : dev@netbeans.apache.org
Cc : Petr Zajac <pe...@gmail.com>
Objet : Using test distribution & Apache's Jenkins pipelines was: Time it takes for Test PHP Cluster

Dne pátek 16. dubna 2021 18:53:31 CEST, Tomáš Procházka napsal(a):
> I stareted conversion of other jobs
> (https://github.com/apache/netbeans/pull/2708) but stopped given 
> current status of Github Actions
> (https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+sta
> tus)

Hello guys,
maybe there is a time to improve the structure of build jobs and to increase the throughput by eliminating duplicated tasks.

These days every job builds the IDE, builds the tests and then it executes some tests. We can do better than that! There has been a "test distribution" 
in NetBeans for years and according to its creator Petr Zajac, it is still working!

It would be good to have one job that builds the ZIP of the IDE and builds the ZIP for the test distribution. Only then other "testing" jobs are started, download the two ZIP files and use them to perform the tests only.

Btw. When at it, shouldn't we consider to use https://ci-builds.apache.org/ ? 
It is an infrastructure that fully runs on Apache's hardware - e.g. we aren't going to be influenced deals with external providers like Travis or Azure. 
Moreover Jenkins supports pipelines to pass the ZIPs from one job to another...

Together we can save the planet!
-jt




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists





---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Using test distribution & Apache's Jenkins pipelines was: Time it takes for Test PHP Cluster

Posted by Jaroslav Tulach <ja...@gmail.com>.
Dne pátek 16. dubna 2021 18:53:31 CEST, Tomáš Procházka napsal(a):
> I stareted conversion of other jobs
> (https://github.com/apache/netbeans/pull/2708) but stopped given current
> status of Github Actions
> (https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status)

Hello guys,
maybe there is a time to improve the structure of build jobs and to increase 
the throughput by eliminating duplicated tasks.

These days every job builds the IDE, builds the tests and then it executes 
some tests. We can do better than that! There has been a "test distribution" 
in NetBeans for years and according to its creator Petr Zajac, it is still 
working!

It would be good to have one job that builds the ZIP of the IDE and builds the 
ZIP for the test distribution. Only then other "testing" jobs are started, 
download the two ZIP files and use them to perform the tests only.

Btw. When at it, shouldn't we consider to use https://ci-builds.apache.org/ ? 
It is an infrastructure that fully runs on Apache's hardware - e.g. we aren't 
going to be influenced deals with external providers like Travis or Azure. 
Moreover Jenkins supports pipelines to pass the ZIPs from one job to 
another...

Together we can save the planet!
-jt




---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Time it takes for Test PHP Cluster

Posted by antonio <an...@vieiro.net>.
Hi all,

There's a recent dscussion (Apr 7) in the "builds.apache.org" mailing 
list [1] entitled "Increase the number of parallel jobs in GitHub 
Actions at ASF organization level" that may be of interest to us.

What Greg Stein (ASF Infra, I think) says is that we should be grateful 
for the free minutes we have in Github Actions, provided by Microsoft, 
and that these free minutes are finite :-).

Moving unit tests to Github Actions is something that will decrease that 
finite amount of free minutes, if I understand correctly. Once those 
free minutes end, the thing will fail (or take longer?).

Kind regards,
Antonio

P.D.: Unit tests are good for us and for our users. Thanks, Junichi, for 
those in the PHP cluster! Way to go!

[1]
http://mail-archives.apache.org/mod_mbox/www-builds/202104.mbox/%3CCABD8fLXAtLGn5aENjUKWtttVjpE1ezTDteDKMP7NaiBBtRcXRw%40mail.gmail.com%3E

On 16/04/2021 18:53, Tomáš Procházka wrote:
> Hi,
> I moved all PHP tests from Travis to Github Actions because tests for 
> PHP Editor were running too long. Travis always timed out.
> It was easier to setup Windows build for PHP Editor in Github Actions.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Time it takes for Test PHP Cluster

Posted by Tomáš Procházka <ka...@razdva.cz>.
Hi,
I moved all PHP tests from Travis to Github Actions because tests for 
PHP Editor were running too long. Travis always timed out.
It was easier to setup Windows build for PHP Editor in Github Actions.

Some tests were OK under Linux but broken when run on Windows, some 
tests were broken from different reasons (files missing from donation, 
different dorectory separator).

Rough overview of fixes: 
https://github.com/apache/netbeans/pulls?q=is%3Apr+author%3AKacerCZ+is%3Aclosed+milestone%3A12.1+tests

I stareted conversion of other jobs 
(https://github.com/apache/netbeans/pull/2708) but stopped given current 
status of Github Actions 
(https://cwiki.apache.org/confluence/display/BUILDS/GitHub+Actions+status)

Regards,
Tom

On 16. 04. 21 16:25, Eric Bresie wrote:
>> Where is the configuration for these build checks setup?  Is there a
>> specific yaml or part of a build script someplace?
> 
> Did more poking here and see
> 
>     - .github/workflows/main.xml file
>        - Assume  this drives the initial "check" jobs.
>        - This has the "Test" job with limited sub-tests defined, compared to
>        the "Test PHP Cluster" which defines a lot more of the php sub
> test cases.
>     - travis.yaml file
>        - Assume this is used in the last phase in the "Travis CI" portion.
>        - This has a lot more test coverage defined here for other areas.
> 
> So at the moment some testing is done at the github action context and some
> done in the "travis" context.
> 
> So there are more tests in the github contexts for php and less for other
> areas while in the travis CI context there is more test coverage in other
> areas and less in php.  Assume there is additional time that I was not
> seeing in the travis context for others tests compared to more time for php
> in the github action context.
> 
> Assume this split between the two areas for testing is related to the
> discussion on "TravisCI vs Jenkins vs GitHub Actions" [1].  Sounded like
> there may be some resource concerns and/or Apache infrastructure
> 
> Will the builds continue to use github actions + travis or are there any
> plans to migrate in one way or another?
> 
> If moving towards github action based, assume other tests would need to be
> moved to the github testing (i.e. more "Test x Cluster" with applicable run
> tests for each as applicable in main.xml).
> 
> Was anything decided on "Jenkins" usage?
> 
> References
> [1]
> https://lists.apache.org/thread.html/rc3a7ad290b004557c69d8cac7f5bd4825ca3d230b626641a8565164b%40%3Cdev.netbeans.apache.org%3E
> 
> Eric Bresie
> ebresie@gmail.com
> 
> 
> 
> On Fri, Apr 16, 2021 at 8:10 AM Geertjan Wielenga
> <ge...@googlemail.com.invalid> wrote:
> 
>> Because the others have less tests.
>>
>> Gj
>>
>> On Fri, Apr 16, 2021 at 2:59 PM Eric Bresie <eb...@gmail.com> wrote:
>>
>>> Let me change the question a little...Why are all the others quicker
>>> compared to the php side?
>>> Do the other clusters / jobs / checks not get tested in the same way
>> (i.e.
>>> only a subset of tests compare to all tests in php test if so why)?  Are
>>> there fewer tests (I would assume not as I've run some of the unit tests
>> on
>>> just one project which ?  Do the tests in other context require some "UI"
>>> based tests not possible at build time compared to php which can be UI
>> less
>>> (i.e. request/response)?Are the other tests faster compared to slower php
>>> tests?  Does the php test require starting/stopping some php server for
>> use
>>> during the test?
>>>
>>> Just a snapshot of few occurrences, not the "Test PHP Cluster" check
>>> compared to others
>>>
>>> For the Windows build
>>>
>>> [image: image.png]
>>>
>>> For the Linux of PHP cluster
>>>
>>> [image: image.png]
>>>
>>> Where is the configuration for these build checks setup?  Is there a
>>> specific yaml or part of a build script someplace?
>>>
>>> Eric Bresie
>>> ebresie@gmail.com
>>>
>>>
>>> On Thu, Apr 15, 2021 at 2:51 PM Junichi Yamamoto <ju...@gmail.com>
>>> wrote:
>>>
>>>> Because PHP editor has many unit tests.
>>>>
>>>> Junichi
>>>>
>>>>
>>>> On Mon, Apr 12, 2021 at 5:43 AM Eric Bresie <eb...@gmail.com> wrote:
>>>>>
>>>>> I was curious...any reason during the build checks that Test PHP
>>>> Cluster takes so long? It seems like most of the checks are about 1 to
>> 10
>>>> minutes but this seems to take closer to an hour.
>>>>>
>>>>> I see a lot of things like the below.
>>>>>
>>>>> [junit] ------------- Standard Error -----------------
>>>>> 9931 (https://github.com/apache/netbeans/runs/2318602490#step:7:9931)
>>>> [junit] Apr 11, 2021 8:36:45 PM
>>>> org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform
>>>>>
>>>>> 9932 (https://github.com/apache/netbeans/runs/2318602490#step:7:9932)
>>>> [junit] INFO: Native file watcher is disabled
>>>>>
>>>>> 9933 (https://github.com/apache/netbeans/runs/2318602490#step:7:9933)
>>>> [junit] Apr 11, 2021 8:36:46 PM
>>>> org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>
>>>>>
>>>>> 9934 (https://github.com/apache/netbeans/runs/2318602490#step:7:9934)
>>>> [junit] WARNING: Not listening on module system
>>>>>
>>>>> 9935 (https://github.com/apache/netbeans/runs/2318602490#step:7:9935)
>>>> [junit] Apr 11, 2021 8:36:47 PM
>>>> org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>
>>>>>
>>>>> 9936 (https://github.com/apache/netbeans/runs/2318602490#step:7:9936)
>>>> [junit] INFO: Parsing of big PHP files enabled: false (max size:
>> 5,000,000)
>>>>>
>>>>> Eric Bresie
>>>>> Ebresie@gmail.com (mailto:Ebresie@gmail.com)
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
>>>> For additional commands, e-mail: dev-help@netbeans.apache.org
>>>>
>>>> For further information about the NetBeans mailing lists, visit:
>>>> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>>>>
>>>>
>>>>
>>>>
>>
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Time it takes for Test PHP Cluster

Posted by Eric Bresie <eb...@gmail.com>.
> Where is the configuration for these build checks setup?  Is there a
> specific yaml or part of a build script someplace?

Did more poking here and see

   - .github/workflows/main.xml file
      - Assume  this drives the initial "check" jobs.
      - This has the "Test" job with limited sub-tests defined, compared to
      the "Test PHP Cluster" which defines a lot more of the php sub
test cases.
   - travis.yaml file
      - Assume this is used in the last phase in the "Travis CI" portion.
      - This has a lot more test coverage defined here for other areas.

So at the moment some testing is done at the github action context and some
done in the "travis" context.

So there are more tests in the github contexts for php and less for other
areas while in the travis CI context there is more test coverage in other
areas and less in php.  Assume there is additional time that I was not
seeing in the travis context for others tests compared to more time for php
in the github action context.

Assume this split between the two areas for testing is related to the
discussion on "TravisCI vs Jenkins vs GitHub Actions" [1].  Sounded like
there may be some resource concerns and/or Apache infrastructure

Will the builds continue to use github actions + travis or are there any
plans to migrate in one way or another?

If moving towards github action based, assume other tests would need to be
moved to the github testing (i.e. more "Test x Cluster" with applicable run
tests for each as applicable in main.xml).

Was anything decided on "Jenkins" usage?

References
[1]
https://lists.apache.org/thread.html/rc3a7ad290b004557c69d8cac7f5bd4825ca3d230b626641a8565164b%40%3Cdev.netbeans.apache.org%3E

Eric Bresie
ebresie@gmail.com



On Fri, Apr 16, 2021 at 8:10 AM Geertjan Wielenga
<ge...@googlemail.com.invalid> wrote:

> Because the others have less tests.
>
> Gj
>
> On Fri, Apr 16, 2021 at 2:59 PM Eric Bresie <eb...@gmail.com> wrote:
>
> > Let me change the question a little...Why are all the others quicker
> > compared to the php side?
> > Do the other clusters / jobs / checks not get tested in the same way
> (i.e.
> > only a subset of tests compare to all tests in php test if so why)?  Are
> > there fewer tests (I would assume not as I've run some of the unit tests
> on
> > just one project which ?  Do the tests in other context require some "UI"
> > based tests not possible at build time compared to php which can be UI
> less
> > (i.e. request/response)?Are the other tests faster compared to slower php
> > tests?  Does the php test require starting/stopping some php server for
> use
> > during the test?
> >
> > Just a snapshot of few occurrences, not the "Test PHP Cluster" check
> > compared to others
> >
> > For the Windows build
> >
> > [image: image.png]
> >
> > For the Linux of PHP cluster
> >
> > [image: image.png]
> >
> > Where is the configuration for these build checks setup?  Is there a
> > specific yaml or part of a build script someplace?
> >
> > Eric Bresie
> > ebresie@gmail.com
> >
> >
> > On Thu, Apr 15, 2021 at 2:51 PM Junichi Yamamoto <ju...@gmail.com>
> > wrote:
> >
> >> Because PHP editor has many unit tests.
> >>
> >> Junichi
> >>
> >>
> >> On Mon, Apr 12, 2021 at 5:43 AM Eric Bresie <eb...@gmail.com> wrote:
> >> >
> >> > I was curious...any reason during the build checks that Test PHP
> >> Cluster takes so long? It seems like most of the checks are about 1 to
> 10
> >> minutes but this seems to take closer to an hour.
> >> >
> >> > I see a lot of things like the below.
> >> >
> >> > [junit] ------------- Standard Error -----------------
> >> > 9931 (https://github.com/apache/netbeans/runs/2318602490#step:7:9931)
> >> [junit] Apr 11, 2021 8:36:45 PM
> >> org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform
> >> >
> >> > 9932 (https://github.com/apache/netbeans/runs/2318602490#step:7:9932)
> >> [junit] INFO: Native file watcher is disabled
> >> >
> >> > 9933 (https://github.com/apache/netbeans/runs/2318602490#step:7:9933)
> >> [junit] Apr 11, 2021 8:36:46 PM
> >> org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>
> >> >
> >> > 9934 (https://github.com/apache/netbeans/runs/2318602490#step:7:9934)
> >> [junit] WARNING: Not listening on module system
> >> >
> >> > 9935 (https://github.com/apache/netbeans/runs/2318602490#step:7:9935)
> >> [junit] Apr 11, 2021 8:36:47 PM
> >> org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>
> >> >
> >> > 9936 (https://github.com/apache/netbeans/runs/2318602490#step:7:9936)
> >> [junit] INFO: Parsing of big PHP files enabled: false (max size:
> 5,000,000)
> >> >
> >> > Eric Bresie
> >> > Ebresie@gmail.com (mailto:Ebresie@gmail.com)
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> >> For additional commands, e-mail: dev-help@netbeans.apache.org
> >>
> >> For further information about the NetBeans mailing lists, visit:
> >> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
> >>
> >>
> >>
> >>
>

Re: Time it takes for Test PHP Cluster

Posted by Geertjan Wielenga <ge...@googlemail.com.INVALID>.
Because the others have less tests.

Gj

On Fri, Apr 16, 2021 at 2:59 PM Eric Bresie <eb...@gmail.com> wrote:

> Let me change the question a little...Why are all the others quicker
> compared to the php side?
> Do the other clusters / jobs / checks not get tested in the same way (i.e.
> only a subset of tests compare to all tests in php test if so why)?  Are
> there fewer tests (I would assume not as I've run some of the unit tests on
> just one project which ?  Do the tests in other context require some "UI"
> based tests not possible at build time compared to php which can be UI less
> (i.e. request/response)?Are the other tests faster compared to slower php
> tests?  Does the php test require starting/stopping some php server for use
> during the test?
>
> Just a snapshot of few occurrences, not the "Test PHP Cluster" check
> compared to others
>
> For the Windows build
>
> [image: image.png]
>
> For the Linux of PHP cluster
>
> [image: image.png]
>
> Where is the configuration for these build checks setup?  Is there a
> specific yaml or part of a build script someplace?
>
> Eric Bresie
> ebresie@gmail.com
>
>
> On Thu, Apr 15, 2021 at 2:51 PM Junichi Yamamoto <ju...@gmail.com>
> wrote:
>
>> Because PHP editor has many unit tests.
>>
>> Junichi
>>
>>
>> On Mon, Apr 12, 2021 at 5:43 AM Eric Bresie <eb...@gmail.com> wrote:
>> >
>> > I was curious...any reason during the build checks that Test PHP
>> Cluster takes so long? It seems like most of the checks are about 1 to 10
>> minutes but this seems to take closer to an hour.
>> >
>> > I see a lot of things like the below.
>> >
>> > [junit] ------------- Standard Error -----------------
>> > 9931 (https://github.com/apache/netbeans/runs/2318602490#step:7:9931)
>> [junit] Apr 11, 2021 8:36:45 PM
>> org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform
>> >
>> > 9932 (https://github.com/apache/netbeans/runs/2318602490#step:7:9932)
>> [junit] INFO: Native file watcher is disabled
>> >
>> > 9933 (https://github.com/apache/netbeans/runs/2318602490#step:7:9933)
>> [junit] Apr 11, 2021 8:36:46 PM
>> org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>
>> >
>> > 9934 (https://github.com/apache/netbeans/runs/2318602490#step:7:9934)
>> [junit] WARNING: Not listening on module system
>> >
>> > 9935 (https://github.com/apache/netbeans/runs/2318602490#step:7:9935)
>> [junit] Apr 11, 2021 8:36:47 PM
>> org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>
>> >
>> > 9936 (https://github.com/apache/netbeans/runs/2318602490#step:7:9936)
>> [junit] INFO: Parsing of big PHP files enabled: false (max size: 5,000,000)
>> >
>> > Eric Bresie
>> > Ebresie@gmail.com (mailto:Ebresie@gmail.com)
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
>> For additional commands, e-mail: dev-help@netbeans.apache.org
>>
>> For further information about the NetBeans mailing lists, visit:
>> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>>
>>
>>
>>

Re: Time it takes for Test PHP Cluster

Posted by Eric Bresie <eb...@gmail.com>.
Let me change the question a little...Why are all the others quicker
compared to the php side?
Do the other clusters / jobs / checks not get tested in the same way (i.e.
only a subset of tests compare to all tests in php test if so why)?  Are
there fewer tests (I would assume not as I've run some of the unit tests on
just one project which ?  Do the tests in other context require some "UI"
based tests not possible at build time compared to php which can be UI less
(i.e. request/response)?Are the other tests faster compared to slower php
tests?  Does the php test require starting/stopping some php server for use
during the test?

Just a snapshot of few occurrences, not the "Test PHP Cluster" check
compared to others

For the Windows build

[image: image.png]

For the Linux of PHP cluster

[image: image.png]

Where is the configuration for these build checks setup?  Is there a
specific yaml or part of a build script someplace?

Eric Bresie
ebresie@gmail.com


On Thu, Apr 15, 2021 at 2:51 PM Junichi Yamamoto <ju...@gmail.com>
wrote:

> Because PHP editor has many unit tests.
>
> Junichi
>
>
> On Mon, Apr 12, 2021 at 5:43 AM Eric Bresie <eb...@gmail.com> wrote:
> >
> > I was curious...any reason during the build checks that Test PHP Cluster
> takes so long? It seems like most of the checks are about 1 to 10 minutes
> but this seems to take closer to an hour.
> >
> > I see a lot of things like the below.
> >
> > [junit] ------------- Standard Error -----------------
> > 9931 (https://github.com/apache/netbeans/runs/2318602490#step:7:9931)
> [junit] Apr 11, 2021 8:36:45 PM
> org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform
> >
> > 9932 (https://github.com/apache/netbeans/runs/2318602490#step:7:9932)
> [junit] INFO: Native file watcher is disabled
> >
> > 9933 (https://github.com/apache/netbeans/runs/2318602490#step:7:9933)
> [junit] Apr 11, 2021 8:36:46 PM
> org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>
> >
> > 9934 (https://github.com/apache/netbeans/runs/2318602490#step:7:9934)
> [junit] WARNING: Not listening on module system
> >
> > 9935 (https://github.com/apache/netbeans/runs/2318602490#step:7:9935)
> [junit] Apr 11, 2021 8:36:47 PM
> org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>
> >
> > 9936 (https://github.com/apache/netbeans/runs/2318602490#step:7:9936)
> [junit] INFO: Parsing of big PHP files enabled: false (max size: 5,000,000)
> >
> > Eric Bresie
> > Ebresie@gmail.com (mailto:Ebresie@gmail.com)
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
>

Re: Time it takes for Test PHP Cluster

Posted by Junichi Yamamoto <ju...@gmail.com>.
Because PHP editor has many unit tests.

Junichi


On Mon, Apr 12, 2021 at 5:43 AM Eric Bresie <eb...@gmail.com> wrote:
>
> I was curious...any reason during the build checks that Test PHP Cluster takes so long? It seems like most of the checks are about 1 to 10 minutes but this seems to take closer to an hour.
>
> I see a lot of things like the below.
>
> [junit] ------------- Standard Error -----------------
> 9931 (https://github.com/apache/netbeans/runs/2318602490#step:7:9931) [junit] Apr 11, 2021 8:36:45 PM org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform
>
> 9932 (https://github.com/apache/netbeans/runs/2318602490#step:7:9932) [junit] INFO: Native file watcher is disabled
>
> 9933 (https://github.com/apache/netbeans/runs/2318602490#step:7:9933) [junit] Apr 11, 2021 8:36:46 PM org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>
>
> 9934 (https://github.com/apache/netbeans/runs/2318602490#step:7:9934) [junit] WARNING: Not listening on module system
>
> 9935 (https://github.com/apache/netbeans/runs/2318602490#step:7:9935) [junit] Apr 11, 2021 8:36:47 PM org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>
>
> 9936 (https://github.com/apache/netbeans/runs/2318602490#step:7:9936) [junit] INFO: Parsing of big PHP files enabled: false (max size: 5,000,000)
>
> Eric Bresie
> Ebresie@gmail.com (mailto:Ebresie@gmail.com)
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org

For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists




Re: Time it takes for Test PHP Cluster

Posted by Eric Bresie <eb...@gmail.com>.
Is it worth making a ticket for this?  In the end, the build checks seem to
be a majority of the time in the builds.

Eric Bresie
ebresie@gmail.com


On Thu, Apr 15, 2021 at 9:48 AM Eric Bresie <eb...@gmail.com> wrote:

> Any further thoughts on this?  Is there a way to speed up the process?
>
> Eric Bresie
> ebresie@gmail.com
>
>
> On Sun, Apr 11, 2021 at 3:42 PM Eric Bresie <eb...@gmail.com> wrote:
>
>> I was curious...any reason during the build checks that Test PHP Cluster
>> takes so long?  It seems like most of the checks are about 1 to 10 minutes
>> but this seems to take closer to an hour.
>>
>> I see a lot of things like the below.
>>
>>
>> [junit] ------------- Standard Error -----------------
>> 9931 <https://github.com/apache/netbeans/runs/2318602490#step:7:9931>
>> [junit] Apr 11, 2021 8:36:45 PM
>> org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform
>> 9932 <https://github.com/apache/netbeans/runs/2318602490#step:7:9932>
>> [junit] INFO: Native file watcher is disabled
>> 9933 <https://github.com/apache/netbeans/runs/2318602490#step:7:9933>
>> [junit] Apr 11, 2021 8:36:46 PM
>> org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>
>> 9934 <https://github.com/apache/netbeans/runs/2318602490#step:7:9934>
>> [junit] WARNING: Not listening on module system
>> 9935 <https://github.com/apache/netbeans/runs/2318602490#step:7:9935>
>> [junit] Apr 11, 2021 8:36:47 PM
>> org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>
>> 9936 <https://github.com/apache/netbeans/runs/2318602490#step:7:9936>
>> [junit] INFO: Parsing of big PHP files enabled: false (max size: 5,000,000)
>>
>> Eric Bresie
>> Ebresie@gmail.com
>>
>

Re: Time it takes for Test PHP Cluster

Posted by Eric Bresie <eb...@gmail.com>.
Any further thoughts on this?  Is there a way to speed up the process?

Eric Bresie
ebresie@gmail.com


On Sun, Apr 11, 2021 at 3:42 PM Eric Bresie <eb...@gmail.com> wrote:

> I was curious...any reason during the build checks that Test PHP Cluster
> takes so long?  It seems like most of the checks are about 1 to 10 minutes
> but this seems to take closer to an hour.
>
> I see a lot of things like the below.
>
>
> [junit] ------------- Standard Error -----------------
> 9931 <https://github.com/apache/netbeans/runs/2318602490#step:7:9931>
> [junit] Apr 11, 2021 8:36:45 PM
> org.netbeans.modules.masterfs.watcher.Watcher getNotifierForPlatform
> 9932 <https://github.com/apache/netbeans/runs/2318602490#step:7:9932>
> [junit] INFO: Native file watcher is disabled
> 9933 <https://github.com/apache/netbeans/runs/2318602490#step:7:9933>
> [junit] Apr 11, 2021 8:36:46 PM
> org.netbeans.modules.settings.RecognizeInstanceObjects$MSL <clinit>
> 9934 <https://github.com/apache/netbeans/runs/2318602490#step:7:9934>
> [junit] WARNING: Not listening on module system
> 9935 <https://github.com/apache/netbeans/runs/2318602490#step:7:9935>
> [junit] Apr 11, 2021 8:36:47 PM
> org.netbeans.modules.php.editor.parser.GSFPHPParser <clinit>
> 9936 <https://github.com/apache/netbeans/runs/2318602490#step:7:9936>
> [junit] INFO: Parsing of big PHP files enabled: false (max size: 5,000,000)
>
> Eric Bresie
> Ebresie@gmail.com
>