You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cordova.apache.org by Dmitry Blotsky <db...@microsoft.com> on 2015/06/19 23:27:29 UTC
[CI] Weekly Update
Hi all,
CI has been doing pretty well this week. Some notable things are:
- We’re hitting scaling issues (a buffer of 70+ pending builds happened a few times this week)
- We’re still waiting on INFRA-9811 and INFRA-9678 for PR-triggered builds
Next uses for the CI that are impeded by the scaling issues:
- Static analysis (FB Infer)
- Automated releases
- Documentation generation
- Testing on more emulators, platforms, and devices
- FirefoxOS, BlackBerry, and WebOS (seeing a lot of love recently)
- Moving our platform and tools CI from Travis and AppVeyor to Buildbot, so all our builds are on Apache infrastructure
Thoughts on improving scalability:
- Builds can be greatly sped up if we can alleviate the path-length limit issue that prevents us from packaging up a Cordova build
- Does anyone know of a tool that can zip up too-long paths?
- Does anyone know if we can reduce our path lengths?
- Does anyone know where we can store a buffer of builds? Cordova-vm is an option (~30GB of space).
- We can run more builds in parallel if folks connect more slaves to the CI (instructions here: https://github.com/apache/cordova-medic/blob/master/SLAVES.md)
- We can split the build process into levels (e.g. smoke, regression, end-to-end) and run them on different occasions
Bug status:
~1 on Android OSX
~2 on Android Windows
~20 on iOS
0 on Windows Phone 8.1
0 on Windows 8.1
0 on Windows 8.0
0 on Windows Phone 8.0
Kindly,
Dmitry
Re: [CI] Weekly Update
Posted by Dmitry Blotsky <db...@microsoft.com>.
@Steve: It’s really just a nice-to-have in the long-term: all CI in one place. They’re fine where they are right now though. Buildbot just runs on our own hardware, and we can control our slave environments more than those of Travis and AppVeyor. Also we don’t need to administer two extra accounts/
@Jesse: The path-length issue is as follows: right now all artifacts of a mobilespec from-source build (i.e. cloning, installing, and linking lib, cli, plugman, js, mobilespec, etc.) cannot be zipped on Windows because there are paths that are too long. If they were to be zipped, there would be only 2 “build mobilespec” builds (one for each OS) at ~6 minutes each, and 7 “test” builds at ~3 minutes each. So, a full E2E build would take up about a third of the time it takes now.
Adding more machines would also relieve some of the build pressure though.
Kindly,
Dmitry
> On Jun 19, 2015, at 3:12 PM, Jesse <pu...@gmail.com> wrote:
>
> Steve, travis+appveyor are about running platform tests while buildbot is
> running all the plugin tests as well.
>
> Dmitry, can you elaborate on the specifics of the path-length issue?
> Is it one platform in particular?
>
>
> My team is hiring!
> @purplecabbage
> risingj.com
>
> On Fri, Jun 19, 2015 at 2:36 PM, Steven Gill <st...@gmail.com> wrote:
>
>> Quick question, why don't keep travis + appveyor as well as build bot? I
>> don't see the reason to remove that functionality.
>>
>> On Fri, Jun 19, 2015 at 2:27 PM, Dmitry Blotsky <db...@microsoft.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> CI has been doing pretty well this week. Some notable things are:
>>> - We’re hitting scaling issues (a buffer of 70+ pending builds happened a
>>> few times this week)
>>> - We’re still waiting on INFRA-9811 and INFRA-9678 for PR-triggered
>> builds
>>>
>>> Next uses for the CI that are impeded by the scaling issues:
>>> - Static analysis (FB Infer)
>>> - Automated releases
>>> - Documentation generation
>>> - Testing on more emulators, platforms, and devices
>>> - FirefoxOS, BlackBerry, and WebOS (seeing a lot of love recently)
>>> - Moving our platform and tools CI from Travis and AppVeyor to Buildbot,
>>> so all our builds are on Apache infrastructure
>>>
>>> Thoughts on improving scalability:
>>> - Builds can be greatly sped up if we can alleviate the path-length limit
>>> issue that prevents us from packaging up a Cordova build
>>> - Does anyone know of a tool that can zip up too-long paths?
>>> - Does anyone know if we can reduce our path lengths?
>>> - Does anyone know where we can store a buffer of builds? Cordova-vm is
>> an
>>> option (~30GB of space).
>>> - We can run more builds in parallel if folks connect more slaves to the
>>> CI (instructions here:
>>> https://github.com/apache/cordova-medic/blob/master/SLAVES.md)
>>> - We can split the build process into levels (e.g. smoke, regression,
>>> end-to-end) and run them on different occasions
>>>
>>> Bug status:
>>> ~1 on Android OSX
>>> ~2 on Android Windows
>>> ~20 on iOS
>>> 0 on Windows Phone 8.1
>>> 0 on Windows 8.1
>>> 0 on Windows 8.0
>>> 0 on Windows Phone 8.0
>>>
>>> Kindly,
>>> Dmitry
>>>
>>
Re: [CI] Weekly Update
Posted by Jesse <pu...@gmail.com>.
Steve, travis+appveyor are about running platform tests while buildbot is
running all the plugin tests as well.
Dmitry, can you elaborate on the specifics of the path-length issue?
Is it one platform in particular?
My team is hiring!
@purplecabbage
risingj.com
On Fri, Jun 19, 2015 at 2:36 PM, Steven Gill <st...@gmail.com> wrote:
> Quick question, why don't keep travis + appveyor as well as build bot? I
> don't see the reason to remove that functionality.
>
> On Fri, Jun 19, 2015 at 2:27 PM, Dmitry Blotsky <db...@microsoft.com>
> wrote:
>
> > Hi all,
> >
> > CI has been doing pretty well this week. Some notable things are:
> > - We’re hitting scaling issues (a buffer of 70+ pending builds happened a
> > few times this week)
> > - We’re still waiting on INFRA-9811 and INFRA-9678 for PR-triggered
> builds
> >
> > Next uses for the CI that are impeded by the scaling issues:
> > - Static analysis (FB Infer)
> > - Automated releases
> > - Documentation generation
> > - Testing on more emulators, platforms, and devices
> > - FirefoxOS, BlackBerry, and WebOS (seeing a lot of love recently)
> > - Moving our platform and tools CI from Travis and AppVeyor to Buildbot,
> > so all our builds are on Apache infrastructure
> >
> > Thoughts on improving scalability:
> > - Builds can be greatly sped up if we can alleviate the path-length limit
> > issue that prevents us from packaging up a Cordova build
> > - Does anyone know of a tool that can zip up too-long paths?
> > - Does anyone know if we can reduce our path lengths?
> > - Does anyone know where we can store a buffer of builds? Cordova-vm is
> an
> > option (~30GB of space).
> > - We can run more builds in parallel if folks connect more slaves to the
> > CI (instructions here:
> > https://github.com/apache/cordova-medic/blob/master/SLAVES.md)
> > - We can split the build process into levels (e.g. smoke, regression,
> > end-to-end) and run them on different occasions
> >
> > Bug status:
> > ~1 on Android OSX
> > ~2 on Android Windows
> > ~20 on iOS
> > 0 on Windows Phone 8.1
> > 0 on Windows 8.1
> > 0 on Windows 8.0
> > 0 on Windows Phone 8.0
> >
> > Kindly,
> > Dmitry
> >
>
Re: [CI] Weekly Update
Posted by Steven Gill <st...@gmail.com>.
Quick question, why don't keep travis + appveyor as well as build bot? I
don't see the reason to remove that functionality.
On Fri, Jun 19, 2015 at 2:27 PM, Dmitry Blotsky <db...@microsoft.com>
wrote:
> Hi all,
>
> CI has been doing pretty well this week. Some notable things are:
> - We’re hitting scaling issues (a buffer of 70+ pending builds happened a
> few times this week)
> - We’re still waiting on INFRA-9811 and INFRA-9678 for PR-triggered builds
>
> Next uses for the CI that are impeded by the scaling issues:
> - Static analysis (FB Infer)
> - Automated releases
> - Documentation generation
> - Testing on more emulators, platforms, and devices
> - FirefoxOS, BlackBerry, and WebOS (seeing a lot of love recently)
> - Moving our platform and tools CI from Travis and AppVeyor to Buildbot,
> so all our builds are on Apache infrastructure
>
> Thoughts on improving scalability:
> - Builds can be greatly sped up if we can alleviate the path-length limit
> issue that prevents us from packaging up a Cordova build
> - Does anyone know of a tool that can zip up too-long paths?
> - Does anyone know if we can reduce our path lengths?
> - Does anyone know where we can store a buffer of builds? Cordova-vm is an
> option (~30GB of space).
> - We can run more builds in parallel if folks connect more slaves to the
> CI (instructions here:
> https://github.com/apache/cordova-medic/blob/master/SLAVES.md)
> - We can split the build process into levels (e.g. smoke, regression,
> end-to-end) and run them on different occasions
>
> Bug status:
> ~1 on Android OSX
> ~2 on Android Windows
> ~20 on iOS
> 0 on Windows Phone 8.1
> 0 on Windows 8.1
> 0 on Windows 8.0
> 0 on Windows Phone 8.0
>
> Kindly,
> Dmitry
>