Re: [QUESTION] How Many Pre-Built Binaries are Enough?

[Andrea Pescetti <pe...@apache.org>]

On 21/10/2015 Dennis E. Hamilton wrote:
>    4 flavors for Linux, taking 67%
>    1 flavor for MacOS, for 18%
>    1 flavor for Windows (win32 x86), for 15%. ...
> when is it time to reduce those that represent inordinate demands to the needs for QA, distribution, and support?

The time is now. Not in terms of QA and support (we can cope with that), 
but in terms of packaging. A very significant bottleneck we have is the 
upload process to make binaries available: true, we are in 2015, but 40 
GBytes are still a big amount of data to move around. Uploading RC1 took 
more than four days of attempts; then things were streamlined with the 
help of Infra, but still very painful at times.

We need to reduce it to somewhere between 20 and 30 GBytes. We could be 
much more aggressive, but reducing to 20-30 GBytes would have resulted 
in several days saved when evaluating/testing the 3 RCs we made for 
OpenOffice 4.1.2.

The good thing is that, for Linux, it seems we can rearrange packages in 
a way that:
1) Does not require any changes to download scripts
2) Does not require major changes to the installation experience
3) Allows to reduce disk space for a full release by at least 10 GBytes

I didn't have time for completing the tests last weekend, but if this 
succeeds it will be worth evaluating. From a release management point of 
view, this is the only concern: creating binary packages for Linux-based 
systems, testing them and supporting them is covered. Disk space 
(actually, over-the-network file transfers) is the main issue.

Regards,
   Andrea.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org

RE: [QUESTION] How Many Pre-Built Binaries are Enough?

["Dennis E. Hamilton" <or...@apache.org>]

Thanks for looking into this.

There is an alternative to what Rory proposes on a different thread that does not require active integration on the part of a download server.  For security and authentication reasons I don't think off-Apache download integration that would be acceptable.

Depending on how language packs work, and ability to have multiple language packs installed, etc., another approach would be to provide a main distribution that carries a small number of language packs for selection among as part of the installation process.  Additional language packs could be downloadable almost as extensions, and even at install time if the primary language is other than one of the ones carried with the primary install.

Yes, that's speculative.  I do think the quantity and variation is the problem almost more than the size, since it is a choke point on the building of a full distribution of binaries, even for release candidates and their QA.  Smaller is still better -- it takes bandwidth on the part of our mirror sites, not just storage space.

It is also necessary to take into consideration third-party construction of distributions and how they might take advantage of a language-selection mechanism or not.  It would be useful to engage such distributors in this conversation.

Based on what the impact is on the deployment pipeline for release candidates and QA, it would be valuable to streamline and simplify the deployment process, including preparation of candidates.

Just pondering ...

I look forward to the results of experimentation in this area.

 - Dennis

PS: I agree that reducing the space for Linux distributions is important, but the bandwidth impact is primarily for Windows and then Macintosh and finally Linux.  And the release-related distribution pipeline issue for binaries is for all of them.  Am I missing something?

> -----Original Message-----
> From: Andrea Pescetti [mailto:pescetti@apache.org]
> Sent: Sunday, November 8, 2015 23:20
> To: dev@openoffice.apache.org
> Subject: Re: [QUESTION] How Many Pre-Built Binaries are Enough?
> 
> On 21/10/2015 Dennis E. Hamilton wrote:
> >    4 flavors for Linux, taking 67%
> >    1 flavor for MacOS, for 18%
> >    1 flavor for Windows (win32 x86), for 15%. ...
> > when is it time to reduce those that represent inordinate demands to
> the needs for QA, distribution, and support?
> 
> The time is now. Not in terms of QA and support (we can cope with that),
> but in terms of packaging. A very significant bottleneck we have is the
> upload process to make binaries available: true, we are in 2015, but 40
> GBytes are still a big amount of data to move around. Uploading RC1 took
> more than four days of attempts; then things were streamlined with the
> help of Infra, but still very painful at times.
> 
> We need to reduce it to somewhere between 20 and 30 GBytes. We could be
> much more aggressive, but reducing to 20-30 GBytes would have resulted
> in several days saved when evaluating/testing the 3 RCs we made for
> OpenOffice 4.1.2.
> 
> The good thing is that, for Linux, it seems we can rearrange packages in
> a way that:
> 1) Does not require any changes to download scripts
> 2) Does not require major changes to the installation experience
> 3) Allows to reduce disk space for a full release by at least 10 GBytes
> 
> I didn't have time for completing the tests last weekend, but if this
> succeeds it will be worth evaluating. From a release management point of
> view, this is the only concern: creating binary packages for Linux-based
> systems, testing them and supporting them is covered. Disk space
> (actually, over-the-network file transfers) is the main issue.
> 
> Regards,
>    Andrea.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org