You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by "Dennis E. Hamilton" <or...@apache.org> on 2015/10/21 02:25:37 UTC
[QUESTION] How Many Pre-Built Binaries are Enough?
I recently started a download of all of the AOO 4.1.2 rc2 candidates, then see how long it was going to take and let it run overnight. In the morning, the download had failed to get them all.
What struck me however is how big the downloads are.
There are full pre-built, installable binaries for 41 separate languages, and separately-introducible smaller language packs for each of them as well.
The pre-built binaries for each single language take 1s gigabyte, in six sets:
4 flavors for Linux, taking 67%
1 flavor for MacOS, for 18%
1 flavor for Windows (win32 x86), for 15%.
Yet the popularity of pre-built binaries, based on download statistics, is inverted, with about 88% for Windows, 9% for MacOS, and 3% for everything else.
Now 3.2% (the actual fraction) of 41 million downloads of AOO 4.1.1 is still 1.3 million that include those Linux cases.
QUESTION: Considering only pre-built binaries for downloading, my question is, when is it time to reduce those that represent inordinate demands to the needs for QA, distribution, and support?
EXCLUDED:
1. This question is not about the relative use of pre-built full binaries for each of 41 languages. There are other questions we could ask about that. (The language packs for given languages are much smaller than the full binaries though, taking about 20MB per flavor.)
2. This question is not about the source codes and the SDK. It is also not about the value of building and testing for different platforms. It is only about distributing pre-built binaries.
I suspect there are more details, and better questions, that could be raised.
I'm all ears. What are your concerns? What do you see being overlooked in this calculus?
- Dennis
> -----Original Message-----
> From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org]
> Sent: Wednesday, September 23, 2015 18:38
> To: dev@openoffice.apache.org
> Subject: RE: [DISCUSS] Apache OpenOffice ODF in the Marketplace -
> Downloading
>
> Here is a rough, top-level view of Apache OpenOffice by the operating
> systems it is downloaded for. This should be no surprise. To have some
> grounding on the immediate situation, here are the downloading
> statistics of Apache OpenOffice 4.1.1 so far.
>
> From Sourceforge,
> <http://sourceforge.net/projects/openofficeorg.mirror/files/4.1.1/stats/
> os?dates=2014-08-01+to+2015-08-31>.
>
> Of the 41 million downloads that have occurred since release of 4.1.1,
> until the end of August,
>
> 87.7% are for Windows
> 9.1% are for Macintosh
> 3.2% is everything else, including Linux
>
[ ... ]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
RE: [QUESTION] How Many Pre-Built Binaries are Enough?
Posted by "Dennis E. Hamilton" <or...@apache.org>.
Thanks for looking into this.
There is an alternative to what Rory proposes on a different thread that does not require active integration on the part of a download server. For security and authentication reasons I don't think off-Apache download integration that would be acceptable.
Depending on how language packs work, and ability to have multiple language packs installed, etc., another approach would be to provide a main distribution that carries a small number of language packs for selection among as part of the installation process. Additional language packs could be downloadable almost as extensions, and even at install time if the primary language is other than one of the ones carried with the primary install.
Yes, that's speculative. I do think the quantity and variation is the problem almost more than the size, since it is a choke point on the building of a full distribution of binaries, even for release candidates and their QA. Smaller is still better -- it takes bandwidth on the part of our mirror sites, not just storage space.
It is also necessary to take into consideration third-party construction of distributions and how they might take advantage of a language-selection mechanism or not. It would be useful to engage such distributors in this conversation.
Based on what the impact is on the deployment pipeline for release candidates and QA, it would be valuable to streamline and simplify the deployment process, including preparation of candidates.
Just pondering ...
I look forward to the results of experimentation in this area.
- Dennis
PS: I agree that reducing the space for Linux distributions is important, but the bandwidth impact is primarily for Windows and then Macintosh and finally Linux. And the release-related distribution pipeline issue for binaries is for all of them. Am I missing something?
> -----Original Message-----
> From: Andrea Pescetti [mailto:pescetti@apache.org]
> Sent: Sunday, November 8, 2015 23:20
> To: dev@openoffice.apache.org
> Subject: Re: [QUESTION] How Many Pre-Built Binaries are Enough?
>
> On 21/10/2015 Dennis E. Hamilton wrote:
> > 4 flavors for Linux, taking 67%
> > 1 flavor for MacOS, for 18%
> > 1 flavor for Windows (win32 x86), for 15%. ...
> > when is it time to reduce those that represent inordinate demands to
> the needs for QA, distribution, and support?
>
> The time is now. Not in terms of QA and support (we can cope with that),
> but in terms of packaging. A very significant bottleneck we have is the
> upload process to make binaries available: true, we are in 2015, but 40
> GBytes are still a big amount of data to move around. Uploading RC1 took
> more than four days of attempts; then things were streamlined with the
> help of Infra, but still very painful at times.
>
> We need to reduce it to somewhere between 20 and 30 GBytes. We could be
> much more aggressive, but reducing to 20-30 GBytes would have resulted
> in several days saved when evaluating/testing the 3 RCs we made for
> OpenOffice 4.1.2.
>
> The good thing is that, for Linux, it seems we can rearrange packages in
> a way that:
> 1) Does not require any changes to download scripts
> 2) Does not require major changes to the installation experience
> 3) Allows to reduce disk space for a full release by at least 10 GBytes
>
> I didn't have time for completing the tests last weekend, but if this
> succeeds it will be worth evaluating. From a release management point of
> view, this is the only concern: creating binary packages for Linux-based
> systems, testing them and supporting them is covered. Disk space
> (actually, over-the-network file transfers) is the main issue.
>
> Regards,
> Andrea.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [QUESTION] How Many Pre-Built Binaries are Enough?
Posted by Andrea Pescetti <pe...@apache.org>.
On 21/10/2015 Dennis E. Hamilton wrote:
> 4 flavors for Linux, taking 67%
> 1 flavor for MacOS, for 18%
> 1 flavor for Windows (win32 x86), for 15%. ...
> when is it time to reduce those that represent inordinate demands to the needs for QA, distribution, and support?
The time is now. Not in terms of QA and support (we can cope with that),
but in terms of packaging. A very significant bottleneck we have is the
upload process to make binaries available: true, we are in 2015, but 40
GBytes are still a big amount of data to move around. Uploading RC1 took
more than four days of attempts; then things were streamlined with the
help of Infra, but still very painful at times.
We need to reduce it to somewhere between 20 and 30 GBytes. We could be
much more aggressive, but reducing to 20-30 GBytes would have resulted
in several days saved when evaluating/testing the 3 RCs we made for
OpenOffice 4.1.2.
The good thing is that, for Linux, it seems we can rearrange packages in
a way that:
1) Does not require any changes to download scripts
2) Does not require major changes to the installation experience
3) Allows to reduce disk space for a full release by at least 10 GBytes
I didn't have time for completing the tests last weekend, but if this
succeeds it will be worth evaluating. From a release management point of
view, this is the only concern: creating binary packages for Linux-based
systems, testing them and supporting them is covered. Disk space
(actually, over-the-network file transfers) is the main issue.
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [QUESTION] How Many Pre-Built Binaries are Enough?
Posted by Kay Schenk <ka...@gmail.com>.
On 10/21/2015 01:16 AM, Jürgen Schmidt wrote:
> On 21/10/15 09:55, Marcus wrote:
>> Am 10/21/2015 02:25 AM, schrieb Dennis E. Hamilton:
>>> I recently started a download of all of the AOO 4.1.2 rc2 candidates,
>>> then see how long it was going to take and let it run overnight. In
>>> the morning, the download had failed to get them all.
>>>
>>> What struck me however is how big the downloads are.
>>>
>>> There are full pre-built, installable binaries for 41 separate
>>> languages, and separately-introducible smaller language packs for each
>>> of them as well.
>>>
>>> The pre-built binaries for each single language take 1s gigabyte, in
>>> six sets:
>>>
>>> 4 flavors for Linux, taking 67%
>>> 1 flavor for MacOS, for 18%
>>> 1 flavor for Windows (win32 x86), for 15%.
>>>
>>> Yet the popularity of pre-built binaries, based on download
>>> statistics, is inverted, with about 88% for Windows, 9% for MacOS, and
>>> 3% for everything else.
>>>
>>> Now 3.2% (the actual fraction) of 41 million downloads of AOO 4.1.1 is
>>> still 1.3 million that include those Linux cases.
>>>
>>> QUESTION: Considering only pre-built binaries for downloading, my
>>> question is, when is it time to reduce those that represent inordinate
>>> demands to the needs for QA, distribution, and support?
So, which of these represent "inordinate demands"? Unless you're
saying that the space considerations for each platform are out of
line with the download percentages?
One subject we had barely broached in the past was the elimination
of the Linux-32 builds. We may want to revisit that. But, I think
the rpm and deb aspects are likely to remain. Maybe packaging Linux
could be improved?
>>>
>>> EXCLUDED:
>>>
>>> 1. This question is not about the relative use of pre-built full
>>> binaries for each of 41 languages. There are other questions we could
>>> ask about that. (The language packs for given languages are much
>>> smaller than the full binaries though, taking about 20MB per flavor.)
>>>
>>> 2. This question is not about the source codes and the SDK. It is
>>> also not about the value of building and testing for different
>>> platforms. It is only about distributing pre-built binaries.
>>>
>>> I suspect there are more details, and better questions, that could be
>>> raised.
>>>
>>> I'm all ears. What are your concerns? What do you see being
>>> overlooked in this calculus?
>>
>> in other words you are asking the question if and how the file sizes of
>> the install bits can be reduced? And when it is time to do this? Is this
>> right or have I understood something wrong?
>>
>> Thanks
>>
>> Marcus
>>
>>
>
> no new question and discussion. This came up at the beginning at Apache
> and before. I remember that Oracle engineers worked on a new installer
> that had a multilingual install piece and download everything else on
> demand (or from a disc I think).
>
> I believe we all agree that it doesn't make too much sense as it is
> today and a smarter packaging and installation would be nice.
>
> The point is that many users rely on the localized version and can't
> really start with an English version only and install the necessary
> language pack. It's all about convenience of an end user product for the
> majority of our users.
>
> A discussion if it make sense or not is useless, I would suggest to
> start a discussion on how to make it better from a technical perspective
> and work on a concept.
>
> Juergen
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
--
--------------------------------------------
MzK
“The journey of a thousand miles begins
with a single step.”
--Lao Tzu
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [QUESTION] How Many Pre-Built Binaries are Enough?
Posted by Jürgen Schmidt <jo...@gmail.com>.
On 21/10/15 09:55, Marcus wrote:
> Am 10/21/2015 02:25 AM, schrieb Dennis E. Hamilton:
>> I recently started a download of all of the AOO 4.1.2 rc2 candidates,
>> then see how long it was going to take and let it run overnight. In
>> the morning, the download had failed to get them all.
>>
>> What struck me however is how big the downloads are.
>>
>> There are full pre-built, installable binaries for 41 separate
>> languages, and separately-introducible smaller language packs for each
>> of them as well.
>>
>> The pre-built binaries for each single language take 1s gigabyte, in
>> six sets:
>>
>> 4 flavors for Linux, taking 67%
>> 1 flavor for MacOS, for 18%
>> 1 flavor for Windows (win32 x86), for 15%.
>>
>> Yet the popularity of pre-built binaries, based on download
>> statistics, is inverted, with about 88% for Windows, 9% for MacOS, and
>> 3% for everything else.
>>
>> Now 3.2% (the actual fraction) of 41 million downloads of AOO 4.1.1 is
>> still 1.3 million that include those Linux cases.
>>
>> QUESTION: Considering only pre-built binaries for downloading, my
>> question is, when is it time to reduce those that represent inordinate
>> demands to the needs for QA, distribution, and support?
>>
>> EXCLUDED:
>>
>> 1. This question is not about the relative use of pre-built full
>> binaries for each of 41 languages. There are other questions we could
>> ask about that. (The language packs for given languages are much
>> smaller than the full binaries though, taking about 20MB per flavor.)
>>
>> 2. This question is not about the source codes and the SDK. It is
>> also not about the value of building and testing for different
>> platforms. It is only about distributing pre-built binaries.
>>
>> I suspect there are more details, and better questions, that could be
>> raised.
>>
>> I'm all ears. What are your concerns? What do you see being
>> overlooked in this calculus?
>
> in other words you are asking the question if and how the file sizes of
> the install bits can be reduced? And when it is time to do this? Is this
> right or have I understood something wrong?
>
> Thanks
>
> Marcus
>
>
no new question and discussion. This came up at the beginning at Apache
and before. I remember that Oracle engineers worked on a new installer
that had a multilingual install piece and download everything else on
demand (or from a disc I think).
I believe we all agree that it doesn't make too much sense as it is
today and a smarter packaging and installation would be nice.
The point is that many users rely on the localized version and can't
really start with an English version only and install the necessary
language pack. It's all about convenience of an end user product for the
majority of our users.
A discussion if it make sense or not is useless, I would suggest to
start a discussion on how to make it better from a technical perspective
and work on a concept.
Juergen
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [QUESTION] How Many Pre-Built Binaries are Enough?
Posted by Marcus <ma...@wtnet.de>.
Am 10/21/2015 02:25 AM, schrieb Dennis E. Hamilton:
> I recently started a download of all of the AOO 4.1.2 rc2 candidates, then see how long it was going to take and let it run overnight. In the morning, the download had failed to get them all.
>
> What struck me however is how big the downloads are.
>
> There are full pre-built, installable binaries for 41 separate languages, and separately-introducible smaller language packs for each of them as well.
>
> The pre-built binaries for each single language take 1s gigabyte, in six sets:
>
> 4 flavors for Linux, taking 67%
> 1 flavor for MacOS, for 18%
> 1 flavor for Windows (win32 x86), for 15%.
>
> Yet the popularity of pre-built binaries, based on download statistics, is inverted, with about 88% for Windows, 9% for MacOS, and 3% for everything else.
>
> Now 3.2% (the actual fraction) of 41 million downloads of AOO 4.1.1 is still 1.3 million that include those Linux cases.
>
> QUESTION: Considering only pre-built binaries for downloading, my question is, when is it time to reduce those that represent inordinate demands to the needs for QA, distribution, and support?
>
> EXCLUDED:
>
> 1. This question is not about the relative use of pre-built full binaries for each of 41 languages. There are other questions we could ask about that. (The language packs for given languages are much smaller than the full binaries though, taking about 20MB per flavor.)
>
> 2. This question is not about the source codes and the SDK. It is also not about the value of building and testing for different platforms. It is only about distributing pre-built binaries.
>
> I suspect there are more details, and better questions, that could be raised.
>
> I'm all ears. What are your concerns? What do you see being overlooked in this calculus?
in other words you are asking the question if and how the file sizes of
the install bits can be reduced? And when it is time to do this? Is this
right or have I understood something wrong?
Thanks
Marcus
>> -----Original Message-----
>> From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org]
>> Sent: Wednesday, September 23, 2015 18:38
>> To: dev@openoffice.apache.org
>> Subject: RE: [DISCUSS] Apache OpenOffice ODF in the Marketplace -
>> Downloading
>>
>> Here is a rough, top-level view of Apache OpenOffice by the operating
>> systems it is downloaded for. This should be no surprise. To have some
>> grounding on the immediate situation, here are the downloading
>> statistics of Apache OpenOffice 4.1.1 so far.
>>
>> From Sourceforge,
>> <http://sourceforge.net/projects/openofficeorg.mirror/files/4.1.1/stats/
>> os?dates=2014-08-01+to+2015-08-31>.
>>
>> Of the 41 million downloads that have occurred since release of 4.1.1,
>> until the end of August,
>>
>> 87.7% are for Windows
>> 9.1% are for Macintosh
>> 3.2% is everything else, including Linux
>>
> [ ... ]
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org