You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openoffice.apache.org by Andrea Pescetti <pe...@apache.org> on 2015/11/08 00:08:09 UTC
[PROPOSAL] Distribute only one source package
We currently distribute 3 source packages at
https://archive.apache.org/dist/openoffice/4.1.2/source/
1) A .tar.bz2 file (209 MBytes)
2) A .tar.gz file (276 Mbytes)
3) A ZIP file (323 MBytes)
The packages are equivalent, so any one would suffice.
As discussed by Regina and Juergen recently, we ship #3 as a convenience
for Windows users but this leads to broken file permissions, so the
recommendation for Windows users is to use #1 or #2, which makes #3 useless.
I suggest, subject to lazy consensus, that we only distribute #1, i.e.,
the .tar.bz2 file.
Reasons:
* If we distribute one source package, it will be clear that we are all
testing and approving the same one
* .tar.bz2 offer better compression than .tar.gz
* bzip2 is ubiquitous today, so I don't believe that there are systems
capable of building OpenOffice which don't have bzip2 available
* better compression formats exist, but they are not as widely supported
as the three we are using now, so I'd stick with bzip2
This of course doesn't apply to 4.1.2, which is already released and
will remain available in all three formats.
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Rory O'Farrell <of...@iol.ie>.
On Mon, 9 Nov 2015 13:01:16 +0100
Andrea Pescetti <pe...@apache.org> wrote:
> Rory O'Farrell wrote:
> > I can see one possible method to reduce the number of prebuilt binaries.
>
> This has nothing to do with this thread though.
I'm sorry for choosing the wrong thread - I realised it was wrong just after I pressed the Send button. It was an early post this morning and by brain was not fully in gear - sorry.
--
Rory O'Farrell <of...@iol.ie>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Andrea Pescetti <pe...@apache.org>.
Rory O'Farrell wrote:
> I can see one possible method to reduce the number of prebuilt binaries.
This has nothing to do with this thread though. This thread is about
what to do about the SOURCE package, which is the package used by
DEVELOPERS who for some reason want to build their own version of
OpenOffice from a source archive, without using our SVN or Git
interfaces. And the thread already derailed enough without the need that
we also take binaries into account!
> I don't know how feasible this might be
Your suggestion makes sense, even though it's unrelated to this thread.
It was considered in the past, just it's not feasible with our setup: we
would need 100 times smaller packages, only one server for download and
a small fraction of the download rate we have today to make it
sustainable. Still, this same idea can be rearranged a bit to make it
feasible in a similar way. More details in another thread, but only when
I have numbers! It isn't useful to discuss this based on pure theory,
since it is a very practical problem.
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Rory O'Farrell <of...@iol.ie>.
I can see one possible method to reduce the number of prebuilt binaries. It would require reconsideration of the download server and of the installer mechanisms for each O.S.
My suggestion is:
Build only language neutral versions for each O.S. Modify the installers and the download server so that all downloads are this language neutral package (about 110/120 MB at a guess) and a language pack of the selected language (20 MB?) - this packaging is the download server's responsibility. The installer installs the language neutral version then silently installs the selected language pack.
I don't know how feasible this might be - bear in mind my knowledge of compilation and linking dates from the days of Z80 assembler and static linking at compile time!
Doing the above suggestion would dramatically reduce the required number of prebuilt binaries.
--
Rory O'Farrell <of...@iol.ie>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
RE: [PROPOSAL] Distribute only one source package
Posted by "Dennis E. Hamilton" <or...@apache.org>.
+1
> -----Original Message-----
> From: Patricia Shanahan [mailto:pats@acm.org]
> Sent: Sunday, November 8, 2015 12:03
> To: dev@openoffice.apache.org
> Subject: Re: [PROPOSAL] Distribute only one source package
[ ... ]
>
> AOO build on Windows currently requires the use of Cygwin which has a
> full range of archive utilities. The downside is that it limits your
> choice of developers to people who are comfortable in a UNIX-like shell
> environment.
[orcmid]
And because the Windows native development tools (as of VS 2008) are used
from within that environment, it gets very weird and is also rather brittle.
>
> My ideal tool chain would be:
>
> Open and extract-all on a zip file OR do an SVN checkout.
>
> Open an appropriate IDE and load the project description file.
>
> Build.
>
> Run.
[orcmid]
I think that is very important to strive for if we are to attract and
sustain development for the platform which is overwhelmingly (87%) used by
our end-users based on the only measures we have.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Patricia Shanahan <pa...@acm.org>.
On 11/8/2015 11:53 AM, Andrea Pescetti wrote:
...
> Do Windows developers really use Windows' built-in utilities for
> unzipping? I really think that the minimal stack for building OpenOffice
> on Windows includes some .tar.bz2-capable programs. We do download and
> expand .tar.bz2 files as part of the build process, so it seems obvious
> that this is not an issue for Windows developers, meaning that this is
> covered by standard tooling.
...
It depends what you mean by standard tooling: tooling as needed
currently for AOO building or normal tooling.
AOO build on Windows currently requires the use of Cygwin which has a
full range of archive utilities. The downside is that it limits your
choice of developers to people who are comfortable in a UNIX-like shell
environment.
My ideal tool chain would be:
Open and extract-all on a zip file OR do an SVN checkout.
Open an appropriate IDE and load the project description file.
Build.
Run.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Rory O'Farrell <of...@iol.ie>.
On Sun, 08 Nov 2015 20:53:44 +0100
Andrea Pescetti <pe...@apache.org> wrote:
> Rory O'Farrell wrote:
> > On Sun, 8 Nov 2015 09:51:07 -0800 "Dennis E. Hamilton" wrote:
> >> There is interesting discussion on this thread that devolves into what compression to use as the single source-package case.
> > My reaction is that most (all?) linux/non-windows builders will be happy with the proposed .bz2 compression.
>
> Last time I had the occasion to see them, all normal file decompressors
> for Windows (Winzip, WinRAR, 7-Zip) were able to extract a .tar.bz2 archive.
My windows experience is way out of date, as instanced by my only knowing it handled .ZIP. If the default archive manager will handle .bz then there is no problem except user education; they may need to be told very positively that the archiver will handle that file type.
Rory
>
> So, speculations aside, is there anyone who has a working stack for
> building OpenOffice on Windows and feels it would be problematic to
> extract a .tar.bz2 archive?
>
> > For them we ought make available a package that opens in the default Windows Archive Manager, whatever that is.
>
> Do Windows developers really use Windows' built-in utilities for
> unzipping? I really think that the minimal stack for building OpenOffice
> on Windows includes some .tar.bz2-capable programs. We do download and
> expand .tar.bz2 files as part of the build process, so it seems obvious
> that this is not an issue for Windows developers, meaning that this is
> covered by standard tooling.
>
> >> MY OFFER: I will happily produce a signed, Windows-acceptable Zip for a source release, using an SVN working copy of the released branch and version.
>
> So long as we (as the project) vote on ONE single source package (the
> .tar.bz2 one), I'm absolutely OK with you doing that. People who want to
> distribute their own "unofficial" archive produced with their utility of
> choice can do that. We can advertise it as a "convenience source
> package" on http://openoffice.apache.org/downloads.html and store it on
> people.apache.org. This is entirely possible.
>
> What we must avoid is that, in theory (since it practice it would be
> interesting to know how many people do that), we ask people who vote on
> a release to download 3 source packages, expand all of them (wasting
> several GBytes of disk space) and ensure they are equivalent. If we have
> one "canonical" source package, everybody knows what we are voting on.
> Then we can have any number of "unofficial" archives in other formats.
>
> >> One produced on Windows for Windows should not present the interoperability and interchange problems that other arrangements introduce.
>
> No idea on this. Maybe yes, maybe not.
>
> >> I am going to appeal to the Apache Project Maturity Model because I believe it is applicable here ...
> >> I think the relevant considerations of what should be *strived*for* are
> >> CD10: The project produces Open Source software, for distribution to the public at no charge.
> >> CD20: The project's code is easily discoverable and publicly accessible.
> >> CD30: The code can be built in a reproducible way using widely available standard tools.
> >> RE10: Releases consist of source code, distributed using standard and open archive formats that are expected to stay readable in the long term.
>
> bzip2 satisfies all of these requirements. We can ask dev@community if
> you have any doubts. For sure, many Apache projects do not provide a ZIP
> file (I admit they tend to prefer .tar.gz to .tar.bz2); no Apache
> project that I know of distributes 3 source packages.
>
> >> My question is, on what platform were the troublesome Zips produced, using what tools?
>
> They were done on a Mac, but this (like most of this discussion) is
> entirely irrelevant. The fact that the .ZIP version has (probably)
> issues is yet another reason to kill it, but my main reason is to make
> it clear what we are voting on.
>
> >> I note also that Zip format is considered standard and open enough that it is the format employed for the ODF packages used by OpenOffice
>
> Here other considerations apply, like decompression speed. But again,
> I'm not proposing to drop ZIP since it's not standard. I'm proposing to
> drop it since it's redundant.
>
> >> Although WinZip *will* unpack a .tar.gz (or .tgz) package, I do not know whether it will unpack a .tar.bz2.
>
> Several years ago it did. I assume it still does.
>
> >> I notice that 7z does handle .rar and .msi and perhaps tar.* compressions but I haven't checked those.
>
> If you want to try in practice, try with this:
>
> http://serf.googlecode.com/files/serf-1.2.1.tar.bz2
>
> but I'm confident that you will be able to expand it on any machine
> where OpenOffice can be built.
>
> By the way, that library is a standard requirement we incorporate in
> OpenOffice, so any system able to build OpenOffice must expand it at
> some point. This is the reason for me to assume that keeping only
> .tar.bz2 is not an issue. But, provided we get consensus on wanting ONE
> "official" source package, if someone has real arguments for preferring
> .tar.gz over .tar.bz2, .tar.gz may work too.
>
> Regards,
> Andrea.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
--
Rory O'Farrell <of...@iol.ie>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Regina Henschel <rb...@t-online.de>.
Hi Andrea,
+1 for your idea of only one source package.
Andrea Pescetti schrieb:
> Rory O'Farrell wrote:
>> On Sun, 8 Nov 2015 09:51:07 -0800 "Dennis E. Hamilton" wrote:
>>> There is interesting discussion on this thread that devolves into
>>> what compression to use as the single source-package case.
>> My reaction is that most (all?) linux/non-windows builders will be
>> happy with the proposed .bz2 compression.
>
> Last time I had the occasion to see them, all normal file decompressors
> for Windows (Winzip, WinRAR, 7-Zip) were able to extract a .tar.bz2
> archive.
>
> So, speculations aside, is there anyone who has a working stack for
> building OpenOffice on Windows and feels it would be problematic to
> extract a .tar.bz2 archive?
You need Cygwin and tar anyway. So there is no problem to extract a
tar.bz2 archive.
>
>> For them we ought make available a package that opens in the default
>> Windows Archive Manager, whatever that is.
>
> Do Windows developers really use Windows' built-in utilities for
> unzipping? I really think that the minimal stack for building OpenOffice
> on Windows includes some .tar.bz2-capable programs. We do download and
> expand .tar.bz2 files as part of the build process, so it seems obvious
> that this is not an issue for Windows developers, meaning that this is
> covered by standard tooling.
Using a tool outside Cygwin is dangerous. Remember the file permission
troubles I had when building 4.1.2. If we don't deliver a .zip archive,
it is less likely, that someone uses a tool outside of Cygwin.
>
>>> MY OFFER: I will happily produce a signed, Windows-acceptable Zip for
>>> a source release, using an SVN working copy of the released branch
>>> and version.
>
> So long as we (as the project) vote on ONE single source package (the
> .tar.bz2 one), I'm absolutely OK with you doing that. People who want to
> distribute their own "unofficial" archive produced with their utility of
> choice can do that. We can advertise it as a "convenience source
> package" on http://openoffice.apache.org/downloads.html and store it on
> people.apache.org. This is entirely possible.
>
> What we must avoid is that, in theory (since it practice it would be
> interesting to know how many people do that), we ask people who vote on
> a release to download 3 source packages, expand all of them (wasting
> several GBytes of disk space) and ensure they are equivalent. If we have
> one "canonical" source package, everybody knows what we are voting on.
> Then we can have any number of "unofficial" archives in other formats.
Having only one official source package would make the work flow and the
voting unambiguous. So I support it. I have no strong opinion whether to
use .bz2 or .gz. They are both easily to handle with tar in Cygwin.
Kind regards
Regina
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Andrea Pescetti <pe...@apache.org>.
Rory O'Farrell wrote:
> On Sun, 8 Nov 2015 09:51:07 -0800 "Dennis E. Hamilton" wrote:
>> There is interesting discussion on this thread that devolves into what compression to use as the single source-package case.
> My reaction is that most (all?) linux/non-windows builders will be happy with the proposed .bz2 compression.
Last time I had the occasion to see them, all normal file decompressors
for Windows (Winzip, WinRAR, 7-Zip) were able to extract a .tar.bz2 archive.
So, speculations aside, is there anyone who has a working stack for
building OpenOffice on Windows and feels it would be problematic to
extract a .tar.bz2 archive?
> For them we ought make available a package that opens in the default Windows Archive Manager, whatever that is.
Do Windows developers really use Windows' built-in utilities for
unzipping? I really think that the minimal stack for building OpenOffice
on Windows includes some .tar.bz2-capable programs. We do download and
expand .tar.bz2 files as part of the build process, so it seems obvious
that this is not an issue for Windows developers, meaning that this is
covered by standard tooling.
>> MY OFFER: I will happily produce a signed, Windows-acceptable Zip for a source release, using an SVN working copy of the released branch and version.
So long as we (as the project) vote on ONE single source package (the
.tar.bz2 one), I'm absolutely OK with you doing that. People who want to
distribute their own "unofficial" archive produced with their utility of
choice can do that. We can advertise it as a "convenience source
package" on http://openoffice.apache.org/downloads.html and store it on
people.apache.org. This is entirely possible.
What we must avoid is that, in theory (since it practice it would be
interesting to know how many people do that), we ask people who vote on
a release to download 3 source packages, expand all of them (wasting
several GBytes of disk space) and ensure they are equivalent. If we have
one "canonical" source package, everybody knows what we are voting on.
Then we can have any number of "unofficial" archives in other formats.
>> One produced on Windows for Windows should not present the interoperability and interchange problems that other arrangements introduce.
No idea on this. Maybe yes, maybe not.
>> I am going to appeal to the Apache Project Maturity Model because I believe it is applicable here ...
>> I think the relevant considerations of what should be *strived*for* are
>> CD10: The project produces Open Source software, for distribution to the public at no charge.
>> CD20: The project's code is easily discoverable and publicly accessible.
>> CD30: The code can be built in a reproducible way using widely available standard tools.
>> RE10: Releases consist of source code, distributed using standard and open archive formats that are expected to stay readable in the long term.
bzip2 satisfies all of these requirements. We can ask dev@community if
you have any doubts. For sure, many Apache projects do not provide a ZIP
file (I admit they tend to prefer .tar.gz to .tar.bz2); no Apache
project that I know of distributes 3 source packages.
>> My question is, on what platform were the troublesome Zips produced, using what tools?
They were done on a Mac, but this (like most of this discussion) is
entirely irrelevant. The fact that the .ZIP version has (probably)
issues is yet another reason to kill it, but my main reason is to make
it clear what we are voting on.
>> I note also that Zip format is considered standard and open enough that it is the format employed for the ODF packages used by OpenOffice
Here other considerations apply, like decompression speed. But again,
I'm not proposing to drop ZIP since it's not standard. I'm proposing to
drop it since it's redundant.
>> Although WinZip *will* unpack a .tar.gz (or .tgz) package, I do not know whether it will unpack a .tar.bz2.
Several years ago it did. I assume it still does.
>> I notice that 7z does handle .rar and .msi and perhaps tar.* compressions but I haven't checked those.
If you want to try in practice, try with this:
http://serf.googlecode.com/files/serf-1.2.1.tar.bz2
but I'm confident that you will be able to expand it on any machine
where OpenOffice can be built.
By the way, that library is a standard requirement we incorporate in
OpenOffice, so any system able to build OpenOffice must expand it at
some point. This is the reason for me to assume that keeping only
.tar.bz2 is not an issue. But, provided we get consensus on wanting ONE
"official" source package, if someone has real arguments for preferring
.tar.gz over .tar.bz2, .tar.gz may work too.
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Rory O'Farrell <of...@iol.ie>.
On Sun, 8 Nov 2015 09:51:07 -0800
"Dennis E. Hamilton" <or...@apache.org> wrote:
> There is interesting discussion on this thread that devolves into what compression to use as the single source-package case.
My reaction is that most (all?) linux/non-windows builders will be happy with the proposed .bz2 compression. However I fear that Windows builders may think only in terms of 7zip (or zip as being compression of choice). For them we ought make available a package that opens in the default Windows Archive Manager, whatever that is.
>
> I do not support this proposal as a first-choice alternative to what may be a surmountable problem.
>
> I want to speak to some higher-level issues, below.
>
> MY OFFER: I will happily produce a signed, Windows-acceptable Zip for a source release, using an SVN working copy of the released branch and version. One produced on Windows for Windows should not present the interoperability and interchange problems that other arrangements introduce. Unzip onto non-Windows platforms should also work properly. These things are relatively easy to check and confirm for a power user such as myself, despite my not being a core developer of the code itself. I will also document the procedure so that anyone can replicate it.
>
> It should be easy for other committers to unpack the zip and confirm its reliability and accuracy on Windows and other platforms. They could even add their counter-signing digital signatures.
>
> If that fails, then it would be time for a Plan B. (A possible unexpected difficulty may have to do with line ends on text files across platforms, depending on whether tools address that as well as SVN clients do.)
>
> - Dennis
>
> - - - - - - - - - - - -
>
> The TL;DR:
>
> I am going to appeal to the Apache Project Maturity Model because I believe it is applicable here, whatever the status of the document at <http://community.apache.org/apache-way/apache-project-maturity-model.html>.
>
> I think the relevant considerations of what should be *strived*for* are
>
> CD10: The project produces Open Source software, for distribution to the public at no charge.
>
> CD20: The project's code is easily discoverable and publicly accessible.
>
> CD30: The code can be built in a reproducible way using widely available standard tools.
>
> RE10: Releases consist of source code, distributed using standard and open archive formats that are expected to stay readable in the long term.
>
> Something that is not in this list, but I see as a corollary, is that how we do these things is part of demonstrating care for the downstream users of the software we produce, for whatever reason they want to consult, examine, learn from, or even develop with, and whenever they choose to do so with a given source release.
>
> - - - - - - - - - - -
>
> MY QUESTION: There should never be a permissions issue in unpacking a Zip file on Windows. The only barriers are (1) file names must be ones that are acceptable and unique when extracted into an NTFS file system and (2) empty directories are not created, even if loaded into later in the stream, and there is no use of Zip extensions that apply only to Unix systems and Unix permissions. There are also concerns about length of full-path file names. (Note that some of these apply to using SVN on Windows too.)
> My question is, on what platform were the troublesome Zips produced, using what tools? We may be seeing an interchange/interoperability problem that comes from inter-platform incompatibilities.
>
> MY CONCERN: These provisions are not just for project developers, but for the public. I don't think only distributing a .tar.bz2 is very adequate with respect to CD20. RE10 is presumably satisfied if there are no uses of patented technologies, and assuming that no deviations from basic .tar formatting are employed (e.g., no launching of shell scripts involved).
>
> I note also that Zip format is considered standard and open enough that it is the format employed for the ODF packages used by OpenOffice and also the packaging of .oxt extensions. It is also the packaging format of choice for OOXML, EPUB, and other domain-specific usages, including distribution of some libraries used in AOO and other projects. (I also notice that sometimes those Zips of libraries don't unpack properly on Windows, but rezipping them on Windows then works everywhere.)
>
> The convention has been to use Zip for Windows oriented access to the source and some flavor of .tar.* for the Unix-oriented world. The basic use of Zip for Windows tends to unzip easily on any platform that can deal with the filename and folder hierarchies.
>
> Although WinZip *will* unpack a .tar.gz (or .tgz) package, I do not know whether it will unpack a .tar.bz2. Expecting a member of the public who operates on Windows to know how to unpack a .tar.* is an unnecessary source of friction. I notice that 7z does handle .rar and .msi and perhaps tar.* compressions but I haven't checked those.
>
>
>
> - - - - - - - - - - - - -
>
> > -----Original Message-----
> > From: Andrea Pescetti [mailto:pescetti@apache.org]
> > Sent: Saturday, November 7, 2015 15:08
> > To: dev@openoffice.apache.org
> > Subject: [PROPOSAL] Distribute only one source package
> >
> > We currently distribute 3 source packages at
> > https://archive.apache.org/dist/openoffice/4.1.2/source/
> > 1) A .tar.bz2 file (209 MBytes)
> > 2) A .tar.gz file (276 Mbytes)
> > 3) A ZIP file (323 MBytes)
> >
> > The packages are equivalent, so any one would suffice.
> >
> > As discussed by Regina and Juergen recently, we ship #3 as a convenience
> > for Windows users but this leads to broken file permissions, so the
> > recommendation for Windows users is to use #1 or #2, which makes #3
> > useless.
> >
> > I suggest, subject to lazy consensus, that we only distribute #1, i.e.,
> > the .tar.bz2 file.
> >
> > Reasons:
> > * If we distribute one source package, it will be clear that we are all
> > testing and approving the same one
> > * .tar.bz2 offer better compression than .tar.gz
> > * bzip2 is ubiquitous today, so I don't believe that there are systems
> > capable of building OpenOffice which don't have bzip2 available
> > * better compression formats exist, but they are not as widely supported
> > as the three we are using now, so I'd stick with bzip2
> >
> > This of course doesn't apply to 4.1.2, which is already released and
> > will remain available in all three formats.
> >
> > Regards,
> > Andrea.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> > For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
--
Rory O'Farrell <of...@iol.ie>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
RE: [PROPOSAL] Distribute only one source package
Posted by "Dennis E. Hamilton" <or...@apache.org>.
There is interesting discussion on this thread that devolves into what compression to use as the single source-package case.
I do not support this proposal as a first-choice alternative to what may be a surmountable problem.
I want to speak to some higher-level issues, below.
MY OFFER: I will happily produce a signed, Windows-acceptable Zip for a source release, using an SVN working copy of the released branch and version. One produced on Windows for Windows should not present the interoperability and interchange problems that other arrangements introduce. Unzip onto non-Windows platforms should also work properly. These things are relatively easy to check and confirm for a power user such as myself, despite my not being a core developer of the code itself. I will also document the procedure so that anyone can replicate it.
It should be easy for other committers to unpack the zip and confirm its reliability and accuracy on Windows and other platforms. They could even add their counter-signing digital signatures.
If that fails, then it would be time for a Plan B. (A possible unexpected difficulty may have to do with line ends on text files across platforms, depending on whether tools address that as well as SVN clients do.)
- Dennis
- - - - - - - - - - - -
The TL;DR:
I am going to appeal to the Apache Project Maturity Model because I believe it is applicable here, whatever the status of the document at <http://community.apache.org/apache-way/apache-project-maturity-model.html>.
I think the relevant considerations of what should be *strived*for* are
CD10: The project produces Open Source software, for distribution to the public at no charge.
CD20: The project's code is easily discoverable and publicly accessible.
CD30: The code can be built in a reproducible way using widely available standard tools.
RE10: Releases consist of source code, distributed using standard and open archive formats that are expected to stay readable in the long term.
Something that is not in this list, but I see as a corollary, is that how we do these things is part of demonstrating care for the downstream users of the software we produce, for whatever reason they want to consult, examine, learn from, or even develop with, and whenever they choose to do so with a given source release.
- - - - - - - - - - -
MY QUESTION: There should never be a permissions issue in unpacking a Zip file on Windows. The only barriers are (1) file names must be ones that are acceptable and unique when extracted into an NTFS file system and (2) empty directories are not created, even if loaded into later in the stream, and there is no use of Zip extensions that apply only to Unix systems and Unix permissions. There are also concerns about length of full-path file names. (Note that some of these apply to using SVN on Windows too.)
My question is, on what platform were the troublesome Zips produced, using what tools? We may be seeing an interchange/interoperability problem that comes from inter-platform incompatibilities.
MY CONCERN: These provisions are not just for project developers, but for the public. I don't think only distributing a .tar.bz2 is very adequate with respect to CD20. RE10 is presumably satisfied if there are no uses of patented technologies, and assuming that no deviations from basic .tar formatting are employed (e.g., no launching of shell scripts involved).
I note also that Zip format is considered standard and open enough that it is the format employed for the ODF packages used by OpenOffice and also the packaging of .oxt extensions. It is also the packaging format of choice for OOXML, EPUB, and other domain-specific usages, including distribution of some libraries used in AOO and other projects. (I also notice that sometimes those Zips of libraries don't unpack properly on Windows, but rezipping them on Windows then works everywhere.)
The convention has been to use Zip for Windows oriented access to the source and some flavor of .tar.* for the Unix-oriented world. The basic use of Zip for Windows tends to unzip easily on any platform that can deal with the filename and folder hierarchies.
Although WinZip *will* unpack a .tar.gz (or .tgz) package, I do not know whether it will unpack a .tar.bz2. Expecting a member of the public who operates on Windows to know how to unpack a .tar.* is an unnecessary source of friction. I notice that 7z does handle .rar and .msi and perhaps tar.* compressions but I haven't checked those.
- - - - - - - - - - - - -
> -----Original Message-----
> From: Andrea Pescetti [mailto:pescetti@apache.org]
> Sent: Saturday, November 7, 2015 15:08
> To: dev@openoffice.apache.org
> Subject: [PROPOSAL] Distribute only one source package
>
> We currently distribute 3 source packages at
> https://archive.apache.org/dist/openoffice/4.1.2/source/
> 1) A .tar.bz2 file (209 MBytes)
> 2) A .tar.gz file (276 Mbytes)
> 3) A ZIP file (323 MBytes)
>
> The packages are equivalent, so any one would suffice.
>
> As discussed by Regina and Juergen recently, we ship #3 as a convenience
> for Windows users but this leads to broken file permissions, so the
> recommendation for Windows users is to use #1 or #2, which makes #3
> useless.
>
> I suggest, subject to lazy consensus, that we only distribute #1, i.e.,
> the .tar.bz2 file.
>
> Reasons:
> * If we distribute one source package, it will be clear that we are all
> testing and approving the same one
> * .tar.bz2 offer better compression than .tar.gz
> * bzip2 is ubiquitous today, so I don't believe that there are systems
> capable of building OpenOffice which don't have bzip2 available
> * better compression formats exist, but they are not as widely supported
> as the three we are using now, so I'd stick with bzip2
>
> This of course doesn't apply to 4.1.2, which is already released and
> will remain available in all three formats.
>
> Regards,
> Andrea.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Andrea Pescetti <pe...@apache.org>.
On 08/11/2015 Marcus wrote:
> Puh, too many compression formats that do the same. ;-)
Yes, there are at least a dozen compression formats better than
.tar.bz2, but it's too easy to get lost in comparing them...
So take this as a proposal to drop ZIP and drop .tar.gz ; then .tar.bz2
will be left, and if one wants to change it to something else this is
another story, since all kinds of issues, from widespread support to
hidden patent traps, from resource requirements to compression and
decompression times, would have to be examined then.
Realistically, sources are about 2% of the size of a release, so the
compressed size is not a major concern. Dropping ZIP and .tar.gz already
cuts the source package size by 75%, which is good enough: we don't need
a better compression right now, we need one "canonical" source package
that can work on all systems where OpenOffice can be built.
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Marcus <ma...@wtnet.de>.
Am 11/08/2015 11:34 AM, schrieb Damjan Jovanovic:
> On Sun, Nov 8, 2015 at 12:19 PM, Marcus<ma...@wtnet.de> wrote:
>
>> Am 11/08/2015 12:08 AM, schrieb Andrea Pescetti:
>>
>> Thanks for bringing this topic again on the table. I remember there was at
>> least 1 request to re-think the distribution policy about the source code
>> files. And also to consider to offer a file with 7Zip compression.
>>
>> We currently distribute 3 source packages at
>>> https://archive.apache.org/dist/openoffice/4.1.2/source/
>>> 1) A .tar.bz2 file (209 MBytes)
>>> 2) A .tar.gz file (276 Mbytes)
>>> 3) A ZIP file (323 MBytes)
>>>
>>> The packages are equivalent, so any one would suffice.
>>>
>>> As discussed by Regina and Juergen recently, we ship #3 as a convenience
>>> for Windows users but this leads to broken file permissions, so the
>>> recommendation for Windows users is to use #1 or #2, which makes #3
>>> useless.
>>>
>>> I suggest, subject to lazy consensus, that we only distribute #1, i.e.,
>>> the .tar.bz2 file.
>>>
>>> Reasons:
>>> * If we distribute one source package, it will be clear that we are all
>>> testing and approving the same one
>>>
>>
>> * It would make the release process a little bit smaller and less complex.
>>
>> * .tar.bz2 offer better compression than .tar.gz
>>>
>>
>> and all types compared together it's even the best compression.
>>
>> * bzip2 is ubiquitous today, so I don't believe that there are systems
>>> capable of building OpenOffice which don't have bzip2 available
>>> * better compression formats exist, but they are not as widely supported
>>> as the three we are using now, so I'd stick with bzip2
>>>
>>
>> Maybe it's worth it to test also with 7Zip. What I've seen and read
>> (outside of OpenOffice files) is promising.
>>
>>
> Do you mean lzma/xz compression (tar.xz) or the .7z file format too?
I meant .7z but it seems ...
> There's a Bugzilla issue about tar.xz:
> https://bz.apache.org/ooo/show_bug.cgi?id=122819
... there is a request for .xz, too.
Puh, too many compression formats that do the same. ;-)
Marcus
>> Otherwise to offer only a .bz2 file is a point I would support.
>>
>> This of course doesn't apply to 4.1.2, which is already released and
>>> will remain available in all three formats
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Damjan Jovanovic <da...@apache.org>.
On Sun, Nov 8, 2015 at 12:19 PM, Marcus <ma...@wtnet.de> wrote:
> Am 11/08/2015 12:08 AM, schrieb Andrea Pescetti:
>
> Thanks for bringing this topic again on the table. I remember there was at
> least 1 request to re-think the distribution policy about the source code
> files. And also to consider to offer a file with 7Zip compression.
>
> We currently distribute 3 source packages at
>> https://archive.apache.org/dist/openoffice/4.1.2/source/
>> 1) A .tar.bz2 file (209 MBytes)
>> 2) A .tar.gz file (276 Mbytes)
>> 3) A ZIP file (323 MBytes)
>>
>> The packages are equivalent, so any one would suffice.
>>
>> As discussed by Regina and Juergen recently, we ship #3 as a convenience
>> for Windows users but this leads to broken file permissions, so the
>> recommendation for Windows users is to use #1 or #2, which makes #3
>> useless.
>>
>> I suggest, subject to lazy consensus, that we only distribute #1, i.e.,
>> the .tar.bz2 file.
>>
>> Reasons:
>> * If we distribute one source package, it will be clear that we are all
>> testing and approving the same one
>>
>
> * It would make the release process a little bit smaller and less complex.
>
> * .tar.bz2 offer better compression than .tar.gz
>>
>
> and all types compared together it's even the best compression.
>
> * bzip2 is ubiquitous today, so I don't believe that there are systems
>> capable of building OpenOffice which don't have bzip2 available
>> * better compression formats exist, but they are not as widely supported
>> as the three we are using now, so I'd stick with bzip2
>>
>
> Maybe it's worth it to test also with 7Zip. What I've seen and read
> (outside of OpenOffice files) is promising.
>
>
Do you mean lzma/xz compression (tar.xz) or the .7z file format too?
There's a Bugzilla issue about tar.xz:
https://bz.apache.org/ooo/show_bug.cgi?id=122819
> Otherwise to offer only a .bz2 file is a point I would support.
>
> This of course doesn't apply to 4.1.2, which is already released and
>> will remain available in all three formats.
>>
>
> Marcus
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
Re: [PROPOSAL] Distribute only one source package
Posted by Marcus <ma...@wtnet.de>.
Am 11/08/2015 12:08 AM, schrieb Andrea Pescetti:
Thanks for bringing this topic again on the table. I remember there was
at least 1 request to re-think the distribution policy about the source
code files. And also to consider to offer a file with 7Zip compression.
> We currently distribute 3 source packages at
> https://archive.apache.org/dist/openoffice/4.1.2/source/
> 1) A .tar.bz2 file (209 MBytes)
> 2) A .tar.gz file (276 Mbytes)
> 3) A ZIP file (323 MBytes)
>
> The packages are equivalent, so any one would suffice.
>
> As discussed by Regina and Juergen recently, we ship #3 as a convenience
> for Windows users but this leads to broken file permissions, so the
> recommendation for Windows users is to use #1 or #2, which makes #3
> useless.
>
> I suggest, subject to lazy consensus, that we only distribute #1, i.e.,
> the .tar.bz2 file.
>
> Reasons:
> * If we distribute one source package, it will be clear that we are all
> testing and approving the same one
* It would make the release process a little bit smaller and less complex.
> * .tar.bz2 offer better compression than .tar.gz
and all types compared together it's even the best compression.
> * bzip2 is ubiquitous today, so I don't believe that there are systems
> capable of building OpenOffice which don't have bzip2 available
> * better compression formats exist, but they are not as widely supported
> as the three we are using now, so I'd stick with bzip2
Maybe it's worth it to test also with 7Zip. What I've seen and read
(outside of OpenOffice files) is promising.
Otherwise to offer only a .bz2 file is a point I would support.
> This of course doesn't apply to 4.1.2, which is already released and
> will remain available in all three formats.
Marcus
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
Re: [PROPOSAL] Distribute only one source package
Posted by Andrea Pescetti <pe...@apache.org>.
Dennis E. Hamilton wrote:
> 1. one of tar.gz/.bz2 is probably the best choice. ...
> That's a pragmatic solution, not a community-building one [;<). So be it.
Well, as we discussed so far, anybody willing to build on Windows has to
download tools capable of handling .tar.bz2 anyway, and someone who
can't expand a .tar.bz2 archive is frankly unlikely to succeed in the
(still quite tedious, as we are seeing) OpenOffice build process. So I
don't see risks to exclude developers by settling on .tar.bz2
> 2. ... tag/branch corresponding to the release, should provide the same files.
This is https://bz.apache.org/ooo/show_bug.cgi?id=126605
Differences are indeed minimal, but we'll fix it in 4.2.0.
The ext_sources are not code we want to ship anyway: the release
contains OpenOffice sources; for convenience we carry some external
sources in SVN, but we don't want to ship them. The "bootstrap" script
will download them from the official site or from repositories that are
under control of the project.
>The 4.1.2 zip source was definitely not made from a native-Windows checkout.
They were all generated on a Mac.
Regards,
Andrea.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
RE: [PROPOSAL] Distribute only one source package
Posted by "Dennis E. Hamilton" <de...@acm.org>.
I have some afterthoughts about this,
1. Because of line-ending issues, and the use of POSIX/*nix permissions or not, a Zip of the kind mainly used by Windows developers and a .tar.gz/.bz2 used by mainly POSIX-oriented developers (e.g., within Cygwin or on a Linux platform) might not be interchangeable. (Note that producing a .zip using a *nix utility does not always produce a result that is interchangeable with Windows. The best way to make a Zip is using Windows to do it)
I do agree that so long as the code is not really buildable using a native windows environment, one of tar.gz/.bz2 is probably the best choice. The tar.bz2 is certainly appealing with regard to the level of achieved compression.
That's a pragmatic solution, not a community-building one [;<). So be it.
2. In my explorations of the source distribution, there is a different problem. Under the release policies, the tag/branch corresponding to the release, should provide the same files. That is not the case for the Apache OpenOffice 4.1.2 source release. However, on the SVN the AOO412 tag includes an ext_sources and a test folder that are not in the source release archive. That needs to be reconciled. Also, the LICENSE, NOTICE, and README files of the Source Release folders are not at the Tag.
- Dennis
PS: I am able to Extract the 4.1.2 .zip and .tar.bz2 and obtain exactly the same set of files.
PPS: If I wanted to get native line-endings on native-Windows SVN checkouts there are many more filetypes I need svn:eol properties on [;<). The 4.1.2 zip source was definitely not made from a native-Windows checkout.
> -----Original Message-----
> From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org]
> Sent: Sunday, November 15, 2015 12:06
> To: dev@openoffice.apache.org
> Subject: RE: [PROPOSAL] Distribute only one source package
>
> I downloaded the tar.bz2 to see if it could be extracted on Windows
> using available tools.
>
> The WinZip I use (admittedly an old version) does not recognize it.
>
> 7z recognizes the bz2 and will decompress it as the .tar file.
>
> 7z will then also extract the .tar if used again in a separate step.
>
> The result from WinZip extraction of the .zip and the z7 two-step
> extraction consists of 60,955 files in 6,420 folders beneath the top-
> level 4.1.2/ folder either way.
>
> I know that WinZip will do a one-step extraction of a .tar.gz if that
> extension is changed to .tgz. I haven't determined if there is a
> similar modification for .tar.bz2 and whether or not 7z supports that
> and .tgz.
>
> Note that none of these exercises involved reliance on CygWin or
> anything similar, such as MSYS2. I did not use any command-line
> versions of the tools.
>
> - Dennis
>
> > -----Original Message-----
> > From: Andrea Pescetti [mailto:pescetti@apache.org]
> > Sent: Saturday, November 7, 2015 15:08
> > To: dev@openoffice.apache.org
> > Subject: [PROPOSAL] Distribute only one source package
> >
> > We currently distribute 3 source packages at
> > https://archive.apache.org/dist/openoffice/4.1.2/source/
> > 1) A .tar.bz2 file (209 MBytes)
> > 2) A .tar.gz file (276 Mbytes)
> > 3) A ZIP file (323 MBytes)
> >
> > The packages are equivalent, so any one would suffice.
> >
> > As discussed by Regina and Juergen recently, we ship #3 as a
> convenience
> > for Windows users but this leads to broken file permissions, so the
> > recommendation for Windows users is to use #1 or #2, which makes #3
> > useless.
> >
> > I suggest, subject to lazy consensus, that we only distribute #1,
> i.e.,
> > the .tar.bz2 file.
> >
> > Reasons:
> > * If we distribute one source package, it will be clear that we are
> all
> > testing and approving the same one
> > * .tar.bz2 offer better compression than .tar.gz
> > * bzip2 is ubiquitous today, so I don't believe that there are systems
> > capable of building OpenOffice which don't have bzip2 available
> > * better compression formats exist, but they are not as widely
> supported
> > as the three we are using now, so I'd stick with bzip2
> >
> > This of course doesn't apply to 4.1.2, which is already released and
> > will remain available in all three formats.
> >
> > Regards,
> > Andrea.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> > For additional commands, e-mail: dev-help@openoffice.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org
RE: [PROPOSAL] Distribute only one source package
Posted by "Dennis E. Hamilton" <de...@acm.org>.
I downloaded the tar.bz2 to see if it could be extracted on Windows using available tools.
The WinZip I use (admittedly an old version) does not recognize it.
7z recognizes the bz2 and will decompress it as the .tar file.
7z will then also extract the .tar if used again in a separate step.
The result from WinZip extraction of the .zip and the z7 two-step extraction consists of 60,955 files in 6,420 folders beneath the top-level 4.1.2/ folder either way.
I know that WinZip will do a one-step extraction of a .tar.gz if that extension is changed to .tgz. I haven't determined if there is a similar modification for .tar.bz2 and whether or not 7z supports that and .tgz.
Note that none of these exercises involved reliance on CygWin or anything similar, such as MSYS2. I did not use any command-line versions of the tools.
- Dennis
> -----Original Message-----
> From: Andrea Pescetti [mailto:pescetti@apache.org]
> Sent: Saturday, November 7, 2015 15:08
> To: dev@openoffice.apache.org
> Subject: [PROPOSAL] Distribute only one source package
>
> We currently distribute 3 source packages at
> https://archive.apache.org/dist/openoffice/4.1.2/source/
> 1) A .tar.bz2 file (209 MBytes)
> 2) A .tar.gz file (276 Mbytes)
> 3) A ZIP file (323 MBytes)
>
> The packages are equivalent, so any one would suffice.
>
> As discussed by Regina and Juergen recently, we ship #3 as a convenience
> for Windows users but this leads to broken file permissions, so the
> recommendation for Windows users is to use #1 or #2, which makes #3
> useless.
>
> I suggest, subject to lazy consensus, that we only distribute #1, i.e.,
> the .tar.bz2 file.
>
> Reasons:
> * If we distribute one source package, it will be clear that we are all
> testing and approving the same one
> * .tar.bz2 offer better compression than .tar.gz
> * bzip2 is ubiquitous today, so I don't believe that there are systems
> capable of building OpenOffice which don't have bzip2 available
> * better compression formats exist, but they are not as widely supported
> as the three we are using now, so I'd stick with bzip2
>
> This of course doesn't apply to 4.1.2, which is already released and
> will remain available in all three formats.
>
> Regards,
> Andrea.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
> For additional commands, e-mail: dev-help@openoffice.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org