PROPOSAL (was Re: Category-B tarballs in SVN )

[Pedro Giffuni <pf...@apache.org>]

Hello fellow indians; ;)

I think there is consensus that this is (at least)
a gray area so I have the following proposal, which
shouldn't get in the way of having this properly
solved by legal but which I think should solve at
least temporarily the issues that we have. It's
actually very simple but who knows, maybe it's even
acceptable as a general incubator policy at the ASF.

The ext_source in shall be divided, according to
the categories of the licenses, into two
directories in SVN, namely:

ext_source_A
ext_source_B

- Ext_source_B shall have a prominent text note that warns
users that the code there is made available only for
builder convenience but that the code is in general
not acceptable for inclusion in Apache source code
releases.

- It is understood that ext_source_B is a quarantine
area. The idea is that the code we have there will
only shrink with time. The code there can be updated
for security reasons but in general no new code should
be brought in without specific consensus (voting, checking
with the PPMC, etc, but not lazy consensus).

NOTE: Consensus for replacing standard OOo 3.4
functionality like the CoinMP solver code is a given
(particularly as the licensing is being worked on) but
we don't want this to be a loophole to bring in MPL'd
code into AOO.

Of course we still have to comply with the standard
Apache policies concerning Category-B : "Code that is
more substantial, more volatile, or not directly consumed
at runtime in source form may only be distributed in
binary form." but at least now it should be pretty clear
and easy for everyone downloading the code from SVN where
they can expect licensing issues.

Pedro.

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Andre Fischer <af...@a-w-f.de>]

On 13.01.2012 21:27, Pedro Giffuni wrote:
> Hello fellow indians; ;)
>
> I think there is consensus that this is (at least)
> a gray area so I have the following proposal, which
> shouldn't get in the way of having this properly
> solved by legal but which I think should solve at
> least temporarily the issues that we have. It's
> actually very simple but who knows, maybe it's even
> acceptable as a general incubator policy at the ASF.
>
> The ext_source in shall be divided, according to
> the categories of the licenses, into two
> directories in SVN, namely:
>
> ext_source_A
> ext_source_B

main/ooo.lst already offers a poor-mans version of this.  It has to 
sections headed with comments

# Libraries with category A license

and

# Libraries with category B license

which reference the tar balls in ext_sources/ that have, surprise, 
category A or category B license.

>
> - Ext_source_B shall have a prominent text note that warns
> users that the code there is made available only for
> builder convenience but that the code is in general
> not acceptable for inclusion in Apache source code
> releases.
>
> - It is understood that ext_source_B is a quarantine
> area. The idea is that the code we have there will
> only shrink with time. The code there can be updated
> for security reasons but in general no new code should
> be brought in without specific consensus (voting, checking
> with the PPMC, etc, but not lazy consensus).

This, I think, is an important point. Discourage people from adding more 
cat-B tar-balls, and encourage developers to replace the existing ones 
with category-A libraries.

We can start a Wiki page for the replacement tasks.  I think some time 
back you mentioned a possible replacement of rhino with V8.

-Andre

>
> NOTE: Consensus for replacing standard OOo 3.4
> functionality like the CoinMP solver code is a given
> (particularly as the licensing is being worked on) but
> we don't want this to be a loophole to bring in MPL'd
> code into AOO.
>
> Of course we still have to comply with the standard
> Apache policies concerning Category-B : "Code that is
> more substantial, more volatile, or not directly consumed
> at runtime in source form may only be distributed in
> binary form." but at least now it should be pretty clear
> and easy for everyone downloading the code from SVN where
> they can expect licensing issues.
>
> Pedro.
>

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Ross Gardler <rg...@opendirective.com>]

As I have asserted before, in order to graduate AOO will need to discuss
this with legal@

My recommendation is to minimise the need to host category B source
wherever possible. Where it is not possible local policy needs to prevent
inadvertent contamination from copyleft provisions. I imagine permission
well be given in such circumstances. Where it is merely a convenience I am
less sure and thus legal@ guidance is required.

Clearly there is disagreement about when it is necessary and until someone
at legal@ examines this specific case we are at an impasse.

As a mentor I want to see this resolved before graduation. In the meantime
a release is allowable as-is.

Ross

Sent from my mobile device, please forgive errors and brevity.
On Jan 18, 2012 2:26 AM, "Pedro Giffuni" <pf...@apache.org> wrote:

>
> --- Lun 16/1/12, Rob Weir ha scritto:
> ...
> > > the the Apache licensing policies:
> > >
> > > http://www.apache.org/legal/3party.html
> > >
> >
> > That URL is for an earlier draft of the policy.  You
> > really should be
> > referencing the latest version of the policy here:
> >
> > http://www.apache.org/legal/resolved.html#category-b
> >
>
> I have looked at this and indeed this appears to be
> the definitive policy (the previous one was being
> revised every 6 months and gave lighter provisions
> for podlings). The definitive policy is much less
> clear than the draft but certainly doesn't seem
> to contradict it.
>
> Still there is no distinction between source packages
> and binary packages and it seems the principles in the
> draft still apply.
>
> Pedro.
>
>

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Pedro Giffuni <pf...@apache.org>]

--- Lun 16/1/12, Rob Weir ha scritto:
...
> > the the Apache licensing policies:
> >
> > http://www.apache.org/legal/3party.html
> >
> 
> That URL is for an earlier draft of the policy.  You
> really should be
> referencing the latest version of the policy here:
> 
> http://www.apache.org/legal/resolved.html#category-b
>

I have looked at this and indeed this appears to be
the definitive policy (the previous one was being
revised every 6 months and gave lighter provisions
for podlings). The definitive policy is much less
clear than the draft but certainly doesn't seem
to contradict it.

Still there is no distinction between source packages
and binary packages and it seems the principles in the
draft still apply.

Pedro.

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Rob Weir <ro...@apache.org>]

On Mon, Jan 16, 2012 at 11:17 AM, Pedro Giffuni <pf...@apache.org> wrote:
> Hi Rob;
>
> I specifically avoided answering to this on Sunday because
> in my religious beliefs it is a day to rest and I didn't
> really want to spend time on this.
>
> Since the time this was posted I think I have seen the light
> (TM) and I am willing to share it with you if you have the
> patience.
>
> The comments that follow are NOT meant for the faint of
> heart if you are likely to have strong feelings about this
> please STOP READING NOW.
>
> Also, if you are still here, remember ... don't kill
> $MESSENGER.
>
> --- Sab 14/1/12, Rob Weir <ro...@apache.org> ha scritto:
>
>>
>> OK, though this is solving a problem we don't really have,
>> right?
>> Although we have not yet built a script to produce a source
>> package per Apache rules, when we do it will not include
>> any of the /ext-src modules, correct?  It won't include
>> the category-a and it will not include the category-b
>> either?  What would be the point, since the
>> build script brings down what it needs via the bootstrap,
>> per the configuration flags used?  So if we really want to
>> give proper notice to the person downloading our source
>> release, this needs to be done:
>
> We do have to include Category-A in the release or the
> release wont build. Separation has to happen.
>

I was talking about source packages.  They are not required to include
everything that needed to produce the binary package. For example, we
don't include Visual C++ or gcc or a copy of Microsoft Windows.
Generally, tools and libraries that are commonly available we treat as
a pre-requisite.

With category-a source code, I think from a policy perspective, we
could include the source in our release, we could require the user to
download manually as a per-requisite, or we could have the build
scripts do this automatically, from an external server or an Apache
server.

> Here is the first big flaw in your reasoning: there is no
> such thing as a source release or a binary release, there
> is simply a release.
>
> Let me explain it this way: long, long ago, before the

<snipping a long historical digression>

Yes, I believe we all know and understand this.  That is why I've been
using the terms "source package" and "binary package".  An Apache
"release" would include at least the source package.  The current plan
for AOO is to have both.


>
>
>>
>> I have no objections if you want to shuffle things around
>> in the directory structure, and update bootstrap logic
>> accordingly.
>
> "shuffling" things is not a problem but I don't think
> updating the bootstrap logic was mandatory. Our releases
> have to build on their own and as you note the sources for
> Category B stuff are not included anyways, but let me point
> out the second big failure in your reasoning.
>
> As I said before there is no such thing as "source releases"
> or "binary releases" and such terms don't appear anywhere in
> the the Apache licensing policies:
>
> http://www.apache.org/legal/3party.html
>

That URL is for an earlier draft of the policy.  You really should be
referencing the latest version of the policy here:

http://www.apache.org/legal/resolved.html#category-b

> Now, this phrase concerning Category-B has received a particular
> wrong reading:
>
> "Code that is more substantial, more volatile, or not directly
> consumed at runtime in source form may only be distributed in
> binary form."
>

That sentence does not exist in the actual policy.   That is only in the draft.

> We, and particularly you, have read this as a prohibition to

Oh, I didn't read it it all, since that is in the draft version only.

> include MPL'd code in "source release" but the truth is that
> it is a prohibition to distribute Category-B software *at
> all*. Distribution certainly includes subversion.
>

http://www.apache.org/legal/resolved.html#category-b

> The point is further clarified:
> "In addition, C-based projects may have difficulty using works
> under these licenses since they would have to deal with
> platform-specific binaries, rather than just distributing
> source that can be built on most any platform."
>

Again, this is not in the real policy.

> This last clarification has an upside: we can include in
> our releases (and therefore in SVN) platform independent files
> like Java bytecode and fonts under a Category-B license.
>

In any case, I think we've found one source of the confusion here.

Take a look at the live version of the policy (it does get updated
from time to time) and let me know if you still see policy concerns
with including category-b in binary form, in the binary packages of
our releases:

http://www.apache.org/legal/resolved.html#category-b

Regards,

-Rob


> Pedro.
>
> PS. My proposal was a step in the right direction but
> given that it's already insufficient I hereby withdraw
> it.
>

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Rob Weir <ro...@apache.org>]

On Mon, Jan 16, 2012 at 6:07 AM, Ross Gardler
<rg...@opendirective.com> wrote:
> On 15 January 2012 03:29, Rob Weir <ro...@apache.org> wrote:
>> On Fri, Jan 13, 2012 at 3:27 PM, Pedro Giffuni <pf...@apache.org> wrote:
>>> Hello fellow indians; ;)
>>>
>>> I think there is consensus that this is (at least)
>>> a gray area so I have the following proposal, which
>>> shouldn't get in the way of having this properly
>>> solved by legal but which I think should solve at
>>> least temporarily the issues that we have. It's
>>> actually very simple but who knows, maybe it's even
>>> acceptable as a general incubator policy at the ASF.
>>>
>>> The ext_source in shall be divided, according to
>>> the categories of the licenses, into two
>>> directories in SVN, namely:
>>>
>>> ext_source_A
>>> ext_source_B
>>>
>>
>> This is assuming all 3rd party modules are pure, under a single
>> license.  I don't believe this is always the case.
>
> Then can /should they be put in the most restrictive category?
>

They certainly could.  But the categories are really an creature of
Apache policy.  They are not necessarily relevant to downstream
consumers, whose policies could be quite different than ours.   That's
why it is good that we give the complete details in the LICENSE file
rather than some Apache-centric view of the world.  That's why I think
it would be far more useful to summarize exactly what license applies.
 We could even make a nice HTML table of this:  component, version,
original URL, license, etc.

>> Why not treat it the way we treat 3rd party modules in releases, e.g.,
>> with a LICENSE and NOTICE file?  We could put a LICENSE file in
>> /ext-src.  That would make it clear to anyone who browses that
>> directory.
>
> +1
>
> On one of the projects I work on we require libraries to have a
> corresponding licences/FOOBAR_license.txt file This then allows some
> simple machine processing to check the license is correctly recorded
> against all libraries included. This is built into the build system
> and automatically flags when something seems to be missing.
>

That could work.

>>> - Ext_source_B shall have a prominent text note that warns
>>> users that the code there is made available only for
>>> builder convenience but that the code is in general
>>> not acceptable for inclusion in Apache source code
>>> releases.
>>>
>>
>> OK, though this is solving a problem we don't really have, right?
>
> I think the point is that it clearly separates out stuff that is OK to
> stay and stuff that needs to be carefully managed. It's a step towards
> satisfying those who are concerned about including this source whilst
> avoiding the need to remove the convenience for developers of having
> the source available.
>

We already do that.  All of these third party modules are already
segregated from the main SVN tree.  This is true regardless of
license.   The legacy project didn't have them under version control.
They just stored them on a web server, totally separated from the
source code for OOo.    That option did not appear to be available to
us at Apache, so we stick them in the ext-src directory in SVN.

> This separation will make it easier for people to evaluate the impact
> of these files when it comes to graduation and will serve to signal
> that there is a process in place for the management of these
> dependencies (or that there needs to be a process).
>

Or we could document exactly what we're doing now, the precautions
we're already taking.  It seems that it would be hard to say whether
what we're doing already is inadequate if you don't know what exactly
we're already doing.  Not you specifically, but in general.

>>  So if we really want to give proper notice
>> to the person downloading our source release, this needs to be done:
>> 1) In the LICENSE and NOTICE files
>
> That has to happen anyway (for cat b licenses).
>

Category-b doesn't go out in released source packages.  And for binary
packages we're already include the required licenses and
notifications.


>> Putting extra verbiage in the SVN tree that is not included in the
>> releases -- I don't see the point.  Is this to protect casual browsers
>> of our SVN tree?
>
> For me the point is not about casual browers it is about developers
> like me who don't download source tars but instead checkout from SVN.
> We can't assume that all such developers will understand the nuances
> of license compatibility or even bother to check.
>

That maybe is the piece of the puzzle that may not be clear unless you
are a developer building AOO.  You do not need to checkout the
category-b source bundles from SVN.  In fact you should not.  You
would only be wasting harddrive space. Our build doc doesn't call for
these modules to be checked-out.  Someone would need to go out of
their way to check them out of SVN explicitly.  The whole idea is that
these are downloaded via the build script, based on the configure
options chosen by the builder.  If they want to build the default
binaries, with no category-b code, then of course, none of these
modules are downloaded.  And if they want to enable the optional
category-b stuff, then they override the configure settings and the
bootstrap script downloads the additional components, according to the
user's platform.  I believe this is done via http requests, not even
via an automated svn co.

We can't prevent someone from modifying the MPL code.  But they would
need to go far out of their way to do this.

> Ross

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Andre Fischer <af...@a-w-f.de>]

On 16.01.2012 12:07, Ross Gardler wrote:
> On 15 January 2012 03:29, Rob Weir<ro...@apache.org>  wrote:
[...]
>> Putting extra verbiage in the SVN tree that is not included in the
>> releases -- I don't see the point.  Is this to protect casual browsers
>> of our SVN tree?
>
> For me the point is not about casual browers it is about developers
> like me who don't download source tars but instead checkout from SVN.
> We can't assume that all such developers will understand the nuances
> of license compatibility or even bother to check.

Maybe the ability of SVN to include/reference an external SVN repository 
from another SVN repository can help (see [1]).

We could put the cat B tar-balls into an SVN repo, that is or is not 
located on an Apache Server, and reference that repo from our OpenOffice 
repository.

The developer, who checks out the source code, can then decide whether 
to automatically checkout the cat B tar-balls together with the rest of 
the code or to just check out the OpenOffice source.

One downside of this approach is the wrong default behavior on checkout:
the command "svn checkout" checks out all referenced external 
repositories.  In order to not include the cat B tar balls you have to 
add the "--ignore-externals" option.  But with a little bit of 
documentation and by adding this option to the Source Control wiki page 
[2] that is maybe OK.

-Andre

[1] http://svnbook.red-bean.com/en/1.5/svn.advanced.externals.html
[2] http://incubator.apache.org/openofficeorg/source.html

[...]

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Ross Gardler <rg...@opendirective.com>]

On 15 January 2012 03:29, Rob Weir <ro...@apache.org> wrote:
> On Fri, Jan 13, 2012 at 3:27 PM, Pedro Giffuni <pf...@apache.org> wrote:
>> Hello fellow indians; ;)
>>
>> I think there is consensus that this is (at least)
>> a gray area so I have the following proposal, which
>> shouldn't get in the way of having this properly
>> solved by legal but which I think should solve at
>> least temporarily the issues that we have. It's
>> actually very simple but who knows, maybe it's even
>> acceptable as a general incubator policy at the ASF.
>>
>> The ext_source in shall be divided, according to
>> the categories of the licenses, into two
>> directories in SVN, namely:
>>
>> ext_source_A
>> ext_source_B
>>
>
> This is assuming all 3rd party modules are pure, under a single
> license.  I don't believe this is always the case.

Then can /should they be put in the most restrictive category?

> Why not treat it the way we treat 3rd party modules in releases, e.g.,
> with a LICENSE and NOTICE file?  We could put a LICENSE file in
> /ext-src.  That would make it clear to anyone who browses that
> directory.

+1

On one of the projects I work on we require libraries to have a
corresponding licences/FOOBAR_license.txt file This then allows some
simple machine processing to check the license is correctly recorded
against all libraries included. This is built into the build system
and automatically flags when something seems to be missing.

>> - Ext_source_B shall have a prominent text note that warns
>> users that the code there is made available only for
>> builder convenience but that the code is in general
>> not acceptable for inclusion in Apache source code
>> releases.
>>
>
> OK, though this is solving a problem we don't really have, right?

I think the point is that it clearly separates out stuff that is OK to
stay and stuff that needs to be carefully managed. It's a step towards
satisfying those who are concerned about including this source whilst
avoiding the need to remove the convenience for developers of having
the source available.

This separation will make it easier for people to evaluate the impact
of these files when it comes to graduation and will serve to signal
that there is a process in place for the management of these
dependencies (or that there needs to be a process).

>  So if we really want to give proper notice
> to the person downloading our source release, this needs to be done:
> 1) In the LICENSE and NOTICE files

That has to happen anyway (for cat b licenses).

> Putting extra verbiage in the SVN tree that is not included in the
> releases -- I don't see the point.  Is this to protect casual browsers
> of our SVN tree?

For me the point is not about casual browers it is about developers
like me who don't download source tars but instead checkout from SVN.
We can't assume that all such developers will understand the nuances
of license compatibility or even bother to check.

Ross

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Pedro Giffuni <pf...@apache.org>]

Hi Rob;

I specifically avoided answering to this on Sunday because
in my religious beliefs it is a day to rest and I didn't
really want to spend time on this.

Since the time this was posted I think I have seen the light
(TM) and I am willing to share it with you if you have the
patience.

The comments that follow are NOT meant for the faint of
heart if you are likely to have strong feelings about this
please STOP READING NOW. 

Also, if you are still here, remember ... don't kill
$MESSENGER.

--- Sab 14/1/12, Rob Weir <ro...@apache.org> ha scritto:

> 
> OK, though this is solving a problem we don't really have,
> right?
> Although we have not yet built a script to produce a source
> package per Apache rules, when we do it will not include
> any of the /ext-src modules, correct?  It won't include
> the category-a and it will not include the category-b
> either?  What would be the point, since the
> build script brings down what it needs via the bootstrap,
> per the configuration flags used?  So if we really want to
> give proper notice to the person downloading our source
> release, this needs to be done:

We do have to include Category-A in the release or the
release wont build. Separation has to happen.

Here is the first big flaw in your reasoning: there is no
such thing as a source release or a binary release, there
is simply a release.

Let me explain it this way: long, long ago, before the
Internet ever was, release engineers would talk about
preparing the distribution stuff they could put into
specific media as a Release. The media then was usually
tapes, later floppies, then CDs, and with the advent of
the Internet new forms of media appeared and new formats
corresponded to those distribution, ZIP files, .tar.gz
archives, you get the idea.

Eventually, new releases and updates were made available
by electronic means without dealing directly with tarballs
or zipfiles and the tendency is indeed to use such modern
methods when possible. One of such methods is called
"subversion" (SVN) and it has been very popular in the
advent of Opensource software, where the source code is
sometimes more important than the accidental binaries.

And the point here is: a release is not just what is
included in a source tarball or a zipfile it is
what is tagged and branched in SVN. Also, if you
check the documentation on how tags and branches
are created you will notice we have to clearly separate
what belongs to the release to what doesn't.



> 
> I have no objections if you want to shuffle things around
> in the directory structure, and update bootstrap logic
> accordingly.

"shuffling" things is not a problem but I don't think
updating the bootstrap logic was mandatory. Our releases
have to build on their own and as you note the sources for
Category B stuff are not included anyways, but let me point
out the second big failure in your reasoning.

As I said before there is no such thing as "source releases"
or "binary releases" and such terms don't appear anywhere in
the the Apache licensing policies:

http://www.apache.org/legal/3party.html

Now, this phrase concerning Category-B has received a particular
wrong reading:

"Code that is more substantial, more volatile, or not directly
consumed at runtime in source form may only be distributed in
binary form."

We, and particularly you, have read this as a prohibition to
include MPL'd code in "source release" but the truth is that
it is a prohibition to distribute Category-B software *at
all*. Distribution certainly includes subversion.

The point is further clarified:
"In addition, C-based projects may have difficulty using works
under these licenses since they would have to deal with
platform-specific binaries, rather than just distributing
source that can be built on most any platform."

This last clarification has an upside: we can include in
our releases (and therefore in SVN) platform independent files
like Java bytecode and fonts under a Category-B license.

Pedro.

PS. My proposal was a step in the right direction but
given that it's already insufficient I hereby withdraw
it.

Re: PROPOSAL (was Re: Category-B tarballs in SVN )

[Rob Weir <ro...@apache.org>]

On Fri, Jan 13, 2012 at 3:27 PM, Pedro Giffuni <pf...@apache.org> wrote:
> Hello fellow indians; ;)
>
> I think there is consensus that this is (at least)
> a gray area so I have the following proposal, which
> shouldn't get in the way of having this properly
> solved by legal but which I think should solve at
> least temporarily the issues that we have. It's
> actually very simple but who knows, maybe it's even
> acceptable as a general incubator policy at the ASF.
>
> The ext_source in shall be divided, according to
> the categories of the licenses, into two
> directories in SVN, namely:
>
> ext_source_A
> ext_source_B
>

This is assuming all 3rd party modules are pure, under a single
license.  I don't believe this is always the case.

Why not treat it the way we treat 3rd party modules in releases, e.g.,
with a LICENSE and NOTICE file?  We could put a LICENSE file in
/ext-src.  That would make it clear to anyone who browses that
directory.

> - Ext_source_B shall have a prominent text note that warns
> users that the code there is made available only for
> builder convenience but that the code is in general
> not acceptable for inclusion in Apache source code
> releases.
>

OK, though this is solving a problem we don't really have, right?
Although we have not yet built a script to produce a source package
per Apache rules, when we do it will not include any of the /ext-src
modules, correct?  It won't include the category-a and it will not
include the category-b either?  What would be the point, since the
build script brings down what it needs via the bootstrap, per the
configuration flags used?  So if we really want to give proper notice
to the person downloading our source release, this needs to be done:
1) In the LICENSE and NOTICE files.  and 2) In the build instructions.

Putting extra verbiage in the SVN tree that is not included in the
releases -- I don't see the point.  Is this to protect casual browsers
of our SVN tree?

I have no objections if you want to shuffle things around in the
directory structure, and update bootstrap logic accordingly.  It is
your time.  But given that these files are not included in our
releases, I don't think this solves the problem you originally set out
to solve.

> - It is understood that ext_source_B is a quarantine
> area. The idea is that the code we have there will
> only shrink with time. The code there can be updated
> for security reasons but in general no new code should
> be brought in without specific consensus (voting, checking
> with the PPMC, etc, but not lazy consensus).
>
> NOTE: Consensus for replacing standard OOo 3.4
> functionality like the CoinMP solver code is a given
> (particularly as the licensing is being worked on) but
> we don't want this to be a loophole to bring in MPL'd
> code into AOO.
>
> Of course we still have to comply with the standard
> Apache policies concerning Category-B : "Code that is
> more substantial, more volatile, or not directly consumed
> at runtime in source form may only be distributed in
> binary form." but at least now it should be pretty clear
> and easy for everyone downloading the code from SVN where
> they can expect licensing issues.
>
> Pedro.
>