You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@incubator.apache.org by Marvin Humphrey <ma...@rectangular.com> on 2014/10/11 14:27:00 UTC
Bloated NOTICE files are evil
On Sat, Oct 11, 2014 at 2:32 AM, Sean Owen <sr...@gmail.com> wrote:
> I am reading http://www.apache.org/dev/licensing-howto.html . Yes
> LICENSE also needs to contain more things as well. Yes, there are
> several situations where NOTICE does not need to change, but this is
> the key sentence:
>
> "Aside from Apache-licensed dependencies which supply NOTICE files of
> their own, it is uncommon for a dependency to require additions to
> NOTICE."
>
> Lots of the dependencies I see here are Apache-licensed dependencies
> with NOTICE files. The Apache License 2 clause 4 means any (relevant)
> parts of the NOTICE files must be included in a distribution, and the
> NOTICE file is the place for that*.
>
> Here's a correct (AFAIK) example from Spark:
>
> https://github.com/apache/spark/blob/master/NOTICE
> https://github.com/apache/spark/blob/master/LICENSE
>
> Some of the same dependencies are included in both.
>
> I don't see why this doesn't apply to Drill too? Unless the guidance
> above is out of date.
I admire the good-faith efforts that the Spark (and Solr) folks have put
in attempting to comply with their interpretation of ASF requirements, but I
don't think we should encourage podlings to emulate the current state of their
licensing documentation.
Complicated NOTICE files impose a burden on downstream consumers trying to
evaluate the licensing of our products. The CYA instinct to throw everything
vaguely notice-ish into NOTICE is understandable, but destructive -- because
the ALv2 section 4(d) imposes more harsh obligations for content in NOTICE
than for content in LICENSE.
http://www.apache.org/licenses/LICENSE-2.0.html#redistribution
d. If the Work includes a "NOTICE" text file as part of its distribution,
then any Derivative Works that You distribute must include a readable
copy of the attribution notices contained within such NOTICE file,
excluding those notices that do not pertain to any part of the
Derivative Works, in at least one of the following places: within a
NOTICE text file distributed as part of the Derivative Works; within
the Source form or documentation, if provided along with the
Derivative Works; or, within a display generated by the Derivative
Works, if and wherever such third-party notices normally appear.
[...]
Recall that the NOTICE file originated through what I would characterize as a
"clever hack" to get the ALv1.1 advertisement clause out of the license so it
wouldn't conflict with the GPL. (See <http://s.apache.org/XAf> and
<http://s.apache.org/jP>.) It's unfortunate that so much extraneous material
has landed in NOTICE in the years since, and that so many hours have been
burned by people trying to figure out how to propagate it.
Here's Roy (in 2008):
http://s.apache.org/wbM
NOTICE is not for describing the license of each artifact.
It is only for required attributions.
[...]
> Furthermore, I assume it is not problematic to have more stuff in the
> NOTICE file(s) than is really required.
Yes, it is problematic. Consider it a tax on all downstream recipients.
> * I am not clear whether distribution a .jar, which has its NOTICE
> file embedded, "counts". This would not be the case in an 'uber' jar
> the the project distributes, and there are some third-party uber jars
> here like hive-exec, but I don't see a Drill uber jar. Maybe that
> counts; maybe it's safer just to populate NOTICE appropriately. Or,
> avoid shipping copies of all these jars directly?
Such confusion is a perfect illustration of why bloated NOTICE files are worse
than bloated LICENSE files.
Marvin Humphrey
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
In any case, the NOTICE files included in every jar are preserved. The
Drill binary release complies *exactly* with the language "include a
readable copy of the attribution notices contained within such NOTICE file".
There is no violation of OSS licenses going on here.
On Sat, Oct 11, 2014 at 6:51 AM, Sean Owen <sr...@gmail.com> wrote:
> On Sat, Oct 11, 2014 at 1:27 PM, Marvin Humphrey <ma...@rectangular.com>
> wrote:
> > I admire the good-faith efforts that the Spark (and Solr) folks have put
> > in attempting to comply with their interpretation of ASF requirements,
> but I
> > don't think we should encourage podlings to emulate the current state of
> their
> > licensing documentation.
>
> Yes, but much more importantly, it is not *violating an OSS license*
> to distribute software that says 'you must include X in a NOTICE file
> if you distribute this' without including X in the NOTICE file?
>
> Yes it means greater downstream burden, yes it may have originated
> from a 'hack', but that doesn't seem to make the license not say what
> it says. Obviously I'd rather not have to do this either.
>
> It's certainly a best practice to not put things in NOTICE that aren't
> actually required. For example upstream NOTICE may refer to components
> that the downstream project does not distribute. I think it's also a
> good reason to think about whether a project actually needs to
> distribute third-party code versus merely depend on it via Maven.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
On Sun, Oct 12, 2014 at 8:16 AM, sebb <se...@gmail.com> wrote:
> > 3. presence or absence of NOTICEs in binary distribution doesn't
> > affect source distribution.
>
> Not true.
>
> > 4. presence or absence of NOTICEs in source distribution doesn't
> > affect binary distribution.
>
> Not true.
>
> ==
>
> The principle is that the NOTICE and LICENSE files in an ASF release
> tarball (whether that is binary or source) must only relate to the
> bits which are actually included in the artifact.
Actually, I think that I was ambiguous a bit in my statement.
My intent was to say that each distribution is independent and the notice
should be considered on the content of the distribution. You are correct
that source may appear in binary distributions and binary in source
distributions and that these situations make my statements hard to
understand.
My point was that the content of one distribution does not inherently imply
the contents of the NOTICE or LICENSE files in a different distribution.
Each must be well-formed based on their own content.
Thank you for the clarifying restatement.
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
On Mon, Oct 13, 2014 at 1:14 PM, sebb <se...@gmail.com> wrote:
> >>> 4. presence or absence of NOTICEs in source distribution doesn't
> >>> affect binary distribution.
> >>
> >> Not true.
> >
> > Ditto.
>
> If you are saying that the NOTICE in an ASF binary release has no
> direct bearing on the NOTICE in the corresponding ASF source release,
> then strictly speaking I agree with you (though of course both will
> have the same pre-amble, and may indeed be identical). The NOTICE in
> an ASF source release does however affect the corresponding ASF binary
> release.
>
> However, if you are saying that a NOTICE in an arbitrary _binary_
> distribution does not affect an ASF _source_ distribution which
> includes said _binary_ distribution, then I don't agree. Likewise if
> you swap _binary_ and _source_ in the previous sentence.
I think that the simpler form of the assertion is just that the contents of
the NOTICE depends on the content of the distribution, be it source, binary
or Martian cookies.
It is reasonably common for the source artifacts to include upstream
artifacts that are not necessarily easily or stably available and not to
include many runtime dependencies. This will increase or decrease the
content of the NOTICE and LICENSE files.
Conversely, it is common for binary artifacts to include upstream binary
artifacts for conveniences and to not include test artifacts. A single
binary artifact may be for a limited purpose, thus shrinking the scope of
NOTICE and LICENSE relative to the corresponding source artifact.
All of these are simpler if you simply look at the bits in the artifact and
declare them appropriately.
Re: Bloated NOTICE files are evil
Posted by sebb <se...@gmail.com>.
On 12 October 2014 17:40, Roman Shaposhnik <rv...@apache.org> wrote:
> On Sun, Oct 12, 2014 at 5:16 AM, sebb <se...@gmail.com> wrote:
>> I don't think that's correct.
>
> Well, since neither of us is a lawyer (at least I am not) at this
> point we're debating interpretations on ALv2.
>
>> A source distribution with NOTICE may be included in an ASF binary
>> distribution - e.g. example code is often included in biary dists.
>> In this case the relevant parts of the source NOTICE would need to be
>> considered for inclusion in the ASF NOTICE for the binary.
>>
>> In other words a source distribution affects all works that include
>> it, not just source distributiions.
>
> My opinion is that you're abusing the notion of the distribution here.
I meant distribution in the sense of being publically available for
distribution, rather than the word as used by Linux providers.
I think my usage agrees with the "Apache License and Distribution FAQ"
http://www.apache.org/foundation/license-faq.html
>> The NOTICE file in an ASF source release must contain all *required*
>> attributions for all included bits, and must not contain attributions
>> for bits that are not included. Only INCLUDED bits affect the NOTICE
>> (and LICENSE) files.
>
> Correct.
I'm glad we agree on something.
Do you agree that the above paragraph also applies to ASF binary releases?
>> The process for determining what goes in the ASF NOTICE file starts
>> with the contents of the associated release artifact.
>
> I still maintain that my interpretation of ALv2 makes it relevant
> what distribution we're talking about source or binary.
That is the first time I have encountered the notion.
Which parts of the ALv2 lead you to that conclusion?
>>> 3. presence or absence of NOTICEs in binary distribution doesn't
>>> affect source distribution.
>>
>> Not true.
>
> That would be your opinion at this point. The one that I don't share.
>
>>> 4. presence or absence of NOTICEs in source distribution doesn't
>>> affect binary distribution.
>>
>> Not true.
>
> Ditto.
If you are saying that the NOTICE in an ASF binary release has no
direct bearing on the NOTICE in the corresponding ASF source release,
then strictly speaking I agree with you (though of course both will
have the same pre-amble, and may indeed be identical). The NOTICE in
an ASF source release does however affect the corresponding ASF binary
release.
However, if you are saying that a NOTICE in an arbitrary _binary_
distribution does not affect an ASF _source_ distribution which
includes said _binary_ distribution, then I don't agree. Likewise if
you swap _binary_ and _source_ in the previous sentence.
> Thanks,
> Roman.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sun, Oct 12, 2014 at 5:16 AM, sebb <se...@gmail.com> wrote:
> I don't think that's correct.
Well, since neither of us is a lawyer (at least I am not) at this
point we're debating interpretations on ALv2.
> A source distribution with NOTICE may be included in an ASF binary
> distribution - e.g. example code is often included in biary dists.
> In this case the relevant parts of the source NOTICE would need to be
> considered for inclusion in the ASF NOTICE for the binary.
>
> In other words a source distribution affects all works that include
> it, not just source distributiions.
My opinion is that you're abusing the notion of the distribution here.
> The NOTICE file in an ASF source release must contain all *required*
> attributions for all included bits, and must not contain attributions
> for bits that are not included. Only INCLUDED bits affect the NOTICE
> (and LICENSE) files.
Correct.
> The process for determining what goes in the ASF NOTICE file starts
> with the contents of the associated release artifact.
I still maintain that my interpretation of ALv2 makes it relevant
what distribution we're talking about source or binary.
>> 3. presence or absence of NOTICEs in binary distribution doesn't
>> affect source distribution.
>
> Not true.
That would be your opinion at this point. The one that I don't share.
>> 4. presence or absence of NOTICEs in source distribution doesn't
>> affect binary distribution.
>
> Not true.
Ditto.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by sebb <se...@gmail.com>.
On 11 October 2014 19:30, Roman Shaposhnik <rv...@apache.org> wrote:
> On Sat, Oct 11, 2014 at 11:08 AM, Sean Owen <sr...@gmail.com> wrote:
>> On Sat, Oct 11, 2014 at 7:02 PM, Ted Dunning <te...@gmail.com> wrote:
>>> Netty's artifacts ("its distribution") do not include a notice. Thus,
>>
>> They most certainly do. Please download the distribution of Netty 4.0.20:
>>
>> https://github.com/netty/netty/releases/tag/netty-4.0.20.Final
>
> This is source code distribution. Assuming that distribution == source
> code distribution is a very ASF-centered view point. Nothing wrong
> with this, but form my interpretation of legal view on this, unless your
> license clearly spells out what "distribution" you can't assume whether
> that is source, binary or both.
>
> Now, ALv2, of course, clearly defines that it is both. And there's the
> rub: it does not, as far as I can tell, infer cross-contamination of these
> types of distributions. IOW the following is true, as far as I can tell:
> 1. if source distribution of ALv2 software has NOTICEs then and only
> then a derviate work distributed in *SOURCE* form must continue
> shipping them.
I don't think that's correct.
A source distribution with NOTICE may be included in an ASF binary
distribution - e.g. example code is often included in biary dists.
In this case the relevant parts of the source NOTICE would need to be
considered for inclusion in the ASF NOTICE for the binary.
In other words a source distribution affects all works that include
it, not just source distributiions.
The NOTICE file in an ASF source release must contain all *required*
attributions for all included bits, and must not contain attributions
for bits that are not included. Only INCLUDED bits affect the NOTICE
(and LICENSE) files.
The process for determining what goes in the ASF NOTICE file starts
with the contents of the associated release artifact.
For each item in the release artifact, you need to determine what
attributions (if any) need to be added to the NOTICE file.
> 2. if binary distribution of ALv2 software has NOTICEs then and only
> then a derviate work distributed in *BINARY* form must continue
> shipping them.
Ditto; binary dists (e.g. images) may be included in ASF source.
> 3. presence or absence of NOTICEs in binary distribution doesn't
> affect source distribution.
Not true.
> 4. presence or absence of NOTICEs in source distribution doesn't
> affect binary distribution.
Not true.
==
The principle is that the NOTICE and LICENSE files in an ASF release
tarball (whether that is binary or source) must only relate to the
bits which are actually included in the artifact.
> Thanks,
> Roman.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Oct 11, 2014 at 11:08 AM, Sean Owen <sr...@gmail.com> wrote:
> On Sat, Oct 11, 2014 at 7:02 PM, Ted Dunning <te...@gmail.com> wrote:
>> Netty's artifacts ("its distribution") do not include a notice. Thus,
>
> They most certainly do. Please download the distribution of Netty 4.0.20:
>
> https://github.com/netty/netty/releases/tag/netty-4.0.20.Final
This is source code distribution. Assuming that distribution == source
code distribution is a very ASF-centered view point. Nothing wrong
with this, but form my interpretation of legal view on this, unless your
license clearly spells out what "distribution" you can't assume whether
that is source, binary or both.
Now, ALv2, of course, clearly defines that it is both. And there's the
rub: it does not, as far as I can tell, infer cross-contamination of these
types of distributions. IOW the following is true, as far as I can tell:
1. if source distribution of ALv2 software has NOTICEs then and only
then a derviate work distributed in *SOURCE* form must continue
shipping them.
2. if binary distribution of ALv2 software has NOTICEs then and only
then a derviate work distributed in *BINARY* form must continue
shipping them.
3. presence or absence of NOTICEs in binary distribution doesn't
affect source distribution.
4. presence or absence of NOTICEs in source distribution doesn't
affect binary distribution.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sat, Oct 11, 2014 at 1:09 PM, Sean Owen <sr...@gmail.com> wrote:
> The meta-issue is that some here seem to be arguing that IP licensing
> can be taken lightly because it's hard or annoying.
That's not it. I am arguing that licensing should be as minimal as possible
because that's in the interest of *users*.
As an IPMC member who reviews releases from time to time, I'm one of those
users. But I also participate in license evaluation for open source products
we incorporate at my day job, and when we come across a 5k long train wreck of
a license, it costs us time and money.
+1 to this suggestion of yours from upthread:
It's certainly a best practice to not put things in NOTICE that aren't
actually required.
Marvin Humphrey
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Sat, Oct 11, 2014 at 1:56 PM, Ross Gardler (MS OPEN TECH)
<Ro...@microsoft.com> wrote:
> Excellent. More eyes on these issues is great. Thanks for spending your time
> on this.
+1
Sean and I have continued our conversation offline and were able to clarify
some aspects of our (very imperfect) documentation. I believe we are now
largely in agreement, and I'm grateful for his conscientiousness and tenacity.
Here's my summary of where we ended up:
* Think of NOTICE not as a vehicle for conveying information to downstream
consumers, but as a mechanism for compelling redistributors to relay
information.
* Some of the advice in the outdated draft document
<http://www.apache.org/legal/3party.html> is misguided in that it attempts
to use NOTICE in ways which are inappropriate given the obligations
imposed by the ALv2. For instance, 3party.html instructs that
all "Category B" dependencies get a specific treatment in NOTICE; the
final version at <http://www.apache.org/legal/resolved.html> omits this
requirement while still implicitly allowing use of NOTICE when legally
necessary.
* Stuff shouldn't end up in NOTICE unless it was required by a copyright
owner. (See <http://s.apache.org/ZEA>.)
* The only original text that we should be adding to NOTICE in our ASF
capacity is the ALv1.1 advertising clause's replacement.
* Another example of stuff that ends up in NOTICE is a contributor's
copyright notice that they or their agent have relocated to NOTICE while
deleting such notices from individual source files.
* Not much else passes the test.
Marvin Humphrey
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
RE: Bloated NOTICE files are evil
Posted by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com>.
Excellent. More eyes on these issues is great. Thanks for spending your time on this.
Sent from my Windows Phone
________________________________
From: Ted Dunning<ma...@gmail.com>
Sent: 10/11/2014 1:34 PM
To: general@incubator.apache.org<ma...@incubator.apache.org>
Subject: Re: Bloated NOTICE files are evil
On Sat, Oct 11, 2014 at 1:09 PM, Sean Owen <sr...@gmail.com> wrote:
> Should I just propose a PR since I'm making trouble about it?
>
Great idea.
Keep in mind that the binary NOTICE file in Drill is generated
automatically by the build code in the source distribution so any PR should
be against that mechanism rather than the final result.
This is analogous to requesting that patches be made to the source rather
than the binary. For the purposes here, the NOTICE and LICENSE file can be
considered the product of the source code.
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
On Sat, Oct 11, 2014 at 1:09 PM, Sean Owen <sr...@gmail.com> wrote:
> Should I just propose a PR since I'm making trouble about it?
>
Great idea.
Keep in mind that the binary NOTICE file in Drill is generated
automatically by the build code in the source distribution so any PR should
be against that mechanism rather than the final result.
This is analogous to requesting that patches be made to the source rather
than the binary. For the purposes here, the NOTICE and LICENSE file can be
considered the product of the source code.
Re: Bloated NOTICE files are evil
Posted by Sean Owen <sr...@gmail.com>.
Sounds OK to me too if that is the prevailing sentiment; I personally
do not operate (non-ASF) OSS projects that way. It seems just within
the letter of the law.
What about this little transitive dep of Netty I mentioned? This is
not a NOTICE issue, and I *think* this one is beyond interpreting
away, even if it looks so small. There are more, and other deps
besides Netty. Or, I'd be happy to know why the license doesn't say
what I think it says, in which case I'll apologize to everyone and buy
you a beer for your time.
The meta-issue is that some here seem to be arguing that IP licensing
can be taken lightly because it's hard or annoying. The subtext I get
is that the AL2 clause 4d is considered so minor and exasperating to
comply with as to be practically ignorable? It would surprise me if
that's supposed to be the takeaway, even if, hey, would kind of make
some practical sense. But AL2 says what it says.
Concretely, I'm suggesting a good-faith effort to review and if needed
fix Drill's license issues, since a casual review seems to have
identified a handful of problems (TBC: at least, we're on to: Netty's
embedded dependencies not appearing in LICENSE). If someone's in a
rush to get this release out, OK, deal with it after?
Should I just propose a PR since I'm making trouble about it?
On Sat, Oct 11, 2014 at 8:58 PM, Ross Gardler (MS OPEN TECH)
<Ro...@microsoft.com> wrote:
> +1, lets not second guess the intention of a third party project. Lets simply ensure *our* projects do what is required.
>
> If anyone here is concerned about the third party being unaware of the results of their distribution practices then that part of this discussion should move to the third party project.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
RE: Bloated NOTICE files are evil
Posted by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com>.
+1, lets not second guess the intention of a third party project. Lets simply ensure *our* projects do what is required.
If anyone here is concerned about the third party being unaware of the results of their distribution practices then that part of this discussion should move to the third party project.
Sent from my Windows Phone
________________________________
From: Roman Shaposhnik<ma...@apache.org>
Sent: 10/11/2014 12:09 PM
To: general@incubator.apache.org<ma...@incubator.apache.org>
Subject: Re: Bloated NOTICE files are evil
On Sat, Oct 11, 2014 at 11:47 AM, Sean Owen <sr...@gmail.com> wrote:
>> You are confusing different distributions. Netty provides a source
>> distribution which does include a NOTICE file. Netty also provides binary
>> (jar) distributions. These do not include a NOTICE file.
>
> I think this is a fair question. Did the Netty project intend the
> NOTICE file to pertain only to the source distribution? from its
> contents, it pertains to the binary distro too, since the binary form
> contains the elements referenced in the NOTICE.
This one really strikes me as an academic exercise. I am not
sure second guessing a motivation of a non-ASF project would
be fruitful for our discussion.
The situation is *really* simple:
1. it seems that for the stuff in Netty's binary distro Drill is
doing the right thing with its binary distro
2. it seems that for the stuff in Netty's source distro Drill is
doing the right thing with its source distro
Is there anything else I am missing?
> I supposed I'd expect erring a bit on the side of the intent and
> spirit from the ASF in interpreting these things, but hey, let's stick
> to technicalities. Just taking the first example -- Netty contains
> among other things a modified version of Webbit, a BSD-licensed
> library. Drill is distributes this code. Where is this in LICENSE?
> It's not even in NOTICE which would be "close" and reference its own
> LICENSE, but you don't distribute the NOTICE even. This is the problem
> with trying to cut it so fine.
I find it unfair to put this burden on Drill. If you really want to help Netty
with the spirit of the law -- why don't you talk to the Netty developers
and straighten these issues with them first?
> It's such a rabbit hole to be sure, and the little downside to being
> blessed with freely accessing so many others' projects. I struggled
> for a while on Spark with this and still probably don't have it all
> right. I mean, shouldn't someone take a look at the many other
> dependencies? this is just one I ran into as a spectator. Why the
> hostility? just stick to the discussion of the license please.
I haven't notice any hostility, really (perhaps participating in some of the
more boisterous ASF communities equipped me with thicker skin).
That said, I do suggest we stay on topic and not try to boil the ocean
here. We are in charge of our own software -- we should do the
right thing with it. With projects outside of ASF we can only do
so much.
On a related note: with every legal council I ever work with, one
of the first conversations I have is around the fact that you never
ever trust somebody else's legal judgement. Which means
that regardless of what the LICENSE or NOTICE say you are
on the hook to 'trust by verify'. Hence BlackDuck and Palamida
scans. When you distribute something as a commercial vendor
it is your responsibility to make sure you are not exposing yourself.
Why am I telling you this? The reason is simple: cost. It costs
a LOT to make sure that the exposure is not there.
If you think that a project run by volunteers can achieve the 100%
of cleanless for every single dependency (direct or transitive)
you're simply kidding yourself. Once again, what we need to focus
on is what we directly control. Not more, not less.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Oct 11, 2014 at 11:47 AM, Sean Owen <sr...@gmail.com> wrote:
>> You are confusing different distributions. Netty provides a source
>> distribution which does include a NOTICE file. Netty also provides binary
>> (jar) distributions. These do not include a NOTICE file.
>
> I think this is a fair question. Did the Netty project intend the
> NOTICE file to pertain only to the source distribution? from its
> contents, it pertains to the binary distro too, since the binary form
> contains the elements referenced in the NOTICE.
This one really strikes me as an academic exercise. I am not
sure second guessing a motivation of a non-ASF project would
be fruitful for our discussion.
The situation is *really* simple:
1. it seems that for the stuff in Netty's binary distro Drill is
doing the right thing with its binary distro
2. it seems that for the stuff in Netty's source distro Drill is
doing the right thing with its source distro
Is there anything else I am missing?
> I supposed I'd expect erring a bit on the side of the intent and
> spirit from the ASF in interpreting these things, but hey, let's stick
> to technicalities. Just taking the first example -- Netty contains
> among other things a modified version of Webbit, a BSD-licensed
> library. Drill is distributes this code. Where is this in LICENSE?
> It's not even in NOTICE which would be "close" and reference its own
> LICENSE, but you don't distribute the NOTICE even. This is the problem
> with trying to cut it so fine.
I find it unfair to put this burden on Drill. If you really want to help Netty
with the spirit of the law -- why don't you talk to the Netty developers
and straighten these issues with them first?
> It's such a rabbit hole to be sure, and the little downside to being
> blessed with freely accessing so many others' projects. I struggled
> for a while on Spark with this and still probably don't have it all
> right. I mean, shouldn't someone take a look at the many other
> dependencies? this is just one I ran into as a spectator. Why the
> hostility? just stick to the discussion of the license please.
I haven't notice any hostility, really (perhaps participating in some of the
more boisterous ASF communities equipped me with thicker skin).
That said, I do suggest we stay on topic and not try to boil the ocean
here. We are in charge of our own software -- we should do the
right thing with it. With projects outside of ASF we can only do
so much.
On a related note: with every legal council I ever work with, one
of the first conversations I have is around the fact that you never
ever trust somebody else's legal judgement. Which means
that regardless of what the LICENSE or NOTICE say you are
on the hook to 'trust by verify'. Hence BlackDuck and Palamida
scans. When you distribute something as a commercial vendor
it is your responsibility to make sure you are not exposing yourself.
Why am I telling you this? The reason is simple: cost. It costs
a LOT to make sure that the exposure is not there.
If you think that a project run by volunteers can achieve the 100%
of cleanless for every single dependency (direct or transitive)
you're simply kidding yourself. Once again, what we need to focus
on is what we directly control. Not more, not less.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Sean Owen <sr...@gmail.com>.
On Sat, Oct 11, 2014 at 7:26 PM, Ted Dunning <te...@gmail.com> wrote:
> On Sat, Oct 11, 2014 at 11:08 AM, Sean Owen <sr...@gmail.com> wrote:
>
>> On Sat, Oct 11, 2014 at 7:02 PM, Ted Dunning <te...@gmail.com>
>> wrote:
>> > Netty's artifacts ("its distribution") do not include a notice. Thus,
>>
>> They most certainly do. Please download the distribution of Netty 4.0.20:
>>
>> https://github.com/netty/netty/releases/tag/netty-4.0.20.Final
>>
>> and find the NOTICE.txt file.
>>
>
> You are confusing different distributions. Netty provides a source
> distribution which does include a NOTICE file. Netty also provides binary
> (jar) distributions. These do not include a NOTICE file.
I think this is a fair question. Did the Netty project intend the
NOTICE file to pertain only to the source distribution? from its
contents, it pertains to the binary distro too, since the binary form
contains the elements referenced in the NOTICE.
I supposed I'd expect erring a bit on the side of the intent and
spirit from the ASF in interpreting these things, but hey, let's stick
to technicalities. Just taking the first example -- Netty contains
among other things a modified version of Webbit, a BSD-licensed
library. Drill is distributes this code. Where is this in LICENSE?
It's not even in NOTICE which would be "close" and reference its own
LICENSE, but you don't distribute the NOTICE even. This is the problem
with trying to cut it so fine.
It's such a rabbit hole to be sure, and the little downside to being
blessed with freely accessing so many others' projects. I struggled
for a while on Spark with this and still probably don't have it all
right. I mean, shouldn't someone take a look at the many other
dependencies? this is just one I ran into as a spectator. Why the
hostility? just stick to the discussion of the license please.
It does make you think twice about redistributing all this third party
code, once you see what it requires.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
On Sat, Oct 11, 2014 at 11:08 AM, Sean Owen <sr...@gmail.com> wrote:
> On Sat, Oct 11, 2014 at 7:02 PM, Ted Dunning <te...@gmail.com>
> wrote:
> > Netty's artifacts ("its distribution") do not include a notice. Thus,
>
> They most certainly do. Please download the distribution of Netty 4.0.20:
>
> https://github.com/netty/netty/releases/tag/netty-4.0.20.Final
>
> and find the NOTICE.txt file.
>
You are confusing different distributions. Netty provides a source
distribution which does include a NOTICE file. Netty also provides binary
(jar) distributions. These do not include a NOTICE file.
Just as the NOTICE file is different in the source and binary distributions
of Drill, so is there a difference in the source and binary distributions
of Netty. The Drill project draws this distinction deliberately and I can
only presume the Netty project does this same. If you think that the Netty
developers are constructing their different distributions incorrectly, you
should probably take that up with them.
Drill includes the binary distribution, not the source. Drill faithfully
propagates all NOTICE files from that binary distribution (i.e. none).
I am not sure why you insist on confusing these two very separate things,
but you do seem to quite persistent in doing so.
Re: Bloated NOTICE files are evil
Posted by Sean Owen <sr...@gmail.com>.
On Sat, Oct 11, 2014 at 7:02 PM, Ted Dunning <te...@gmail.com> wrote:
> Netty's artifacts ("its distribution") do not include a notice. Thus,
They most certainly do. Please download the distribution of Netty 4.0.20:
https://github.com/netty/netty/releases/tag/netty-4.0.20.Final
and find the NOTICE.txt file.
I hope that helps. The rest of what you suggest I suggest does sound
like madness, and I'm not suggesting it.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
On Sat, Oct 11, 2014 at 8:13 AM, Sean Owen <sr...@gmail.com> wrote:
> Unfortunately, Netty does not include NOTICE.txt in any of its jars
> for you. This text does not appear therefore in the Drill binary
> distro. At least, I grepped up and down the whole distro and didn't
> see it, and it's not in the Netty jars.
>
> What's the theory that this meets the requirements of Netty's AL2 license?
>
The relevant part of the AL2 license says:
and If the Work includes a "NOTICE" text file as part of its distribution,
> then any Derivative Works that You distribute must include a readable copy
> of the attribution notices contained within such NOTICE file, excluding
> those notices that do not pertain to any part of the Derivative Works, in
> at least one of the following places: within a NOTICE text file distributed
> as part of the Derivative Works; within the Source form or documentation,
> if provided along with the Derivative Works; or, within a display generated
> by the Derivative Works, if and wherever such third-party notices normally
> appear.
Netty's artifacts ("its distribution") do not include a notice. Thus,
Drill does not create one. As far as I can tell, Netty is doing precisely
as Marvin suggests ... they have an attribution in the source and have made
a determination that they do not need a NOTICE file in the jar artifact.
The fact that they do not have such a NOTICE indicates that they do not
require us to propagate said non-existent NOTICE.
Were we to include their sources, I would expect that we would preserve
their NOTICE file since their source code includes such a thing.
To summarize:
ALv2 says that *if* a NOTICE exists in a distribution, then inclusion of
that distribution requires propagation of the content of that NOTICE.
Clearly, if such a NOTICE does not exist, there is no such burden.
Roy (in 2008, one of the original authors of the ASL) says propagating
NOTICE content willy nilly is bad and not a good thing, thus confirming the
inference that lack of a NOTICE in an included artifact implies no further
burden.
Netty does not include a NOTICE in their jars, apparently in compliance
with the license and Roy's interpretation.
Sean says that since Netty does not have such a NOTICE in their
distribution, we have to go further back up the family tree to find or
invent one and that not inventing the missing NOTICE somehow violates
Netty's ASL in spite of the fact that their license says that we don't have
to do that.
Sean further seems to imply that we have to go further back up the family
tree to find any defect in any handling of license software and take it on
ourselves to repair this defect by inventing new NOTICE content.
I say that this way lies madness.
Re: Bloated NOTICE files are evil
Posted by Sean Owen <sr...@gmail.com>.
On Sat, Oct 11, 2014 at 6:35 PM, Ted Dunning <te...@gmail.com> wrote:
> On Sat, Oct 11, 2014 at 8:13 AM, Sean Owen <sr...@gmail.com> wrote:
>
>> Here's another example. Drill distributes Netty 4.0.20, which is AL2
>> licensed and contains a substantial NOTICE file with stuff like ...
>>
>>
>> -------------------------------------------------------------------------------
>> This product contains the extensions to Java Collections Framework which
>> has
>> been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene:
>>
>> * LICENSE:
>> * license/LICENSE.jsr166y.txt (Public Domain)
>> * HOMEPAGE:
>> * http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/
>> *
>> http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166
>>
>
> What you quote is an acknowledgment of public domain code.
>
> How is that a problem?
I think the productive way forward is to read clause 4d of the Apache
License 2.0. It doesn't say "you can ignore things in the NOTICE file
that don't seem relevant to you because they're referring to public
domain things". It specifies that the NOTICE file contents are to be
reproduced in a distribution of a Derivative Work. For better or
worse, that's what it says.
Then have a look at the Netty NOTICE file I've pointed out. It
contains much more than this, including MIT, BSD, AL2 licensed
references -- although again, this isn't really relevant according to
AL2, but may help you. I've copied the content below.
This is just one dependency I highlighted with what appears to be a
license problem in this release; I would not expect it is the only
one. I don't think it's good practice to guess ad hoc at reasons to
ignore the issue of OSS licensing. Why not just conduct a review in
good time? I think it's pretty hard to get right given the complexity,
and I doubt every project has it perfect, but a good-faith effort is
not optional.
-------------------------------------------------------------------------------
This product contains the extensions to Java Collections Framework which has
been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene:
* LICENSE:
* license/LICENSE.jsr166y.txt (Public Domain)
* HOMEPAGE:
* http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/
* http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/
This product contains a modified version of Robert Harder's Public Domain
Base64 Encoder and Decoder, which can be obtained at:
* LICENSE:
* license/LICENSE.base64.txt (Public Domain)
* HOMEPAGE:
* http://iharder.sourceforge.net/current/java/base64/
This product contains a modified portion of 'Webbit', an event based
WebSocket and HTTP server, which can be obtained at:
* LICENSE:
* license/LICENSE.webbit.txt (BSD License)
* HOMEPAGE:
* https://github.com/joewalnes/webbit
This product contains a modified portion of 'SLF4J', a simple logging
facade for Java, which can be obtained at:
* LICENSE:
* license/LICENSE.slf4j.txt (MIT License)
* HOMEPAGE:
* http://www.slf4j.org/
This product contains a modified portion of 'ArrayDeque', written by Josh
Bloch of Google, Inc:
* LICENSE:
* license/LICENSE.deque.txt (Public Domain)
This product contains a modified version of Roland Kuhn's ASL2
AbstractNodeQueue, which is based on Dmitriy Vyukov's non-intrusive MPSC queue.
It can be obtained at:
* LICENSE:
* license/LICENSE.abstractnodequeue.txt (Public Domain)
* HOMEPAGE:
* https://github.com/akka/akka/blob/wip-2.2.3-for-scala-2.11/akka-actor/src/main/java/akka/dispatch/AbstractNodeQueue.java
This product optionally depends on 'JZlib', a re-implementation of zlib in
pure Java, which can be obtained at:
* LICENSE:
* license/LICENSE.jzlib.txt (BSD style License)
* HOMEPAGE:
* http://www.jcraft.com/jzlib/
This product optionally depends on 'Protocol Buffers', Google's data
interchange format, which can be obtained at:
* LICENSE:
* license/LICENSE.protobuf.txt (New BSD License)
* HOMEPAGE:
* http://code.google.com/p/protobuf/
This product optionally depends on 'Bouncy Castle Crypto APIs' to generate
a temporary self-signed X.509 certificate when the JVM does not provide the
equivalent functionality. It can be obtained at:
* LICENSE:
* license/LICENSE.bouncycastle.txt (MIT License)
* HOMEPAGE:
* http://www.bouncycastle.org/
This product optionally depends on 'Snappy', a compression library produced
by Google Inc, which can be obtained at:
* LICENSE:
* license/LICENSE.snappy.txt (New BSD License)
* HOMEPAGE:
* http://code.google.com/p/snappy/
This product optionally depends on 'JBoss Marshalling', an alternative Java
serialization API, which can be obtained at:
* LICENSE:
* license/LICENSE.jboss-marshalling.txt (GNU LGPL 2.1)
* HOMEPAGE:
* http://www.jboss.org/jbossmarshalling
This product optionally depends on 'Caliper', Google's micro-
benchmarking framework, which can be obtained at:
* LICENSE:
* license/LICENSE.caliper.txt (Apache License 2.0)
* HOMEPAGE:
* http://code.google.com/p/caliper/
This product optionally depends on 'Apache Commons Logging', a logging
framework, which can be obtained at:
* LICENSE:
* license/LICENSE.commons-logging.txt (Apache License 2.0)
* HOMEPAGE:
* http://commons.apache.org/logging/
This product optionally depends on 'Apache Log4J', a logging framework, which
can be obtained at:
* LICENSE:
* license/LICENSE.log4j.txt (Apache License 2.0)
* HOMEPAGE:
* http://logging.apache.org/log4j/
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
On Sat, Oct 11, 2014 at 8:13 AM, Sean Owen <sr...@gmail.com> wrote:
> Here's another example. Drill distributes Netty 4.0.20, which is AL2
> licensed and contains a substantial NOTICE file with stuff like ...
>
>
> -------------------------------------------------------------------------------
> This product contains the extensions to Java Collections Framework which
> has
> been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene:
>
> * LICENSE:
> * license/LICENSE.jsr166y.txt (Public Domain)
> * HOMEPAGE:
> * http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/
> *
> http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166
>
What you quote is an acknowledgment of public domain code.
How is that a problem?
Re: Bloated NOTICE files are evil
Posted by Sean Owen <sr...@gmail.com>.
On Sat, Oct 11, 2014 at 3:43 PM, Ted Dunning <te...@gmail.com> wrote:
> It is a tortured and patently incorrect reading to assume that the license
> requires TWO copies of such notices.
Sure, but, nobody said that. The question is whether at least one copy
is present, and ideally in the right place.
Here's another example. Drill distributes Netty 4.0.20, which is AL2
licensed and contains a substantial NOTICE file with stuff like ...
-------------------------------------------------------------------------------
This product contains the extensions to Java Collections Framework which has
been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene:
* LICENSE:
* license/LICENSE.jsr166y.txt (Public Domain)
* HOMEPAGE:
* http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/
* http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166
Unfortunately, Netty does not include NOTICE.txt in any of its jars
for you. This text does not appear therefore in the Drill binary
distro. At least, I grepped up and down the whole distro and didn't
see it, and it's not in the Netty jars.
What's the theory that this meets the requirements of Netty's AL2 license?
Seems like such an annoying technicality, and maybe something that can
be addressed rapidly in a next release if it's an issue. I think we'd
all like more clarifying discussion, like this, on what the right
thing is, but the right thing has to be done even if it's irritating.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: Bloated NOTICE files are evil
Posted by Ted Dunning <te...@gmail.com>.
On Sat, Oct 11, 2014 at 6:51 AM, Sean Owen <sr...@gmail.com> wrote:
> Yes it means greater downstream burden, yes it may have originated
> from a 'hack', but that doesn't seem to make the license not say what
> it says. Obviously I'd rather not have to do this either.
>
It is a tortured and patently incorrect reading to assume that the license
requires TWO copies of such notices.
Re: Bloated NOTICE files are evil
Posted by Sean Owen <sr...@gmail.com>.
On Sat, Oct 11, 2014 at 1:27 PM, Marvin Humphrey <ma...@rectangular.com> wrote:
> I admire the good-faith efforts that the Spark (and Solr) folks have put
> in attempting to comply with their interpretation of ASF requirements, but I
> don't think we should encourage podlings to emulate the current state of their
> licensing documentation.
Yes, but much more importantly, it is not *violating an OSS license*
to distribute software that says 'you must include X in a NOTICE file
if you distribute this' without including X in the NOTICE file?
Yes it means greater downstream burden, yes it may have originated
from a 'hack', but that doesn't seem to make the license not say what
it says. Obviously I'd rather not have to do this either.
It's certainly a best practice to not put things in NOTICE that aren't
actually required. For example upstream NOTICE may refer to components
that the downstream project does not distribute. I think it's also a
good reason to think about whether a project actually needs to
distribute third-party code versus merely depend on it via Maven.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org