You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Brian Hulette <bh...@google.com> on 2021/03/17 21:32:09 UTC

Missing copyright notices due to LICENSE change

I just noticed that there was a recent change to our LICENSE file to make
it exactly match the Apache 2.0 License [1]. This seems to be the result of
two conflicting LICENSE issues.

Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
license policies [2]. Prior to the change our documentation didn't show up
there [3].

Java artifact LICENSE issue: The removed text contained information
relevant to "convenience binary distributions". This text was added in [4]
as a result of this dev@ thread [5], where we noticed that copyright
notices were missing in binary artifacts. The suggested solution (that we
went with) was to just add the information to the root (source) LICENSE.

I'm not sure that that solution is consistent with this ASF guide [6] which
states:

> The LICENSE and NOTICE files must *exactly* represent the contents of the
distribution they reside in. Only components and resources that are
actually included in a distribution have any bearing on the content of that
distribution's NOTICE and LICENSE.

I would argue that *just* Apache 2.0 is the correct text for our root
(source) LICENSE, and the correct way to deal with binary artifacts is to
generate per-artifact LICENSE/NOTICE files.


So right now the Go issue is fixed, but the Java artifact issue has
regressed. I can think of two potential solutions to resolve both:
1) Restore the "convenience binary distributions" information, and see if
we can get pkg.go.dev to allow it.
2) Add infrastructure to generate LICENSE and NOTICE files for Java binary
artifacts.

I have no idea how we might implement (2) so (1) seems more tenable, but
less correct since it's adding information not relevant to the source
release.

Brian


[1] https://github.com/apache/beam/pull/11657
[2] https://pkg.go.dev/license-policy
[3]
https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
[4] https://github.com/apache/beam/pull/5461
[5]
https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
[6] https://infra.apache.org/licensing-howto.html

Re: Missing copyright notices due to LICENSE change

Posted by Robert Burke <lo...@apache.org>.
I have a PR to make the proposed change https://github.com/apache/beam/pull/15201
Apache Legal says the approach seems fine as well: https://issues.apache.org/jira/browse/LEGAL-582

So this is likely going to be merged in today, and possibly cherry picked into the freshly cut release to get the Go SDK docs visible sooner. (This lets the BPG refer to those links reliably as well).

Thanks all!

On 2021/06/02 01:59:04, Robert Burke <ro...@frantil.com> wrote: 
> A link should be fine. The scanner is a similarity detector so small diffs
> shouldn't cause issues.
> 
> Though 100% agree that removing the requirement for that license at all
> would be preferable all around.
> 
> On Tue, Jun 1, 2021, 6:53 PM Ahmet Altay <al...@google.com> wrote:
> 
> >
> >
> > On Tue, May 25, 2021 at 9:22 AM Robert Burke <ro...@frantil.com> wrote:
> >
> >> The owners at pkg go.dev say they can't properly recognize the python
> >> license (see https://github.com/golang/go/issues/45095) due to the
> >> license being somewhat domineering (a go project could *only* have that
> >> license if it had that license, apparently).
> >>
> >> We also can't do a directory specific solution for the Go code.
> >> Pkg.go.dev is an automated crawler and not human curated so they need to
> >> be more conservative (all general LICENSE files to the root of the repo are
> >> included)
> >>
> >> One suggestion is that instead of appending the license to a single file,
> >> we could separate that license into it's own LICENSE.python file which
> >> wouldn't be picked up by the tooling.
> >> I'll take a look at our various license handling /publishing code too and
> >> make appropriate changes to make sure it's not lost in those cases.
> >>
> >>
> >> Another alternative that might work (I've asked in the issue) is
> >> maintaining the license in a LICENSE file, but moving that content in
> >> particular to the sdks/python directory. But this is less preferable since
> >> we have python code in more places than that.
> >>
> >> My vote is for LICENSE.python, and if there are no objections by the next
> >> release cut I'll put forth the PR. I'd rather not make the change so close
> >> to it, and we can cherry pick it otherwise.
> >>
> >
> > If we break the license, we will still need to package all licenses for
> > python packaging, right? As a matter of fact,
> > https://issues.apache.org/jira/browse/LEGAL-417 was about this addition
> > to Beam the response we got was : "Add the Python license into either the
> > project LICENSE, or in a separate file in the tar.gz and a link from the
> > project LICENSE.". So, I think we need to both package and also link from
> > the project LICENSE. Would the linking part work with go tooling?
> >
> > (I think a better option would be re-writing that one specific function
> > and removing the additional licensing needs. For reference this happened
> > at: https://github.com/apache/beam/pull/6423)
> >
> >
> >> Robert Burke
> >> Beam Go Busybody
> >>
> >> On Wed, Mar 24, 2021, 9:25 AM Robert Burke <ro...@frantil.com> wrote:
> >>
> >>> I'm less concerned about the Go Doc at this point.
> >>>
> >>> 0. The Go SDK is still experimental (I know I know, almost there),
> >>> making it less severe of an issue.
> >>> 1 The interpretation pkg.go.dev does is live with the release.
> >>> 1.a That is, if they start accepting the python 2.0 license blob that's
> >>> fixed for all affected versions.
> >>> 1.b Further, if we do cherrypick licence textual fixes into the older
> >>> versions for the licence matching, then the site will pick those up again.
> >>> 2. Users can run the godoc tool
> >>> <https://pkg.go.dev/golang.org/x/tools/cmd/godoc> themselves to get
> >>> package documentation if they want the docs of the version they're using,
> >>> or simply have the programming language server (for go, GoPLS) lift that
> >>> into their IDEs as needed.
> >>>
> >>>
> >>> On Mon, 22 Mar 2021 at 16:42, Ahmet Altay <al...@google.com> wrote:
> >>>
> >>>> For visibility: Change is cherry picked to 2.29.0 release branch (
> >>>> https://github.com/apache/beam/pull/14304).
> >>>>
> >>>> On Mon, Mar 22, 2021 at 12:37 PM Kenneth Knowles <ke...@apache.org>
> >>>> wrote:
> >>>>
> >>>>> So if I understand correctly, the options for a correct license in the
> >>>>> released artifacts are:
> >>>>>
> >>>>>  - revert the change
> >>>>>  - build some automation for bundled jars
> >>>>>  - do something manual?
> >>>>>
> >>>>> Kenn
> >>>>>
> >>>>> On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <bh...@google.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Pros/cons for a cherrypick:
> >>>>>> (+) Fixes regression for licenses in released Java artifacts.
> >>>>>> (-) It's possible it will permanently break docs on pkg.go.dev for
> >>>>>> 2.29.0 if https://github.com/golang/go/issues/45095 requires changes
> >>>>>> on our end (e.g. fixing the PSF License text).
> >>>>>>
> >>>>>> My sense is the pro outweighs the con here, but I could be convinced
> >>>>>> otherwise. I guess that makes me +0 for cherrypick.
> >>>>>>
> >>>>>> Brian
> >>>>>>
> >>>>>> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
> >>>>>>>>
> >>>>>>>
> >>>>>>> I do not think so. I have not filed a jira or started a cherry pick
> >>>>>>> pr.
> >>>>>>>
> >>>>>>> Sorry, I was not sure if we agreed to cherry pick or not. Do you
> >>>>>>> want me to do that?
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <
> >>>>>>>> valentyn@google.com> wrote:
> >>>>>>>>
> >>>>>>>>> I also noticed (with a help of an automated tool) that
> >>>>>>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
> >>>>>>>>> includes additional licenses not included in
> >>>>>>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI
> >>>>>>>>> since Dataflow runner is released as a separate jar artifact, and the
> >>>>>>>>> licenses in question (GPL 2.0, CDDL) pertain to its dependencies, or we
> >>>>>>>>> need to include those licenses as well?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <
> >>>>>>>>>> bhulette@google.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
> >>>>>>>>>>>
> >>>>>>>>>>> > and probably cherry pick it into the affected release branches.
> >>>>>>>>>>> Even if we do this, the Java artifacts from the affected
> >>>>>>>>>>> releases are missing the additional LICENSE text.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> IMO we can skip the cherry picks perhaps with the exception of
> >>>>>>>>>> the upcoming 2.29 release.
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> > I do not know how to interpret this ASF guide. As an example
> >>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
> >>>>>>>>>>> licenses directory. There are even overlapping mentions.
> >>>>>>>>>>> Agreed. I am a software engineer, not a lawyer, and even the
> >>>>>>>>>>> ASF's guide that presumably targets engineers is not particularly clear to
> >>>>>>>>>>> me. This was just my tenuous understanding after a quick review.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Agreed. We can ask LEGAL for further clarification.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Thank you Rebo. I agree with reverting first and then figure
> >>>>>>>>>>>> out the next steps.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Here is a PR to revert your change:
> >>>>>>>>>>>> https://github.com/apache/beam/pull/14267
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <
> >>>>>>>>>>>> robert@frantil.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Looking at the history it seems that before the python text
> >>>>>>>>>>>>> was added, pkg.go.dev can parse the license stack just fine.
> >>>>>>>>>>>>> It doesn't recognize the PSF license, and fails closed entirely as a result.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I've filed an issue with pkg.go.dev (
> >>>>>>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is
> >>>>>>>>>>>>> fixed, the affected versions will become visible as well.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> In the meantime, we should revert my change which clobbered
> >>>>>>>>>>>>> the other licenses and probably cherry pick it into the affected release
> >>>>>>>>>>>>> branches.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing
> >>>>>>>>>>>>> but python can use it and call it the PSF license. However it is a
> >>>>>>>>>>>>> redistribution friendly license, which is what matters.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thank you for this email.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <
> >>>>>>>>>>>>>> bhulette@google.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I just noticed that there was a recent change to our LICENSE
> >>>>>>>>>>>>>>> file to make it exactly match the Apache 2.0 License [1]. This seems to be
> >>>>>>>>>>>>>>> the result of two conflicting LICENSE issues.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy
> >>>>>>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our
> >>>>>>>>>>>>>>> documentation didn't show up there [3].
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Java artifact LICENSE issue: The removed text contained
> >>>>>>>>>>>>>>> information relevant to "convenience binary distributions". This text was
> >>>>>>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
> >>>>>>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
> >>>>>>>>>>>>>>> suggested solution (that we went with) was to just add the information to
> >>>>>>>>>>>>>>> the root (source) LICENSE.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Python distribution is missing both files as well. (
> >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'm not sure that that solution is consistent with this ASF
> >>>>>>>>>>>>>>> guide [6] which states:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
> >>>>>>>>>>>>>>> contents of the distribution they reside in. Only components and resources
> >>>>>>>>>>>>>>> that are actually included in a distribution have any bearing on the
> >>>>>>>>>>>>>>> content of that distribution's NOTICE and LICENSE.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for
> >>>>>>>>>>>>>>> our root (source) LICENSE, and the correct way to deal with binary
> >>>>>>>>>>>>>>> artifacts is to generate per-artifact LICENSE/NOTICE files.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I do not know how to interpret this ASF guide. As an example
> >>>>>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
> >>>>>>>>>>>>>> licenses directory. There are even overlapping mentions.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact
> >>>>>>>>>>>>>>> issue has regressed. I can think of two potential solutions to resolve both:
> >>>>>>>>>>>>>>> 1) Restore the "convenience binary distributions"
> >>>>>>>>>>>>>>> information, and see if we can get pkg.go.dev to allow it.
> >>>>>>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files
> >>>>>>>>>>>>>>> for Java binary artifacts.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
> >>>>>>>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
> >>>>>>>>>>>>>>> source release.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Brian
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
> >>>>>>>>>>>>>>> [2] https://pkg.go.dev/license-policy
> >>>>>>>>>>>>>>> [3]
> >>>>>>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
> >>>>>>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
> >>>>>>>>>>>>>>> [5]
> >>>>>>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
> >>>>>>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.htmlI'll take
> >>>>>>>>>>>>>>> a look at our various license handling /publishing code too and make
> >>>>>>>>>>>>>>> appropriate changes to make sure it's not lost in those cases.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> 

Re: Missing copyright notices due to LICENSE change

Posted by Robert Burke <ro...@frantil.com>.
A link should be fine. The scanner is a similarity detector so small diffs
shouldn't cause issues.

Though 100% agree that removing the requirement for that license at all
would be preferable all around.

On Tue, Jun 1, 2021, 6:53 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Tue, May 25, 2021 at 9:22 AM Robert Burke <ro...@frantil.com> wrote:
>
>> The owners at pkg go.dev say they can't properly recognize the python
>> license (see https://github.com/golang/go/issues/45095) due to the
>> license being somewhat domineering (a go project could *only* have that
>> license if it had that license, apparently).
>>
>> We also can't do a directory specific solution for the Go code.
>> Pkg.go.dev is an automated crawler and not human curated so they need to
>> be more conservative (all general LICENSE files to the root of the repo are
>> included)
>>
>> One suggestion is that instead of appending the license to a single file,
>> we could separate that license into it's own LICENSE.python file which
>> wouldn't be picked up by the tooling.
>> I'll take a look at our various license handling /publishing code too and
>> make appropriate changes to make sure it's not lost in those cases.
>>
>>
>> Another alternative that might work (I've asked in the issue) is
>> maintaining the license in a LICENSE file, but moving that content in
>> particular to the sdks/python directory. But this is less preferable since
>> we have python code in more places than that.
>>
>> My vote is for LICENSE.python, and if there are no objections by the next
>> release cut I'll put forth the PR. I'd rather not make the change so close
>> to it, and we can cherry pick it otherwise.
>>
>
> If we break the license, we will still need to package all licenses for
> python packaging, right? As a matter of fact,
> https://issues.apache.org/jira/browse/LEGAL-417 was about this addition
> to Beam the response we got was : "Add the Python license into either the
> project LICENSE, or in a separate file in the tar.gz and a link from the
> project LICENSE.". So, I think we need to both package and also link from
> the project LICENSE. Would the linking part work with go tooling?
>
> (I think a better option would be re-writing that one specific function
> and removing the additional licensing needs. For reference this happened
> at: https://github.com/apache/beam/pull/6423)
>
>
>> Robert Burke
>> Beam Go Busybody
>>
>> On Wed, Mar 24, 2021, 9:25 AM Robert Burke <ro...@frantil.com> wrote:
>>
>>> I'm less concerned about the Go Doc at this point.
>>>
>>> 0. The Go SDK is still experimental (I know I know, almost there),
>>> making it less severe of an issue.
>>> 1 The interpretation pkg.go.dev does is live with the release.
>>> 1.a That is, if they start accepting the python 2.0 license blob that's
>>> fixed for all affected versions.
>>> 1.b Further, if we do cherrypick licence textual fixes into the older
>>> versions for the licence matching, then the site will pick those up again.
>>> 2. Users can run the godoc tool
>>> <https://pkg.go.dev/golang.org/x/tools/cmd/godoc> themselves to get
>>> package documentation if they want the docs of the version they're using,
>>> or simply have the programming language server (for go, GoPLS) lift that
>>> into their IDEs as needed.
>>>
>>>
>>> On Mon, 22 Mar 2021 at 16:42, Ahmet Altay <al...@google.com> wrote:
>>>
>>>> For visibility: Change is cherry picked to 2.29.0 release branch (
>>>> https://github.com/apache/beam/pull/14304).
>>>>
>>>> On Mon, Mar 22, 2021 at 12:37 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> So if I understand correctly, the options for a correct license in the
>>>>> released artifacts are:
>>>>>
>>>>>  - revert the change
>>>>>  - build some automation for bundled jars
>>>>>  - do something manual?
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <bh...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Pros/cons for a cherrypick:
>>>>>> (+) Fixes regression for licenses in released Java artifacts.
>>>>>> (-) It's possible it will permanently break docs on pkg.go.dev for
>>>>>> 2.29.0 if https://github.com/golang/go/issues/45095 requires changes
>>>>>> on our end (e.g. fixing the PSF License text).
>>>>>>
>>>>>> My sense is the pro outweighs the con here, but I could be convinced
>>>>>> otherwise. I guess that makes me +0 for cherrypick.
>>>>>>
>>>>>> Brian
>>>>>>
>>>>>> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>>>>>>>>
>>>>>>>
>>>>>>> I do not think so. I have not filed a jira or started a cherry pick
>>>>>>> pr.
>>>>>>>
>>>>>>> Sorry, I was not sure if we agreed to cherry pick or not. Do you
>>>>>>> want me to do that?
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <
>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>
>>>>>>>>> I also noticed (with a help of an automated tool) that
>>>>>>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>>>>>>>>> includes additional licenses not included in
>>>>>>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI
>>>>>>>>> since Dataflow runner is released as a separate jar artifact, and the
>>>>>>>>> licenses in question (GPL 2.0, CDDL) pertain to its dependencies, or we
>>>>>>>>> need to include those licenses as well?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <
>>>>>>>>>> bhulette@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>>>>>>>>
>>>>>>>>>>> > and probably cherry pick it into the affected release branches.
>>>>>>>>>>> Even if we do this, the Java artifacts from the affected
>>>>>>>>>>> releases are missing the additional LICENSE text.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> IMO we can skip the cherry picks perhaps with the exception of
>>>>>>>>>> the upcoming 2.29 release.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> > I do not know how to interpret this ASF guide. As an example
>>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>>> Agreed. I am a software engineer, not a lawyer, and even the
>>>>>>>>>>> ASF's guide that presumably targets engineers is not particularly clear to
>>>>>>>>>>> me. This was just my tenuous understanding after a quick review.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Agreed. We can ask LEGAL for further clarification.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thank you Rebo. I agree with reverting first and then figure
>>>>>>>>>>>> out the next steps.
>>>>>>>>>>>>
>>>>>>>>>>>> Here is a PR to revert your change:
>>>>>>>>>>>> https://github.com/apache/beam/pull/14267
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <
>>>>>>>>>>>> robert@frantil.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Looking at the history it seems that before the python text
>>>>>>>>>>>>> was added, pkg.go.dev can parse the license stack just fine.
>>>>>>>>>>>>> It doesn't recognize the PSF license, and fails closed entirely as a result.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've filed an issue with pkg.go.dev (
>>>>>>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is
>>>>>>>>>>>>> fixed, the affected versions will become visible as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> In the meantime, we should revert my change which clobbered
>>>>>>>>>>>>> the other licenses and probably cherry pick it into the affected release
>>>>>>>>>>>>> branches.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing
>>>>>>>>>>>>> but python can use it and call it the PSF license. However it is a
>>>>>>>>>>>>> redistribution friendly license, which is what matters.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you for this email.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <
>>>>>>>>>>>>>> bhulette@google.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I just noticed that there was a recent change to our LICENSE
>>>>>>>>>>>>>>> file to make it exactly match the Apache 2.0 License [1]. This seems to be
>>>>>>>>>>>>>>> the result of two conflicting LICENSE issues.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy
>>>>>>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our
>>>>>>>>>>>>>>> documentation didn't show up there [3].
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Java artifact LICENSE issue: The removed text contained
>>>>>>>>>>>>>>> information relevant to "convenience binary distributions". This text was
>>>>>>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
>>>>>>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
>>>>>>>>>>>>>>> suggested solution (that we went with) was to just add the information to
>>>>>>>>>>>>>>> the root (source) LICENSE.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Python distribution is missing both files as well. (
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm not sure that that solution is consistent with this ASF
>>>>>>>>>>>>>>> guide [6] which states:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for
>>>>>>>>>>>>>>> our root (source) LICENSE, and the correct way to deal with binary
>>>>>>>>>>>>>>> artifacts is to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I do not know how to interpret this ASF guide. As an example
>>>>>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact
>>>>>>>>>>>>>>> issue has regressed. I can think of two potential solutions to resolve both:
>>>>>>>>>>>>>>> 1) Restore the "convenience binary distributions"
>>>>>>>>>>>>>>> information, and see if we can get pkg.go.dev to allow it.
>>>>>>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files
>>>>>>>>>>>>>>> for Java binary artifacts.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>>>>>>>>> source release.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Brian
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>>>>>>>>> [5]
>>>>>>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.htmlI'll take
>>>>>>>>>>>>>>> a look at our various license handling /publishing code too and make
>>>>>>>>>>>>>>> appropriate changes to make sure it's not lost in those cases.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Ahmet Altay <al...@google.com>.
On Tue, May 25, 2021 at 9:22 AM Robert Burke <ro...@frantil.com> wrote:

> The owners at pkg go.dev say they can't properly recognize the python
> license (see https://github.com/golang/go/issues/45095) due to the
> license being somewhat domineering (a go project could *only* have that
> license if it had that license, apparently).
>
> We also can't do a directory specific solution for the Go code. Pkg.go.dev
> is an automated crawler and not human curated so they need to be more
> conservative (all general LICENSE files to the root of the repo are
> included)
>
> One suggestion is that instead of appending the license to a single file,
> we could separate that license into it's own LICENSE.python file which
> wouldn't be picked up by the tooling.
> I'll take a look at our various license handling /publishing code too and
> make appropriate changes to make sure it's not lost in those cases.
>
>
> Another alternative that might work (I've asked in the issue) is
> maintaining the license in a LICENSE file, but moving that content in
> particular to the sdks/python directory. But this is less preferable since
> we have python code in more places than that.
>
> My vote is for LICENSE.python, and if there are no objections by the next
> release cut I'll put forth the PR. I'd rather not make the change so close
> to it, and we can cherry pick it otherwise.
>

If we break the license, we will still need to package all licenses for
python packaging, right? As a matter of fact,
https://issues.apache.org/jira/browse/LEGAL-417 was about this addition to
Beam the response we got was : "Add the Python license into either the
project LICENSE, or in a separate file in the tar.gz and a link from the
project LICENSE.". So, I think we need to both package and also link from
the project LICENSE. Would the linking part work with go tooling?

(I think a better option would be re-writing that one specific function and
removing the additional licensing needs. For reference this happened at:
https://github.com/apache/beam/pull/6423)


> Robert Burke
> Beam Go Busybody
>
> On Wed, Mar 24, 2021, 9:25 AM Robert Burke <ro...@frantil.com> wrote:
>
>> I'm less concerned about the Go Doc at this point.
>>
>> 0. The Go SDK is still experimental (I know I know, almost there), making
>> it less severe of an issue.
>> 1 The interpretation pkg.go.dev does is live with the release.
>> 1.a That is, if they start accepting the python 2.0 license blob that's
>> fixed for all affected versions.
>> 1.b Further, if we do cherrypick licence textual fixes into the older
>> versions for the licence matching, then the site will pick those up again.
>> 2. Users can run the godoc tool
>> <https://pkg.go.dev/golang.org/x/tools/cmd/godoc> themselves to get
>> package documentation if they want the docs of the version they're using,
>> or simply have the programming language server (for go, GoPLS) lift that
>> into their IDEs as needed.
>>
>>
>> On Mon, 22 Mar 2021 at 16:42, Ahmet Altay <al...@google.com> wrote:
>>
>>> For visibility: Change is cherry picked to 2.29.0 release branch (
>>> https://github.com/apache/beam/pull/14304).
>>>
>>> On Mon, Mar 22, 2021 at 12:37 PM Kenneth Knowles <ke...@apache.org>
>>> wrote:
>>>
>>>> So if I understand correctly, the options for a correct license in the
>>>> released artifacts are:
>>>>
>>>>  - revert the change
>>>>  - build some automation for bundled jars
>>>>  - do something manual?
>>>>
>>>> Kenn
>>>>
>>>> On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <bh...@google.com>
>>>> wrote:
>>>>
>>>>> Pros/cons for a cherrypick:
>>>>> (+) Fixes regression for licenses in released Java artifacts.
>>>>> (-) It's possible it will permanently break docs on pkg.go.dev for
>>>>> 2.29.0 if https://github.com/golang/go/issues/45095 requires changes
>>>>> on our end (e.g. fixing the PSF License text).
>>>>>
>>>>> My sense is the pro outweighs the con here, but I could be convinced
>>>>> otherwise. I guess that makes me +0 for cherrypick.
>>>>>
>>>>> Brian
>>>>>
>>>>> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>>>>>>>
>>>>>>
>>>>>> I do not think so. I have not filed a jira or started a cherry pick
>>>>>> pr.
>>>>>>
>>>>>> Sorry, I was not sure if we agreed to cherry pick or not. Do you want
>>>>>> me to do that?
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <
>>>>>>> valentyn@google.com> wrote:
>>>>>>>
>>>>>>>> I also noticed (with a help of an automated tool) that
>>>>>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>>>>>>>> includes additional licenses not included in
>>>>>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI
>>>>>>>> since Dataflow runner is released as a separate jar artifact, and the
>>>>>>>> licenses in question (GPL 2.0, CDDL) pertain to its dependencies, or we
>>>>>>>> need to include those licenses as well?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>>>>>>>
>>>>>>>>>> > and probably cherry pick it into the affected release branches.
>>>>>>>>>> Even if we do this, the Java artifacts from the affected releases
>>>>>>>>>> are missing the additional LICENSE text.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> IMO we can skip the cherry picks perhaps with the exception of the
>>>>>>>>> upcoming 2.29 release.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> > I do not know how to interpret this ASF guide. As an example
>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>> Agreed. I am a software engineer, not a lawyer, and even the
>>>>>>>>>> ASF's guide that presumably targets engineers is not particularly clear to
>>>>>>>>>> me. This was just my tenuous understanding after a quick review.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Agreed. We can ask LEGAL for further clarification.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank you Rebo. I agree with reverting first and then figure out
>>>>>>>>>>> the next steps.
>>>>>>>>>>>
>>>>>>>>>>> Here is a PR to revert your change:
>>>>>>>>>>> https://github.com/apache/beam/pull/14267
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Looking at the history it seems that before the python text was
>>>>>>>>>>>> added, pkg.go.dev can parse the license stack just fine. It
>>>>>>>>>>>> doesn't recognize the PSF license, and fails closed entirely as a result.
>>>>>>>>>>>>
>>>>>>>>>>>> I've filed an issue with pkg.go.dev (
>>>>>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is
>>>>>>>>>>>> fixed, the affected versions will become visible as well.
>>>>>>>>>>>>
>>>>>>>>>>>> In the meantime, we should revert my change which clobbered the
>>>>>>>>>>>> other licenses and probably cherry pick it into the affected release
>>>>>>>>>>>> branches.
>>>>>>>>>>>>
>>>>>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing
>>>>>>>>>>>> but python can use it and call it the PSF license. However it is a
>>>>>>>>>>>> redistribution friendly license, which is what matters.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you for this email.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <
>>>>>>>>>>>>> bhulette@google.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I just noticed that there was a recent change to our LICENSE
>>>>>>>>>>>>>> file to make it exactly match the Apache 2.0 License [1]. This seems to be
>>>>>>>>>>>>>> the result of two conflicting LICENSE issues.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy
>>>>>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our
>>>>>>>>>>>>>> documentation didn't show up there [3].
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Java artifact LICENSE issue: The removed text contained
>>>>>>>>>>>>>> information relevant to "convenience binary distributions". This text was
>>>>>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
>>>>>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
>>>>>>>>>>>>>> suggested solution (that we went with) was to just add the information to
>>>>>>>>>>>>>> the root (source) LICENSE.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Python distribution is missing both files as well. (
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm not sure that that solution is consistent with this ASF
>>>>>>>>>>>>>> guide [6] which states:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for
>>>>>>>>>>>>>> our root (source) LICENSE, and the correct way to deal with binary
>>>>>>>>>>>>>> artifacts is to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I do not know how to interpret this ASF guide. As an example
>>>>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact
>>>>>>>>>>>>>> issue has regressed. I can think of two potential solutions to resolve both:
>>>>>>>>>>>>>> 1) Restore the "convenience binary distributions"
>>>>>>>>>>>>>> information, and see if we can get pkg.go.dev to allow it.
>>>>>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files
>>>>>>>>>>>>>> for Java binary artifacts.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>>>>>>>> source release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Brian
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>>>>>>>> [3]
>>>>>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>>>>>>>> [5]
>>>>>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.htmlI'll take a
>>>>>>>>>>>>>> look at our various license handling /publishing code too and make
>>>>>>>>>>>>>> appropriate changes to make sure it's not lost in those cases.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Robert Burke <ro...@frantil.com>.
The owners at pkg go.dev say they can't properly recognize the python
license (see https://github.com/golang/go/issues/45095) due to the license
being somewhat domineering (a go project could *only* have that license if
it had that license, apparently).

We also can't do a directory specific solution for the Go code. Pkg.go.dev
is an automated crawler and not human curated so they need to be more
conservative (all general LICENSE files to the root of the repo are
included)

One suggestion is that instead of appending the license to a single file,
we could separate that license into it's own LICENSE.python file which
wouldn't be picked up by the tooling.
I'll take a look at our various license handling /publishing code too and
make appropriate changes to make sure it's not lost in those cases.


Another alternative that might work (I've asked in the issue) is
maintaining the license in a LICENSE file, but moving that content in
particular to the sdks/python directory. But this is less preferable since
we have python code in more places than that.

My vote is for LICENSE.python, and if there are no objections by the next
release cut I'll put forth the PR. I'd rather not make the change so close
to it, and we can cherry pick it otherwise.

Robert Burke
Beam Go Busybody

On Wed, Mar 24, 2021, 9:25 AM Robert Burke <ro...@frantil.com> wrote:

> I'm less concerned about the Go Doc at this point.
>
> 0. The Go SDK is still experimental (I know I know, almost there), making
> it less severe of an issue.
> 1 The interpretation pkg.go.dev does is live with the release.
> 1.a That is, if they start accepting the python 2.0 license blob that's
> fixed for all affected versions.
> 1.b Further, if we do cherrypick licence textual fixes into the older
> versions for the licence matching, then the site will pick those up again.
> 2. Users can run the godoc tool
> <https://pkg.go.dev/golang.org/x/tools/cmd/godoc> themselves to get
> package documentation if they want the docs of the version they're using,
> or simply have the programming language server (for go, GoPLS) lift that
> into their IDEs as needed.
>
>
> On Mon, 22 Mar 2021 at 16:42, Ahmet Altay <al...@google.com> wrote:
>
>> For visibility: Change is cherry picked to 2.29.0 release branch (
>> https://github.com/apache/beam/pull/14304).
>>
>> On Mon, Mar 22, 2021 at 12:37 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> So if I understand correctly, the options for a correct license in the
>>> released artifacts are:
>>>
>>>  - revert the change
>>>  - build some automation for bundled jars
>>>  - do something manual?
>>>
>>> Kenn
>>>
>>> On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <bh...@google.com>
>>> wrote:
>>>
>>>> Pros/cons for a cherrypick:
>>>> (+) Fixes regression for licenses in released Java artifacts.
>>>> (-) It's possible it will permanently break docs on pkg.go.dev for
>>>> 2.29.0 if https://github.com/golang/go/issues/45095 requires changes
>>>> on our end (e.g. fixing the PSF License text).
>>>>
>>>> My sense is the pro outweighs the con here, but I could be convinced
>>>> otherwise. I guess that makes me +0 for cherrypick.
>>>>
>>>> Brian
>>>>
>>>> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>>>>>>
>>>>>
>>>>> I do not think so. I have not filed a jira or started a cherry pick
>>>>> pr.
>>>>>
>>>>> Sorry, I was not sure if we agreed to cherry pick or not. Do you want
>>>>> me to do that?
>>>>>
>>>>>
>>>>>>
>>>>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <
>>>>>> valentyn@google.com> wrote:
>>>>>>
>>>>>>> I also noticed (with a help of an automated tool) that
>>>>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>>>>>>> includes additional licenses not included in
>>>>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI
>>>>>>> since Dataflow runner is released as a separate jar artifact, and the
>>>>>>> licenses in question (GPL 2.0, CDDL) pertain to its dependencies, or we
>>>>>>> need to include those licenses as well?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>>>>>>
>>>>>>>>> > and probably cherry pick it into the affected release branches.
>>>>>>>>> Even if we do this, the Java artifacts from the affected releases
>>>>>>>>> are missing the additional LICENSE text.
>>>>>>>>>
>>>>>>>>
>>>>>>>> IMO we can skip the cherry picks perhaps with the exception of the
>>>>>>>> upcoming 2.29 release.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> > I do not know how to interpret this ASF guide. As an example
>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's
>>>>>>>>> guide that presumably targets engineers is not particularly clear to me.
>>>>>>>>> This was just my tenuous understanding after a quick review.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Agreed. We can ask LEGAL for further clarification.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thank you Rebo. I agree with reverting first and then figure out
>>>>>>>>>> the next steps.
>>>>>>>>>>
>>>>>>>>>> Here is a PR to revert your change:
>>>>>>>>>> https://github.com/apache/beam/pull/14267
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Looking at the history it seems that before the python text was
>>>>>>>>>>> added, pkg.go.dev can parse the license stack just fine. It
>>>>>>>>>>> doesn't recognize the PSF license, and fails closed entirely as a result.
>>>>>>>>>>>
>>>>>>>>>>> I've filed an issue with pkg.go.dev (
>>>>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is
>>>>>>>>>>> fixed, the affected versions will become visible as well.
>>>>>>>>>>>
>>>>>>>>>>> In the meantime, we should revert my change which clobbered the
>>>>>>>>>>> other licenses and probably cherry pick it into the affected release
>>>>>>>>>>> branches.
>>>>>>>>>>>
>>>>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing
>>>>>>>>>>> but python can use it and call it the PSF license. However it is a
>>>>>>>>>>> redistribution friendly license, which is what matters.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thank you for this email.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <
>>>>>>>>>>>> bhulette@google.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I just noticed that there was a recent change to our LICENSE
>>>>>>>>>>>>> file to make it exactly match the Apache 2.0 License [1]. This seems to be
>>>>>>>>>>>>> the result of two conflicting LICENSE issues.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy
>>>>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our
>>>>>>>>>>>>> documentation didn't show up there [3].
>>>>>>>>>>>>>
>>>>>>>>>>>>> Java artifact LICENSE issue: The removed text contained
>>>>>>>>>>>>> information relevant to "convenience binary distributions". This text was
>>>>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
>>>>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
>>>>>>>>>>>>> suggested solution (that we went with) was to just add the information to
>>>>>>>>>>>>> the root (source) LICENSE.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Python distribution is missing both files as well. (
>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm not sure that that solution is consistent with this ASF
>>>>>>>>>>>>> guide [6] which states:
>>>>>>>>>>>>>
>>>>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for
>>>>>>>>>>>>> our root (source) LICENSE, and the correct way to deal with binary
>>>>>>>>>>>>> artifacts is to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I do not know how to interpret this ASF guide. As an example
>>>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact
>>>>>>>>>>>>> issue has regressed. I can think of two potential solutions to resolve both:
>>>>>>>>>>>>> 1) Restore the "convenience binary distributions" information,
>>>>>>>>>>>>> and see if we can get pkg.go.dev to allow it.
>>>>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for
>>>>>>>>>>>>> Java binary artifacts.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>>>>>>> source release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Brian
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>>>>>>> [3]
>>>>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>>>>>>> [5]
>>>>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.htmlI'll take a
>>>>>>>>>>>>> look at our various license handling /publishing code too and make
>>>>>>>>>>>>> appropriate changes to make sure it's not lost in those cases.
>>>>>>>>>>>>>
>>>>>>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Robert Burke <ro...@frantil.com>.
I'm less concerned about the Go Doc at this point.

0. The Go SDK is still experimental (I know I know, almost there), making
it less severe of an issue.
1 The interpretation pkg.go.dev does is live with the release.
1.a That is, if they start accepting the python 2.0 license blob that's
fixed for all affected versions.
1.b Further, if we do cherrypick licence textual fixes into the older
versions for the licence matching, then the site will pick those up again.
2. Users can run the godoc tool
<https://pkg.go.dev/golang.org/x/tools/cmd/godoc> themselves to get package
documentation if they want the docs of the version they're using, or simply
have the programming language server (for go, GoPLS) lift that into their
IDEs as needed.


On Mon, 22 Mar 2021 at 16:42, Ahmet Altay <al...@google.com> wrote:

> For visibility: Change is cherry picked to 2.29.0 release branch (
> https://github.com/apache/beam/pull/14304).
>
> On Mon, Mar 22, 2021 at 12:37 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> So if I understand correctly, the options for a correct license in the
>> released artifacts are:
>>
>>  - revert the change
>>  - build some automation for bundled jars
>>  - do something manual?
>>
>> Kenn
>>
>> On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <bh...@google.com>
>> wrote:
>>
>>> Pros/cons for a cherrypick:
>>> (+) Fixes regression for licenses in released Java artifacts.
>>> (-) It's possible it will permanently break docs on pkg.go.dev for
>>> 2.29.0 if https://github.com/golang/go/issues/45095 requires changes on
>>> our end (e.g. fixing the PSF License text).
>>>
>>> My sense is the pro outweighs the con here, but I could be convinced
>>> otherwise. I guess that makes me +0 for cherrypick.
>>>
>>> Brian
>>>
>>> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>>>>>
>>>>
>>>> I do not think so. I have not filed a jira or started a cherry pick pr.
>>>>
>>>> Sorry, I was not sure if we agreed to cherry pick or not. Do you want
>>>> me to do that?
>>>>
>>>>
>>>>>
>>>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <
>>>>> valentyn@google.com> wrote:
>>>>>
>>>>>> I also noticed (with a help of an automated tool) that
>>>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>>>>>> includes additional licenses not included in
>>>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI
>>>>>> since Dataflow runner is released as a separate jar artifact, and the
>>>>>> licenses in question (GPL 2.0, CDDL) pertain to its dependencies, or we
>>>>>> need to include those licenses as well?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>>>>>
>>>>>>>> > and probably cherry pick it into the affected release branches.
>>>>>>>> Even if we do this, the Java artifacts from the affected releases
>>>>>>>> are missing the additional LICENSE text.
>>>>>>>>
>>>>>>>
>>>>>>> IMO we can skip the cherry picks perhaps with the exception of the
>>>>>>> upcoming 2.29 release.
>>>>>>>
>>>>>>>>
>>>>>>>> > I do not know how to interpret this ASF guide. As an example from
>>>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's
>>>>>>>> guide that presumably targets engineers is not particularly clear to me.
>>>>>>>> This was just my tenuous understanding after a quick review.
>>>>>>>>
>>>>>>>
>>>>>>> Agreed. We can ask LEGAL for further clarification.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thank you Rebo. I agree with reverting first and then figure out
>>>>>>>>> the next steps.
>>>>>>>>>
>>>>>>>>> Here is a PR to revert your change:
>>>>>>>>> https://github.com/apache/beam/pull/14267
>>>>>>>>>
>>>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Looking at the history it seems that before the python text was
>>>>>>>>>> added, pkg.go.dev can parse the license stack just fine. It
>>>>>>>>>> doesn't recognize the PSF license, and fails closed entirely as a result.
>>>>>>>>>>
>>>>>>>>>> I've filed an issue with pkg.go.dev (
>>>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is fixed,
>>>>>>>>>> the affected versions will become visible as well.
>>>>>>>>>>
>>>>>>>>>> In the meantime, we should revert my change which clobbered the
>>>>>>>>>> other licenses and probably cherry pick it into the affected release
>>>>>>>>>> branches.
>>>>>>>>>>
>>>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing
>>>>>>>>>> but python can use it and call it the PSF license. However it is a
>>>>>>>>>> redistribution friendly license, which is what matters.
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank you for this email.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <
>>>>>>>>>>> bhulette@google.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I just noticed that there was a recent change to our LICENSE
>>>>>>>>>>>> file to make it exactly match the Apache 2.0 License [1]. This seems to be
>>>>>>>>>>>> the result of two conflicting LICENSE issues.
>>>>>>>>>>>>
>>>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy
>>>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our
>>>>>>>>>>>> documentation didn't show up there [3].
>>>>>>>>>>>>
>>>>>>>>>>>> Java artifact LICENSE issue: The removed text contained
>>>>>>>>>>>> information relevant to "convenience binary distributions". This text was
>>>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
>>>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
>>>>>>>>>>>> suggested solution (that we went with) was to just add the information to
>>>>>>>>>>>> the root (source) LICENSE.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Python distribution is missing both files as well. (
>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not sure that that solution is consistent with this ASF
>>>>>>>>>>>> guide [6] which states:
>>>>>>>>>>>>
>>>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>>>>>
>>>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for
>>>>>>>>>>>> our root (source) LICENSE, and the correct way to deal with binary
>>>>>>>>>>>> artifacts is to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I do not know how to interpret this ASF guide. As an example
>>>>>>>>>>> from another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact issue
>>>>>>>>>>>> has regressed. I can think of two potential solutions to resolve both:
>>>>>>>>>>>> 1) Restore the "convenience binary distributions" information,
>>>>>>>>>>>> and see if we can get pkg.go.dev to allow it.
>>>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for
>>>>>>>>>>>> Java binary artifacts.
>>>>>>>>>>>>
>>>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>>>>>> source release.
>>>>>>>>>>>>
>>>>>>>>>>>> Brian
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>>>>>> [3]
>>>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>>>>>> [5]
>>>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>>>>>>>>
>>>>>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Ahmet Altay <al...@google.com>.
For visibility: Change is cherry picked to 2.29.0 release branch (
https://github.com/apache/beam/pull/14304).

On Mon, Mar 22, 2021 at 12:37 PM Kenneth Knowles <ke...@apache.org> wrote:

> So if I understand correctly, the options for a correct license in the
> released artifacts are:
>
>  - revert the change
>  - build some automation for bundled jars
>  - do something manual?
>
> Kenn
>
> On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <bh...@google.com>
> wrote:
>
>> Pros/cons for a cherrypick:
>> (+) Fixes regression for licenses in released Java artifacts.
>> (-) It's possible it will permanently break docs on pkg.go.dev for
>> 2.29.0 if https://github.com/golang/go/issues/45095 requires changes on
>> our end (e.g. fixing the PSF License text).
>>
>> My sense is the pro outweighs the con here, but I could be convinced
>> otherwise. I guess that makes me +0 for cherrypick.
>>
>> Brian
>>
>> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org>
>>> wrote:
>>>
>>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>>>>
>>>
>>> I do not think so. I have not filed a jira or started a cherry pick pr.
>>>
>>> Sorry, I was not sure if we agreed to cherry pick or not. Do you want me
>>> to do that?
>>>
>>>
>>>>
>>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <
>>>> valentyn@google.com> wrote:
>>>>
>>>>> I also noticed (with a help of an automated tool) that
>>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>>>>> includes additional licenses not included in
>>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI since
>>>>> Dataflow runner is released as a separate jar artifact, and the licenses in
>>>>> question (GPL 2.0, CDDL) pertain to its dependencies, or we need to include
>>>>> those licenses as well?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>>>>
>>>>>>> > and probably cherry pick it into the affected release branches.
>>>>>>> Even if we do this, the Java artifacts from the affected releases
>>>>>>> are missing the additional LICENSE text.
>>>>>>>
>>>>>>
>>>>>> IMO we can skip the cherry picks perhaps with the exception of the
>>>>>> upcoming 2.29 release.
>>>>>>
>>>>>>>
>>>>>>> > I do not know how to interpret this ASF guide. As an example from
>>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's
>>>>>>> guide that presumably targets engineers is not particularly clear to me.
>>>>>>> This was just my tenuous understanding after a quick review.
>>>>>>>
>>>>>>
>>>>>> Agreed. We can ask LEGAL for further clarification.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thank you Rebo. I agree with reverting first and then figure out
>>>>>>>> the next steps.
>>>>>>>>
>>>>>>>> Here is a PR to revert your change:
>>>>>>>> https://github.com/apache/beam/pull/14267
>>>>>>>>
>>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Looking at the history it seems that before the python text was
>>>>>>>>> added, pkg.go.dev can parse the license stack just fine. It
>>>>>>>>> doesn't recognize the PSF license, and fails closed entirely as a result.
>>>>>>>>>
>>>>>>>>> I've filed an issue with pkg.go.dev (
>>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is fixed,
>>>>>>>>> the affected versions will become visible as well.
>>>>>>>>>
>>>>>>>>> In the meantime, we should revert my change which clobbered the
>>>>>>>>> other licenses and probably cherry pick it into the affected release
>>>>>>>>> branches.
>>>>>>>>>
>>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing but
>>>>>>>>> python can use it and call it the PSF license. However it is a
>>>>>>>>> redistribution friendly license, which is what matters.
>>>>>>>>>
>>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thank you for this email.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <
>>>>>>>>>> bhulette@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I just noticed that there was a recent change to our LICENSE
>>>>>>>>>>> file to make it exactly match the Apache 2.0 License [1]. This seems to be
>>>>>>>>>>> the result of two conflicting LICENSE issues.
>>>>>>>>>>>
>>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy
>>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our
>>>>>>>>>>> documentation didn't show up there [3].
>>>>>>>>>>>
>>>>>>>>>>> Java artifact LICENSE issue: The removed text contained
>>>>>>>>>>> information relevant to "convenience binary distributions". This text was
>>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
>>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
>>>>>>>>>>> suggested solution (that we went with) was to just add the information to
>>>>>>>>>>> the root (source) LICENSE.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Python distribution is missing both files as well. (
>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure that that solution is consistent with this ASF
>>>>>>>>>>> guide [6] which states:
>>>>>>>>>>>
>>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>>>>
>>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for our
>>>>>>>>>>> root (source) LICENSE, and the correct way to deal with binary artifacts is
>>>>>>>>>>> to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I do not know how to interpret this ASF guide. As an example from
>>>>>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact issue
>>>>>>>>>>> has regressed. I can think of two potential solutions to resolve both:
>>>>>>>>>>> 1) Restore the "convenience binary distributions" information,
>>>>>>>>>>> and see if we can get pkg.go.dev to allow it.
>>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for
>>>>>>>>>>> Java binary artifacts.
>>>>>>>>>>>
>>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>>>>> source release.
>>>>>>>>>>>
>>>>>>>>>>> Brian
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>>>>> [3]
>>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>>>>> [5]
>>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>>>>>>>
>>>>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Kenneth Knowles <ke...@apache.org>.
So if I understand correctly, the options for a correct license in the
released artifacts are:

 - revert the change
 - build some automation for bundled jars
 - do something manual?

Kenn

On Mon, Mar 22, 2021 at 10:57 AM Brian Hulette <bh...@google.com> wrote:

> Pros/cons for a cherrypick:
> (+) Fixes regression for licenses in released Java artifacts.
> (-) It's possible it will permanently break docs on pkg.go.dev for 2.29.0
> if https://github.com/golang/go/issues/45095 requires changes on our end
> (e.g. fixing the PSF License text).
>
> My sense is the pro outweighs the con here, but I could be convinced
> otherwise. I guess that makes me +0 for cherrypick.
>
> Brian
>
> On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>>>
>>
>> I do not think so. I have not filed a jira or started a cherry pick pr.
>>
>> Sorry, I was not sure if we agreed to cherry pick or not. Do you want me
>> to do that?
>>
>>
>>>
>>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <va...@google.com>
>>> wrote:
>>>
>>>> I also noticed (with a help of an automated tool) that
>>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>>>> includes additional licenses not included in
>>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI since
>>>> Dataflow runner is released as a separate jar artifact, and the licenses in
>>>> question (GPL 2.0, CDDL) pertain to its dependencies, or we need to include
>>>> those licenses as well?
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>>>
>>>>>> > and probably cherry pick it into the affected release branches.
>>>>>> Even if we do this, the Java artifacts from the affected releases are
>>>>>> missing the additional LICENSE text.
>>>>>>
>>>>>
>>>>> IMO we can skip the cherry picks perhaps with the exception of the
>>>>> upcoming 2.29 release.
>>>>>
>>>>>>
>>>>>> > I do not know how to interpret this ASF guide. As an example from
>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>> licenses directory. There are even overlapping mentions.
>>>>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's
>>>>>> guide that presumably targets engineers is not particularly clear to me.
>>>>>> This was just my tenuous understanding after a quick review.
>>>>>>
>>>>>
>>>>> Agreed. We can ask LEGAL for further clarification.
>>>>>
>>>>>
>>>>>>
>>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> Thank you Rebo. I agree with reverting first and then figure out the
>>>>>>> next steps.
>>>>>>>
>>>>>>> Here is a PR to revert your change:
>>>>>>> https://github.com/apache/beam/pull/14267
>>>>>>>
>>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Looking at the history it seems that before the python text was
>>>>>>>> added, pkg.go.dev can parse the license stack just fine. It
>>>>>>>> doesn't recognize the PSF license, and fails closed entirely as a result.
>>>>>>>>
>>>>>>>> I've filed an issue with pkg.go.dev (
>>>>>>>> https://github.com/golang/go/issues/45095). If the bug is fixed,
>>>>>>>> the affected versions will become visible as well.
>>>>>>>>
>>>>>>>> In the meantime, we should revert my change which clobbered the
>>>>>>>> other licenses and probably cherry pick it into the affected release
>>>>>>>> branches.
>>>>>>>>
>>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing but
>>>>>>>> python can use it and call it the PSF license. However it is a
>>>>>>>> redistribution friendly license, which is what matters.
>>>>>>>>
>>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>>>
>>>>>>>>> Thank you for this email.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I just noticed that there was a recent change to our LICENSE file
>>>>>>>>>> to make it exactly match the Apache 2.0 License [1]. This seems to be the
>>>>>>>>>> result of two conflicting LICENSE issues.
>>>>>>>>>>
>>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy
>>>>>>>>>> pkg.go.dev's license policies [2]. Prior to the change our
>>>>>>>>>> documentation didn't show up there [3].
>>>>>>>>>>
>>>>>>>>>> Java artifact LICENSE issue: The removed text contained
>>>>>>>>>> information relevant to "convenience binary distributions". This text was
>>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
>>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
>>>>>>>>>> suggested solution (that we went with) was to just add the information to
>>>>>>>>>> the root (source) LICENSE.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Python distribution is missing both files as well. (
>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'm not sure that that solution is consistent with this ASF guide
>>>>>>>>>> [6] which states:
>>>>>>>>>>
>>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>>>
>>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for our
>>>>>>>>>> root (source) LICENSE, and the correct way to deal with binary artifacts is
>>>>>>>>>> to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I do not know how to interpret this ASF guide. As an example from
>>>>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So right now the Go issue is fixed, but the Java artifact issue
>>>>>>>>>> has regressed. I can think of two potential solutions to resolve both:
>>>>>>>>>> 1) Restore the "convenience binary distributions" information,
>>>>>>>>>> and see if we can get pkg.go.dev to allow it.
>>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for
>>>>>>>>>> Java binary artifacts.
>>>>>>>>>>
>>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>>>> source release.
>>>>>>>>>>
>>>>>>>>>> Brian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>>>> [3]
>>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>>>> [5]
>>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>>>>>>
>>>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Brian Hulette <bh...@google.com>.
Pros/cons for a cherrypick:
(+) Fixes regression for licenses in released Java artifacts.
(-) It's possible it will permanently break docs on pkg.go.dev for 2.29.0
if https://github.com/golang/go/issues/45095 requires changes on our end
(e.g. fixing the PSF License text).

My sense is the pro outweighs the con here, but I could be convinced
otherwise. I guess that makes me +0 for cherrypick.

Brian

On Mon, Mar 22, 2021 at 10:43 AM Ahmet Altay <al...@google.com> wrote:

>
>
> On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>>
>
> I do not think so. I have not filed a jira or started a cherry pick pr.
>
> Sorry, I was not sure if we agreed to cherry pick or not. Do you want me
> to do that?
>
>
>>
>> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <va...@google.com>
>> wrote:
>>
>>> I also noticed (with a help of an automated tool) that
>>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>>> includes additional licenses not included in
>>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI since
>>> Dataflow runner is released as a separate jar artifact, and the licenses in
>>> question (GPL 2.0, CDDL) pertain to its dependencies, or we need to include
>>> those licenses as well?
>>>
>>>
>>>
>>>
>>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>>>> wrote:
>>>>
>>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>>
>>>>> > and probably cherry pick it into the affected release branches.
>>>>> Even if we do this, the Java artifacts from the affected releases are
>>>>> missing the additional LICENSE text.
>>>>>
>>>>
>>>> IMO we can skip the cherry picks perhaps with the exception of the
>>>> upcoming 2.29 release.
>>>>
>>>>>
>>>>> > I do not know how to interpret this ASF guide. As an example from
>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>> licenses directory. There are even overlapping mentions.
>>>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's
>>>>> guide that presumably targets engineers is not particularly clear to me.
>>>>> This was just my tenuous understanding after a quick review.
>>>>>
>>>>
>>>> Agreed. We can ask LEGAL for further clarification.
>>>>
>>>>
>>>>>
>>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>> Thank you Rebo. I agree with reverting first and then figure out the
>>>>>> next steps.
>>>>>>
>>>>>> Here is a PR to revert your change:
>>>>>> https://github.com/apache/beam/pull/14267
>>>>>>
>>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Looking at the history it seems that before the python text was
>>>>>>> added, pkg.go.dev can parse the license stack just fine. It doesn't
>>>>>>> recognize the PSF license, and fails closed entirely as a result.
>>>>>>>
>>>>>>> I've filed an issue with pkg.go.dev (
>>>>>>> https://github.com/golang/go/issues/45095). If the bug is fixed,
>>>>>>> the affected versions will become visible as well.
>>>>>>>
>>>>>>> In the meantime, we should revert my change which clobbered the
>>>>>>> other licenses and probably cherry pick it into the affected release
>>>>>>> branches.
>>>>>>>
>>>>>>> The PSF license is annoying as it's explicitly unique. Nothing but
>>>>>>> python can use it and call it the PSF license. However it is a
>>>>>>> redistribution friendly license, which is what matters.
>>>>>>>
>>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>>
>>>>>>>> Thank you for this email.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I just noticed that there was a recent change to our LICENSE file
>>>>>>>>> to make it exactly match the Apache 2.0 License [1]. This seems to be the
>>>>>>>>> result of two conflicting LICENSE issues.
>>>>>>>>>
>>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>>>>>>>>> license policies [2]. Prior to the change our documentation didn't show up
>>>>>>>>> there [3].
>>>>>>>>>
>>>>>>>>> Java artifact LICENSE issue: The removed text contained
>>>>>>>>> information relevant to "convenience binary distributions". This text was
>>>>>>>>> added in [4] as a result of this dev@ thread [5], where we
>>>>>>>>> noticed that copyright notices were missing in binary artifacts. The
>>>>>>>>> suggested solution (that we went with) was to just add the information to
>>>>>>>>> the root (source) LICENSE.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Python distribution is missing both files as well. (
>>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm not sure that that solution is consistent with this ASF guide
>>>>>>>>> [6] which states:
>>>>>>>>>
>>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>>
>>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for our
>>>>>>>>> root (source) LICENSE, and the correct way to deal with binary artifacts is
>>>>>>>>> to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I do not know how to interpret this ASF guide. As an example from
>>>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So right now the Go issue is fixed, but the Java artifact issue
>>>>>>>>> has regressed. I can think of two potential solutions to resolve both:
>>>>>>>>> 1) Restore the "convenience binary distributions" information, and
>>>>>>>>> see if we can get pkg.go.dev to allow it.
>>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for
>>>>>>>>> Java binary artifacts.
>>>>>>>>>
>>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>>> source release.
>>>>>>>>>
>>>>>>>>> Brian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>>> [3]
>>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>>> [5]
>>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>>>>>
>>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Ahmet Altay <al...@google.com>.
On Mon, Mar 22, 2021 at 10:31 AM Kenneth Knowles <ke...@apache.org> wrote:

> Is there a Jira marked as blocking 2.29.0 for the cherrypick?
>

I do not think so. I have not filed a jira or started a cherry pick pr.

Sorry, I was not sure if we agreed to cherry pick or not. Do you want me to
do that?


>
> On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <va...@google.com>
> wrote:
>
>> I also noticed (with a help of an automated tool) that
>> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
>> includes additional licenses not included in
>> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI since
>> Dataflow runner is released as a separate jar artifact, and the licenses in
>> question (GPL 2.0, CDDL) pertain to its dependencies, or we need to include
>> those licenses as well?
>>
>>
>>
>>
>> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>>> wrote:
>>>
>>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>>
>>>> > and probably cherry pick it into the affected release branches.
>>>> Even if we do this, the Java artifacts from the affected releases are
>>>> missing the additional LICENSE text.
>>>>
>>>
>>> IMO we can skip the cherry picks perhaps with the exception of the
>>> upcoming 2.29 release.
>>>
>>>>
>>>> > I do not know how to interpret this ASF guide. As an example from
>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>> licenses directory. There are even overlapping mentions.
>>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's
>>>> guide that presumably targets engineers is not particularly clear to me.
>>>> This was just my tenuous understanding after a quick review.
>>>>
>>>
>>> Agreed. We can ask LEGAL for further clarification.
>>>
>>>
>>>>
>>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> Thank you Rebo. I agree with reverting first and then figure out the
>>>>> next steps.
>>>>>
>>>>> Here is a PR to revert your change:
>>>>> https://github.com/apache/beam/pull/14267
>>>>>
>>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>>> wrote:
>>>>>
>>>>>> Looking at the history it seems that before the python text was
>>>>>> added, pkg.go.dev can parse the license stack just fine. It doesn't
>>>>>> recognize the PSF license, and fails closed entirely as a result.
>>>>>>
>>>>>> I've filed an issue with pkg.go.dev (
>>>>>> https://github.com/golang/go/issues/45095). If the bug is fixed, the
>>>>>> affected versions will become visible as well.
>>>>>>
>>>>>> In the meantime, we should revert my change which clobbered the other
>>>>>> licenses and probably cherry pick it into the affected release branches.
>>>>>>
>>>>>> The PSF license is annoying as it's explicitly unique. Nothing but
>>>>>> python can use it and call it the PSF license. However it is a
>>>>>> redistribution friendly license, which is what matters.
>>>>>>
>>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> Thank you for this email.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I just noticed that there was a recent change to our LICENSE file
>>>>>>>> to make it exactly match the Apache 2.0 License [1]. This seems to be the
>>>>>>>> result of two conflicting LICENSE issues.
>>>>>>>>
>>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>>>>>>>> license policies [2]. Prior to the change our documentation didn't show up
>>>>>>>> there [3].
>>>>>>>>
>>>>>>>> Java artifact LICENSE issue: The removed text contained information
>>>>>>>> relevant to "convenience binary distributions". This text was added in [4]
>>>>>>>> as a result of this dev@ thread [5], where we noticed that
>>>>>>>> copyright notices were missing in binary artifacts. The suggested solution
>>>>>>>> (that we went with) was to just add the information to the root (source)
>>>>>>>> LICENSE.
>>>>>>>>
>>>>>>>
>>>>>>> Python distribution is missing both files as well. (
>>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> I'm not sure that that solution is consistent with this ASF guide
>>>>>>>> [6] which states:
>>>>>>>>
>>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the
>>>>>>>> contents of the distribution they reside in. Only components and resources
>>>>>>>> that are actually included in a distribution have any bearing on the
>>>>>>>> content of that distribution's NOTICE and LICENSE.
>>>>>>>>
>>>>>>>> I would argue that *just* Apache 2.0 is the correct text for our
>>>>>>>> root (source) LICENSE, and the correct way to deal with binary artifacts is
>>>>>>>> to generate per-artifact LICENSE/NOTICE files.
>>>>>>>>
>>>>>>>
>>>>>>> I do not know how to interpret this ASF guide. As an example from
>>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> So right now the Go issue is fixed, but the Java artifact issue has
>>>>>>>> regressed. I can think of two potential solutions to resolve both:
>>>>>>>> 1) Restore the "convenience binary distributions" information, and
>>>>>>>> see if we can get pkg.go.dev to allow it.
>>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for Java
>>>>>>>> binary artifacts.
>>>>>>>>
>>>>>>>> I have no idea how we might implement (2) so (1) seems more
>>>>>>>> tenable, but less correct since it's adding information not relevant to the
>>>>>>>> source release.
>>>>>>>>
>>>>>>>> Brian
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>>> [3]
>>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>>> [5]
>>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>>>>
>>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Kenneth Knowles <ke...@apache.org>.
Is there a Jira marked as blocking 2.29.0 for the cherrypick?

On Fri, Mar 19, 2021 at 6:16 PM Valentyn Tymofieiev <va...@google.com>
wrote:

> I also noticed (with a help of an automated tool) that
> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
> includes additional licenses not included in
> https://github.com/apache/beam/blob/master/LICENSE. Is that WAI since
> Dataflow runner is released as a separate jar artifact, and the licenses in
> question (GPL 2.0, CDDL) pertain to its dependencies, or we need to include
> those licenses as well?
>
>
>
>
> On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com>
>> wrote:
>>
>>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>>
>>> > and probably cherry pick it into the affected release branches.
>>> Even if we do this, the Java artifacts from the affected releases are
>>> missing the additional LICENSE text.
>>>
>>
>> IMO we can skip the cherry picks perhaps with the exception of the
>> upcoming 2.29 release.
>>
>>>
>>> > I do not know how to interpret this ASF guide. As an example from
>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>> licenses directory. There are even overlapping mentions.
>>> Agreed. I am a software engineer, not a lawyer, and even the ASF's guide
>>> that presumably targets engineers is not particularly clear to me. This was
>>> just my tenuous understanding after a quick review.
>>>
>>
>> Agreed. We can ask LEGAL for further clarification.
>>
>>
>>>
>>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> Thank you Rebo. I agree with reverting first and then figure out the
>>>> next steps.
>>>>
>>>> Here is a PR to revert your change:
>>>> https://github.com/apache/beam/pull/14267
>>>>
>>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com>
>>>> wrote:
>>>>
>>>>> Looking at the history it seems that before the python text was added,
>>>>> pkg.go.dev can parse the license stack just fine. It doesn't
>>>>> recognize the PSF license, and fails closed entirely as a result.
>>>>>
>>>>> I've filed an issue with pkg.go.dev (
>>>>> https://github.com/golang/go/issues/45095). If the bug is fixed, the
>>>>> affected versions will become visible as well.
>>>>>
>>>>> In the meantime, we should revert my change which clobbered the other
>>>>> licenses and probably cherry pick it into the affected release branches.
>>>>>
>>>>> The PSF license is annoying as it's explicitly unique. Nothing but
>>>>> python can use it and call it the PSF license. However it is a
>>>>> redistribution friendly license, which is what matters.
>>>>>
>>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>> Thank you for this email.
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I just noticed that there was a recent change to our LICENSE file to
>>>>>>> make it exactly match the Apache 2.0 License [1]. This seems to be the
>>>>>>> result of two conflicting LICENSE issues.
>>>>>>>
>>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>>>>>>> license policies [2]. Prior to the change our documentation didn't show up
>>>>>>> there [3].
>>>>>>>
>>>>>>> Java artifact LICENSE issue: The removed text contained information
>>>>>>> relevant to "convenience binary distributions". This text was added in [4]
>>>>>>> as a result of this dev@ thread [5], where we noticed that
>>>>>>> copyright notices were missing in binary artifacts. The suggested solution
>>>>>>> (that we went with) was to just add the information to the root (source)
>>>>>>> LICENSE.
>>>>>>>
>>>>>>
>>>>>> Python distribution is missing both files as well. (
>>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> I'm not sure that that solution is consistent with this ASF guide
>>>>>>> [6] which states:
>>>>>>>
>>>>>>> > The LICENSE and NOTICE files must *exactly* represent the contents
>>>>>>> of the distribution they reside in. Only components and resources that are
>>>>>>> actually included in a distribution have any bearing on the content of that
>>>>>>> distribution's NOTICE and LICENSE.
>>>>>>>
>>>>>>> I would argue that *just* Apache 2.0 is the correct text for our
>>>>>>> root (source) LICENSE, and the correct way to deal with binary artifacts is
>>>>>>> to generate per-artifact LICENSE/NOTICE files.
>>>>>>>
>>>>>>
>>>>>> I do not know how to interpret this ASF guide. As an example from
>>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>>> licenses directory. There are even overlapping mentions.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> So right now the Go issue is fixed, but the Java artifact issue has
>>>>>>> regressed. I can think of two potential solutions to resolve both:
>>>>>>> 1) Restore the "convenience binary distributions" information, and
>>>>>>> see if we can get pkg.go.dev to allow it.
>>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for Java
>>>>>>> binary artifacts.
>>>>>>>
>>>>>>> I have no idea how we might implement (2) so (1) seems more tenable,
>>>>>>> but less correct since it's adding information not relevant to the source
>>>>>>> release.
>>>>>>>
>>>>>>> Brian
>>>>>>>
>>>>>>>
>>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>>> [3]
>>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>>> [5]
>>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>>>
>>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Valentyn Tymofieiev <va...@google.com>.
I also noticed (with a help of an automated tool) that
https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/resources/NOTICES
includes additional licenses not included in
https://github.com/apache/beam/blob/master/LICENSE. Is that WAI since
Dataflow runner is released as a separate jar artifact, and the licenses in
question (GPL 2.0, CDDL) pertain to its dependencies, or we need to include
those licenses as well?




On Thu, Mar 18, 2021 at 9:51 AM Ahmet Altay <al...@google.com> wrote:

>
>
> On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com> wrote:
>
>> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>>
>> > and probably cherry pick it into the affected release branches.
>> Even if we do this, the Java artifacts from the affected releases are
>> missing the additional LICENSE text.
>>
>
> IMO we can skip the cherry picks perhaps with the exception of the
> upcoming 2.29 release.
>
>>
>> > I do not know how to interpret this ASF guide. As an example from
>> another project: airflow also has a LICENSE file, NOTICE file, and a
>> licenses directory. There are even overlapping mentions.
>> Agreed. I am a software engineer, not a lawyer, and even the ASF's guide
>> that presumably targets engineers is not particularly clear to me. This was
>> just my tenuous understanding after a quick review.
>>
>
> Agreed. We can ask LEGAL for further clarification.
>
>
>>
>> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Thank you Rebo. I agree with reverting first and then figure out the
>>> next steps.
>>>
>>> Here is a PR to revert your change:
>>> https://github.com/apache/beam/pull/14267
>>>
>>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com> wrote:
>>>
>>>> Looking at the history it seems that before the python text was added,
>>>> pkg.go.dev can parse the license stack just fine. It doesn't recognize
>>>> the PSF license, and fails closed entirely as a result.
>>>>
>>>> I've filed an issue with pkg.go.dev (
>>>> https://github.com/golang/go/issues/45095). If the bug is fixed, the
>>>> affected versions will become visible as well.
>>>>
>>>> In the meantime, we should revert my change which clobbered the other
>>>> licenses and probably cherry pick it into the affected release branches.
>>>>
>>>> The PSF license is annoying as it's explicitly unique. Nothing but
>>>> python can use it and call it the PSF license. However it is a
>>>> redistribution friendly license, which is what matters.
>>>>
>>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> Thank you for this email.
>>>>>
>>>>>
>>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I just noticed that there was a recent change to our LICENSE file to
>>>>>> make it exactly match the Apache 2.0 License [1]. This seems to be the
>>>>>> result of two conflicting LICENSE issues.
>>>>>>
>>>>>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>>>>>> license policies [2]. Prior to the change our documentation didn't show up
>>>>>> there [3].
>>>>>>
>>>>>> Java artifact LICENSE issue: The removed text contained information
>>>>>> relevant to "convenience binary distributions". This text was added in [4]
>>>>>> as a result of this dev@ thread [5], where we noticed that copyright
>>>>>> notices were missing in binary artifacts. The suggested solution (that we
>>>>>> went with) was to just add the information to the root (source) LICENSE.
>>>>>>
>>>>>
>>>>> Python distribution is missing both files as well. (
>>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>>
>>>>>
>>>>>>
>>>>>> I'm not sure that that solution is consistent with this ASF guide [6]
>>>>>> which states:
>>>>>>
>>>>>> > The LICENSE and NOTICE files must *exactly* represent the contents
>>>>>> of the distribution they reside in. Only components and resources that are
>>>>>> actually included in a distribution have any bearing on the content of that
>>>>>> distribution's NOTICE and LICENSE.
>>>>>>
>>>>>> I would argue that *just* Apache 2.0 is the correct text for our root
>>>>>> (source) LICENSE, and the correct way to deal with binary artifacts is to
>>>>>> generate per-artifact LICENSE/NOTICE files.
>>>>>>
>>>>>
>>>>> I do not know how to interpret this ASF guide. As an example from
>>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>>> licenses directory. There are even overlapping mentions.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> So right now the Go issue is fixed, but the Java artifact issue has
>>>>>> regressed. I can think of two potential solutions to resolve both:
>>>>>> 1) Restore the "convenience binary distributions" information, and
>>>>>> see if we can get pkg.go.dev to allow it.
>>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for Java
>>>>>> binary artifacts.
>>>>>>
>>>>>> I have no idea how we might implement (2) so (1) seems more tenable,
>>>>>> but less correct since it's adding information not relevant to the source
>>>>>> release.
>>>>>>
>>>>>> Brian
>>>>>>
>>>>>>
>>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>>> [2] https://pkg.go.dev/license-policy
>>>>>> [3]
>>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>>> [5]
>>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>>
>>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Ahmet Altay <al...@google.com>.
On Thu, Mar 18, 2021 at 6:39 AM Brian Hulette <bh...@google.com> wrote:

> Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev
>
> > and probably cherry pick it into the affected release branches.
> Even if we do this, the Java artifacts from the affected releases are
> missing the additional LICENSE text.
>

IMO we can skip the cherry picks perhaps with the exception of the upcoming
2.29 release.

>
> > I do not know how to interpret this ASF guide. As an example from
> another project: airflow also has a LICENSE file, NOTICE file, and a
> licenses directory. There are even overlapping mentions.
> Agreed. I am a software engineer, not a lawyer, and even the ASF's guide
> that presumably targets engineers is not particularly clear to me. This was
> just my tenuous understanding after a quick review.
>

Agreed. We can ask LEGAL for further clarification.


>
> On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:
>
>> Thank you Rebo. I agree with reverting first and then figure out the next
>> steps.
>>
>> Here is a PR to revert your change:
>> https://github.com/apache/beam/pull/14267
>>
>> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com> wrote:
>>
>>> Looking at the history it seems that before the python text was added,
>>> pkg.go.dev can parse the license stack just fine. It doesn't recognize
>>> the PSF license, and fails closed entirely as a result.
>>>
>>> I've filed an issue with pkg.go.dev (
>>> https://github.com/golang/go/issues/45095). If the bug is fixed, the
>>> affected versions will become visible as well.
>>>
>>> In the meantime, we should revert my change which clobbered the other
>>> licenses and probably cherry pick it into the affected release branches.
>>>
>>> The PSF license is annoying as it's explicitly unique. Nothing but
>>> python can use it and call it the PSF license. However it is a
>>> redistribution friendly license, which is what matters.
>>>
>>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> Thank you for this email.
>>>>
>>>>
>>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>>>> wrote:
>>>>
>>>>> I just noticed that there was a recent change to our LICENSE file to
>>>>> make it exactly match the Apache 2.0 License [1]. This seems to be the
>>>>> result of two conflicting LICENSE issues.
>>>>>
>>>>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>>>>> license policies [2]. Prior to the change our documentation didn't show up
>>>>> there [3].
>>>>>
>>>>> Java artifact LICENSE issue: The removed text contained information
>>>>> relevant to "convenience binary distributions". This text was added in [4]
>>>>> as a result of this dev@ thread [5], where we noticed that copyright
>>>>> notices were missing in binary artifacts. The suggested solution (that we
>>>>> went with) was to just add the information to the root (source) LICENSE.
>>>>>
>>>>
>>>> Python distribution is missing both files as well. (
>>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>>
>>>>
>>>>>
>>>>> I'm not sure that that solution is consistent with this ASF guide [6]
>>>>> which states:
>>>>>
>>>>> > The LICENSE and NOTICE files must *exactly* represent the contents
>>>>> of the distribution they reside in. Only components and resources that are
>>>>> actually included in a distribution have any bearing on the content of that
>>>>> distribution's NOTICE and LICENSE.
>>>>>
>>>>> I would argue that *just* Apache 2.0 is the correct text for our root
>>>>> (source) LICENSE, and the correct way to deal with binary artifacts is to
>>>>> generate per-artifact LICENSE/NOTICE files.
>>>>>
>>>>
>>>> I do not know how to interpret this ASF guide. As an example from
>>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>>> licenses directory. There are even overlapping mentions.
>>>>
>>>>
>>>>>
>>>>>
>>>>> So right now the Go issue is fixed, but the Java artifact issue has
>>>>> regressed. I can think of two potential solutions to resolve both:
>>>>> 1) Restore the "convenience binary distributions" information, and see
>>>>> if we can get pkg.go.dev to allow it.
>>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for Java
>>>>> binary artifacts.
>>>>>
>>>>> I have no idea how we might implement (2) so (1) seems more tenable,
>>>>> but less correct since it's adding information not relevant to the source
>>>>> release.
>>>>>
>>>>> Brian
>>>>>
>>>>>
>>>>> [1] https://github.com/apache/beam/pull/11657
>>>>> [2] https://pkg.go.dev/license-policy
>>>>> [3]
>>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>>> [4] https://github.com/apache/beam/pull/5461
>>>>> [5]
>>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>>
>>>>

Re: Missing copyright notices due to LICENSE change

Posted by Brian Hulette <bh...@google.com>.
Thanks Robert! I'm +1 for reverting and engaging pkg.go.dev

> and probably cherry pick it into the affected release branches.
Even if we do this, the Java artifacts from the affected releases are
missing the additional LICENSE text.

> I do not know how to interpret this ASF guide. As an example from another
project: airflow also has a LICENSE file, NOTICE file, and a licenses
directory. There are even overlapping mentions.
Agreed. I am a software engineer, not a lawyer, and even the ASF's guide
that presumably targets engineers is not particularly clear to me. This was
just my tenuous understanding after a quick review.

On Wed, Mar 17, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:

> Thank you Rebo. I agree with reverting first and then figure out the next
> steps.
>
> Here is a PR to revert your change:
> https://github.com/apache/beam/pull/14267
>
> On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com> wrote:
>
>> Looking at the history it seems that before the python text was added,
>> pkg.go.dev can parse the license stack just fine. It doesn't recognize
>> the PSF license, and fails closed entirely as a result.
>>
>> I've filed an issue with pkg.go.dev (
>> https://github.com/golang/go/issues/45095). If the bug is fixed, the
>> affected versions will become visible as well.
>>
>> In the meantime, we should revert my change which clobbered the other
>> licenses and probably cherry pick it into the affected release branches.
>>
>> The PSF license is annoying as it's explicitly unique. Nothing but python
>> can use it and call it the PSF license. However it is a redistribution
>> friendly license, which is what matters.
>>
>> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Thank you for this email.
>>>
>>>
>>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>>> wrote:
>>>
>>>> I just noticed that there was a recent change to our LICENSE file to
>>>> make it exactly match the Apache 2.0 License [1]. This seems to be the
>>>> result of two conflicting LICENSE issues.
>>>>
>>>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>>>> license policies [2]. Prior to the change our documentation didn't show up
>>>> there [3].
>>>>
>>>> Java artifact LICENSE issue: The removed text contained information
>>>> relevant to "convenience binary distributions". This text was added in [4]
>>>> as a result of this dev@ thread [5], where we noticed that copyright
>>>> notices were missing in binary artifacts. The suggested solution (that we
>>>> went with) was to just add the information to the root (source) LICENSE.
>>>>
>>>
>>> Python distribution is missing both files as well. (
>>> https://issues.apache.org/jira/browse/BEAM-1746)
>>>
>>>
>>>>
>>>> I'm not sure that that solution is consistent with this ASF guide [6]
>>>> which states:
>>>>
>>>> > The LICENSE and NOTICE files must *exactly* represent the contents of
>>>> the distribution they reside in. Only components and resources that are
>>>> actually included in a distribution have any bearing on the content of that
>>>> distribution's NOTICE and LICENSE.
>>>>
>>>> I would argue that *just* Apache 2.0 is the correct text for our root
>>>> (source) LICENSE, and the correct way to deal with binary artifacts is to
>>>> generate per-artifact LICENSE/NOTICE files.
>>>>
>>>
>>> I do not know how to interpret this ASF guide. As an example from
>>> another project: airflow also has a LICENSE file, NOTICE file, and a
>>> licenses directory. There are even overlapping mentions.
>>>
>>>
>>>>
>>>>
>>>> So right now the Go issue is fixed, but the Java artifact issue has
>>>> regressed. I can think of two potential solutions to resolve both:
>>>> 1) Restore the "convenience binary distributions" information, and see
>>>> if we can get pkg.go.dev to allow it.
>>>> 2) Add infrastructure to generate LICENSE and NOTICE files for Java
>>>> binary artifacts.
>>>>
>>>> I have no idea how we might implement (2) so (1) seems more tenable,
>>>> but less correct since it's adding information not relevant to the source
>>>> release.
>>>>
>>>> Brian
>>>>
>>>>
>>>> [1] https://github.com/apache/beam/pull/11657
>>>> [2] https://pkg.go.dev/license-policy
>>>> [3]
>>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>>> [4] https://github.com/apache/beam/pull/5461
>>>> [5]
>>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>>> [6] https://infra.apache.org/licensing-howto.html
>>>>
>>>

Re: Missing copyright notices due to LICENSE change

Posted by Ahmet Altay <al...@google.com>.
Thank you Rebo. I agree with reverting first and then figure out the next
steps.

Here is a PR to revert your change:
https://github.com/apache/beam/pull/14267

On Wed, Mar 17, 2021 at 4:02 PM Robert Burke <ro...@frantil.com> wrote:

> Looking at the history it seems that before the python text was added,
> pkg.go.dev can parse the license stack just fine. It doesn't recognize
> the PSF license, and fails closed entirely as a result.
>
> I've filed an issue with pkg.go.dev (
> https://github.com/golang/go/issues/45095). If the bug is fixed, the
> affected versions will become visible as well.
>
> In the meantime, we should revert my change which clobbered the other
> licenses and probably cherry pick it into the affected release branches.
>
> The PSF license is annoying as it's explicitly unique. Nothing but python
> can use it and call it the PSF license. However it is a redistribution
> friendly license, which is what matters.
>
> On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:
>
>> Thank you for this email.
>>
>>
>> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com>
>> wrote:
>>
>>> I just noticed that there was a recent change to our LICENSE file to
>>> make it exactly match the Apache 2.0 License [1]. This seems to be the
>>> result of two conflicting LICENSE issues.
>>>
>>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>>> license policies [2]. Prior to the change our documentation didn't show up
>>> there [3].
>>>
>>> Java artifact LICENSE issue: The removed text contained information
>>> relevant to "convenience binary distributions". This text was added in [4]
>>> as a result of this dev@ thread [5], where we noticed that copyright
>>> notices were missing in binary artifacts. The suggested solution (that we
>>> went with) was to just add the information to the root (source) LICENSE.
>>>
>>
>> Python distribution is missing both files as well. (
>> https://issues.apache.org/jira/browse/BEAM-1746)
>>
>>
>>>
>>> I'm not sure that that solution is consistent with this ASF guide [6]
>>> which states:
>>>
>>> > The LICENSE and NOTICE files must *exactly* represent the contents of
>>> the distribution they reside in. Only components and resources that are
>>> actually included in a distribution have any bearing on the content of that
>>> distribution's NOTICE and LICENSE.
>>>
>>> I would argue that *just* Apache 2.0 is the correct text for our root
>>> (source) LICENSE, and the correct way to deal with binary artifacts is to
>>> generate per-artifact LICENSE/NOTICE files.
>>>
>>
>> I do not know how to interpret this ASF guide. As an example from another
>> project: airflow also has a LICENSE file, NOTICE file, and a licenses
>> directory. There are even overlapping mentions.
>>
>>
>>>
>>>
>>> So right now the Go issue is fixed, but the Java artifact issue has
>>> regressed. I can think of two potential solutions to resolve both:
>>> 1) Restore the "convenience binary distributions" information, and see
>>> if we can get pkg.go.dev to allow it.
>>> 2) Add infrastructure to generate LICENSE and NOTICE files for Java
>>> binary artifacts.
>>>
>>> I have no idea how we might implement (2) so (1) seems more tenable, but
>>> less correct since it's adding information not relevant to the source
>>> release.
>>>
>>> Brian
>>>
>>>
>>> [1] https://github.com/apache/beam/pull/11657
>>> [2] https://pkg.go.dev/license-policy
>>> [3]
>>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>>> [4] https://github.com/apache/beam/pull/5461
>>> [5]
>>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>>> [6] https://infra.apache.org/licensing-howto.html
>>>
>>

Re: Missing copyright notices due to LICENSE change

Posted by Robert Burke <ro...@frantil.com>.
Looking at the history it seems that before the python text was added,
pkg.go.dev can parse the license stack just fine. It doesn't recognize the
PSF license, and fails closed entirely as a result.

I've filed an issue with pkg.go.dev (
https://github.com/golang/go/issues/45095). If the bug is fixed, the
affected versions will become visible as well.

In the meantime, we should revert my change which clobbered the other
licenses and probably cherry pick it into the affected release branches.

The PSF license is annoying as it's explicitly unique. Nothing but python
can use it and call it the PSF license. However it is a redistribution
friendly license, which is what matters.

On Wed, Mar 17, 2021, 3:00 PM Ahmet Altay <al...@google.com> wrote:

> Thank you for this email.
>
>
> On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com> wrote:
>
>> I just noticed that there was a recent change to our LICENSE file to make
>> it exactly match the Apache 2.0 License [1]. This seems to be the result of
>> two conflicting LICENSE issues.
>>
>> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
>> license policies [2]. Prior to the change our documentation didn't show up
>> there [3].
>>
>> Java artifact LICENSE issue: The removed text contained information
>> relevant to "convenience binary distributions". This text was added in [4]
>> as a result of this dev@ thread [5], where we noticed that copyright
>> notices were missing in binary artifacts. The suggested solution (that we
>> went with) was to just add the information to the root (source) LICENSE.
>>
>
> Python distribution is missing both files as well. (
> https://issues.apache.org/jira/browse/BEAM-1746)
>
>
>>
>> I'm not sure that that solution is consistent with this ASF guide [6]
>> which states:
>>
>> > The LICENSE and NOTICE files must *exactly* represent the contents of
>> the distribution they reside in. Only components and resources that are
>> actually included in a distribution have any bearing on the content of that
>> distribution's NOTICE and LICENSE.
>>
>> I would argue that *just* Apache 2.0 is the correct text for our root
>> (source) LICENSE, and the correct way to deal with binary artifacts is to
>> generate per-artifact LICENSE/NOTICE files.
>>
>
> I do not know how to interpret this ASF guide. As an example from another
> project: airflow also has a LICENSE file, NOTICE file, and a licenses
> directory. There are even overlapping mentions.
>
>
>>
>>
>> So right now the Go issue is fixed, but the Java artifact issue has
>> regressed. I can think of two potential solutions to resolve both:
>> 1) Restore the "convenience binary distributions" information, and see if
>> we can get pkg.go.dev to allow it.
>> 2) Add infrastructure to generate LICENSE and NOTICE files for Java
>> binary artifacts.
>>
>> I have no idea how we might implement (2) so (1) seems more tenable, but
>> less correct since it's adding information not relevant to the source
>> release.
>>
>> Brian
>>
>>
>> [1] https://github.com/apache/beam/pull/11657
>> [2] https://pkg.go.dev/license-policy
>> [3]
>> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
>> [4] https://github.com/apache/beam/pull/5461
>> [5]
>> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
>> [6] https://infra.apache.org/licensing-howto.html
>>
>

Re: Missing copyright notices due to LICENSE change

Posted by Ahmet Altay <al...@google.com>.
Thank you for this email.


On Wed, Mar 17, 2021 at 2:32 PM Brian Hulette <bh...@google.com> wrote:

> I just noticed that there was a recent change to our LICENSE file to make
> it exactly match the Apache 2.0 License [1]. This seems to be the result of
> two conflicting LICENSE issues.
>
> Go LICENSE issue: The motivation for [1] was to satisfy pkg.go.dev's
> license policies [2]. Prior to the change our documentation didn't show up
> there [3].
>
> Java artifact LICENSE issue: The removed text contained information
> relevant to "convenience binary distributions". This text was added in [4]
> as a result of this dev@ thread [5], where we noticed that copyright
> notices were missing in binary artifacts. The suggested solution (that we
> went with) was to just add the information to the root (source) LICENSE.
>

Python distribution is missing both files as well. (
https://issues.apache.org/jira/browse/BEAM-1746)


>
> I'm not sure that that solution is consistent with this ASF guide [6]
> which states:
>
> > The LICENSE and NOTICE files must *exactly* represent the contents of
> the distribution they reside in. Only components and resources that are
> actually included in a distribution have any bearing on the content of that
> distribution's NOTICE and LICENSE.
>
> I would argue that *just* Apache 2.0 is the correct text for our root
> (source) LICENSE, and the correct way to deal with binary artifacts is to
> generate per-artifact LICENSE/NOTICE files.
>

I do not know how to interpret this ASF guide. As an example from another
project: airflow also has a LICENSE file, NOTICE file, and a licenses
directory. There are even overlapping mentions.


>
>
> So right now the Go issue is fixed, but the Java artifact issue has
> regressed. I can think of two potential solutions to resolve both:
> 1) Restore the "convenience binary distributions" information, and see if
> we can get pkg.go.dev to allow it.
> 2) Add infrastructure to generate LICENSE and NOTICE files for Java binary
> artifacts.
>
> I have no idea how we might implement (2) so (1) seems more tenable, but
> less correct since it's adding information not relevant to the source
> release.
>
> Brian
>
>
> [1] https://github.com/apache/beam/pull/11657
> [2] https://pkg.go.dev/license-policy
> [3]
> https://pkg.go.dev/github.com/apache/beam@v2.21.0+incompatible/sdks/go/pkg/beam
> [4] https://github.com/apache/beam/pull/5461
> [5]
> https://lists.apache.org/thread.html/6ef6630e908147ee83e1f1efd4befbda43efb2a59271c5cb49473103@%3Cdev.beam.apache.org%3E
> [6] https://infra.apache.org/licensing-howto.html
>