You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Till Rohrmann <tr...@apache.org> on 2016/06/29 15:17:45 UTC
Web dashboard binaries and licensing
Hi Flink community,
while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed that
according to the LICENSE file Flink contains many java script files.
However, tracking the corresponding files back was not so easy, because
they are actually all merged into
flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large java
script file which is part of the result of the bower build process.
I was wondering why we added a "binary" file which is auto-generated to our
code base. Was/Is there any specific reason for it?
The problem is that the java script files contained in the vendor.js are
not exactly the set of declared bower dependencies in
flink-runtime-web/web-dashboard/bower.json. I suspect that also transitive
dependencies will be included. At least that is my only explanation why
we're referencing lodash.js, for example, in our LICENSE file (you can find
it deeply hidden in the vendor.js file).
Wouldn't it be easier to auto-generate the web-dashboard related files
while building Flink? This would not only clean up our repository but also
make the traceability of contained 3rd party code in our code base much
easier. Maybe this maven plugin [1] could help us.
[1] https://github.com/eirslett/frontend-maven-plugin
Cheers,
Till
Re: Web dashboard binaries and licensing
Posted by Ufuk Celebi <uc...@apache.org>.
That's good news. :-) Thanks for looking into it Till and Stephan.
On Tue, Jul 5, 2016 at 5:37 PM, Till Rohrmann <tr...@apache.org> wrote:
> I talked to Stephan and he pointed out that the flink-dist binary file,
> which is Flink's fat jar, is not part of the official Flink release. We do
> offer to download this file as part of a zip file from the Flink website.
> However, this is only for convenience. In contrast to that, other binary
> files, such as the jars which are released on maven-central, are part of
> the official release. Consequently, these files must contain appropriate
> LICENSE and NOTICE files. If I remember correctly, then this is relevant
> for modules which shade dependencies away, because the shaded dependency is
> then contained in the jar.
>
> That was also the reason why we removed the license entries for all the
> dependencies contained in flink-dist. Given that this reasoning is sound,
> we can keep the LICENSE and NOTICE file as it is modulo the changes we
> introduced between Flink 1.0 and 1.1.
>
> On Sat, Jul 2, 2016 at 6:07 PM, Ufuk Celebi <uc...@apache.org> wrote:
>
>> On Fri, Jul 1, 2016 at 6:50 PM, Aljoscha Krettek <al...@apache.org>
>> wrote:
>> > Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our
>> > binary releases.
>>
>> True... to quote the linked ASF page: "As far as LICENSE and NOTICE
>> are concerned, only bundled bits matter."
>>
Re: Web dashboard binaries and licensing
Posted by Till Rohrmann <tr...@apache.org>.
I talked to Stephan and he pointed out that the flink-dist binary file,
which is Flink's fat jar, is not part of the official Flink release. We do
offer to download this file as part of a zip file from the Flink website.
However, this is only for convenience. In contrast to that, other binary
files, such as the jars which are released on maven-central, are part of
the official release. Consequently, these files must contain appropriate
LICENSE and NOTICE files. If I remember correctly, then this is relevant
for modules which shade dependencies away, because the shaded dependency is
then contained in the jar.
That was also the reason why we removed the license entries for all the
dependencies contained in flink-dist. Given that this reasoning is sound,
we can keep the LICENSE and NOTICE file as it is modulo the changes we
introduced between Flink 1.0 and 1.1.
On Sat, Jul 2, 2016 at 6:07 PM, Ufuk Celebi <uc...@apache.org> wrote:
> On Fri, Jul 1, 2016 at 6:50 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
> > Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our
> > binary releases.
>
> True... to quote the linked ASF page: "As far as LICENSE and NOTICE
> are concerned, only bundled bits matter."
>
Re: Web dashboard binaries and licensing
Posted by Ufuk Celebi <uc...@apache.org>.
On Fri, Jul 1, 2016 at 6:50 PM, Aljoscha Krettek <al...@apache.org> wrote:
> Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our
> binary releases.
True... to quote the linked ASF page: "As far as LICENSE and NOTICE
are concerned, only bundled bits matter."
Re: Web dashboard binaries and licensing
Posted by Aljoscha Krettek <al...@apache.org>.
Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our
binary releases.
On Fri, 1 Jul 2016 at 18:06 Greg Hogan <co...@greghogan.com> wrote:
> These external libraries will still be shipped in the binaries, which
> complicates the issue since source and binary licenses must be tracked
> separately.
> http://www.apache.org/dev/licensing-howto.html#binary
>
> I do agree that this would be a nice improvement to the build system.
>
> On Fri, Jul 1, 2016 at 11:16 AM, Maximilian Michels <mx...@apache.org>
> wrote:
>
> > Ah, now I see that a compile-time dependency resolution would make
> > sense. Then we don't have to check license compatibility for web
> > dependencies which are downloaded during compile time and are not part
> > of the source distribution.
> >
> > +1 Would be worth the effort to integrate this in our build system,
> > e.g. via the proposed plugin.
> >
> > On Thu, Jun 30, 2016 at 2:48 PM, Till Rohrmann <tr...@apache.org>
> > wrote:
> > > Yes, that is also how I understood the Apache License requirements.
> > Having
> > > a mere dependency on some external library which is not shipped as part
> > of
> > > the source release does not require to include it in the LICENSE/NOTICE
> > > file. I think that you only have to include a reference in the
> > > LICENSE/NOTICE file, if your source release contains actual code which
> > has
> > > a copyright. With the vendor.js file, the latter case applies.
> > >
> > > I'm not a bower expert, so I cannot tell for sure which javascript
> files
> > > end up in the vendor.js file. I would assume that they are the bower
> > > dependencies defined in bower.json and their transitive dependencies.
> But
> > > parsing the vendor.js file will not be feasible, since it contains
> 80000+
> > > LOC.
> > >
> > > Thus my main concern, why I initiated this discussion, is the legal
> > aspect
> > > of including a monolithic file of javascript code in our code base. The
> > > Flink community might release code which violates other people's
> > copyright.
> > >
> > > Additionally, it might be worth the effort to harden our build system
> > > anyway if you say that it's fragile. Then we could also properly
> > integrate
> > > the web-dashboard generation into the build process.
> > >
> > > On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <aljoscha@apache.org
> >
> > > wrote:
> > >
> > >> AFAIK the Apache License requires that we have an entry in the
> > >> LICENSE/NOTICE file for all external code that we ship with our
> source.
> > If
> > >> we have vendor.js in our source we have to include everything that is
> in
> > >> there. If we don't have any actual external code in our source but
> only
> > >> specified dependencies we should not be required to include them in
> the
> > >> LICENSE/NOTICE file.
> > >>
> > >> On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <mx...@apache.org>
> wrote:
> > >>
> > >> > I'm afraid I don't think integration in the Maven build process
> makes
> > >> > a difference in terms of licensing. It is already clearly specified
> > >> > what dependencies the web frontend uses (see README, bower.json,
> > >> > package.json). It won't get any easier with something integrated in
> > >> > the build process. We still have to leverage the Javascript build
> > >> > systems which resolve transitive dependencies in the same way Maven
> > >> > does. In the end, we just have to properly check out licenses which
> > >> > takes time.
> > >> >
> > >> > In what sense do you think it would make licensing easier?
> > >> >
> > >> > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <
> > aljoscha@apache.org>
> > >> > wrote:
> > >> > > I think it's not a question of easy-of-use but one of licensing. I
> > >> don't
> > >> > > think anyone really knows what code ends up in vendor.js, so it is
> > very
> > >> > > hard to figure out what we have to put into our LICENSE file.
> > >> > >
> > >> > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <mx...@apache.org>
> > >> wrote:
> > >> > >
> > >> > >> Hi Till,
> > >> > >>
> > >> > >> Thanks for checking the licenses for our web frontend.
> > >> > >>
> > >> > >> I think the reason why we added a big binary Javascript blob into
> > our
> > >> > >> repository was that it was easiest for most developers to deal
> > with.
> > >> > >> We don't have much Javascript expertise in the Flink community.
> > >> > >> Incorporating the web frontend in the build process would require
> > some
> > >> > >> additional work and make it even more complicated. It this sense
> > the
> > >> > >> solution we have currently is quite nice because it doesn't
> require
> > >> > >> developers to fiddle around with Javascript as long as they are
> not
> > >> > >> working on the webfrontend. If they do, they can simply use NPM
> and
> > >> > >> Bower which install the listed dependencies. The disadvantage is
> a
> > >> > >> slight increase of our repository because we commit a new
> > "vendor.js"
> > >> > >> for every recompile, but that is negligible in my opinion.
> > >> > >>
> > >> > >> It would be cleaner to build the Javascript parts in the Maven
> > build
> > >> > >> process but it will complicate the build process further.
> Frankly,
> > I
> > >> > >> don't see it'll add much value. Considering the fragility of our
> > build
> > >> > >> system I might be a bit conservative here :)
> > >> > >>
> > >> > >> Cheers,
> > >> > >> Max
> > >> > >>
> > >> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <
> > trohrmann@apache.org>
> > >> > >> wrote:
> > >> > >> > Hi Flink community,
> > >> > >> >
> > >> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I
> > >> noticed
> > >> > >> that
> > >> > >> > according to the LICENSE file Flink contains many java script
> > files.
> > >> > >> > However, tracking the corresponding files back was not so easy,
> > >> > because
> > >> > >> > they are actually all merged into
> > >> > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is
> a
> > >> large
> > >> > >> java
> > >> > >> > script file which is part of the result of the bower build
> > process.
> > >> > >> >
> > >> > >> > I was wondering why we added a "binary" file which is
> > auto-generated
> > >> > to
> > >> > >> our
> > >> > >> > code base. Was/Is there any specific reason for it?
> > >> > >> >
> > >> > >> > The problem is that the java script files contained in the
> > vendor.js
> > >> > are
> > >> > >> > not exactly the set of declared bower dependencies in
> > >> > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also
> > >> > >> transitive
> > >> > >> > dependencies will be included. At least that is my only
> > explanation
> > >> > why
> > >> > >> > we're referencing lodash.js, for example, in our LICENSE file
> > (you
> > >> can
> > >> > >> find
> > >> > >> > it deeply hidden in the vendor.js file).
> > >> > >> >
> > >> > >> > Wouldn't it be easier to auto-generate the web-dashboard
> related
> > >> files
> > >> > >> > while building Flink? This would not only clean up our
> repository
> > >> but
> > >> > >> also
> > >> > >> > make the traceability of contained 3rd party code in our code
> > base
> > >> > much
> > >> > >> > easier. Maybe this maven plugin [1] could help us.
> > >> > >> >
> > >> > >> > [1] https://github.com/eirslett/frontend-maven-plugin
> > >> > >> >
> > >> > >> > Cheers,
> > >> > >> > Till
> > >> > >>
> > >> >
> > >>
> >
>
Re: Web dashboard binaries and licensing
Posted by Greg Hogan <co...@greghogan.com>.
These external libraries will still be shipped in the binaries, which
complicates the issue since source and binary licenses must be tracked
separately.
http://www.apache.org/dev/licensing-howto.html#binary
I do agree that this would be a nice improvement to the build system.
On Fri, Jul 1, 2016 at 11:16 AM, Maximilian Michels <mx...@apache.org> wrote:
> Ah, now I see that a compile-time dependency resolution would make
> sense. Then we don't have to check license compatibility for web
> dependencies which are downloaded during compile time and are not part
> of the source distribution.
>
> +1 Would be worth the effort to integrate this in our build system,
> e.g. via the proposed plugin.
>
> On Thu, Jun 30, 2016 at 2:48 PM, Till Rohrmann <tr...@apache.org>
> wrote:
> > Yes, that is also how I understood the Apache License requirements.
> Having
> > a mere dependency on some external library which is not shipped as part
> of
> > the source release does not require to include it in the LICENSE/NOTICE
> > file. I think that you only have to include a reference in the
> > LICENSE/NOTICE file, if your source release contains actual code which
> has
> > a copyright. With the vendor.js file, the latter case applies.
> >
> > I'm not a bower expert, so I cannot tell for sure which javascript files
> > end up in the vendor.js file. I would assume that they are the bower
> > dependencies defined in bower.json and their transitive dependencies. But
> > parsing the vendor.js file will not be feasible, since it contains 80000+
> > LOC.
> >
> > Thus my main concern, why I initiated this discussion, is the legal
> aspect
> > of including a monolithic file of javascript code in our code base. The
> > Flink community might release code which violates other people's
> copyright.
> >
> > Additionally, it might be worth the effort to harden our build system
> > anyway if you say that it's fragile. Then we could also properly
> integrate
> > the web-dashboard generation into the build process.
> >
> > On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> >
> >> AFAIK the Apache License requires that we have an entry in the
> >> LICENSE/NOTICE file for all external code that we ship with our source.
> If
> >> we have vendor.js in our source we have to include everything that is in
> >> there. If we don't have any actual external code in our source but only
> >> specified dependencies we should not be required to include them in the
> >> LICENSE/NOTICE file.
> >>
> >> On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <mx...@apache.org> wrote:
> >>
> >> > I'm afraid I don't think integration in the Maven build process makes
> >> > a difference in terms of licensing. It is already clearly specified
> >> > what dependencies the web frontend uses (see README, bower.json,
> >> > package.json). It won't get any easier with something integrated in
> >> > the build process. We still have to leverage the Javascript build
> >> > systems which resolve transitive dependencies in the same way Maven
> >> > does. In the end, we just have to properly check out licenses which
> >> > takes time.
> >> >
> >> > In what sense do you think it would make licensing easier?
> >> >
> >> > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <
> aljoscha@apache.org>
> >> > wrote:
> >> > > I think it's not a question of easy-of-use but one of licensing. I
> >> don't
> >> > > think anyone really knows what code ends up in vendor.js, so it is
> very
> >> > > hard to figure out what we have to put into our LICENSE file.
> >> > >
> >> > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <mx...@apache.org>
> >> wrote:
> >> > >
> >> > >> Hi Till,
> >> > >>
> >> > >> Thanks for checking the licenses for our web frontend.
> >> > >>
> >> > >> I think the reason why we added a big binary Javascript blob into
> our
> >> > >> repository was that it was easiest for most developers to deal
> with.
> >> > >> We don't have much Javascript expertise in the Flink community.
> >> > >> Incorporating the web frontend in the build process would require
> some
> >> > >> additional work and make it even more complicated. It this sense
> the
> >> > >> solution we have currently is quite nice because it doesn't require
> >> > >> developers to fiddle around with Javascript as long as they are not
> >> > >> working on the webfrontend. If they do, they can simply use NPM and
> >> > >> Bower which install the listed dependencies. The disadvantage is a
> >> > >> slight increase of our repository because we commit a new
> "vendor.js"
> >> > >> for every recompile, but that is negligible in my opinion.
> >> > >>
> >> > >> It would be cleaner to build the Javascript parts in the Maven
> build
> >> > >> process but it will complicate the build process further. Frankly,
> I
> >> > >> don't see it'll add much value. Considering the fragility of our
> build
> >> > >> system I might be a bit conservative here :)
> >> > >>
> >> > >> Cheers,
> >> > >> Max
> >> > >>
> >> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <
> trohrmann@apache.org>
> >> > >> wrote:
> >> > >> > Hi Flink community,
> >> > >> >
> >> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I
> >> noticed
> >> > >> that
> >> > >> > according to the LICENSE file Flink contains many java script
> files.
> >> > >> > However, tracking the corresponding files back was not so easy,
> >> > because
> >> > >> > they are actually all merged into
> >> > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a
> >> large
> >> > >> java
> >> > >> > script file which is part of the result of the bower build
> process.
> >> > >> >
> >> > >> > I was wondering why we added a "binary" file which is
> auto-generated
> >> > to
> >> > >> our
> >> > >> > code base. Was/Is there any specific reason for it?
> >> > >> >
> >> > >> > The problem is that the java script files contained in the
> vendor.js
> >> > are
> >> > >> > not exactly the set of declared bower dependencies in
> >> > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also
> >> > >> transitive
> >> > >> > dependencies will be included. At least that is my only
> explanation
> >> > why
> >> > >> > we're referencing lodash.js, for example, in our LICENSE file
> (you
> >> can
> >> > >> find
> >> > >> > it deeply hidden in the vendor.js file).
> >> > >> >
> >> > >> > Wouldn't it be easier to auto-generate the web-dashboard related
> >> files
> >> > >> > while building Flink? This would not only clean up our repository
> >> but
> >> > >> also
> >> > >> > make the traceability of contained 3rd party code in our code
> base
> >> > much
> >> > >> > easier. Maybe this maven plugin [1] could help us.
> >> > >> >
> >> > >> > [1] https://github.com/eirslett/frontend-maven-plugin
> >> > >> >
> >> > >> > Cheers,
> >> > >> > Till
> >> > >>
> >> >
> >>
>
Re: Web dashboard binaries and licensing
Posted by Maximilian Michels <mx...@apache.org>.
Ah, now I see that a compile-time dependency resolution would make
sense. Then we don't have to check license compatibility for web
dependencies which are downloaded during compile time and are not part
of the source distribution.
+1 Would be worth the effort to integrate this in our build system,
e.g. via the proposed plugin.
On Thu, Jun 30, 2016 at 2:48 PM, Till Rohrmann <tr...@apache.org> wrote:
> Yes, that is also how I understood the Apache License requirements. Having
> a mere dependency on some external library which is not shipped as part of
> the source release does not require to include it in the LICENSE/NOTICE
> file. I think that you only have to include a reference in the
> LICENSE/NOTICE file, if your source release contains actual code which has
> a copyright. With the vendor.js file, the latter case applies.
>
> I'm not a bower expert, so I cannot tell for sure which javascript files
> end up in the vendor.js file. I would assume that they are the bower
> dependencies defined in bower.json and their transitive dependencies. But
> parsing the vendor.js file will not be feasible, since it contains 80000+
> LOC.
>
> Thus my main concern, why I initiated this discussion, is the legal aspect
> of including a monolithic file of javascript code in our code base. The
> Flink community might release code which violates other people's copyright.
>
> Additionally, it might be worth the effort to harden our build system
> anyway if you say that it's fragile. Then we could also properly integrate
> the web-dashboard generation into the build process.
>
> On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
>> AFAIK the Apache License requires that we have an entry in the
>> LICENSE/NOTICE file for all external code that we ship with our source. If
>> we have vendor.js in our source we have to include everything that is in
>> there. If we don't have any actual external code in our source but only
>> specified dependencies we should not be required to include them in the
>> LICENSE/NOTICE file.
>>
>> On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <mx...@apache.org> wrote:
>>
>> > I'm afraid I don't think integration in the Maven build process makes
>> > a difference in terms of licensing. It is already clearly specified
>> > what dependencies the web frontend uses (see README, bower.json,
>> > package.json). It won't get any easier with something integrated in
>> > the build process. We still have to leverage the Javascript build
>> > systems which resolve transitive dependencies in the same way Maven
>> > does. In the end, we just have to properly check out licenses which
>> > takes time.
>> >
>> > In what sense do you think it would make licensing easier?
>> >
>> > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <al...@apache.org>
>> > wrote:
>> > > I think it's not a question of easy-of-use but one of licensing. I
>> don't
>> > > think anyone really knows what code ends up in vendor.js, so it is very
>> > > hard to figure out what we have to put into our LICENSE file.
>> > >
>> > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <mx...@apache.org>
>> wrote:
>> > >
>> > >> Hi Till,
>> > >>
>> > >> Thanks for checking the licenses for our web frontend.
>> > >>
>> > >> I think the reason why we added a big binary Javascript blob into our
>> > >> repository was that it was easiest for most developers to deal with.
>> > >> We don't have much Javascript expertise in the Flink community.
>> > >> Incorporating the web frontend in the build process would require some
>> > >> additional work and make it even more complicated. It this sense the
>> > >> solution we have currently is quite nice because it doesn't require
>> > >> developers to fiddle around with Javascript as long as they are not
>> > >> working on the webfrontend. If they do, they can simply use NPM and
>> > >> Bower which install the listed dependencies. The disadvantage is a
>> > >> slight increase of our repository because we commit a new "vendor.js"
>> > >> for every recompile, but that is negligible in my opinion.
>> > >>
>> > >> It would be cleaner to build the Javascript parts in the Maven build
>> > >> process but it will complicate the build process further. Frankly, I
>> > >> don't see it'll add much value. Considering the fragility of our build
>> > >> system I might be a bit conservative here :)
>> > >>
>> > >> Cheers,
>> > >> Max
>> > >>
>> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <tr...@apache.org>
>> > >> wrote:
>> > >> > Hi Flink community,
>> > >> >
>> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I
>> noticed
>> > >> that
>> > >> > according to the LICENSE file Flink contains many java script files.
>> > >> > However, tracking the corresponding files back was not so easy,
>> > because
>> > >> > they are actually all merged into
>> > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a
>> large
>> > >> java
>> > >> > script file which is part of the result of the bower build process.
>> > >> >
>> > >> > I was wondering why we added a "binary" file which is auto-generated
>> > to
>> > >> our
>> > >> > code base. Was/Is there any specific reason for it?
>> > >> >
>> > >> > The problem is that the java script files contained in the vendor.js
>> > are
>> > >> > not exactly the set of declared bower dependencies in
>> > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also
>> > >> transitive
>> > >> > dependencies will be included. At least that is my only explanation
>> > why
>> > >> > we're referencing lodash.js, for example, in our LICENSE file (you
>> can
>> > >> find
>> > >> > it deeply hidden in the vendor.js file).
>> > >> >
>> > >> > Wouldn't it be easier to auto-generate the web-dashboard related
>> files
>> > >> > while building Flink? This would not only clean up our repository
>> but
>> > >> also
>> > >> > make the traceability of contained 3rd party code in our code base
>> > much
>> > >> > easier. Maybe this maven plugin [1] could help us.
>> > >> >
>> > >> > [1] https://github.com/eirslett/frontend-maven-plugin
>> > >> >
>> > >> > Cheers,
>> > >> > Till
>> > >>
>> >
>>
Re: Web dashboard binaries and licensing
Posted by Till Rohrmann <tr...@apache.org>.
Yes, that is also how I understood the Apache License requirements. Having
a mere dependency on some external library which is not shipped as part of
the source release does not require to include it in the LICENSE/NOTICE
file. I think that you only have to include a reference in the
LICENSE/NOTICE file, if your source release contains actual code which has
a copyright. With the vendor.js file, the latter case applies.
I'm not a bower expert, so I cannot tell for sure which javascript files
end up in the vendor.js file. I would assume that they are the bower
dependencies defined in bower.json and their transitive dependencies. But
parsing the vendor.js file will not be feasible, since it contains 80000+
LOC.
Thus my main concern, why I initiated this discussion, is the legal aspect
of including a monolithic file of javascript code in our code base. The
Flink community might release code which violates other people's copyright.
Additionally, it might be worth the effort to harden our build system
anyway if you say that it's fragile. Then we could also properly integrate
the web-dashboard generation into the build process.
On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <al...@apache.org>
wrote:
> AFAIK the Apache License requires that we have an entry in the
> LICENSE/NOTICE file for all external code that we ship with our source. If
> we have vendor.js in our source we have to include everything that is in
> there. If we don't have any actual external code in our source but only
> specified dependencies we should not be required to include them in the
> LICENSE/NOTICE file.
>
> On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <mx...@apache.org> wrote:
>
> > I'm afraid I don't think integration in the Maven build process makes
> > a difference in terms of licensing. It is already clearly specified
> > what dependencies the web frontend uses (see README, bower.json,
> > package.json). It won't get any easier with something integrated in
> > the build process. We still have to leverage the Javascript build
> > systems which resolve transitive dependencies in the same way Maven
> > does. In the end, we just have to properly check out licenses which
> > takes time.
> >
> > In what sense do you think it would make licensing easier?
> >
> > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <al...@apache.org>
> > wrote:
> > > I think it's not a question of easy-of-use but one of licensing. I
> don't
> > > think anyone really knows what code ends up in vendor.js, so it is very
> > > hard to figure out what we have to put into our LICENSE file.
> > >
> > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <mx...@apache.org>
> wrote:
> > >
> > >> Hi Till,
> > >>
> > >> Thanks for checking the licenses for our web frontend.
> > >>
> > >> I think the reason why we added a big binary Javascript blob into our
> > >> repository was that it was easiest for most developers to deal with.
> > >> We don't have much Javascript expertise in the Flink community.
> > >> Incorporating the web frontend in the build process would require some
> > >> additional work and make it even more complicated. It this sense the
> > >> solution we have currently is quite nice because it doesn't require
> > >> developers to fiddle around with Javascript as long as they are not
> > >> working on the webfrontend. If they do, they can simply use NPM and
> > >> Bower which install the listed dependencies. The disadvantage is a
> > >> slight increase of our repository because we commit a new "vendor.js"
> > >> for every recompile, but that is negligible in my opinion.
> > >>
> > >> It would be cleaner to build the Javascript parts in the Maven build
> > >> process but it will complicate the build process further. Frankly, I
> > >> don't see it'll add much value. Considering the fragility of our build
> > >> system I might be a bit conservative here :)
> > >>
> > >> Cheers,
> > >> Max
> > >>
> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <tr...@apache.org>
> > >> wrote:
> > >> > Hi Flink community,
> > >> >
> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I
> noticed
> > >> that
> > >> > according to the LICENSE file Flink contains many java script files.
> > >> > However, tracking the corresponding files back was not so easy,
> > because
> > >> > they are actually all merged into
> > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a
> large
> > >> java
> > >> > script file which is part of the result of the bower build process.
> > >> >
> > >> > I was wondering why we added a "binary" file which is auto-generated
> > to
> > >> our
> > >> > code base. Was/Is there any specific reason for it?
> > >> >
> > >> > The problem is that the java script files contained in the vendor.js
> > are
> > >> > not exactly the set of declared bower dependencies in
> > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also
> > >> transitive
> > >> > dependencies will be included. At least that is my only explanation
> > why
> > >> > we're referencing lodash.js, for example, in our LICENSE file (you
> can
> > >> find
> > >> > it deeply hidden in the vendor.js file).
> > >> >
> > >> > Wouldn't it be easier to auto-generate the web-dashboard related
> files
> > >> > while building Flink? This would not only clean up our repository
> but
> > >> also
> > >> > make the traceability of contained 3rd party code in our code base
> > much
> > >> > easier. Maybe this maven plugin [1] could help us.
> > >> >
> > >> > [1] https://github.com/eirslett/frontend-maven-plugin
> > >> >
> > >> > Cheers,
> > >> > Till
> > >>
> >
>
Re: Web dashboard binaries and licensing
Posted by Aljoscha Krettek <al...@apache.org>.
AFAIK the Apache License requires that we have an entry in the
LICENSE/NOTICE file for all external code that we ship with our source. If
we have vendor.js in our source we have to include everything that is in
there. If we don't have any actual external code in our source but only
specified dependencies we should not be required to include them in the
LICENSE/NOTICE file.
On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <mx...@apache.org> wrote:
> I'm afraid I don't think integration in the Maven build process makes
> a difference in terms of licensing. It is already clearly specified
> what dependencies the web frontend uses (see README, bower.json,
> package.json). It won't get any easier with something integrated in
> the build process. We still have to leverage the Javascript build
> systems which resolve transitive dependencies in the same way Maven
> does. In the end, we just have to properly check out licenses which
> takes time.
>
> In what sense do you think it would make licensing easier?
>
> On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
> > I think it's not a question of easy-of-use but one of licensing. I don't
> > think anyone really knows what code ends up in vendor.js, so it is very
> > hard to figure out what we have to put into our LICENSE file.
> >
> > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <mx...@apache.org> wrote:
> >
> >> Hi Till,
> >>
> >> Thanks for checking the licenses for our web frontend.
> >>
> >> I think the reason why we added a big binary Javascript blob into our
> >> repository was that it was easiest for most developers to deal with.
> >> We don't have much Javascript expertise in the Flink community.
> >> Incorporating the web frontend in the build process would require some
> >> additional work and make it even more complicated. It this sense the
> >> solution we have currently is quite nice because it doesn't require
> >> developers to fiddle around with Javascript as long as they are not
> >> working on the webfrontend. If they do, they can simply use NPM and
> >> Bower which install the listed dependencies. The disadvantage is a
> >> slight increase of our repository because we commit a new "vendor.js"
> >> for every recompile, but that is negligible in my opinion.
> >>
> >> It would be cleaner to build the Javascript parts in the Maven build
> >> process but it will complicate the build process further. Frankly, I
> >> don't see it'll add much value. Considering the fragility of our build
> >> system I might be a bit conservative here :)
> >>
> >> Cheers,
> >> Max
> >>
> >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <tr...@apache.org>
> >> wrote:
> >> > Hi Flink community,
> >> >
> >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed
> >> that
> >> > according to the LICENSE file Flink contains many java script files.
> >> > However, tracking the corresponding files back was not so easy,
> because
> >> > they are actually all merged into
> >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large
> >> java
> >> > script file which is part of the result of the bower build process.
> >> >
> >> > I was wondering why we added a "binary" file which is auto-generated
> to
> >> our
> >> > code base. Was/Is there any specific reason for it?
> >> >
> >> > The problem is that the java script files contained in the vendor.js
> are
> >> > not exactly the set of declared bower dependencies in
> >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also
> >> transitive
> >> > dependencies will be included. At least that is my only explanation
> why
> >> > we're referencing lodash.js, for example, in our LICENSE file (you can
> >> find
> >> > it deeply hidden in the vendor.js file).
> >> >
> >> > Wouldn't it be easier to auto-generate the web-dashboard related files
> >> > while building Flink? This would not only clean up our repository but
> >> also
> >> > make the traceability of contained 3rd party code in our code base
> much
> >> > easier. Maybe this maven plugin [1] could help us.
> >> >
> >> > [1] https://github.com/eirslett/frontend-maven-plugin
> >> >
> >> > Cheers,
> >> > Till
> >>
>
Re: Web dashboard binaries and licensing
Posted by Maximilian Michels <mx...@apache.org>.
I'm afraid I don't think integration in the Maven build process makes
a difference in terms of licensing. It is already clearly specified
what dependencies the web frontend uses (see README, bower.json,
package.json). It won't get any easier with something integrated in
the build process. We still have to leverage the Javascript build
systems which resolve transitive dependencies in the same way Maven
does. In the end, we just have to properly check out licenses which
takes time.
In what sense do you think it would make licensing easier?
On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <al...@apache.org> wrote:
> I think it's not a question of easy-of-use but one of licensing. I don't
> think anyone really knows what code ends up in vendor.js, so it is very
> hard to figure out what we have to put into our LICENSE file.
>
> On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <mx...@apache.org> wrote:
>
>> Hi Till,
>>
>> Thanks for checking the licenses for our web frontend.
>>
>> I think the reason why we added a big binary Javascript blob into our
>> repository was that it was easiest for most developers to deal with.
>> We don't have much Javascript expertise in the Flink community.
>> Incorporating the web frontend in the build process would require some
>> additional work and make it even more complicated. It this sense the
>> solution we have currently is quite nice because it doesn't require
>> developers to fiddle around with Javascript as long as they are not
>> working on the webfrontend. If they do, they can simply use NPM and
>> Bower which install the listed dependencies. The disadvantage is a
>> slight increase of our repository because we commit a new "vendor.js"
>> for every recompile, but that is negligible in my opinion.
>>
>> It would be cleaner to build the Javascript parts in the Maven build
>> process but it will complicate the build process further. Frankly, I
>> don't see it'll add much value. Considering the fragility of our build
>> system I might be a bit conservative here :)
>>
>> Cheers,
>> Max
>>
>> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <tr...@apache.org>
>> wrote:
>> > Hi Flink community,
>> >
>> > while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed
>> that
>> > according to the LICENSE file Flink contains many java script files.
>> > However, tracking the corresponding files back was not so easy, because
>> > they are actually all merged into
>> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large
>> java
>> > script file which is part of the result of the bower build process.
>> >
>> > I was wondering why we added a "binary" file which is auto-generated to
>> our
>> > code base. Was/Is there any specific reason for it?
>> >
>> > The problem is that the java script files contained in the vendor.js are
>> > not exactly the set of declared bower dependencies in
>> > flink-runtime-web/web-dashboard/bower.json. I suspect that also
>> transitive
>> > dependencies will be included. At least that is my only explanation why
>> > we're referencing lodash.js, for example, in our LICENSE file (you can
>> find
>> > it deeply hidden in the vendor.js file).
>> >
>> > Wouldn't it be easier to auto-generate the web-dashboard related files
>> > while building Flink? This would not only clean up our repository but
>> also
>> > make the traceability of contained 3rd party code in our code base much
>> > easier. Maybe this maven plugin [1] could help us.
>> >
>> > [1] https://github.com/eirslett/frontend-maven-plugin
>> >
>> > Cheers,
>> > Till
>>
Re: Web dashboard binaries and licensing
Posted by Aljoscha Krettek <al...@apache.org>.
I think it's not a question of easy-of-use but one of licensing. I don't
think anyone really knows what code ends up in vendor.js, so it is very
hard to figure out what we have to put into our LICENSE file.
On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <mx...@apache.org> wrote:
> Hi Till,
>
> Thanks for checking the licenses for our web frontend.
>
> I think the reason why we added a big binary Javascript blob into our
> repository was that it was easiest for most developers to deal with.
> We don't have much Javascript expertise in the Flink community.
> Incorporating the web frontend in the build process would require some
> additional work and make it even more complicated. It this sense the
> solution we have currently is quite nice because it doesn't require
> developers to fiddle around with Javascript as long as they are not
> working on the webfrontend. If they do, they can simply use NPM and
> Bower which install the listed dependencies. The disadvantage is a
> slight increase of our repository because we commit a new "vendor.js"
> for every recompile, but that is negligible in my opinion.
>
> It would be cleaner to build the Javascript parts in the Maven build
> process but it will complicate the build process further. Frankly, I
> don't see it'll add much value. Considering the fragility of our build
> system I might be a bit conservative here :)
>
> Cheers,
> Max
>
> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <tr...@apache.org>
> wrote:
> > Hi Flink community,
> >
> > while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed
> that
> > according to the LICENSE file Flink contains many java script files.
> > However, tracking the corresponding files back was not so easy, because
> > they are actually all merged into
> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large
> java
> > script file which is part of the result of the bower build process.
> >
> > I was wondering why we added a "binary" file which is auto-generated to
> our
> > code base. Was/Is there any specific reason for it?
> >
> > The problem is that the java script files contained in the vendor.js are
> > not exactly the set of declared bower dependencies in
> > flink-runtime-web/web-dashboard/bower.json. I suspect that also
> transitive
> > dependencies will be included. At least that is my only explanation why
> > we're referencing lodash.js, for example, in our LICENSE file (you can
> find
> > it deeply hidden in the vendor.js file).
> >
> > Wouldn't it be easier to auto-generate the web-dashboard related files
> > while building Flink? This would not only clean up our repository but
> also
> > make the traceability of contained 3rd party code in our code base much
> > easier. Maybe this maven plugin [1] could help us.
> >
> > [1] https://github.com/eirslett/frontend-maven-plugin
> >
> > Cheers,
> > Till
>
Re: Web dashboard binaries and licensing
Posted by Maximilian Michels <mx...@apache.org>.
Hi Till,
Thanks for checking the licenses for our web frontend.
I think the reason why we added a big binary Javascript blob into our
repository was that it was easiest for most developers to deal with.
We don't have much Javascript expertise in the Flink community.
Incorporating the web frontend in the build process would require some
additional work and make it even more complicated. It this sense the
solution we have currently is quite nice because it doesn't require
developers to fiddle around with Javascript as long as they are not
working on the webfrontend. If they do, they can simply use NPM and
Bower which install the listed dependencies. The disadvantage is a
slight increase of our repository because we commit a new "vendor.js"
for every recompile, but that is negligible in my opinion.
It would be cleaner to build the Javascript parts in the Maven build
process but it will complicate the build process further. Frankly, I
don't see it'll add much value. Considering the fragility of our build
system I might be a bit conservative here :)
Cheers,
Max
On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <tr...@apache.org> wrote:
> Hi Flink community,
>
> while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed that
> according to the LICENSE file Flink contains many java script files.
> However, tracking the corresponding files back was not so easy, because
> they are actually all merged into
> flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large java
> script file which is part of the result of the bower build process.
>
> I was wondering why we added a "binary" file which is auto-generated to our
> code base. Was/Is there any specific reason for it?
>
> The problem is that the java script files contained in the vendor.js are
> not exactly the set of declared bower dependencies in
> flink-runtime-web/web-dashboard/bower.json. I suspect that also transitive
> dependencies will be included. At least that is my only explanation why
> we're referencing lodash.js, for example, in our LICENSE file (you can find
> it deeply hidden in the vendor.js file).
>
> Wouldn't it be easier to auto-generate the web-dashboard related files
> while building Flink? This would not only clean up our repository but also
> make the traceability of contained 3rd party code in our code base much
> easier. Maybe this maven plugin [1] could help us.
>
> [1] https://github.com/eirslett/frontend-maven-plugin
>
> Cheers,
> Till