You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@quickstep.apache.org by Marc Spehlmann <sp...@gmail.com> on 2017/01/05 16:43:58 UTC
release: third_party/
I double checked the libraries in thirdy_party. They adhere to Apache's 3rd
party requirement as they are all apache 2 or opensourced by Google. The
only issue I saw was that IWYU has no documentation. Anyone know of its
source?
Library
Ver
License
Notes
benchmark
Apache 2.0
cpplint
Google
Header states that reuse is unconditional so long as the copyright header
stays intact.
Farmhash
Google
No restrictions so long as COPYING file is preserved. See COPYING
gflags
Google
No restrictions so long as COPYING file is preserved. See COPYING
glog
Google
No restrictions so long as COPYING file is preserved. See COPYING
gtest
Google
No restrictions so long as COPYING file is preserved. See COPYING in
subprojects.
gperftools
Google
No restrictions so long as COPYING file is preserved.
iwyu
No license present
linenoise
Google
No restrictions so long as LICENSE file is preserved.
protobuf
Google
No restrictions so long as LICENSE file is preserved.
RE2
Google
No restrictions so long as LICENSE file is preserved.
tmb
Apache 2.0
README: TMB is part of the Quickstep project (copyright Pivotal Software,
Inc.) and is distributed under the same license terms.
Re: release: third_party/
Posted by Zuyu Zhang <zu...@cs.wisc.edu>.
Hi all,
If I remembered correctly, the IWYU script (iwyu/iwyu_helper.py) was added
by Shoban.
As of the protobuf version, the distributed Quickstep needs protobuf 3 (on
which gRPC depends). And there is no hard requirements for the single node
version, at least 2.6 works.
Sorry for that I missed the discussion regarding changing submodules for
re2 and gtest to downloading a release. I would suggest to track a release
commit in a submodule. For example, when we added it as a submodule (
https://github.com/pivotalsoftware/quickstep/pull/156), there is a TODO in
the comments to track v1.8 once ready.
Cheers,
Zuyu
Re: release: third_party/
Posted by Jignesh Patel <jm...@gmail.com>.
I second that!! Big thanks Harshad and Marc for taking the lead on this release!
Cheers,
Jignesh
On 1/31/17, 6:15 PM, "Julian Hyde" <jh...@apache.org> wrote:
Well done! I know this wasn’t easy.
> On Jan 31, 2017, at 3:41 PM, Harshad Deshmukh <ha...@cs.wisc.edu> wrote:
>
> Hello all,
>
> At long last, we have dealt with the third party issue. Some highlights:
>
> 1. Most of the libraries are now downloaded through a shell script.
>
> 2. The download links point to the release versions of the libraries.
>
> 3. After downloading the source code, we apply appropriate patches.
>
> Some libraries don't have an official release yet, so we will wait until they have a release. Until then, we have copied their entire source code to our third party directory. I have updated the build instructions based on these changes.
>
> Thanks Julian, Marc and Zuyu for your help.
>
> On 01/23/2017 09:57 PM, Jignesh Patel wrote:
>> Thanks Zuyu for the nice summary! For this release, I’d support going with Harshad’s lead, which is a single script to download the third party libraries.
>>
>> Dear Harshad: If your life is simpler with any of the other option (e.g. the issue you are having with an old version of Ubuntu in Travis), when feel free to go with the Mesos approach.
>>
>> Cheers,
>> Jignesh
>>
>> On 1/23/17, 12:26 AM, "Zuyu Zhang" <hitzzy@gmail.com on behalf of zuyu@cs.wisc.edu> wrote:
>>
>> FYI, there are some Apache projects in C++ (
>> https://projects.apache.org/projects.html?language), and more in github (
>> https://github.com/apache?language=c%2B%2B), including incubator projects.
>> I summaries how typically they deal with the third parties and the release.
>> - Apache Mesos (https://github.com/apache/mesos) has most third parties
>> in release tar balls, along with patches.
>> - Apache Kudu (https://github.com/apache/kudu) has multiple scripts to
>> download and build third parties.
>> - Apache NiFi - MiNiFi (https://github.com/apache/nifi-minifi-cpp)
>> includes the whole codebase of third parties.
>> Cheers,
>> Zuyu
>>
>>
>
> --
> Thanks,
> Harshad
>
Re: release: third_party/
Posted by Julian Hyde <jh...@apache.org>.
Well done! I know this wasn’t easy.
> On Jan 31, 2017, at 3:41 PM, Harshad Deshmukh <ha...@cs.wisc.edu> wrote:
>
> Hello all,
>
> At long last, we have dealt with the third party issue. Some highlights:
>
> 1. Most of the libraries are now downloaded through a shell script.
>
> 2. The download links point to the release versions of the libraries.
>
> 3. After downloading the source code, we apply appropriate patches.
>
> Some libraries don't have an official release yet, so we will wait until they have a release. Until then, we have copied their entire source code to our third party directory. I have updated the build instructions based on these changes.
>
> Thanks Julian, Marc and Zuyu for your help.
>
> On 01/23/2017 09:57 PM, Jignesh Patel wrote:
>> Thanks Zuyu for the nice summary! For this release, I’d support going with Harshad’s lead, which is a single script to download the third party libraries.
>>
>> Dear Harshad: If your life is simpler with any of the other option (e.g. the issue you are having with an old version of Ubuntu in Travis), when feel free to go with the Mesos approach.
>>
>> Cheers,
>> Jignesh
>>
>> On 1/23/17, 12:26 AM, "Zuyu Zhang" <hitzzy@gmail.com on behalf of zuyu@cs.wisc.edu> wrote:
>>
>> FYI, there are some Apache projects in C++ (
>> https://projects.apache.org/projects.html?language), and more in github (
>> https://github.com/apache?language=c%2B%2B), including incubator projects.
>> I summaries how typically they deal with the third parties and the release.
>> - Apache Mesos (https://github.com/apache/mesos) has most third parties
>> in release tar balls, along with patches.
>> - Apache Kudu (https://github.com/apache/kudu) has multiple scripts to
>> download and build third parties.
>> - Apache NiFi - MiNiFi (https://github.com/apache/nifi-minifi-cpp)
>> includes the whole codebase of third parties.
>> Cheers,
>> Zuyu
>>
>>
>
> --
> Thanks,
> Harshad
>
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
Hi Zuyu,
Can you please start another thread for the travis problems for
distributed version? This thread is old and it's hard to keep track of
things on this one.
On 03/27/2017 04:45 PM, Zuyu Zhang wrote:
> Hi Harshad -- I did reply regarding the protobuf version (link
> <https://lists.apache.org/thread.html/3842f5853566b817fb99e79e39dc8097c5621449150a1dc19d439707@%3Cdev.quickstep.apache.org%3E>).
> \u200b
> Anyway, it is ok for me to keep the submodule. And I have tested that
> upgrading to protobuf 3 as a git submodule works in the Travis CI
> (https://github.com/apache/incubator-quickstep/pull/223/
> <https://github.com/apache/incubator-quickstep/pull/223/>).
>
> But there are serval issues need to fix to test the distributed
> version in Travis CI:
>
> * 50 minute timeout, even if we only run the distributed unit tests:
> https://github.com/apache/incubator-quickstep/pull/220
> <https://github.com/apache/incubator-quickstep/pull/220>. We may
> use ccache to speed up the whole process.
> * What is the best way to install grpc in Travis: download in Travis
> script, or download in the third_party? I don't think we could
> install using a pre-built binary package. Again, downloading in
> Travis script gets stuck due to the timeout.
>
> Cheers,
> Zuyu
--
Thanks,
Harshad
Re: release: third_party/
Posted by Zuyu Zhang <zu...@cs.wisc.edu>.
Hi Harshad -- I did reply regarding the protobuf version (link
<https://lists.apache.org/thread.html/3842f5853566b817fb99e79e39dc8097c5621449150a1dc19d439707@%3Cdev.quickstep.apache.org%3E>
).
Anyway, it is ok for me to keep the submodule. And I have tested that
upgrading to protobuf 3 as a git submodule works in the Travis CI (
https://github.com/apache/incubator-quickstep/pull/223/).
But there are serval issues need to fix to test the distributed version in
Travis CI:
- 50 minute timeout, even if we only run the distributed unit tests:
https://github.com/apache/incubator-quickstep/pull/220
<https://github.com/apache/incubator-quickstep/pull/220>. We may use
ccache to speed up the whole process.
- What is the best way to install grpc in Travis: download in Travis
script, or download in the third_party? I don't think we could install
using a pre-built binary package. Again, downloading in Travis script gets
stuck due to the timeout.
Cheers,
Zuyu
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
Hi Zuyu,
The protobuf issue had come when I was working on making the third party
library leaner, when I had asked you about the appropriate version for
protobuf. (Email link
<https://lists.apache.org/thread.html/271ac803d315c0ebbbf96862747e741dff1594c306a57253ffd4d481@%3Cdev.quickstep.apache.org%3E>).
If you want to remove it as a submodule and make it a part of the
download script, please feel free to make appropriate changes.
On 03/27/2017 03:04 PM, Zuyu Zhang wrote:
> Hi Harshad,
>
> A quick question: any reason why we still keep protobuf as a git
> submodule?
>
> To test the distributed version in the Travis CI, we need at least
> protobuf 3.0 and grpc.
>
> Cheers,
> Zuyu\u200b
--
Thanks,
Harshad
Re: release: third_party/
Posted by Zuyu Zhang <zu...@cs.wisc.edu>.
Hi Harshad,
A quick question: any reason why we still keep protobuf as a git submodule?
To test the distributed version in the Travis CI, we need at least protobuf
3.0 and grpc.
Cheers,
Zuyu
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
Hello all,
At long last, we have dealt with the third party issue. Some highlights:
1. Most of the libraries are now downloaded through a shell script.
2. The download links point to the release versions of the libraries.
3. After downloading the source code, we apply appropriate patches.
Some libraries don't have an official release yet, so we will wait until
they have a release. Until then, we have copied their entire source code
to our third party directory. I have updated the build instructions
based on these changes.
Thanks Julian, Marc and Zuyu for your help.
On 01/23/2017 09:57 PM, Jignesh Patel wrote:
> Thanks Zuyu for the nice summary! For this release, I\u2019d support going with Harshad\u2019s lead, which is a single script to download the third party libraries.
>
> Dear Harshad: If your life is simpler with any of the other option (e.g. the issue you are having with an old version of Ubuntu in Travis), when feel free to go with the Mesos approach.
>
> Cheers,
> Jignesh
>
> On 1/23/17, 12:26 AM, "Zuyu Zhang" <hitzzy@gmail.com on behalf of zuyu@cs.wisc.edu> wrote:
>
> FYI, there are some Apache projects in C++ (
> https://projects.apache.org/projects.html?language), and more in github (
> https://github.com/apache?language=c%2B%2B), including incubator projects.
>
> I summaries how typically they deal with the third parties and the release.
>
> - Apache Mesos (https://github.com/apache/mesos) has most third parties
> in release tar balls, along with patches.
> - Apache Kudu (https://github.com/apache/kudu) has multiple scripts to
> download and build third parties.
> - Apache NiFi - MiNiFi (https://github.com/apache/nifi-minifi-cpp)
> includes the whole codebase of third parties.
>
> Cheers,
> Zuyu
>
>
>
--
Thanks,
Harshad
Re: release: third_party/
Posted by Jignesh Patel <jm...@gmail.com>.
Thanks Zuyu for the nice summary! For this release, I’d support going with Harshad’s lead, which is a single script to download the third party libraries.
Dear Harshad: If your life is simpler with any of the other option (e.g. the issue you are having with an old version of Ubuntu in Travis), when feel free to go with the Mesos approach.
Cheers,
Jignesh
On 1/23/17, 12:26 AM, "Zuyu Zhang" <hitzzy@gmail.com on behalf of zuyu@cs.wisc.edu> wrote:
FYI, there are some Apache projects in C++ (
https://projects.apache.org/projects.html?language), and more in github (
https://github.com/apache?language=c%2B%2B), including incubator projects.
I summaries how typically they deal with the third parties and the release.
- Apache Mesos (https://github.com/apache/mesos) has most third parties
in release tar balls, along with patches.
- Apache Kudu (https://github.com/apache/kudu) has multiple scripts to
download and build third parties.
- Apache NiFi - MiNiFi (https://github.com/apache/nifi-minifi-cpp)
includes the whole codebase of third parties.
Cheers,
Zuyu
Re: release: third_party/
Posted by Zuyu Zhang <zu...@cs.wisc.edu>.
FYI, there are some Apache projects in C++ (
https://projects.apache.org/projects.html?language), and more in github (
https://github.com/apache?language=c%2B%2B), including incubator projects.
I summaries how typically they deal with the third parties and the release.
- Apache Mesos (https://github.com/apache/mesos) has most third parties
in release tar balls, along with patches.
- Apache Kudu (https://github.com/apache/kudu) has multiple scripts to
download and build third parties.
- Apache NiFi - MiNiFi (https://github.com/apache/nifi-minifi-cpp)
includes the whole codebase of third parties.
Cheers,
Zuyu
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
Hi all,
An update on the release and third party related issues. I have created
a branch called "lean-third-party" (link:
https://github.com/apache/incubator-quickstep/tree/lean-third-party).
Some highlights -
1. I removed the source code for many libraries and added patches for
some libraries.
2. There is a shell script that downloads the source code for the third
party libraries and optionally applies patches. No more submodules for
such libraries.
3. Each download link points to a release version of the library. There
are some libraries for which there is no official release version yet,
for such libraries I haven't modified the code and let it as is.
Upgrading the version in the future will require minor modification in
the shell script.
Can I have some volunteers for trying out this branch and ensuring that
the scripts download the libraries and apply the patches correctly? Once
we get green signals from few of the volunteers, I can create a pull
request and update the master branch.
@Zuyu: We don't know what's the right version for protobuf that we need.
Can you please let us know?
Thanks Julian for the inputs on cleaning up the third party library.
On 01/05/2017 01:03 PM, Jignesh Patel wrote:
> I think it was Zuyu. Zuyu \u2013 can you comment?
>
> Cheers,
> Jignesh
>
> On 1/5/17, 12:56 PM, "Harshad Deshmukh" <ha...@cs.wisc.edu> wrote:
>
> If I recall correctly, one of the Quickstep developers wrote the python
> script.
>
>
> On 01/05/2017 11:43 AM, Marc Spehlmann wrote:
> > That seems to be the repo with the LLVM code for implementing IWYU. I think
> > what we have in our repo is scripts ontop of that library. I'm wondering
> > where the scripts came from.
> >
> > Thanks,
> > Marc
> >
> > On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> > wrote:
> >
> >> Hi Marc,
> >>
> >> How about this one for IWYU?
> >>
> >> https://github.com/include-what-you-use/include-what-you-use
> >> /blob/master/LICENSE.TXT
> >>
> >>
> >> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
> >>
> >>> I double checked the libraries in thirdy_party. They adhere to Apache's
> >>> 3rd
> >>> party requirement as they are all apache 2 or opensourced by Google. The
> >>> only issue I saw was that IWYU has no documentation. Anyone know of its
> >>> source?
> >>>
> >>> Library
> >>>
> >>> Ver
> >>>
> >>> License
> >>>
> >>> Notes
> >>>
> >>> benchmark
> >>>
> >>> Apache 2.0
> >>>
> >>> cpplint
> >>>
> >>> Google
> >>>
> >>> Header states that reuse is unconditional so long as the copyright header
> >>> stays intact.
> >>>
> >>> Farmhash
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>
> >>> gflags
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>
> >>> glog
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>
> >>> gtest
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as COPYING file is preserved. See COPYING in
> >>> subprojects.
> >>>
> >>> gperftools
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as COPYING file is preserved.
> >>>
> >>> iwyu
> >>>
> >>>
> >>> No license present
> >>>
> >>> linenoise
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as LICENSE file is preserved.
> >>>
> >>> protobuf
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as LICENSE file is preserved.
> >>>
> >>> RE2
> >>>
> >>> Google
> >>>
> >>> No restrictions so long as LICENSE file is preserved.
> >>>
> >>> tmb
> >>>
> >>> Apache 2.0
> >>>
> >>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
> >>> Inc.) and is distributed under the same license terms.
> >>>
> >>>
> >> --
> >> Thanks,
> >> Harshad
> >>
> >>
>
> --
> Thanks,
> Harshad
>
>
>
>
--
Thanks,
Harshad
Re: release: third_party/
Posted by Jignesh Patel <jm...@gmail.com>.
I think it was Zuyu. Zuyu – can you comment?
Cheers,
Jignesh
On 1/5/17, 12:56 PM, "Harshad Deshmukh" <ha...@cs.wisc.edu> wrote:
If I recall correctly, one of the Quickstep developers wrote the python
script.
On 01/05/2017 11:43 AM, Marc Spehlmann wrote:
> That seems to be the repo with the LLVM code for implementing IWYU. I think
> what we have in our repo is scripts ontop of that library. I'm wondering
> where the scripts came from.
>
> Thanks,
> Marc
>
> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> wrote:
>
>> Hi Marc,
>>
>> How about this one for IWYU?
>>
>> https://github.com/include-what-you-use/include-what-you-use
>> /blob/master/LICENSE.TXT
>>
>>
>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>
>>> I double checked the libraries in thirdy_party. They adhere to Apache's
>>> 3rd
>>> party requirement as they are all apache 2 or opensourced by Google. The
>>> only issue I saw was that IWYU has no documentation. Anyone know of its
>>> source?
>>>
>>> Library
>>>
>>> Ver
>>>
>>> License
>>>
>>> Notes
>>>
>>> benchmark
>>>
>>> Apache 2.0
>>>
>>> cpplint
>>>
>>> Google
>>>
>>> Header states that reuse is unconditional so long as the copyright header
>>> stays intact.
>>>
>>> Farmhash
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> gflags
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> glog
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> gtest
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING in
>>> subprojects.
>>>
>>> gperftools
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved.
>>>
>>> iwyu
>>>
>>>
>>> No license present
>>>
>>> linenoise
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> protobuf
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> RE2
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> tmb
>>>
>>> Apache 2.0
>>>
>>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
>>> Inc.) and is distributed under the same license terms.
>>>
>>>
>> --
>> Thanks,
>> Harshad
>>
>>
--
Thanks,
Harshad
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
If I recall correctly, one of the Quickstep developers wrote the python
script.
On 01/05/2017 11:43 AM, Marc Spehlmann wrote:
> That seems to be the repo with the LLVM code for implementing IWYU. I think
> what we have in our repo is scripts ontop of that library. I'm wondering
> where the scripts came from.
>
> Thanks,
> Marc
>
> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> wrote:
>
>> Hi Marc,
>>
>> How about this one for IWYU?
>>
>> https://github.com/include-what-you-use/include-what-you-use
>> /blob/master/LICENSE.TXT
>>
>>
>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>
>>> I double checked the libraries in thirdy_party. They adhere to Apache's
>>> 3rd
>>> party requirement as they are all apache 2 or opensourced by Google. The
>>> only issue I saw was that IWYU has no documentation. Anyone know of its
>>> source?
>>>
>>> Library
>>>
>>> Ver
>>>
>>> License
>>>
>>> Notes
>>>
>>> benchmark
>>>
>>> Apache 2.0
>>>
>>> cpplint
>>>
>>> Google
>>>
>>> Header states that reuse is unconditional so long as the copyright header
>>> stays intact.
>>>
>>> Farmhash
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> gflags
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> glog
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> gtest
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING in
>>> subprojects.
>>>
>>> gperftools
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved.
>>>
>>> iwyu
>>>
>>>
>>> No license present
>>>
>>> linenoise
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> protobuf
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> RE2
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> tmb
>>>
>>> Apache 2.0
>>>
>>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
>>> Inc.) and is distributed under the same license terms.
>>>
>>>
>> --
>> Thanks,
>> Harshad
>>
>>
--
Thanks,
Harshad
Re: release: third_party/
Posted by Marc Spehlmann <sp...@gmail.com>.
Also, IWYU script was written by Shoban
On Mon, Jan 9, 2017 at 3:39 PM, Marc Spehlmann <sp...@gmail.com>
wrote:
> Thanks for finding these! Impala is more or less doing what we're doing,
> except it doesn't use submodules. One thing is that if you download their
> release (http://impala.apache.org/downloads.html) it's not clear where
> all the thirdparty dependencies went.
>
> On Mon, Jan 9, 2017 at 12:00 PM, Hakan Memisoglu <
> hakanmemisoglu@apache.org> wrote:
>
>> Hi Marc,
>>
>> Here are two Apache projects using C++:
>>
>> Impala: https://github.com/cloudera/Impala <https://github.com/cloudera/I
>> mpala>
>>
>> Kudu: https://github.com/cloudera/kudu <https://github.com/cloudera/kudu>
>>
>> Kudu has a download script for dependencies in third_party folder.
>> It seems to be a good example for our case.
>>
>> > On Jan 8, 2017, at 2:17 PM, Marc Spehlmann <sp...@gmail.com>
>> wrote:
>> >
>> > Regarding the IWYU script. It sounds safe then. If anything, maybe it
>> > belongs in another folder. ("tools/" ?)
>> >
>> > On Thu, Jan 5, 2017 at 3:07 PM, Julian Hyde <jh...@apache.org> wrote:
>> >
>> >> Better still, make a release depend only releases. Snapshots and even
>> >> commit IDs can disappear.
>> >>
>> >> And consider the state of the IP. In Apache and many other open source
>> >> projects, the IP is only guaranteed "clean" when it has been formally
>> >> released.
>> >>
>> >> Sorry this is all painful. But conversely, this is the value-add of
>> >> making a release.
>> >>
>> >> On Thu, Jan 5, 2017 at 11:57 AM, Harshad Deshmukh <harshad@cs.wisc.edu
>> >
>> >> wrote:
>> >>>> Also, since you’re relying on the latest googletest, protobuf and re2
>> >> your
>> >>>> release will work today and will break at some point in the future.
>> >> That’s
>> >>>> not OK. A release can depend only on releases, not snapshots or live
>> >>>> repositories.
>> >>>
>> >>> Thanks for pointing that. It is possible to fetch a particular release
>> >>> (based on its commit ID) of the submodule. We will add the release
>> >> commit ID
>> >>> to the git submodule file.
>> >>>
>> >>>
>> >>>
>> >>> On 01/05/2017 01:44 PM, Julian Hyde wrote:
>> >>>>
>> >>>> Git sub-modules are useful, I agree. But for a source release, the
>> goal
>> >> is
>> >>>> that someone should be able to download the source tar-ball and go:
>> >>>>
>> >>>> $ curl -O …/apache-quickstep-x.x-incubating.tar.gz
>> >>>> $ tar xvfz apache-quickstep-x.x-incubating.tar.gz
>> >>>> $ cd apache-quickstep-x.x-incubating/build
>> >>>> $ ./download-thirdparty.sh
>> >>>> $ cmake etc.
>> >>>>
>> >>>> I tried to find other Apache projects that use git submodules and see
>> >> what
>> >>>> they do for source releases. Not much luck. Maybe someone else can
>> find
>> >>>> something.
>> >>>>
>> >>>> Also, since you’re relying on the latest googletest, protobuf and re2
>> >> your
>> >>>> release will work today and will break at some point in the future.
>> >> That’s
>> >>>> not OK. A release can depend only on releases, not snapshots or live
>> >>>> repositories.
>> >>>>
>> >>>> Julian
>> >>>>
>> >>>>
>> >>>>
>> >>>>> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>> >>>>> wrote:
>> >>>>>
>> >>>>> Thanks for the review Julian.
>> >>>>>
>> >>>>> For some of the third party libraries used in Quickstep (e.g.
>> >> googletest,
>> >>>>> protobuf and re2) we use the submodules feature of git. For such
>> >> libraries,
>> >>>>> the developer has to initialize the submodule only once, which pulls
>> >> code
>> >>>>> from the third party repo to the Quickstep third party directory.
>> >>>>>
>> >>>>> I don't know of a centralized repo for C++ projects. Does the git
>> >>>>> submodule method sound similar to the maven central approach you
>> >> mentioned?
>> >>>>>
>> >>>>> On 01/05/2017 12:43 PM, Julian Hyde wrote:
>> >>>>>>
>> >>>>>> I took a quick look at third_party and there don’t seem to be any
>> >>>>>> binaries in there. That’s good. You definitely cannot include
>> >> binaries in a
>> >>>>>> source release.
>> >>>>>>
>> >>>>>> The more you can remove from third_party, the better. It doesn’t
>> have
>> >> to
>> >>>>>> be done this release, but the less IP there is to review, the
>> easier
>> >> for
>> >>>>>> everyone.
>> >>>>>>
>> >>>>>> Consider pulling from an external source the first time the
>> developer
>> >>>>>> builds in a sandbox, then apply patches. The patches will be the
>> only
>> >> thing
>> >>>>>> checked in to quickstep. (My expertise is in Java projects, which
>> >> these days
>> >>>>>> get their dependencies from a Maven repo such as Maven central; I
>> >> don’t know
>> >>>>>> whether there is an equivalent place to pull C and C++ source code.
>> >> Might be
>> >>>>>> worth a review of what other C and C++ based Apache projects do for
>> >> their
>> >>>>>> third-party dependencies.)
>> >>>>>>
>> >>>>>> Julian
>> >>>>>>
>> >>>>>>
>> >>>>>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <
>> spehl.apache@gmail.com>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>> That seems to be the repo with the LLVM code for implementing
>> IWYU. I
>> >>>>>>> think
>> >>>>>>> what we have in our repo is scripts ontop of that library. I'm
>> >>>>>>> wondering
>> >>>>>>> where the scripts came from.
>> >>>>>>>
>> >>>>>>> Thanks,
>> >>>>>>> Marc
>> >>>>>>>
>> >>>>>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <
>> >> harshad@cs.wisc.edu>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Hi Marc,
>> >>>>>>>>
>> >>>>>>>> How about this one for IWYU?
>> >>>>>>>>
>> >>>>>>>> https://github.com/include-what-you-use/include-what-you-use
>> >>>>>>>> /blob/master/LICENSE.TXT
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>> >>>>>>>>
>> >>>>>>>>> I double checked the libraries in thirdy_party. They adhere to
>> >>>>>>>>> Apache's
>> >>>>>>>>> 3rd
>> >>>>>>>>> party requirement as they are all apache 2 or opensourced by
>> >> Google.
>> >>>>>>>>> The
>> >>>>>>>>> only issue I saw was that IWYU has no documentation. Anyone
>> know of
>> >>>>>>>>> its
>> >>>>>>>>> source?
>> >>>>>>>>>
>> >>>>>>>>> Library
>> >>>>>>>>>
>> >>>>>>>>> Ver
>> >>>>>>>>>
>> >>>>>>>>> License
>> >>>>>>>>>
>> >>>>>>>>> Notes
>> >>>>>>>>>
>> >>>>>>>>> benchmark
>> >>>>>>>>>
>> >>>>>>>>> Apache 2.0
>> >>>>>>>>>
>> >>>>>>>>> cpplint
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> Header states that reuse is unconditional so long as the
>> copyright
>> >>>>>>>>> header
>> >>>>>>>>> stays intact.
>> >>>>>>>>>
>> >>>>>>>>> Farmhash
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as COPYING file is preserved. See
>> COPYING
>> >>>>>>>>>
>> >>>>>>>>> gflags
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as COPYING file is preserved. See
>> COPYING
>> >>>>>>>>>
>> >>>>>>>>> glog
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as COPYING file is preserved. See
>> COPYING
>> >>>>>>>>>
>> >>>>>>>>> gtest
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as COPYING file is preserved. See
>> COPYING
>> >> in
>> >>>>>>>>> subprojects.
>> >>>>>>>>>
>> >>>>>>>>> gperftools
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as COPYING file is preserved.
>> >>>>>>>>>
>> >>>>>>>>> iwyu
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> No license present
>> >>>>>>>>>
>> >>>>>>>>> linenoise
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as LICENSE file is preserved.
>> >>>>>>>>>
>> >>>>>>>>> protobuf
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as LICENSE file is preserved.
>> >>>>>>>>>
>> >>>>>>>>> RE2
>> >>>>>>>>>
>> >>>>>>>>> Google
>> >>>>>>>>>
>> >>>>>>>>> No restrictions so long as LICENSE file is preserved.
>> >>>>>>>>>
>> >>>>>>>>> tmb
>> >>>>>>>>>
>> >>>>>>>>> Apache 2.0
>> >>>>>>>>>
>> >>>>>>>>> README: TMB is part of the Quickstep project (copyright Pivotal
>> >>>>>>>>> Software,
>> >>>>>>>>> Inc.) and is distributed under the same license terms.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Thanks,
>> >>>>>>>> Harshad
>> >>>>>>>>
>> >>>>>>>>
>> >>>>> --
>> >>>>> Thanks,
>> >>>>> Harshad
>> >>>>>
>> >>>
>> >>> --
>> >>> Thanks,
>> >>> Harshad
>> >>>
>> >>
>>
>>
>
Re: release: third_party/
Posted by Marc Spehlmann <sp...@gmail.com>.
Thanks for finding these! Impala is more or less doing what we're doing,
except it doesn't use submodules. One thing is that if you download their
release (http://impala.apache.org/downloads.html) it's not clear where all
the thirdparty dependencies went.
On Mon, Jan 9, 2017 at 12:00 PM, Hakan Memisoglu <ha...@apache.org>
wrote:
> Hi Marc,
>
> Here are two Apache projects using C++:
>
> Impala: https://github.com/cloudera/Impala <https://github.com/cloudera/
> Impala>
>
> Kudu: https://github.com/cloudera/kudu <https://github.com/cloudera/kudu>
>
> Kudu has a download script for dependencies in third_party folder.
> It seems to be a good example for our case.
>
> > On Jan 8, 2017, at 2:17 PM, Marc Spehlmann <sp...@gmail.com>
> wrote:
> >
> > Regarding the IWYU script. It sounds safe then. If anything, maybe it
> > belongs in another folder. ("tools/" ?)
> >
> > On Thu, Jan 5, 2017 at 3:07 PM, Julian Hyde <jh...@apache.org> wrote:
> >
> >> Better still, make a release depend only releases. Snapshots and even
> >> commit IDs can disappear.
> >>
> >> And consider the state of the IP. In Apache and many other open source
> >> projects, the IP is only guaranteed "clean" when it has been formally
> >> released.
> >>
> >> Sorry this is all painful. But conversely, this is the value-add of
> >> making a release.
> >>
> >> On Thu, Jan 5, 2017 at 11:57 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> >> wrote:
> >>>> Also, since you’re relying on the latest googletest, protobuf and re2
> >> your
> >>>> release will work today and will break at some point in the future.
> >> That’s
> >>>> not OK. A release can depend only on releases, not snapshots or live
> >>>> repositories.
> >>>
> >>> Thanks for pointing that. It is possible to fetch a particular release
> >>> (based on its commit ID) of the submodule. We will add the release
> >> commit ID
> >>> to the git submodule file.
> >>>
> >>>
> >>>
> >>> On 01/05/2017 01:44 PM, Julian Hyde wrote:
> >>>>
> >>>> Git sub-modules are useful, I agree. But for a source release, the
> goal
> >> is
> >>>> that someone should be able to download the source tar-ball and go:
> >>>>
> >>>> $ curl -O …/apache-quickstep-x.x-incubating.tar.gz
> >>>> $ tar xvfz apache-quickstep-x.x-incubating.tar.gz
> >>>> $ cd apache-quickstep-x.x-incubating/build
> >>>> $ ./download-thirdparty.sh
> >>>> $ cmake etc.
> >>>>
> >>>> I tried to find other Apache projects that use git submodules and see
> >> what
> >>>> they do for source releases. Not much luck. Maybe someone else can
> find
> >>>> something.
> >>>>
> >>>> Also, since you’re relying on the latest googletest, protobuf and re2
> >> your
> >>>> release will work today and will break at some point in the future.
> >> That’s
> >>>> not OK. A release can depend only on releases, not snapshots or live
> >>>> repositories.
> >>>>
> >>>> Julian
> >>>>
> >>>>
> >>>>
> >>>>> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> >>>>> wrote:
> >>>>>
> >>>>> Thanks for the review Julian.
> >>>>>
> >>>>> For some of the third party libraries used in Quickstep (e.g.
> >> googletest,
> >>>>> protobuf and re2) we use the submodules feature of git. For such
> >> libraries,
> >>>>> the developer has to initialize the submodule only once, which pulls
> >> code
> >>>>> from the third party repo to the Quickstep third party directory.
> >>>>>
> >>>>> I don't know of a centralized repo for C++ projects. Does the git
> >>>>> submodule method sound similar to the maven central approach you
> >> mentioned?
> >>>>>
> >>>>> On 01/05/2017 12:43 PM, Julian Hyde wrote:
> >>>>>>
> >>>>>> I took a quick look at third_party and there don’t seem to be any
> >>>>>> binaries in there. That’s good. You definitely cannot include
> >> binaries in a
> >>>>>> source release.
> >>>>>>
> >>>>>> The more you can remove from third_party, the better. It doesn’t
> have
> >> to
> >>>>>> be done this release, but the less IP there is to review, the easier
> >> for
> >>>>>> everyone.
> >>>>>>
> >>>>>> Consider pulling from an external source the first time the
> developer
> >>>>>> builds in a sandbox, then apply patches. The patches will be the
> only
> >> thing
> >>>>>> checked in to quickstep. (My expertise is in Java projects, which
> >> these days
> >>>>>> get their dependencies from a Maven repo such as Maven central; I
> >> don’t know
> >>>>>> whether there is an equivalent place to pull C and C++ source code.
> >> Might be
> >>>>>> worth a review of what other C and C++ based Apache projects do for
> >> their
> >>>>>> third-party dependencies.)
> >>>>>>
> >>>>>> Julian
> >>>>>>
> >>>>>>
> >>>>>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <spehl.apache@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> That seems to be the repo with the LLVM code for implementing
> IWYU. I
> >>>>>>> think
> >>>>>>> what we have in our repo is scripts ontop of that library. I'm
> >>>>>>> wondering
> >>>>>>> where the scripts came from.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Marc
> >>>>>>>
> >>>>>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <
> >> harshad@cs.wisc.edu>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Marc,
> >>>>>>>>
> >>>>>>>> How about this one for IWYU?
> >>>>>>>>
> >>>>>>>> https://github.com/include-what-you-use/include-what-you-use
> >>>>>>>> /blob/master/LICENSE.TXT
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
> >>>>>>>>
> >>>>>>>>> I double checked the libraries in thirdy_party. They adhere to
> >>>>>>>>> Apache's
> >>>>>>>>> 3rd
> >>>>>>>>> party requirement as they are all apache 2 or opensourced by
> >> Google.
> >>>>>>>>> The
> >>>>>>>>> only issue I saw was that IWYU has no documentation. Anyone know
> of
> >>>>>>>>> its
> >>>>>>>>> source?
> >>>>>>>>>
> >>>>>>>>> Library
> >>>>>>>>>
> >>>>>>>>> Ver
> >>>>>>>>>
> >>>>>>>>> License
> >>>>>>>>>
> >>>>>>>>> Notes
> >>>>>>>>>
> >>>>>>>>> benchmark
> >>>>>>>>>
> >>>>>>>>> Apache 2.0
> >>>>>>>>>
> >>>>>>>>> cpplint
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> Header states that reuse is unconditional so long as the
> copyright
> >>>>>>>>> header
> >>>>>>>>> stays intact.
> >>>>>>>>>
> >>>>>>>>> Farmhash
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>>>>>>>
> >>>>>>>>> gflags
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>>>>>>>
> >>>>>>>>> glog
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>>>>>>>
> >>>>>>>>> gtest
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> >> in
> >>>>>>>>> subprojects.
> >>>>>>>>>
> >>>>>>>>> gperftools
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as COPYING file is preserved.
> >>>>>>>>>
> >>>>>>>>> iwyu
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> No license present
> >>>>>>>>>
> >>>>>>>>> linenoise
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as LICENSE file is preserved.
> >>>>>>>>>
> >>>>>>>>> protobuf
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as LICENSE file is preserved.
> >>>>>>>>>
> >>>>>>>>> RE2
> >>>>>>>>>
> >>>>>>>>> Google
> >>>>>>>>>
> >>>>>>>>> No restrictions so long as LICENSE file is preserved.
> >>>>>>>>>
> >>>>>>>>> tmb
> >>>>>>>>>
> >>>>>>>>> Apache 2.0
> >>>>>>>>>
> >>>>>>>>> README: TMB is part of the Quickstep project (copyright Pivotal
> >>>>>>>>> Software,
> >>>>>>>>> Inc.) and is distributed under the same license terms.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Thanks,
> >>>>>>>> Harshad
> >>>>>>>>
> >>>>>>>>
> >>>>> --
> >>>>> Thanks,
> >>>>> Harshad
> >>>>>
> >>>
> >>> --
> >>> Thanks,
> >>> Harshad
> >>>
> >>
>
>
Re: release: third_party/
Posted by Hakan Memisoglu <ha...@apache.org>.
Hi Marc,
Here are two Apache projects using C++:
Impala: https://github.com/cloudera/Impala <https://github.com/cloudera/Impala>
Kudu: https://github.com/cloudera/kudu <https://github.com/cloudera/kudu>
Kudu has a download script for dependencies in third_party folder.
It seems to be a good example for our case.
> On Jan 8, 2017, at 2:17 PM, Marc Spehlmann <sp...@gmail.com> wrote:
>
> Regarding the IWYU script. It sounds safe then. If anything, maybe it
> belongs in another folder. ("tools/" ?)
>
> On Thu, Jan 5, 2017 at 3:07 PM, Julian Hyde <jh...@apache.org> wrote:
>
>> Better still, make a release depend only releases. Snapshots and even
>> commit IDs can disappear.
>>
>> And consider the state of the IP. In Apache and many other open source
>> projects, the IP is only guaranteed "clean" when it has been formally
>> released.
>>
>> Sorry this is all painful. But conversely, this is the value-add of
>> making a release.
>>
>> On Thu, Jan 5, 2017 at 11:57 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>> wrote:
>>>> Also, since you’re relying on the latest googletest, protobuf and re2
>> your
>>>> release will work today and will break at some point in the future.
>> That’s
>>>> not OK. A release can depend only on releases, not snapshots or live
>>>> repositories.
>>>
>>> Thanks for pointing that. It is possible to fetch a particular release
>>> (based on its commit ID) of the submodule. We will add the release
>> commit ID
>>> to the git submodule file.
>>>
>>>
>>>
>>> On 01/05/2017 01:44 PM, Julian Hyde wrote:
>>>>
>>>> Git sub-modules are useful, I agree. But for a source release, the goal
>> is
>>>> that someone should be able to download the source tar-ball and go:
>>>>
>>>> $ curl -O …/apache-quickstep-x.x-incubating.tar.gz
>>>> $ tar xvfz apache-quickstep-x.x-incubating.tar.gz
>>>> $ cd apache-quickstep-x.x-incubating/build
>>>> $ ./download-thirdparty.sh
>>>> $ cmake etc.
>>>>
>>>> I tried to find other Apache projects that use git submodules and see
>> what
>>>> they do for source releases. Not much luck. Maybe someone else can find
>>>> something.
>>>>
>>>> Also, since you’re relying on the latest googletest, protobuf and re2
>> your
>>>> release will work today and will break at some point in the future.
>> That’s
>>>> not OK. A release can depend only on releases, not snapshots or live
>>>> repositories.
>>>>
>>>> Julian
>>>>
>>>>
>>>>
>>>>> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>>>>> wrote:
>>>>>
>>>>> Thanks for the review Julian.
>>>>>
>>>>> For some of the third party libraries used in Quickstep (e.g.
>> googletest,
>>>>> protobuf and re2) we use the submodules feature of git. For such
>> libraries,
>>>>> the developer has to initialize the submodule only once, which pulls
>> code
>>>>> from the third party repo to the Quickstep third party directory.
>>>>>
>>>>> I don't know of a centralized repo for C++ projects. Does the git
>>>>> submodule method sound similar to the maven central approach you
>> mentioned?
>>>>>
>>>>> On 01/05/2017 12:43 PM, Julian Hyde wrote:
>>>>>>
>>>>>> I took a quick look at third_party and there don’t seem to be any
>>>>>> binaries in there. That’s good. You definitely cannot include
>> binaries in a
>>>>>> source release.
>>>>>>
>>>>>> The more you can remove from third_party, the better. It doesn’t have
>> to
>>>>>> be done this release, but the less IP there is to review, the easier
>> for
>>>>>> everyone.
>>>>>>
>>>>>> Consider pulling from an external source the first time the developer
>>>>>> builds in a sandbox, then apply patches. The patches will be the only
>> thing
>>>>>> checked in to quickstep. (My expertise is in Java projects, which
>> these days
>>>>>> get their dependencies from a Maven repo such as Maven central; I
>> don’t know
>>>>>> whether there is an equivalent place to pull C and C++ source code.
>> Might be
>>>>>> worth a review of what other C and C++ based Apache projects do for
>> their
>>>>>> third-party dependencies.)
>>>>>>
>>>>>> Julian
>>>>>>
>>>>>>
>>>>>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <sp...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> That seems to be the repo with the LLVM code for implementing IWYU. I
>>>>>>> think
>>>>>>> what we have in our repo is scripts ontop of that library. I'm
>>>>>>> wondering
>>>>>>> where the scripts came from.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Marc
>>>>>>>
>>>>>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <
>> harshad@cs.wisc.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Marc,
>>>>>>>>
>>>>>>>> How about this one for IWYU?
>>>>>>>>
>>>>>>>> https://github.com/include-what-you-use/include-what-you-use
>>>>>>>> /blob/master/LICENSE.TXT
>>>>>>>>
>>>>>>>>
>>>>>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>>>>>>>
>>>>>>>>> I double checked the libraries in thirdy_party. They adhere to
>>>>>>>>> Apache's
>>>>>>>>> 3rd
>>>>>>>>> party requirement as they are all apache 2 or opensourced by
>> Google.
>>>>>>>>> The
>>>>>>>>> only issue I saw was that IWYU has no documentation. Anyone know of
>>>>>>>>> its
>>>>>>>>> source?
>>>>>>>>>
>>>>>>>>> Library
>>>>>>>>>
>>>>>>>>> Ver
>>>>>>>>>
>>>>>>>>> License
>>>>>>>>>
>>>>>>>>> Notes
>>>>>>>>>
>>>>>>>>> benchmark
>>>>>>>>>
>>>>>>>>> Apache 2.0
>>>>>>>>>
>>>>>>>>> cpplint
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> Header states that reuse is unconditional so long as the copyright
>>>>>>>>> header
>>>>>>>>> stays intact.
>>>>>>>>>
>>>>>>>>> Farmhash
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>>>>
>>>>>>>>> gflags
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>>>>
>>>>>>>>> glog
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>>>>
>>>>>>>>> gtest
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>> in
>>>>>>>>> subprojects.
>>>>>>>>>
>>>>>>>>> gperftools
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as COPYING file is preserved.
>>>>>>>>>
>>>>>>>>> iwyu
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> No license present
>>>>>>>>>
>>>>>>>>> linenoise
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>>>>
>>>>>>>>> protobuf
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>>>>
>>>>>>>>> RE2
>>>>>>>>>
>>>>>>>>> Google
>>>>>>>>>
>>>>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>>>>
>>>>>>>>> tmb
>>>>>>>>>
>>>>>>>>> Apache 2.0
>>>>>>>>>
>>>>>>>>> README: TMB is part of the Quickstep project (copyright Pivotal
>>>>>>>>> Software,
>>>>>>>>> Inc.) and is distributed under the same license terms.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks,
>>>>>>>> Harshad
>>>>>>>>
>>>>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Harshad
>>>>>
>>>
>>> --
>>> Thanks,
>>> Harshad
>>>
>>
Re: release: third_party/
Posted by Marc Spehlmann <sp...@gmail.com>.
Regarding the IWYU script. It sounds safe then. If anything, maybe it
belongs in another folder. ("tools/" ?)
On Thu, Jan 5, 2017 at 3:07 PM, Julian Hyde <jh...@apache.org> wrote:
> Better still, make a release depend only releases. Snapshots and even
> commit IDs can disappear.
>
> And consider the state of the IP. In Apache and many other open source
> projects, the IP is only guaranteed "clean" when it has been formally
> released.
>
> Sorry this is all painful. But conversely, this is the value-add of
> making a release.
>
> On Thu, Jan 5, 2017 at 11:57 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> wrote:
> >> Also, since you’re relying on the latest googletest, protobuf and re2
> your
> >> release will work today and will break at some point in the future.
> That’s
> >> not OK. A release can depend only on releases, not snapshots or live
> >> repositories.
> >
> > Thanks for pointing that. It is possible to fetch a particular release
> > (based on its commit ID) of the submodule. We will add the release
> commit ID
> > to the git submodule file.
> >
> >
> >
> > On 01/05/2017 01:44 PM, Julian Hyde wrote:
> >>
> >> Git sub-modules are useful, I agree. But for a source release, the goal
> is
> >> that someone should be able to download the source tar-ball and go:
> >>
> >> $ curl -O …/apache-quickstep-x.x-incubating.tar.gz
> >> $ tar xvfz apache-quickstep-x.x-incubating.tar.gz
> >> $ cd apache-quickstep-x.x-incubating/build
> >> $ ./download-thirdparty.sh
> >> $ cmake etc.
> >>
> >> I tried to find other Apache projects that use git submodules and see
> what
> >> they do for source releases. Not much luck. Maybe someone else can find
> >> something.
> >>
> >> Also, since you’re relying on the latest googletest, protobuf and re2
> your
> >> release will work today and will break at some point in the future.
> That’s
> >> not OK. A release can depend only on releases, not snapshots or live
> >> repositories.
> >>
> >> Julian
> >>
> >>
> >>
> >>> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> >>> wrote:
> >>>
> >>> Thanks for the review Julian.
> >>>
> >>> For some of the third party libraries used in Quickstep (e.g.
> googletest,
> >>> protobuf and re2) we use the submodules feature of git. For such
> libraries,
> >>> the developer has to initialize the submodule only once, which pulls
> code
> >>> from the third party repo to the Quickstep third party directory.
> >>>
> >>> I don't know of a centralized repo for C++ projects. Does the git
> >>> submodule method sound similar to the maven central approach you
> mentioned?
> >>>
> >>> On 01/05/2017 12:43 PM, Julian Hyde wrote:
> >>>>
> >>>> I took a quick look at third_party and there don’t seem to be any
> >>>> binaries in there. That’s good. You definitely cannot include
> binaries in a
> >>>> source release.
> >>>>
> >>>> The more you can remove from third_party, the better. It doesn’t have
> to
> >>>> be done this release, but the less IP there is to review, the easier
> for
> >>>> everyone.
> >>>>
> >>>> Consider pulling from an external source the first time the developer
> >>>> builds in a sandbox, then apply patches. The patches will be the only
> thing
> >>>> checked in to quickstep. (My expertise is in Java projects, which
> these days
> >>>> get their dependencies from a Maven repo such as Maven central; I
> don’t know
> >>>> whether there is an equivalent place to pull C and C++ source code.
> Might be
> >>>> worth a review of what other C and C++ based Apache projects do for
> their
> >>>> third-party dependencies.)
> >>>>
> >>>> Julian
> >>>>
> >>>>
> >>>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <sp...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>> That seems to be the repo with the LLVM code for implementing IWYU. I
> >>>>> think
> >>>>> what we have in our repo is scripts ontop of that library. I'm
> >>>>> wondering
> >>>>> where the scripts came from.
> >>>>>
> >>>>> Thanks,
> >>>>> Marc
> >>>>>
> >>>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <
> harshad@cs.wisc.edu>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Marc,
> >>>>>>
> >>>>>> How about this one for IWYU?
> >>>>>>
> >>>>>> https://github.com/include-what-you-use/include-what-you-use
> >>>>>> /blob/master/LICENSE.TXT
> >>>>>>
> >>>>>>
> >>>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
> >>>>>>
> >>>>>>> I double checked the libraries in thirdy_party. They adhere to
> >>>>>>> Apache's
> >>>>>>> 3rd
> >>>>>>> party requirement as they are all apache 2 or opensourced by
> Google.
> >>>>>>> The
> >>>>>>> only issue I saw was that IWYU has no documentation. Anyone know of
> >>>>>>> its
> >>>>>>> source?
> >>>>>>>
> >>>>>>> Library
> >>>>>>>
> >>>>>>> Ver
> >>>>>>>
> >>>>>>> License
> >>>>>>>
> >>>>>>> Notes
> >>>>>>>
> >>>>>>> benchmark
> >>>>>>>
> >>>>>>> Apache 2.0
> >>>>>>>
> >>>>>>> cpplint
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> Header states that reuse is unconditional so long as the copyright
> >>>>>>> header
> >>>>>>> stays intact.
> >>>>>>>
> >>>>>>> Farmhash
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>>>>>
> >>>>>>> gflags
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>>>>>
> >>>>>>> glog
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> >>>>>>>
> >>>>>>> gtest
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
> in
> >>>>>>> subprojects.
> >>>>>>>
> >>>>>>> gperftools
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as COPYING file is preserved.
> >>>>>>>
> >>>>>>> iwyu
> >>>>>>>
> >>>>>>>
> >>>>>>> No license present
> >>>>>>>
> >>>>>>> linenoise
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as LICENSE file is preserved.
> >>>>>>>
> >>>>>>> protobuf
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as LICENSE file is preserved.
> >>>>>>>
> >>>>>>> RE2
> >>>>>>>
> >>>>>>> Google
> >>>>>>>
> >>>>>>> No restrictions so long as LICENSE file is preserved.
> >>>>>>>
> >>>>>>> tmb
> >>>>>>>
> >>>>>>> Apache 2.0
> >>>>>>>
> >>>>>>> README: TMB is part of the Quickstep project (copyright Pivotal
> >>>>>>> Software,
> >>>>>>> Inc.) and is distributed under the same license terms.
> >>>>>>>
> >>>>>>>
> >>>>>> --
> >>>>>> Thanks,
> >>>>>> Harshad
> >>>>>>
> >>>>>>
> >>> --
> >>> Thanks,
> >>> Harshad
> >>>
> >
> > --
> > Thanks,
> > Harshad
> >
>
Re: release: third_party/
Posted by Julian Hyde <jh...@apache.org>.
Better still, make a release depend only releases. Snapshots and even
commit IDs can disappear.
And consider the state of the IP. In Apache and many other open source
projects, the IP is only guaranteed "clean" when it has been formally
released.
Sorry this is all painful. But conversely, this is the value-add of
making a release.
On Thu, Jan 5, 2017 at 11:57 AM, Harshad Deshmukh <ha...@cs.wisc.edu> wrote:
>> Also, since you’re relying on the latest googletest, protobuf and re2 your
>> release will work today and will break at some point in the future. That’s
>> not OK. A release can depend only on releases, not snapshots or live
>> repositories.
>
> Thanks for pointing that. It is possible to fetch a particular release
> (based on its commit ID) of the submodule. We will add the release commit ID
> to the git submodule file.
>
>
>
> On 01/05/2017 01:44 PM, Julian Hyde wrote:
>>
>> Git sub-modules are useful, I agree. But for a source release, the goal is
>> that someone should be able to download the source tar-ball and go:
>>
>> $ curl -O …/apache-quickstep-x.x-incubating.tar.gz
>> $ tar xvfz apache-quickstep-x.x-incubating.tar.gz
>> $ cd apache-quickstep-x.x-incubating/build
>> $ ./download-thirdparty.sh
>> $ cmake etc.
>>
>> I tried to find other Apache projects that use git submodules and see what
>> they do for source releases. Not much luck. Maybe someone else can find
>> something.
>>
>> Also, since you’re relying on the latest googletest, protobuf and re2 your
>> release will work today and will break at some point in the future. That’s
>> not OK. A release can depend only on releases, not snapshots or live
>> repositories.
>>
>> Julian
>>
>>
>>
>>> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>>> wrote:
>>>
>>> Thanks for the review Julian.
>>>
>>> For some of the third party libraries used in Quickstep (e.g. googletest,
>>> protobuf and re2) we use the submodules feature of git. For such libraries,
>>> the developer has to initialize the submodule only once, which pulls code
>>> from the third party repo to the Quickstep third party directory.
>>>
>>> I don't know of a centralized repo for C++ projects. Does the git
>>> submodule method sound similar to the maven central approach you mentioned?
>>>
>>> On 01/05/2017 12:43 PM, Julian Hyde wrote:
>>>>
>>>> I took a quick look at third_party and there don’t seem to be any
>>>> binaries in there. That’s good. You definitely cannot include binaries in a
>>>> source release.
>>>>
>>>> The more you can remove from third_party, the better. It doesn’t have to
>>>> be done this release, but the less IP there is to review, the easier for
>>>> everyone.
>>>>
>>>> Consider pulling from an external source the first time the developer
>>>> builds in a sandbox, then apply patches. The patches will be the only thing
>>>> checked in to quickstep. (My expertise is in Java projects, which these days
>>>> get their dependencies from a Maven repo such as Maven central; I don’t know
>>>> whether there is an equivalent place to pull C and C++ source code. Might be
>>>> worth a review of what other C and C++ based Apache projects do for their
>>>> third-party dependencies.)
>>>>
>>>> Julian
>>>>
>>>>
>>>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <sp...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> That seems to be the repo with the LLVM code for implementing IWYU. I
>>>>> think
>>>>> what we have in our repo is scripts ontop of that library. I'm
>>>>> wondering
>>>>> where the scripts came from.
>>>>>
>>>>> Thanks,
>>>>> Marc
>>>>>
>>>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>>>>> wrote:
>>>>>
>>>>>> Hi Marc,
>>>>>>
>>>>>> How about this one for IWYU?
>>>>>>
>>>>>> https://github.com/include-what-you-use/include-what-you-use
>>>>>> /blob/master/LICENSE.TXT
>>>>>>
>>>>>>
>>>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>>>>>
>>>>>>> I double checked the libraries in thirdy_party. They adhere to
>>>>>>> Apache's
>>>>>>> 3rd
>>>>>>> party requirement as they are all apache 2 or opensourced by Google.
>>>>>>> The
>>>>>>> only issue I saw was that IWYU has no documentation. Anyone know of
>>>>>>> its
>>>>>>> source?
>>>>>>>
>>>>>>> Library
>>>>>>>
>>>>>>> Ver
>>>>>>>
>>>>>>> License
>>>>>>>
>>>>>>> Notes
>>>>>>>
>>>>>>> benchmark
>>>>>>>
>>>>>>> Apache 2.0
>>>>>>>
>>>>>>> cpplint
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> Header states that reuse is unconditional so long as the copyright
>>>>>>> header
>>>>>>> stays intact.
>>>>>>>
>>>>>>> Farmhash
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>>
>>>>>>> gflags
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>>
>>>>>>> glog
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>>
>>>>>>> gtest
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as COPYING file is preserved. See COPYING in
>>>>>>> subprojects.
>>>>>>>
>>>>>>> gperftools
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as COPYING file is preserved.
>>>>>>>
>>>>>>> iwyu
>>>>>>>
>>>>>>>
>>>>>>> No license present
>>>>>>>
>>>>>>> linenoise
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>>
>>>>>>> protobuf
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>>
>>>>>>> RE2
>>>>>>>
>>>>>>> Google
>>>>>>>
>>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>>
>>>>>>> tmb
>>>>>>>
>>>>>>> Apache 2.0
>>>>>>>
>>>>>>> README: TMB is part of the Quickstep project (copyright Pivotal
>>>>>>> Software,
>>>>>>> Inc.) and is distributed under the same license terms.
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Thanks,
>>>>>> Harshad
>>>>>>
>>>>>>
>>> --
>>> Thanks,
>>> Harshad
>>>
>
> --
> Thanks,
> Harshad
>
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
> Also, since you\u2019re relying on the latest googletest, protobuf and re2
your release will work today and will break at some point in the future.
That\u2019s not OK. A release can depend only on releases, not snapshots or
live repositories.
Thanks for pointing that. It is possible to fetch a particular release
(based on its commit ID) of the submodule. We will add the release
commit ID to the git submodule file.
On 01/05/2017 01:44 PM, Julian Hyde wrote:
> Git sub-modules are useful, I agree. But for a source release, the goal is that someone should be able to download the source tar-ball and go:
>
> $ curl -O \u2026/apache-quickstep-x.x-incubating.tar.gz
> $ tar xvfz apache-quickstep-x.x-incubating.tar.gz
> $ cd apache-quickstep-x.x-incubating/build
> $ ./download-thirdparty.sh
> $ cmake etc.
>
> I tried to find other Apache projects that use git submodules and see what they do for source releases. Not much luck. Maybe someone else can find something.
>
> Also, since you\u2019re relying on the latest googletest, protobuf and re2 your release will work today and will break at some point in the future. That\u2019s not OK. A release can depend only on releases, not snapshots or live repositories.
>
> Julian
>
>
>
>> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <ha...@cs.wisc.edu> wrote:
>>
>> Thanks for the review Julian.
>>
>> For some of the third party libraries used in Quickstep (e.g. googletest, protobuf and re2) we use the submodules feature of git. For such libraries, the developer has to initialize the submodule only once, which pulls code from the third party repo to the Quickstep third party directory.
>>
>> I don't know of a centralized repo for C++ projects. Does the git submodule method sound similar to the maven central approach you mentioned?
>>
>> On 01/05/2017 12:43 PM, Julian Hyde wrote:
>>> I took a quick look at third_party and there don\u2019t seem to be any binaries in there. That\u2019s good. You definitely cannot include binaries in a source release.
>>>
>>> The more you can remove from third_party, the better. It doesn\u2019t have to be done this release, but the less IP there is to review, the easier for everyone.
>>>
>>> Consider pulling from an external source the first time the developer builds in a sandbox, then apply patches. The patches will be the only thing checked in to quickstep. (My expertise is in Java projects, which these days get their dependencies from a Maven repo such as Maven central; I don\u2019t know whether there is an equivalent place to pull C and C++ source code. Might be worth a review of what other C and C++ based Apache projects do for their third-party dependencies.)
>>>
>>> Julian
>>>
>>>
>>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <sp...@gmail.com> wrote:
>>>>
>>>> That seems to be the repo with the LLVM code for implementing IWYU. I think
>>>> what we have in our repo is scripts ontop of that library. I'm wondering
>>>> where the scripts came from.
>>>>
>>>> Thanks,
>>>> Marc
>>>>
>>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>>>> wrote:
>>>>
>>>>> Hi Marc,
>>>>>
>>>>> How about this one for IWYU?
>>>>>
>>>>> https://github.com/include-what-you-use/include-what-you-use
>>>>> /blob/master/LICENSE.TXT
>>>>>
>>>>>
>>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>>>>
>>>>>> I double checked the libraries in thirdy_party. They adhere to Apache's
>>>>>> 3rd
>>>>>> party requirement as they are all apache 2 or opensourced by Google. The
>>>>>> only issue I saw was that IWYU has no documentation. Anyone know of its
>>>>>> source?
>>>>>>
>>>>>> Library
>>>>>>
>>>>>> Ver
>>>>>>
>>>>>> License
>>>>>>
>>>>>> Notes
>>>>>>
>>>>>> benchmark
>>>>>>
>>>>>> Apache 2.0
>>>>>>
>>>>>> cpplint
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> Header states that reuse is unconditional so long as the copyright header
>>>>>> stays intact.
>>>>>>
>>>>>> Farmhash
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>
>>>>>> gflags
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>
>>>>>> glog
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>>
>>>>>> gtest
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as COPYING file is preserved. See COPYING in
>>>>>> subprojects.
>>>>>>
>>>>>> gperftools
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as COPYING file is preserved.
>>>>>>
>>>>>> iwyu
>>>>>>
>>>>>>
>>>>>> No license present
>>>>>>
>>>>>> linenoise
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>
>>>>>> protobuf
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>
>>>>>> RE2
>>>>>>
>>>>>> Google
>>>>>>
>>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>>
>>>>>> tmb
>>>>>>
>>>>>> Apache 2.0
>>>>>>
>>>>>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
>>>>>> Inc.) and is distributed under the same license terms.
>>>>>>
>>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Harshad
>>>>>
>>>>>
>> --
>> Thanks,
>> Harshad
>>
--
Thanks,
Harshad
Re: release: third_party/
Posted by Julian Hyde <jh...@apache.org>.
Git sub-modules are useful, I agree. But for a source release, the goal is that someone should be able to download the source tar-ball and go:
$ curl -O …/apache-quickstep-x.x-incubating.tar.gz
$ tar xvfz apache-quickstep-x.x-incubating.tar.gz
$ cd apache-quickstep-x.x-incubating/build
$ ./download-thirdparty.sh
$ cmake etc.
I tried to find other Apache projects that use git submodules and see what they do for source releases. Not much luck. Maybe someone else can find something.
Also, since you’re relying on the latest googletest, protobuf and re2 your release will work today and will break at some point in the future. That’s not OK. A release can depend only on releases, not snapshots or live repositories.
Julian
> On Jan 5, 2017, at 11:03 AM, Harshad Deshmukh <ha...@cs.wisc.edu> wrote:
>
> Thanks for the review Julian.
>
> For some of the third party libraries used in Quickstep (e.g. googletest, protobuf and re2) we use the submodules feature of git. For such libraries, the developer has to initialize the submodule only once, which pulls code from the third party repo to the Quickstep third party directory.
>
> I don't know of a centralized repo for C++ projects. Does the git submodule method sound similar to the maven central approach you mentioned?
>
> On 01/05/2017 12:43 PM, Julian Hyde wrote:
>> I took a quick look at third_party and there don’t seem to be any binaries in there. That’s good. You definitely cannot include binaries in a source release.
>>
>> The more you can remove from third_party, the better. It doesn’t have to be done this release, but the less IP there is to review, the easier for everyone.
>>
>> Consider pulling from an external source the first time the developer builds in a sandbox, then apply patches. The patches will be the only thing checked in to quickstep. (My expertise is in Java projects, which these days get their dependencies from a Maven repo such as Maven central; I don’t know whether there is an equivalent place to pull C and C++ source code. Might be worth a review of what other C and C++ based Apache projects do for their third-party dependencies.)
>>
>> Julian
>>
>>
>>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <sp...@gmail.com> wrote:
>>>
>>> That seems to be the repo with the LLVM code for implementing IWYU. I think
>>> what we have in our repo is scripts ontop of that library. I'm wondering
>>> where the scripts came from.
>>>
>>> Thanks,
>>> Marc
>>>
>>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>>> wrote:
>>>
>>>> Hi Marc,
>>>>
>>>> How about this one for IWYU?
>>>>
>>>> https://github.com/include-what-you-use/include-what-you-use
>>>> /blob/master/LICENSE.TXT
>>>>
>>>>
>>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>>>
>>>>> I double checked the libraries in thirdy_party. They adhere to Apache's
>>>>> 3rd
>>>>> party requirement as they are all apache 2 or opensourced by Google. The
>>>>> only issue I saw was that IWYU has no documentation. Anyone know of its
>>>>> source?
>>>>>
>>>>> Library
>>>>>
>>>>> Ver
>>>>>
>>>>> License
>>>>>
>>>>> Notes
>>>>>
>>>>> benchmark
>>>>>
>>>>> Apache 2.0
>>>>>
>>>>> cpplint
>>>>>
>>>>> Google
>>>>>
>>>>> Header states that reuse is unconditional so long as the copyright header
>>>>> stays intact.
>>>>>
>>>>> Farmhash
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>
>>>>> gflags
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>
>>>>> glog
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>>
>>>>> gtest
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as COPYING file is preserved. See COPYING in
>>>>> subprojects.
>>>>>
>>>>> gperftools
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as COPYING file is preserved.
>>>>>
>>>>> iwyu
>>>>>
>>>>>
>>>>> No license present
>>>>>
>>>>> linenoise
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>
>>>>> protobuf
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>
>>>>> RE2
>>>>>
>>>>> Google
>>>>>
>>>>> No restrictions so long as LICENSE file is preserved.
>>>>>
>>>>> tmb
>>>>>
>>>>> Apache 2.0
>>>>>
>>>>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
>>>>> Inc.) and is distributed under the same license terms.
>>>>>
>>>>>
>>>> --
>>>> Thanks,
>>>> Harshad
>>>>
>>>>
>
> --
> Thanks,
> Harshad
>
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
Thanks for the review Julian.
For some of the third party libraries used in Quickstep (e.g.
googletest, protobuf and re2) we use the submodules feature of git. For
such libraries, the developer has to initialize the submodule only once,
which pulls code from the third party repo to the Quickstep third party
directory.
I don't know of a centralized repo for C++ projects. Does the git
submodule method sound similar to the maven central approach you mentioned?
On 01/05/2017 12:43 PM, Julian Hyde wrote:
> I took a quick look at third_party and there don\u2019t seem to be any binaries in there. That\u2019s good. You definitely cannot include binaries in a source release.
>
> The more you can remove from third_party, the better. It doesn\u2019t have to be done this release, but the less IP there is to review, the easier for everyone.
>
> Consider pulling from an external source the first time the developer builds in a sandbox, then apply patches. The patches will be the only thing checked in to quickstep. (My expertise is in Java projects, which these days get their dependencies from a Maven repo such as Maven central; I don\u2019t know whether there is an equivalent place to pull C and C++ source code. Might be worth a review of what other C and C++ based Apache projects do for their third-party dependencies.)
>
> Julian
>
>
>> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <sp...@gmail.com> wrote:
>>
>> That seems to be the repo with the LLVM code for implementing IWYU. I think
>> what we have in our repo is scripts ontop of that library. I'm wondering
>> where the scripts came from.
>>
>> Thanks,
>> Marc
>>
>> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
>> wrote:
>>
>>> Hi Marc,
>>>
>>> How about this one for IWYU?
>>>
>>> https://github.com/include-what-you-use/include-what-you-use
>>> /blob/master/LICENSE.TXT
>>>
>>>
>>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>>
>>>> I double checked the libraries in thirdy_party. They adhere to Apache's
>>>> 3rd
>>>> party requirement as they are all apache 2 or opensourced by Google. The
>>>> only issue I saw was that IWYU has no documentation. Anyone know of its
>>>> source?
>>>>
>>>> Library
>>>>
>>>> Ver
>>>>
>>>> License
>>>>
>>>> Notes
>>>>
>>>> benchmark
>>>>
>>>> Apache 2.0
>>>>
>>>> cpplint
>>>>
>>>> Google
>>>>
>>>> Header states that reuse is unconditional so long as the copyright header
>>>> stays intact.
>>>>
>>>> Farmhash
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>
>>>> gflags
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>
>>>> glog
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>>
>>>> gtest
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as COPYING file is preserved. See COPYING in
>>>> subprojects.
>>>>
>>>> gperftools
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as COPYING file is preserved.
>>>>
>>>> iwyu
>>>>
>>>>
>>>> No license present
>>>>
>>>> linenoise
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as LICENSE file is preserved.
>>>>
>>>> protobuf
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as LICENSE file is preserved.
>>>>
>>>> RE2
>>>>
>>>> Google
>>>>
>>>> No restrictions so long as LICENSE file is preserved.
>>>>
>>>> tmb
>>>>
>>>> Apache 2.0
>>>>
>>>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
>>>> Inc.) and is distributed under the same license terms.
>>>>
>>>>
>>> --
>>> Thanks,
>>> Harshad
>>>
>>>
--
Thanks,
Harshad
Re: release: third_party/
Posted by Julian Hyde <jh...@apache.org>.
I took a quick look at third_party and there don’t seem to be any binaries in there. That’s good. You definitely cannot include binaries in a source release.
The more you can remove from third_party, the better. It doesn’t have to be done this release, but the less IP there is to review, the easier for everyone.
Consider pulling from an external source the first time the developer builds in a sandbox, then apply patches. The patches will be the only thing checked in to quickstep. (My expertise is in Java projects, which these days get their dependencies from a Maven repo such as Maven central; I don’t know whether there is an equivalent place to pull C and C++ source code. Might be worth a review of what other C and C++ based Apache projects do for their third-party dependencies.)
Julian
> On Jan 5, 2017, at 9:43 AM, Marc Spehlmann <sp...@gmail.com> wrote:
>
> That seems to be the repo with the LLVM code for implementing IWYU. I think
> what we have in our repo is scripts ontop of that library. I'm wondering
> where the scripts came from.
>
> Thanks,
> Marc
>
> On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
> wrote:
>
>> Hi Marc,
>>
>> How about this one for IWYU?
>>
>> https://github.com/include-what-you-use/include-what-you-use
>> /blob/master/LICENSE.TXT
>>
>>
>> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>>
>>> I double checked the libraries in thirdy_party. They adhere to Apache's
>>> 3rd
>>> party requirement as they are all apache 2 or opensourced by Google. The
>>> only issue I saw was that IWYU has no documentation. Anyone know of its
>>> source?
>>>
>>> Library
>>>
>>> Ver
>>>
>>> License
>>>
>>> Notes
>>>
>>> benchmark
>>>
>>> Apache 2.0
>>>
>>> cpplint
>>>
>>> Google
>>>
>>> Header states that reuse is unconditional so long as the copyright header
>>> stays intact.
>>>
>>> Farmhash
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> gflags
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> glog
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING
>>>
>>> gtest
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved. See COPYING in
>>> subprojects.
>>>
>>> gperftools
>>>
>>> Google
>>>
>>> No restrictions so long as COPYING file is preserved.
>>>
>>> iwyu
>>>
>>>
>>> No license present
>>>
>>> linenoise
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> protobuf
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> RE2
>>>
>>> Google
>>>
>>> No restrictions so long as LICENSE file is preserved.
>>>
>>> tmb
>>>
>>> Apache 2.0
>>>
>>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
>>> Inc.) and is distributed under the same license terms.
>>>
>>>
>> --
>> Thanks,
>> Harshad
>>
>>
Re: release: third_party/
Posted by Marc Spehlmann <sp...@gmail.com>.
That seems to be the repo with the LLVM code for implementing IWYU. I think
what we have in our repo is scripts ontop of that library. I'm wondering
where the scripts came from.
Thanks,
Marc
On Thu, Jan 5, 2017 at 10:50 AM, Harshad Deshmukh <ha...@cs.wisc.edu>
wrote:
> Hi Marc,
>
> How about this one for IWYU?
>
> https://github.com/include-what-you-use/include-what-you-use
> /blob/master/LICENSE.TXT
>
>
> On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
>
>> I double checked the libraries in thirdy_party. They adhere to Apache's
>> 3rd
>> party requirement as they are all apache 2 or opensourced by Google. The
>> only issue I saw was that IWYU has no documentation. Anyone know of its
>> source?
>>
>> Library
>>
>> Ver
>>
>> License
>>
>> Notes
>>
>> benchmark
>>
>> Apache 2.0
>>
>> cpplint
>>
>> Google
>>
>> Header states that reuse is unconditional so long as the copyright header
>> stays intact.
>>
>> Farmhash
>>
>> Google
>>
>> No restrictions so long as COPYING file is preserved. See COPYING
>>
>> gflags
>>
>> Google
>>
>> No restrictions so long as COPYING file is preserved. See COPYING
>>
>> glog
>>
>> Google
>>
>> No restrictions so long as COPYING file is preserved. See COPYING
>>
>> gtest
>>
>> Google
>>
>> No restrictions so long as COPYING file is preserved. See COPYING in
>> subprojects.
>>
>> gperftools
>>
>> Google
>>
>> No restrictions so long as COPYING file is preserved.
>>
>> iwyu
>>
>>
>> No license present
>>
>> linenoise
>>
>> Google
>>
>> No restrictions so long as LICENSE file is preserved.
>>
>> protobuf
>>
>> Google
>>
>> No restrictions so long as LICENSE file is preserved.
>>
>> RE2
>>
>> Google
>>
>> No restrictions so long as LICENSE file is preserved.
>>
>> tmb
>>
>> Apache 2.0
>>
>> README: TMB is part of the Quickstep project (copyright Pivotal Software,
>> Inc.) and is distributed under the same license terms.
>>
>>
> --
> Thanks,
> Harshad
>
>
Re: release: third_party/
Posted by Harshad Deshmukh <ha...@cs.wisc.edu>.
Hi Marc,
How about this one for IWYU?
https://github.com/include-what-you-use/include-what-you-use/blob/master/LICENSE.TXT
On 01/05/2017 10:43 AM, Marc Spehlmann wrote:
> I double checked the libraries in thirdy_party. They adhere to Apache's 3rd
> party requirement as they are all apache 2 or opensourced by Google. The
> only issue I saw was that IWYU has no documentation. Anyone know of its
> source?
>
> Library
>
> Ver
>
> License
>
> Notes
>
> benchmark
>
> Apache 2.0
>
> cpplint
>
> Google
>
> Header states that reuse is unconditional so long as the copyright header
> stays intact.
>
> Farmhash
>
> Google
>
> No restrictions so long as COPYING file is preserved. See COPYING
>
> gflags
>
> Google
>
> No restrictions so long as COPYING file is preserved. See COPYING
>
> glog
>
> Google
>
> No restrictions so long as COPYING file is preserved. See COPYING
>
> gtest
>
> Google
>
> No restrictions so long as COPYING file is preserved. See COPYING in
> subprojects.
>
> gperftools
>
> Google
>
> No restrictions so long as COPYING file is preserved.
>
> iwyu
>
>
> No license present
>
> linenoise
>
> Google
>
> No restrictions so long as LICENSE file is preserved.
>
> protobuf
>
> Google
>
> No restrictions so long as LICENSE file is preserved.
>
> RE2
>
> Google
>
> No restrictions so long as LICENSE file is preserved.
>
> tmb
>
> Apache 2.0
>
> README: TMB is part of the Quickstep project (copyright Pivotal Software,
> Inc.) and is distributed under the same license terms.
>
--
Thanks,
Harshad