Re: Install some 3rdparty packages needed for building Mesos modules

[Evers Benno <be...@yandex-team.ru>]

I just wanted to note that there are some existing packages doing this
already.

For example the thunderbird-dev package on ubuntu (>= trusty) has
several private library headers in e.g. /usr/include/thunderbird/gmock
or /usr/include/thunderbird/google/protobuf

On 27.01.2016 16:28, Kapil Arya wrote:
> On Wed, Jan 27, 2016 at 8:04 AM, Alexander Rojas
> <al...@mesosphere.io> wrote:
>> While valid, I feel that is a very un-unix way of doing this. There are a couple of ideas from me here.
>>
>> 1. We should separate between a dev release and a normal release. The dev release will include headers for people who want to build modules and other mesos stuff, and the normal release only includes the executables.
> 
> The dev and normal releases are from the packaging point of view, when
> someone installs pre-compiled Mesos binaries. Here we are talking
> about users who run `./configure; make; make install`. For packaging,
> I totally agree that we shouldn't bundle the 3rdparty stuff (and
> distros won't let us do that anyways).
> 
>> 2. When building Mesos, we should avoid exporting unnecessary symbols. This happens to me when building a framework which used a newer version of boost, but when trying to link to mesos, it was taken boost symbols from Mesos, rather annoying (I gave up at the end). I had this problem in the past but I just cannot remember how it was solved.
> 
> True. The boost problem that you encountered is exactly the kind of
> problem this effort is trying to address.
>>
>> With respect to Alex issue, I don’t see them as much of a problem as long as we keep the standard behavior of the `--prefix` flag, which allows installation of different versions in different directores.
> 
> Yup, we are not proposing to change any semantics of --prefix at all.
> 
>>
>>
>>> On 26 Jan 2016, at 19:54, Kapil Arya <ka...@mesosphere.io> wrote:
>>>
>>> Thanks for the feedback, Alex! I have inlined my responses.
>>>
>>>> First, I think we should support two use cases: installing 3rdparty
>>>> packages and exposing them in the local build. As a Mesos (module)
>>>> developer, I may have different versions of Mesos on my machine and I
>>>> install neither of them to avoid conflicts. If I develop a module for a
>>>> particular Mesos version (i.e. build), I would like to use artifacts of
>>>> that build without installing them anywhere.
>>>
>>> That's a valid use case. How about "installing" the 3rdparty packages
>>> by setting their $PREFIX to $(top_builddir)/3rdparty? That way, one
>>> can append "-I$(top_builddir)/3rdparty/include" to CPPFLAGS and
>>> "$(top_builddir)/3rdparty/lib" to their LD_LIBRARY_PATH when building
>>> modules? We can also update libprocess/Mesos to also use these
>>> locations instead of passing a dozen "-I" and "-L" flags during
>>> compilation. I am guessing this won't be too intrusive to the overall
>>> build system.
>>>
>>>> Second, additionally 3rdparty packages, how about modules we provide with
>>>> Mesos, e.g. FixedResourceEstimator? Also if in the future we decide to
>>>> refactor default implementations into modules (e.g. hierarchical
>>>> allocator), we should install them somewhere.
>>>
>>> We have a ticket (https://issues.apache.org/jira/browse/MESOS-1940)
>>> for it :-). We should just install them in $(pkglibdir). That's the
>>> default location for distro packaging as well. That's not too hard to
>>> do. Just that we need to decide which modules should be installed.
>>>
>>> Best,
>>> Kapil
>>>
>>>
>>>> On Wed, Jan 20, 2016 at 8:09 AM, Kapil Arya <ka...@mesosphere.io> wrote:
>>>>
>>>>> On Wed, Jan 20, 2016 at 2:04 AM, Marco Massenzio <m....@gmail.com>
>>>>> wrote:
>>>>>> Great thinking, Kapil!
>>>>>> (I'm one who got the headache :)
>>>>>>
>>>>>> However, having recently gone through the effort of having to figure out
>>>>> it
>>>>>> all, my +1 goes for *good documentation* of what is necessary.
>>>>>
>>>>> Totally with you on this :)
>>>>>
>>>>>
>>>>>> When installing stuff / magic happening behind the scenes, it is always
>>>>>> difficult to ensure it works on all "supported" platforms (and let's not
>>>>>> even go into non-supported ones) and we would also run the risk that
>>>>> future
>>>>>> changes may inadvertently break it.
>>>>>>
>>>>>> Bear also in mind that folks (who may not be using the --prefix) may be
>>>>>> "surprised" to find incompatible/unexpected versions of
>>>>> glog/protobuf/etc.
>>>>>> in the /usr/local system dir.
>>>>>
>>>>> This is the reason why we want to put it inside Mesos "owned"
>>>>> directories (e.g., /usr/local/include/mesos/3rdparty,
>>>>> /usr/lib64/mesos/3rdparty) so that this whole operation is side-effect
>>>>> free for other applications/packages installed on the system.
>>>>>
>>>>>> We could also provide "exemplary scripts" that automate (most of) the
>>>>>> tedious work and example build files, alongside documentation.
>>>>>> If folks agree that this is a desirable alternative, I'm happy to help
>>>>> out
>>>>>> - as mentioned, I've recently been through this, so memory is still
>>>>> fresh.
>>>>>
>>>>> I think several of us have developed such scripts by now. However, the
>>>>> problem is maintainability as they get out-of-sync whenever a 3rdparty
>>>>> component is updated in Mesos :-/.
>>>>>
>>>>>>
>>>>>> My 2c.
>>>>>>
>>>>>> --
>>>>>> *Marco Massenzio*
>>>>>> http://codetrips.com
>>>>>>
>>>>>> On Tue, Jan 19, 2016 at 10:22 PM, Kapil Arya <ka...@mesosphere.io>
>>>>> wrote:
>>>>>>
>>>>>>> On Tue, Jan 19, 2016 at 11:58 PM, James Peach <jo...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> On Jan 19, 2016, at 2:03 PM, Kapil Arya <ka...@mesosphere.io> wrote:
>>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I wanted to get your opinion on installing the 3rdparty packages
>>>>> glog,
>>>>>>>>> protobuf, boost and picojson[1] when installing Mesos itself. These
>>>>>>>>> packages are required to build Mesos modules.
>>>>>>>>
>>>>>>>> An alternative approach could be to hide these headers so they are
>>>>>>> internal to Mesos and not incidentally required by innocent modules.
>>>>> IIRC
>>>>>>> this should be fairly easy for picojson, but (much) harder for glog,
>>>>>>> protobuf and boost.
>>>>>>>
>>>>>>> That's a lot of work and super hard to maintain IMHO :(.
>>>>>>>
>>>>>>>>> Currently, a module write has to manually install these 3rdparty
>>>>>>>>> packages, either system-wide or locally, and update the compilation
>>>>>>>>> flags such as CPPFLAGS to point to the installation which is
>>>>>>>>> error-prone. Further, one might have a system-wide installation with
>>>>>>>>> the wrong package version, causing even more headache.
>>>>>>>>
>>>>>>>> If you are looking at this, could you please also look at these:
>>>>>>>>        https://issues.apache.org/jira/browse/MESOS-2537
>>>>>>>>        https://issues.apache.org/jira/browse/MESOS-4096
>>>>>>>>
>>>>>>>> MESOS-2537 provides consistent behaviour for building against
>>>>> optionally
>>>>>>> bundled dependencies (and fixes the --enable-foo semantics).
>>>>>>>
>>>>>>> I'll take a look at this one.
>>>>>>>
>>>>>>>> MESOS-4096 makes it impossible to run stout tests against a protobuf
>>>>>>> that is not version 2.5.0.
>>>>>>>
>>>>>>> At some point, AlexR and I tried to work on it, but with the stout
>>>>>>> directory structure, it got harder to fix then it seemed at first.
>>>>>>>
>>>>>>>>
>>>>>>>>> The proposal here is to install these 3rdparty packages when
>>>>>>>>> installing Mesos. To avoid any conflicts with system-wide or local
>>>>>>>>> installation, we can install them as follows:
>>>>>>>>>
>>>>>>>>> ${PREFIX}/include/mesos/3rdparty -- for header files
>>>>>>>>> ${PREFIX}/<LIBDIR>/mesos/3rdparty -- for library files (LIBDIR can be
>>>>>>>>> lib or lib64 depending upon the installation)
>>>>>>>>>
>>>>>>>>> where PREFIX refers to the `--prefix` flag for Mesos configure
>>>>> script.
>>>>>>>>>
>>>>>>>>> We would then update `mesos.pc` with the correct flags so that a
>>>>>>>>> module write can simply use `pkg-config` to get all the required
>>>>>>>>> flags.
>>>>>>>>>
>>>>>>>>> I have created an issue
>>>>>>>>> https://issues.apache.org/jira/browse/MESOS-4434 to track this.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Kapil
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1]: picojson is currently installed in ${PREFIX}/include. See
>>>>>>>>> https://issues.apache.org/jira/browse/MESOS-3909
>>>>>>>>
>>>>>>>
>>>>>
>>