You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Tomas Barton <ba...@gmail.com> on 2017/11/03 15:23:11 UTC
DC/OS (Mesos) portability
Hi,
I'd like to contribute to DC/OS with a Debian/Suse/... support.
Surprisingly on Debian most of the compatibility issues could be solved by
a sequence of symlinks.
Why Mesos dev list? :)
Currently the biggest issue is connected to distributing libmesos-bundle
tar archive, which contain the libmesos.so library and several others. The
library is dynamically linked with certain libcurl, libssl, libsvn etc.
that might differ between distributions.
I can think of a few solutions:
1. Compile Mesos (master and agent) using static build (which as I
understood aren't currently fully supported/propagated).
2. Generate bundle during automatic builds for certain supported
distributions.
3. Include libmesos in standard distribution channels - rpm, deb packages
(that might take same time).
The last solution would be the best, but Mesos release cycle is very
different from distributions release cycle. It might be complicated to
synchronize.
I coudn't find scripts for generating libmesos-bundle, but it's a archive
with libraries from build server, e.g.
https://downloads.mesosphere.io/libmesos-bundle/libmesos-bundle-1.10-1.4-63e0814.tar.gz
(32MB).
So the question is, whether Mesos website could provide prebuild libmesos
bundle for each release and platform, that could be afterwards used e.g. in
DC/OS packages?
Last issue might be connected to an executor that eventually might need OS
family ENV variable with OS release version, so that it can fetch
corresponding libbundle archive. Such information is typically parsed from
`uname -a` or `lsb_release -sri` (if available). This way DC/OS could be
running on a cluster with diverse OS versions/distributions.
Thanks for your time! I'd like to hear your opinion.
Regards,
Tomas Barton
Re: DC/OS (Mesos) portability
Posted by Judith Malnick <jm...@mesosphere.io>.
That's the one, thank you!
On Tue, Nov 21, 2017 at 3:56 PM, Tomas Barton <ba...@gmail.com>
wrote:
> Hi,
>
> there ain't much differences between Debian stable and Ubuntu LTS. Ubuntu
> 18.04 should be pretty much the same as Debian 9 (as far as Mesos
> dependencies concerned).
>
> Currently in DC/OS if you install a framework such as Elastic it will
> fetch incorrect libmesos-bundle (built for CentOS 7). libmesos-1.4.0.so
> will be dynamically linked to libssl.so.1.0.0 while in Debian 9 there's
> libssl.so.1.0.2. Similar problem for libcurl and few others. If DC/OS
> wouldn't bundle libcurl and simply require appropriate distribution package
> during installation, there wouldn't be any compatibility issue. Or is there
> a need for using different curl versions in each DC/OS package?
>
> DC/OS 1.11 will no longer overwrite LD_LIBRARY_PATH in Mesos executor,
> which eliminates one of current issues (Thx Alex!)
>
> In fact if libmesos would be loaded from system path instead of fetching a
> resource from an URI during task deployment we wouldn't need to include
> libmesos in the library bundle at all.
>
> I think we can move to discussion to an approprite DC/OS ticket and
> resolve issues on DC/OS side first, then come back to Mesos if necessary.
>
> Probably https://jira.mesosphere.com/browse/DCOS_OSS-25 ?
>
> Tomas
>
> On 22 November 2017 at 00:01, Alex Rukletsov <al...@mesosphere.com> wrote:
>
>> I think Tomas means Mesos dependencies, like libcurl, and not libmesos. If
>> I understand him correctly, he is saying that part of Mesos dependencies
>> is
>> not distributed with Mesos binaries, and, if not included into a
>> distribution, might complicate installation process.
>>
>> On Fri, Nov 3, 2017 at 8:54 PM, Joseph Wu <jo...@mesosphere.io> wrote:
>>
>> > It isn't clear to me how DC/OS would benefit from (ongoing) work to
>> > create/push Mesos packages. DC/OS downloads and builds all of its
>> > component parts from source.
>> >
>> > Also, we (Mesos devs) are hoping to get more frameworks to move away
>> from
>> > using libmesos (including the API shims), in favor of using the HTTP
>> APIs
>> > instead. So we have a dis-incentive to provide a libmesos bundle.
>> >
>> > On Fri, Nov 3, 2017 at 8:23 AM, Tomas Barton <ba...@gmail.com>
>> > wrote:
>> >
>> > > Hi,
>> > >
>> > > I'd like to contribute to DC/OS with a Debian/Suse/... support.
>> > > Surprisingly on Debian most of the compatibility issues could be
>> solved
>> > by
>> > > a sequence of symlinks.
>> > >
>> > > Why Mesos dev list? :)
>> > >
>> > > Currently the biggest issue is connected to distributing
>> libmesos-bundle
>> > > tar archive, which contain the libmesos.so library and several others.
>> > The
>> > > library is dynamically linked with certain libcurl, libssl, libsvn
>> etc.
>> > > that might differ between distributions.
>> > >
>> > > I can think of a few solutions:
>> > > 1. Compile Mesos (master and agent) using static build (which as I
>> > > understood aren't currently fully supported/propagated).
>> > > 2. Generate bundle during automatic builds for certain supported
>> > > distributions.
>> > > 3. Include libmesos in standard distribution channels - rpm, deb
>> > packages
>> > > (that might take same time).
>> > >
>> > > The last solution would be the best, but Mesos release cycle is very
>> > > different from distributions release cycle. It might be complicated to
>> > > synchronize.
>> > >
>> > > I coudn't find scripts for generating libmesos-bundle, but it's a
>> archive
>> > > with libraries from build server, e.g.
>> > > https://downloads.mesosphere.io/libmesos-bundle/libmesos-
>> > > bundle-1.10-1.4-63e0814.tar.gz
>> > > (32MB).
>> > >
>> > > So the question is, whether Mesos website could provide prebuild
>> libmesos
>> > > bundle for each release and platform, that could be afterwards used
>> e.g.
>> > in
>> > > DC/OS packages?
>> > >
>> > > Last issue might be connected to an executor that eventually might
>> need
>> > OS
>> > > family ENV variable with OS release version, so that it can fetch
>> > > corresponding libbundle archive. Such information is typically parsed
>> > from
>> > > `uname -a` or `lsb_release -sri` (if available). This way DC/OS could
>> be
>> > > running on a cluster with diverse OS versions/distributions.
>> > >
>> > > Thanks for your time! I'd like to hear your opinion.
>> > >
>> > > Regards,
>> > > Tomas Barton
>> > >
>> >
>>
>
>
--
Judith Malnick
Community Manager
310-709-1517
Re: DC/OS (Mesos) portability
Posted by Tomas Barton <ba...@gmail.com>.
Hi,
there ain't much differences between Debian stable and Ubuntu LTS. Ubuntu
18.04 should be pretty much the same as Debian 9 (as far as Mesos
dependencies concerned).
Currently in DC/OS if you install a framework such as Elastic it will fetch
incorrect libmesos-bundle (built for CentOS 7). libmesos-1.4.0.so will be
dynamically linked to libssl.so.1.0.0 while in Debian 9 there's
libssl.so.1.0.2. Similar problem for libcurl and few others. If DC/OS
wouldn't bundle libcurl and simply require appropriate distribution package
during installation, there wouldn't be any compatibility issue. Or is there
a need for using different curl versions in each DC/OS package?
DC/OS 1.11 will no longer overwrite LD_LIBRARY_PATH in Mesos executor,
which eliminates one of current issues (Thx Alex!)
In fact if libmesos would be loaded from system path instead of fetching a
resource from an URI during task deployment we wouldn't need to include
libmesos in the library bundle at all.
I think we can move to discussion to an approprite DC/OS ticket and resolve
issues on DC/OS side first, then come back to Mesos if necessary.
Probably https://jira.mesosphere.com/browse/DCOS_OSS-25 ?
Tomas
On 22 November 2017 at 00:01, Alex Rukletsov <al...@mesosphere.com> wrote:
> I think Tomas means Mesos dependencies, like libcurl, and not libmesos. If
> I understand him correctly, he is saying that part of Mesos dependencies is
> not distributed with Mesos binaries, and, if not included into a
> distribution, might complicate installation process.
>
> On Fri, Nov 3, 2017 at 8:54 PM, Joseph Wu <jo...@mesosphere.io> wrote:
>
> > It isn't clear to me how DC/OS would benefit from (ongoing) work to
> > create/push Mesos packages. DC/OS downloads and builds all of its
> > component parts from source.
> >
> > Also, we (Mesos devs) are hoping to get more frameworks to move away from
> > using libmesos (including the API shims), in favor of using the HTTP APIs
> > instead. So we have a dis-incentive to provide a libmesos bundle.
> >
> > On Fri, Nov 3, 2017 at 8:23 AM, Tomas Barton <ba...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I'd like to contribute to DC/OS with a Debian/Suse/... support.
> > > Surprisingly on Debian most of the compatibility issues could be solved
> > by
> > > a sequence of symlinks.
> > >
> > > Why Mesos dev list? :)
> > >
> > > Currently the biggest issue is connected to distributing
> libmesos-bundle
> > > tar archive, which contain the libmesos.so library and several others.
> > The
> > > library is dynamically linked with certain libcurl, libssl, libsvn
> etc.
> > > that might differ between distributions.
> > >
> > > I can think of a few solutions:
> > > 1. Compile Mesos (master and agent) using static build (which as I
> > > understood aren't currently fully supported/propagated).
> > > 2. Generate bundle during automatic builds for certain supported
> > > distributions.
> > > 3. Include libmesos in standard distribution channels - rpm, deb
> > packages
> > > (that might take same time).
> > >
> > > The last solution would be the best, but Mesos release cycle is very
> > > different from distributions release cycle. It might be complicated to
> > > synchronize.
> > >
> > > I coudn't find scripts for generating libmesos-bundle, but it's a
> archive
> > > with libraries from build server, e.g.
> > > https://downloads.mesosphere.io/libmesos-bundle/libmesos-
> > > bundle-1.10-1.4-63e0814.tar.gz
> > > (32MB).
> > >
> > > So the question is, whether Mesos website could provide prebuild
> libmesos
> > > bundle for each release and platform, that could be afterwards used
> e.g.
> > in
> > > DC/OS packages?
> > >
> > > Last issue might be connected to an executor that eventually might need
> > OS
> > > family ENV variable with OS release version, so that it can fetch
> > > corresponding libbundle archive. Such information is typically parsed
> > from
> > > `uname -a` or `lsb_release -sri` (if available). This way DC/OS could
> be
> > > running on a cluster with diverse OS versions/distributions.
> > >
> > > Thanks for your time! I'd like to hear your opinion.
> > >
> > > Regards,
> > > Tomas Barton
> > >
> >
>
Re: DC/OS (Mesos) portability
Posted by Alex Rukletsov <al...@mesosphere.com>.
I think Tomas means Mesos dependencies, like libcurl, and not libmesos. If
I understand him correctly, he is saying that part of Mesos dependencies is
not distributed with Mesos binaries, and, if not included into a
distribution, might complicate installation process.
On Fri, Nov 3, 2017 at 8:54 PM, Joseph Wu <jo...@mesosphere.io> wrote:
> It isn't clear to me how DC/OS would benefit from (ongoing) work to
> create/push Mesos packages. DC/OS downloads and builds all of its
> component parts from source.
>
> Also, we (Mesos devs) are hoping to get more frameworks to move away from
> using libmesos (including the API shims), in favor of using the HTTP APIs
> instead. So we have a dis-incentive to provide a libmesos bundle.
>
> On Fri, Nov 3, 2017 at 8:23 AM, Tomas Barton <ba...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'd like to contribute to DC/OS with a Debian/Suse/... support.
> > Surprisingly on Debian most of the compatibility issues could be solved
> by
> > a sequence of symlinks.
> >
> > Why Mesos dev list? :)
> >
> > Currently the biggest issue is connected to distributing libmesos-bundle
> > tar archive, which contain the libmesos.so library and several others.
> The
> > library is dynamically linked with certain libcurl, libssl, libsvn etc.
> > that might differ between distributions.
> >
> > I can think of a few solutions:
> > 1. Compile Mesos (master and agent) using static build (which as I
> > understood aren't currently fully supported/propagated).
> > 2. Generate bundle during automatic builds for certain supported
> > distributions.
> > 3. Include libmesos in standard distribution channels - rpm, deb
> packages
> > (that might take same time).
> >
> > The last solution would be the best, but Mesos release cycle is very
> > different from distributions release cycle. It might be complicated to
> > synchronize.
> >
> > I coudn't find scripts for generating libmesos-bundle, but it's a archive
> > with libraries from build server, e.g.
> > https://downloads.mesosphere.io/libmesos-bundle/libmesos-
> > bundle-1.10-1.4-63e0814.tar.gz
> > (32MB).
> >
> > So the question is, whether Mesos website could provide prebuild libmesos
> > bundle for each release and platform, that could be afterwards used e.g.
> in
> > DC/OS packages?
> >
> > Last issue might be connected to an executor that eventually might need
> OS
> > family ENV variable with OS release version, so that it can fetch
> > corresponding libbundle archive. Such information is typically parsed
> from
> > `uname -a` or `lsb_release -sri` (if available). This way DC/OS could be
> > running on a cluster with diverse OS versions/distributions.
> >
> > Thanks for your time! I'd like to hear your opinion.
> >
> > Regards,
> > Tomas Barton
> >
>
Re: DC/OS (Mesos) portability
Posted by Joseph Wu <jo...@mesosphere.io>.
It isn't clear to me how DC/OS would benefit from (ongoing) work to
create/push Mesos packages. DC/OS downloads and builds all of its
component parts from source.
Also, we (Mesos devs) are hoping to get more frameworks to move away from
using libmesos (including the API shims), in favor of using the HTTP APIs
instead. So we have a dis-incentive to provide a libmesos bundle.
On Fri, Nov 3, 2017 at 8:23 AM, Tomas Barton <ba...@gmail.com> wrote:
> Hi,
>
> I'd like to contribute to DC/OS with a Debian/Suse/... support.
> Surprisingly on Debian most of the compatibility issues could be solved by
> a sequence of symlinks.
>
> Why Mesos dev list? :)
>
> Currently the biggest issue is connected to distributing libmesos-bundle
> tar archive, which contain the libmesos.so library and several others. The
> library is dynamically linked with certain libcurl, libssl, libsvn etc.
> that might differ between distributions.
>
> I can think of a few solutions:
> 1. Compile Mesos (master and agent) using static build (which as I
> understood aren't currently fully supported/propagated).
> 2. Generate bundle during automatic builds for certain supported
> distributions.
> 3. Include libmesos in standard distribution channels - rpm, deb packages
> (that might take same time).
>
> The last solution would be the best, but Mesos release cycle is very
> different from distributions release cycle. It might be complicated to
> synchronize.
>
> I coudn't find scripts for generating libmesos-bundle, but it's a archive
> with libraries from build server, e.g.
> https://downloads.mesosphere.io/libmesos-bundle/libmesos-
> bundle-1.10-1.4-63e0814.tar.gz
> (32MB).
>
> So the question is, whether Mesos website could provide prebuild libmesos
> bundle for each release and platform, that could be afterwards used e.g. in
> DC/OS packages?
>
> Last issue might be connected to an executor that eventually might need OS
> family ENV variable with OS release version, so that it can fetch
> corresponding libbundle archive. Such information is typically parsed from
> `uname -a` or `lsb_release -sri` (if available). This way DC/OS could be
> running on a cluster with diverse OS versions/distributions.
>
> Thanks for your time! I'd like to hear your opinion.
>
> Regards,
> Tomas Barton
>