You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Tomek Janiszewski <ja...@gmail.com> on 2018/03/19 15:24:14 UTC
Re: Introducing `support/mesos-build.sh`
Dockerfile for ARM with CMake support https://reviews.apache.org/r/66138/
pon., 12 lut 2018 o 15:41 użytkownik Tomek Janiszewski <ja...@gmail.com>
napisał:
> How can I use it for our ARM CI?
>
> JOBS=16 OS=arm64v8/ubuntu:16.04 ./support/mesos-build.sh
> + set -e
> + set -o pipefail
> + : arm64v8/ubuntu:16.04
> + : autotools
> + : gcc
> + : '--verbose --disable-libtool-wrappers'
> + : 'GLOG_v=1 MESOS_VERBOSE=1'
> + : 16
> ++ git rev-parse --show-toplevel
> + MESOS_DIR=/home/janisz/mesos
> ++ git diff-index --quiet HEAD --
> + docker run --rm -v /home/janisz/mesos:/SRC:Z -e BUILDTOOL=autotools -e
> COMPILER=gcc -e 'CONFIGURATION=--verbose --disable-libtool-wrappers' -e
> 'ENVIRONMENT=GLOG_v=1 MESOS_VERBOSE=1' -e JOBS=16
> mesos/mesos-build:arm64v8/ubuntu-16.04
> docker: invalid reference format.
> See 'docker run --help'.
>
>
> czw., 8 lut 2018 o 07:38 użytkownik Michael Park <mp...@apache.org>
> napisał:
>
>> The first run looks good!
>> https://builds.apache.org/job/Mesos-Buildbot/4890/
>>
>> [image: Screen Shot 2018-02-07 at 10.30.51 PM.png]
>> On Wed, Feb 7, 2018 at 8:39 PM Michael Park <mp...@apache.org> wrote:
>>
>> > Yep, Just landed! Waiting for
>> https://builds.apache.org/job/Mesos-Buildbot to
>> > pick it up.
>> >
>> > On Wed, Feb 7, 2018 at 8:27 PM Vinod Kone <vi...@apache.org> wrote:
>> >
>> >> Yay, thanks MPark! Has the change landed already?
>> >>
>> >> On Wed, Feb 7, 2018 at 8:23 PM, Michael Park <mp...@apache.org> wrote:
>> >>
>> >> > Many of you probably know that we currently have
>> >> `support/docker-build.sh`
>> >> > to power our CI for our various configurations. One of the problems
>> for
>> >> us
>> >> > has been that we create a `Dockerfile` ad-hoc and invoke `docker
>> build`
>> >> > with it. This is very inefficient and also leads to flaky issues
>> around
>> >> > `apt-get install`.
>> >> >
>> >> > I've introduced `support/mesos-build.sh` which operates off of docker
>> >> > images hosted on Dockerhub instead, and should aid in bringing us
>> faster
>> >> > and more stable CI results!
>> >> >
>> >> > As a bonus, we now also test Clang on the CentOS 7!
>> >> >
>> >> > Thanks,
>> >> >
>> >> > MPark
>> >> >
>> >>
>> >
>>
>
Re: Introducing `support/mesos-build.sh`
Posted by Tomek Janiszewski <ja...@gmail.com>.
Other jobs finish in 50 min but they do clang/gcc and cmake/auttools checks
so total build time takes 2h.
I hope we can make next release with ARM binaries. Tere was some proposal
of making official Mesos binaries (debs, rpms etc) so we can plug in this
process.
wt., 20 mar 2018 o 15:35 użytkownik Ed Vielmetti <ed...@packet.net> napisał:
> Congratulations Tomek and good job.
>
> How does build time on this system compare to other systems?
>
> When you have something packaged for release I'd love to give it a try.
>
> On Tue, Mar 20, 2018 at 8:54 AM, Tomek Janiszewski <ja...@gmail.com>
> wrote:
>
>> Thank you!
>>
>> ARM builds are passing right now [0]. We are using ubuntu-16.04-arm image
>> [1]. I think from next release we can consider ARM as a supported platform!
>>
>> 0:
>> https://builds.apache.org/job/Mesos-Buildbot-ARM/BUILDTOOL=autotools,COMPILER=gcc,CONFIGURATION=--verbose%20--disable-libtool-wrappers%20--disable-java%20--disable-python,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1%20%20MESOS_TEST_AWAIT_TIMEOUT=60secs%20JOBS=16,label_exp=arm/453/
>> 1: https://hub.docker.com/r/mesos/mesos-build/tags/
>>
>> wt., 20 mar 2018 o 13:30 użytkownik Benjamin Bannier <
>> benjamin.bannier@mesosphere.io> napisał:
>>
>>> Done.
>>>
>>> > On Mar 20, 2018, at 10:46 AM, Tomek Janiszewski <ja...@gmail.com>
>>> wrote:
>>> >
>>> > Thanks for merging. Can we push this image to dockerhub tagged as
>>> > ubuntu-16.04-arm? Then we will need to update CI configuration to:
>>> > JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=cmake COMPILER=gcc
>>> > CONFIGURATION='--disable-java --disable-python
>>> --disable-libtool-wrappers'
>>> > ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ../mesos-build.sh
>>> > or
>>> > JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=autotools COMPILER=gcc
>>> > CONFIGURATION='--disable-java --disable-python
>>> --disable-libtool-wrappers'
>>> > ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ./support/mesos-build.sh
>>> >
>>> >
>>> > pon., 19 mar 2018 o 16:24 użytkownik Tomek Janiszewski <
>>> janiszt@gmail.com>
>>> > napisał:
>>> >
>>> >> Dockerfile for ARM with CMake support
>>> https://reviews.apache.org/r/66138/
>>> >>
>>> >> pon., 12 lut 2018 o 15:41 użytkownik Tomek Janiszewski <
>>> janiszt@gmail.com>
>>> >> napisał:
>>> >>
>>> >>> How can I use it for our ARM CI?
>>> >>>
>>> >>> JOBS=16 OS=arm64v8/ubuntu:16.04 ./support/mesos-build.sh
>>> >>> + set -e
>>> >>> + set -o pipefail
>>> >>> + : arm64v8/ubuntu:16.04
>>> >>> + : autotools
>>> >>> + : gcc
>>> >>> + : '--verbose --disable-libtool-wrappers'
>>> >>> + : 'GLOG_v=1 MESOS_VERBOSE=1'
>>> >>> + : 16
>>> >>> ++ git rev-parse --show-toplevel
>>> >>> + MESOS_DIR=/home/janisz/mesos
>>> >>> ++ git diff-index --quiet HEAD --
>>> >>> + docker run --rm -v /home/janisz/mesos:/SRC:Z -e
>>> BUILDTOOL=autotools -e
>>> >>> COMPILER=gcc -e 'CONFIGURATION=--verbose --disable-libtool-wrappers'
>>> -e
>>> >>> 'ENVIRONMENT=GLOG_v=1 MESOS_VERBOSE=1' -e JOBS=16
>>> >>> mesos/mesos-build:arm64v8/ubuntu-16.04
>>> >>> docker: invalid reference format.
>>> >>> See 'docker run --help'.
>>> >>>
>>> >>>
>>> >>> czw., 8 lut 2018 o 07:38 użytkownik Michael Park <mp...@apache.org>
>>> >>> napisał:
>>> >>>
>>> >>>> The first run looks good!
>>> >>>> https://builds.apache.org/job/Mesos-Buildbot/4890/
>>> >>>>
>>> >>>> [image: Screen Shot 2018-02-07 at 10.30.51 PM.png]
>>> >>>> On Wed, Feb 7, 2018 at 8:39 PM Michael Park <mp...@apache.org>
>>> wrote:
>>> >>>>
>>> >>>>> Yep, Just landed! Waiting for
>>> >>>> https://builds.apache.org/job/Mesos-Buildbot to
>>> >>>>> pick it up.
>>> >>>>>
>>> >>>>> On Wed, Feb 7, 2018 at 8:27 PM Vinod Kone <vi...@apache.org>
>>> >>>> wrote:
>>> >>>>>
>>> >>>>>> Yay, thanks MPark! Has the change landed already?
>>> >>>>>>
>>> >>>>>> On Wed, Feb 7, 2018 at 8:23 PM, Michael Park <mp...@apache.org>
>>> >>>> wrote:
>>> >>>>>>
>>> >>>>>>> Many of you probably know that we currently have
>>> >>>>>> `support/docker-build.sh`
>>> >>>>>>> to power our CI for our various configurations. One of the
>>> problems
>>> >>>> for
>>> >>>>>> us
>>> >>>>>>> has been that we create a `Dockerfile` ad-hoc and invoke `docker
>>> >>>> build`
>>> >>>>>>> with it. This is very inefficient and also leads to flaky issues
>>> >>>> around
>>> >>>>>>> `apt-get install`.
>>> >>>>>>>
>>> >>>>>>> I've introduced `support/mesos-build.sh` which operates off of
>>> >>>> docker
>>> >>>>>>> images hosted on Dockerhub instead, and should aid in bringing us
>>> >>>> faster
>>> >>>>>>> and more stable CI results!
>>> >>>>>>>
>>> >>>>>>> As a bonus, we now also test Clang on the CentOS 7!
>>> >>>>>>>
>>> >>>>>>> Thanks,
>>> >>>>>>>
>>> >>>>>>> MPark
>>> >>>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>>
>>> >>>
>>>
>>>
>
>
> --
> Edward Vielmetti
> +1 734 330 2465 <(734)%20330-2465>
> ed@packet.net
> http://worksonarm.com
>
Re: Introducing `support/mesos-build.sh`
Posted by Tomek Janiszewski <ja...@gmail.com>.
Thank you!
ARM builds are passing right now [0]. We are using ubuntu-16.04-arm image
[1]. I think from next release we can consider ARM as a supported platform!
0:
https://builds.apache.org/job/Mesos-Buildbot-ARM/BUILDTOOL=autotools,COMPILER=gcc,CONFIGURATION=--verbose%20--disable-libtool-wrappers%20--disable-java%20--disable-python,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1%20%20MESOS_TEST_AWAIT_TIMEOUT=60secs%20JOBS=16,label_exp=arm/453/
1: https://hub.docker.com/r/mesos/mesos-build/tags/
wt., 20 mar 2018 o 13:30 użytkownik Benjamin Bannier <
benjamin.bannier@mesosphere.io> napisał:
> Done.
>
> > On Mar 20, 2018, at 10:46 AM, Tomek Janiszewski <ja...@gmail.com>
> wrote:
> >
> > Thanks for merging. Can we push this image to dockerhub tagged as
> > ubuntu-16.04-arm? Then we will need to update CI configuration to:
> > JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=cmake COMPILER=gcc
> > CONFIGURATION='--disable-java --disable-python
> --disable-libtool-wrappers'
> > ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ../mesos-build.sh
> > or
> > JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=autotools COMPILER=gcc
> > CONFIGURATION='--disable-java --disable-python
> --disable-libtool-wrappers'
> > ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ./support/mesos-build.sh
> >
> >
> > pon., 19 mar 2018 o 16:24 użytkownik Tomek Janiszewski <
> janiszt@gmail.com>
> > napisał:
> >
> >> Dockerfile for ARM with CMake support
> https://reviews.apache.org/r/66138/
> >>
> >> pon., 12 lut 2018 o 15:41 użytkownik Tomek Janiszewski <
> janiszt@gmail.com>
> >> napisał:
> >>
> >>> How can I use it for our ARM CI?
> >>>
> >>> JOBS=16 OS=arm64v8/ubuntu:16.04 ./support/mesos-build.sh
> >>> + set -e
> >>> + set -o pipefail
> >>> + : arm64v8/ubuntu:16.04
> >>> + : autotools
> >>> + : gcc
> >>> + : '--verbose --disable-libtool-wrappers'
> >>> + : 'GLOG_v=1 MESOS_VERBOSE=1'
> >>> + : 16
> >>> ++ git rev-parse --show-toplevel
> >>> + MESOS_DIR=/home/janisz/mesos
> >>> ++ git diff-index --quiet HEAD --
> >>> + docker run --rm -v /home/janisz/mesos:/SRC:Z -e BUILDTOOL=autotools
> -e
> >>> COMPILER=gcc -e 'CONFIGURATION=--verbose --disable-libtool-wrappers' -e
> >>> 'ENVIRONMENT=GLOG_v=1 MESOS_VERBOSE=1' -e JOBS=16
> >>> mesos/mesos-build:arm64v8/ubuntu-16.04
> >>> docker: invalid reference format.
> >>> See 'docker run --help'.
> >>>
> >>>
> >>> czw., 8 lut 2018 o 07:38 użytkownik Michael Park <mp...@apache.org>
> >>> napisał:
> >>>
> >>>> The first run looks good!
> >>>> https://builds.apache.org/job/Mesos-Buildbot/4890/
> >>>>
> >>>> [image: Screen Shot 2018-02-07 at 10.30.51 PM.png]
> >>>> On Wed, Feb 7, 2018 at 8:39 PM Michael Park <mp...@apache.org> wrote:
> >>>>
> >>>>> Yep, Just landed! Waiting for
> >>>> https://builds.apache.org/job/Mesos-Buildbot to
> >>>>> pick it up.
> >>>>>
> >>>>> On Wed, Feb 7, 2018 at 8:27 PM Vinod Kone <vi...@apache.org>
> >>>> wrote:
> >>>>>
> >>>>>> Yay, thanks MPark! Has the change landed already?
> >>>>>>
> >>>>>> On Wed, Feb 7, 2018 at 8:23 PM, Michael Park <mp...@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>>> Many of you probably know that we currently have
> >>>>>> `support/docker-build.sh`
> >>>>>>> to power our CI for our various configurations. One of the problems
> >>>> for
> >>>>>> us
> >>>>>>> has been that we create a `Dockerfile` ad-hoc and invoke `docker
> >>>> build`
> >>>>>>> with it. This is very inefficient and also leads to flaky issues
> >>>> around
> >>>>>>> `apt-get install`.
> >>>>>>>
> >>>>>>> I've introduced `support/mesos-build.sh` which operates off of
> >>>> docker
> >>>>>>> images hosted on Dockerhub instead, and should aid in bringing us
> >>>> faster
> >>>>>>> and more stable CI results!
> >>>>>>>
> >>>>>>> As a bonus, we now also test Clang on the CentOS 7!
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> MPark
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
>
>
Re: Introducing `support/mesos-build.sh`
Posted by Benjamin Bannier <be...@mesosphere.io>.
Done.
> On Mar 20, 2018, at 10:46 AM, Tomek Janiszewski <ja...@gmail.com> wrote:
>
> Thanks for merging. Can we push this image to dockerhub tagged as
> ubuntu-16.04-arm? Then we will need to update CI configuration to:
> JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=cmake COMPILER=gcc
> CONFIGURATION='--disable-java --disable-python --disable-libtool-wrappers'
> ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ../mesos-build.sh
> or
> JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=autotools COMPILER=gcc
> CONFIGURATION='--disable-java --disable-python --disable-libtool-wrappers'
> ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ./support/mesos-build.sh
>
>
> pon., 19 mar 2018 o 16:24 użytkownik Tomek Janiszewski <ja...@gmail.com>
> napisał:
>
>> Dockerfile for ARM with CMake support https://reviews.apache.org/r/66138/
>>
>> pon., 12 lut 2018 o 15:41 użytkownik Tomek Janiszewski <ja...@gmail.com>
>> napisał:
>>
>>> How can I use it for our ARM CI?
>>>
>>> JOBS=16 OS=arm64v8/ubuntu:16.04 ./support/mesos-build.sh
>>> + set -e
>>> + set -o pipefail
>>> + : arm64v8/ubuntu:16.04
>>> + : autotools
>>> + : gcc
>>> + : '--verbose --disable-libtool-wrappers'
>>> + : 'GLOG_v=1 MESOS_VERBOSE=1'
>>> + : 16
>>> ++ git rev-parse --show-toplevel
>>> + MESOS_DIR=/home/janisz/mesos
>>> ++ git diff-index --quiet HEAD --
>>> + docker run --rm -v /home/janisz/mesos:/SRC:Z -e BUILDTOOL=autotools -e
>>> COMPILER=gcc -e 'CONFIGURATION=--verbose --disable-libtool-wrappers' -e
>>> 'ENVIRONMENT=GLOG_v=1 MESOS_VERBOSE=1' -e JOBS=16
>>> mesos/mesos-build:arm64v8/ubuntu-16.04
>>> docker: invalid reference format.
>>> See 'docker run --help'.
>>>
>>>
>>> czw., 8 lut 2018 o 07:38 użytkownik Michael Park <mp...@apache.org>
>>> napisał:
>>>
>>>> The first run looks good!
>>>> https://builds.apache.org/job/Mesos-Buildbot/4890/
>>>>
>>>> [image: Screen Shot 2018-02-07 at 10.30.51 PM.png]
>>>> On Wed, Feb 7, 2018 at 8:39 PM Michael Park <mp...@apache.org> wrote:
>>>>
>>>>> Yep, Just landed! Waiting for
>>>> https://builds.apache.org/job/Mesos-Buildbot to
>>>>> pick it up.
>>>>>
>>>>> On Wed, Feb 7, 2018 at 8:27 PM Vinod Kone <vi...@apache.org>
>>>> wrote:
>>>>>
>>>>>> Yay, thanks MPark! Has the change landed already?
>>>>>>
>>>>>> On Wed, Feb 7, 2018 at 8:23 PM, Michael Park <mp...@apache.org>
>>>> wrote:
>>>>>>
>>>>>>> Many of you probably know that we currently have
>>>>>> `support/docker-build.sh`
>>>>>>> to power our CI for our various configurations. One of the problems
>>>> for
>>>>>> us
>>>>>>> has been that we create a `Dockerfile` ad-hoc and invoke `docker
>>>> build`
>>>>>>> with it. This is very inefficient and also leads to flaky issues
>>>> around
>>>>>>> `apt-get install`.
>>>>>>>
>>>>>>> I've introduced `support/mesos-build.sh` which operates off of
>>>> docker
>>>>>>> images hosted on Dockerhub instead, and should aid in bringing us
>>>> faster
>>>>>>> and more stable CI results!
>>>>>>>
>>>>>>> As a bonus, we now also test Clang on the CentOS 7!
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> MPark
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
Re: Introducing `support/mesos-build.sh`
Posted by Tomek Janiszewski <ja...@gmail.com>.
Thanks for merging. Can we push this image to dockerhub tagged as
ubuntu-16.04-arm? Then we will need to update CI configuration to:
JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=cmake COMPILER=gcc
CONFIGURATION='--disable-java --disable-python --disable-libtool-wrappers'
ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ../mesos-build.sh
or
JOBS=16 OS=ubuntu-16.04-arm BUILDTOOL=autotools COMPILER=gcc
CONFIGURATION='--disable-java --disable-python --disable-libtool-wrappers'
ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1' ./support/mesos-build.sh
pon., 19 mar 2018 o 16:24 użytkownik Tomek Janiszewski <ja...@gmail.com>
napisał:
> Dockerfile for ARM with CMake support https://reviews.apache.org/r/66138/
>
> pon., 12 lut 2018 o 15:41 użytkownik Tomek Janiszewski <ja...@gmail.com>
> napisał:
>
>> How can I use it for our ARM CI?
>>
>> JOBS=16 OS=arm64v8/ubuntu:16.04 ./support/mesos-build.sh
>> + set -e
>> + set -o pipefail
>> + : arm64v8/ubuntu:16.04
>> + : autotools
>> + : gcc
>> + : '--verbose --disable-libtool-wrappers'
>> + : 'GLOG_v=1 MESOS_VERBOSE=1'
>> + : 16
>> ++ git rev-parse --show-toplevel
>> + MESOS_DIR=/home/janisz/mesos
>> ++ git diff-index --quiet HEAD --
>> + docker run --rm -v /home/janisz/mesos:/SRC:Z -e BUILDTOOL=autotools -e
>> COMPILER=gcc -e 'CONFIGURATION=--verbose --disable-libtool-wrappers' -e
>> 'ENVIRONMENT=GLOG_v=1 MESOS_VERBOSE=1' -e JOBS=16
>> mesos/mesos-build:arm64v8/ubuntu-16.04
>> docker: invalid reference format.
>> See 'docker run --help'.
>>
>>
>> czw., 8 lut 2018 o 07:38 użytkownik Michael Park <mp...@apache.org>
>> napisał:
>>
>>> The first run looks good!
>>> https://builds.apache.org/job/Mesos-Buildbot/4890/
>>>
>>> [image: Screen Shot 2018-02-07 at 10.30.51 PM.png]
>>> On Wed, Feb 7, 2018 at 8:39 PM Michael Park <mp...@apache.org> wrote:
>>>
>>> > Yep, Just landed! Waiting for
>>> https://builds.apache.org/job/Mesos-Buildbot to
>>> > pick it up.
>>> >
>>> > On Wed, Feb 7, 2018 at 8:27 PM Vinod Kone <vi...@apache.org>
>>> wrote:
>>> >
>>> >> Yay, thanks MPark! Has the change landed already?
>>> >>
>>> >> On Wed, Feb 7, 2018 at 8:23 PM, Michael Park <mp...@apache.org>
>>> wrote:
>>> >>
>>> >> > Many of you probably know that we currently have
>>> >> `support/docker-build.sh`
>>> >> > to power our CI for our various configurations. One of the problems
>>> for
>>> >> us
>>> >> > has been that we create a `Dockerfile` ad-hoc and invoke `docker
>>> build`
>>> >> > with it. This is very inefficient and also leads to flaky issues
>>> around
>>> >> > `apt-get install`.
>>> >> >
>>> >> > I've introduced `support/mesos-build.sh` which operates off of
>>> docker
>>> >> > images hosted on Dockerhub instead, and should aid in bringing us
>>> faster
>>> >> > and more stable CI results!
>>> >> >
>>> >> > As a bonus, we now also test Clang on the CentOS 7!
>>> >> >
>>> >> > Thanks,
>>> >> >
>>> >> > MPark
>>> >> >
>>> >>
>>> >
>>>
>>