You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Olaf Flebbe <of...@oflebbe.de> on 2017/11/24 23:14:23 UTC
AARCH64 Blocker
Hi,
on the ARM64 CI Host (centos-7) docker networking seems to be to instable to be useful.
In a docker machine calling this command (fedora26 on aarch64, but others as well) fails (timeouts) for about 50% of all tries.
curl -v -o /dev/null 'http://conjars.org/repo/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar'
On the docker host this works flawless. Nameserver (8.8.8.8) the same for machine and host.
I am out of options, will have to cancel efforts to port bigtop on aarch64 for now.
docker --version
Docker version 1.12.5, build 047e51b/1.12.5
I have no idea where this docker installation comes from, since it is not enabled in systemd / init.
Olaf
Re: AARCH64 Blocker
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi Marcin,
thank you very much for your effort. Sent via DM. Will share the key when set up
Olaf
> Am 06.12.2017 um 13:42 schrieb Marcin Juszkiewicz <ma...@linaro.org>:
>
> W dniu 25.11.2017 o 00:14, Olaf Flebbe pisze:
>
>> on the ARM64 CI Host (centos-7) docker networking seems to be to
>> instable to be useful.
>>
>> In a docker machine calling this command (fedora26 on aarch64, but
>> others as well) fails (timeouts) for about 50% of all tries.
>>
>> curl -v -o /dev/null
>> 'http://conjars.org/repo/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar'
>>
>> On the docker host this works flawless. Nameserver (8.8.8.8) the
>> same for machine and host.
>>
>> I am out of options, will have to cancel efforts to port bigtop on
>> aarch64 for now.
>>
>> docker --version Docker version 1.12.5, build 047e51b/1.12.5
>>
>> I have no idea where this docker installation comes from, since it is
>> not enabled in systemd / init.
>
> Can I suggest moving to Debian 9 'stretch' as system for CI aarch64
> host? Linaro provides build of Docker CE 17.06 for that distribution and
> (so far) I did not had any issues with Docker behaviour on my servers
> using it.
>
> We have two VM instances prepared for such use - send me ssh public key
> and I will add it to account with sudo permissions.
Re: AARCH64 Blocker
Posted by Marcin Juszkiewicz <ma...@linaro.org>.
W dniu 25.11.2017 o 00:14, Olaf Flebbe pisze:
> on the ARM64 CI Host (centos-7) docker networking seems to be to
> instable to be useful.
>
> In a docker machine calling this command (fedora26 on aarch64, but
> others as well) fails (timeouts) for about 50% of all tries.
>
> curl -v -o /dev/null
> 'http://conjars.org/repo/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar'
>
> On the docker host this works flawless. Nameserver (8.8.8.8) the
> same for machine and host.
>
> I am out of options, will have to cancel efforts to port bigtop on
> aarch64 for now.
>
> docker --version Docker version 1.12.5, build 047e51b/1.12.5
>
> I have no idea where this docker installation comes from, since it is
> not enabled in systemd / init.
Can I suggest moving to Debian 9 'stretch' as system for CI aarch64
host? Linaro provides build of Docker CE 17.06 for that distribution and
(so far) I did not had any issues with Docker behaviour on my servers
using it.
We have two VM instances prepared for such use - send me ssh public key
and I will add it to account with sudo permissions.
Re: AARCH64 Blocker
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi,
Now I understand what you are referring to: --net=host let docker use host networking . It took a while ;-)
Actually, on our x86_64 ci we are using container networking with a sonatype nexus container as a caching proxy for maven artifacts. This results in sharing the network with all compile jobs on the same host.
We had big problems with that in the past related to insane compile projects interfering with each other: The spark version 1 and spark version 2 jobs running on the same host mixed their network based nailgun scala compilation service since they are on the same network. This is fixed by us by turning nailgun off.
When we switch on --net=host, every compile job opening up services is not only visible to each other, it is exposed to the internet, but it is essentially the same as running a compile job on the host itself.
Let's try it for compilation. However, the docker provisioner will not work that way.
Regards,
Olaf
> Am 27.11.2017 um 02:14 schrieb Jun HE <ry...@gmail.com>:
>
> Hi, Evans, Olaf,
>
> I used to meet this issue on Linaro's cloud. The workaround is to run
> container in "host" mode.
> Not sure if this is a workable way for ARM64 CI.
>
> Regards,
>
> Jun
>
> 2017-11-27 2:43 GMT+08:00 Evans Ye <ev...@apache.org>:
>
>> Ganesh can you help to clarify?
>>
>> 2017-11-25 7:14 GMT+08:00 Olaf Flebbe <of...@oflebbe.de>:
>>
>>> Hi,
>>>
>>> on the ARM64 CI Host (centos-7) docker networking seems to be to
>> instable
>>> to be useful.
>>>
>>> In a docker machine calling this command (fedora26 on aarch64, but others
>>> as well) fails (timeouts) for about 50% of all tries.
>>>
>>> curl -v -o /dev/null 'http://conjars.org/repo/org/
>>> pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-
>>> aggdesigner-algorithm-5.1.5-jhyde.jar'
>>>
>>> On the docker host this works flawless. Nameserver (8.8.8.8) the same for
>>> machine and host.
>>>
>>> I am out of options, will have to cancel efforts to port bigtop on
>> aarch64
>>> for now.
>>>
>>> docker --version
>>> Docker version 1.12.5, build 047e51b/1.12.5
>>>
>>> I have no idea where this docker installation comes from, since it is not
>>> enabled in systemd / init.
>>>
>>>
>>> Olaf
>>>
>>
Re: AARCH64 Blocker
Posted by Jun HE <ry...@gmail.com>.
Hi, Evans, Olaf,
I used to meet this issue on Linaro's cloud. The workaround is to run
container in "host" mode.
Not sure if this is a workable way for ARM64 CI.
Regards,
Jun
2017-11-27 2:43 GMT+08:00 Evans Ye <ev...@apache.org>:
> Ganesh can you help to clarify?
>
> 2017-11-25 7:14 GMT+08:00 Olaf Flebbe <of...@oflebbe.de>:
>
> > Hi,
> >
> > on the ARM64 CI Host (centos-7) docker networking seems to be to
> instable
> > to be useful.
> >
> > In a docker machine calling this command (fedora26 on aarch64, but others
> > as well) fails (timeouts) for about 50% of all tries.
> >
> > curl -v -o /dev/null 'http://conjars.org/repo/org/
> > pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-
> > aggdesigner-algorithm-5.1.5-jhyde.jar'
> >
> > On the docker host this works flawless. Nameserver (8.8.8.8) the same for
> > machine and host.
> >
> > I am out of options, will have to cancel efforts to port bigtop on
> aarch64
> > for now.
> >
> > docker --version
> > Docker version 1.12.5, build 047e51b/1.12.5
> >
> > I have no idea where this docker installation comes from, since it is not
> > enabled in systemd / init.
> >
> >
> > Olaf
> >
>
Re: AARCH64 Blocker
Posted by Evans Ye <ev...@apache.org>.
Ganesh can you help to clarify?
2017-11-25 7:14 GMT+08:00 Olaf Flebbe <of...@oflebbe.de>:
> Hi,
>
> on the ARM64 CI Host (centos-7) docker networking seems to be to instable
> to be useful.
>
> In a docker machine calling this command (fedora26 on aarch64, but others
> as well) fails (timeouts) for about 50% of all tries.
>
> curl -v -o /dev/null 'http://conjars.org/repo/org/
> pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-
> aggdesigner-algorithm-5.1.5-jhyde.jar'
>
> On the docker host this works flawless. Nameserver (8.8.8.8) the same for
> machine and host.
>
> I am out of options, will have to cancel efforts to port bigtop on aarch64
> for now.
>
> docker --version
> Docker version 1.12.5, build 047e51b/1.12.5
>
> I have no idea where this docker installation comes from, since it is not
> enabled in systemd / init.
>
>
> Olaf
>