You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Ankur Goenka <go...@google.com> on 2018/10/19 19:44:33 UTC
Docker missing on Beam15
Hi,
Flink Validates Runner test cases are failing on Beam 15 because docker is
not installed.
Failing tasks
https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
Can we install docker on all the machines as the Portable Validates Runner
tests need it.
Thanks,
Ankur
Re: Docker missing on Beam15
Posted by Thomas Weise <th...@apache.org>.
Thanks! There have been a few successful runs now.
On Tue, Oct 23, 2018 at 8:52 AM Yifan Zou <yi...@google.com> wrote:
> FYI, the docker was restarted on beam15.
>
> On Tue, Oct 23, 2018 at 7:08 AM Thomas Weise <th...@apache.org> wrote:
>
>> For the latter (createProcessWorker):
>> https://github.com/apache/beam/pull/6793
>>
>>
>> On Tue, Oct 23, 2018 at 6:47 AM Thomas Weise <th...@apache.org> wrote:
>>
>>> Thanks for taking a look Yifan. Yes, it appears this was an intermittent
>>> issue.
>>>
>>> For beam_PostCommit_Python_VR_Flink we are left with:
>>>
>>> * beam15 docker errors
>>> * segmentation faults
>>> * "Execution failed for task ':beam-sdks-python:createProcessWorker'" -
>>> which should not even execute since we are using Docker
>>>
>>>
>>> On Mon, Oct 22, 2018 at 10:50 PM Yifan Zou <yi...@google.com> wrote:
>>>
>>>> I'm not able to reproduce that error in Beam6 (#459
>>>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/459/>,
>>>> #460
>>>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/460/>),
>>>> it probably due to some outage of Debian [1]. The image was successfully
>>>> built, but the test failed in other reasons.
>>>> And indeed, the beam_PostCommit_Python_VR_Flink is very flaky.
>>>>
>>>> Yifan
>>>>
>>>> [1] https://github.com/docker-library/python/issues/241
>>>>
>>>> On Mon, Oct 22, 2018 at 5:39 PM Thomas Weise <th...@apache.org> wrote:
>>>>
>>>>> Looks like we have more container build related errors.
>>>>>
>>>>> This is from beam6 -
>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/44/
>>>>>
>>>>> Reading package lists...
>>>>> [91mW: The repository 'http://deb.debian.org/debian stretch Release'
>>>>> does not have a Release file.
>>>>>
>>>>> W: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
>>>>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages 404 Not Found
>>>>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/main/binary-amd64/Packages 404 Not Found
>>>>> E: Some index files failed to download. They have been ignored, or old ones used instead.
>>>>>
>>>>>
>>>>> On Mon, Oct 22, 2018 at 2:54 PM Ankur Goenka <go...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Thanks Yifan!
>>>>>>
>>>>>> On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yi...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> So, looks like none of us have the permissions. I filed INFRA-17167
>>>>>>> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra
>>>>>>> team to restart the docker on the beam15.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> Yifan
>>>>>>>
>>>>>>> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I've seen the docker issue pop-up on website pre-commits as well:
>>>>>>>> https://issues.apache.org/jira/browse/BEAM-5783. There were also
>>>>>>>> on beam15.
>>>>>>>>
>>>>>>>> When I searched around the internet I found lots of instances of
>>>>>>>> the same error; it seems to be some unreliability in the guts of Docker
>>>>>>>> [1]. Perhaps restarting the VM or docker daemon could help. Does anybody
>>>>>>>> have permissions to log on and try it?
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://github.com/moby/moby/issues/31849#issuecomment-320236354
>>>>>>>>
>>>>>>>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> There are two issues with
>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/
>>>>>>>>> currently:
>>>>>>>>>
>>>>>>>>> 1) The mentioned issue with docker on beam15 - Jason, can you
>>>>>>>>> possibly advise how to deal with it?
>>>>>>>>>
>>>>>>>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>>>>>>>>> exhibited by
>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>>>>>>>>
>>>>>>>>> The Gradle scan is here:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>>>>>>>>
>>>>>>>>> There are multiple of those in sequence on beam13
>>>>>>>>>
>>>>>>>>> Some more comments:
>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-5467
>>>>>>>>>
>>>>>>>>> Any help to further investigate or fix would be appreciated!
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Thomas
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I got "Failed to restart docker.service: Interactive
>>>>>>>>>> authentication required" while trying to restart the docker on
>>>>>>>>>> beam15.
>>>>>>>>>> Does anyone have the permission to do that? Or, we need to ask
>>>>>>>>>> Apache Infra for help.
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>> Yifan
>>>>>>>>>>
>>>>>>>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Can we restart docker as it seems to have fixed the issue for
>>>>>>>>>>> others https://github.com/moby/moby/issues/31849 ?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Ankur
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> The docker has been installed on all Jenkins VMs. The image
>>>>>>>>>>>> build process was interrupted by a grpc connection issue.
>>>>>>>>>>>>
>>>>>>>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>>>>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> - Yifan
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <
>>>>>>>>>>>> goenka@google.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Flink Validates Runner test cases are failing on Beam 15
>>>>>>>>>>>>> because docker is not installed.
>>>>>>>>>>>>> Failing tasks
>>>>>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>>>>>>>>> Can we install docker on all the machines as the Portable
>>>>>>>>>>>>> Validates Runner tests need it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Ankur
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Got feedback? tinyurl.com/swegner-feedback
>>>>>>>>
>>>>>>>
Re: Docker missing on Beam15
Posted by Yifan Zou <yi...@google.com>.
FYI, the docker was restarted on beam15.
On Tue, Oct 23, 2018 at 7:08 AM Thomas Weise <th...@apache.org> wrote:
> For the latter (createProcessWorker):
> https://github.com/apache/beam/pull/6793
>
>
> On Tue, Oct 23, 2018 at 6:47 AM Thomas Weise <th...@apache.org> wrote:
>
>> Thanks for taking a look Yifan. Yes, it appears this was an intermittent
>> issue.
>>
>> For beam_PostCommit_Python_VR_Flink we are left with:
>>
>> * beam15 docker errors
>> * segmentation faults
>> * "Execution failed for task ':beam-sdks-python:createProcessWorker'" -
>> which should not even execute since we are using Docker
>>
>>
>> On Mon, Oct 22, 2018 at 10:50 PM Yifan Zou <yi...@google.com> wrote:
>>
>>> I'm not able to reproduce that error in Beam6 (#459
>>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/459/>,
>>> #460
>>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/460/>),
>>> it probably due to some outage of Debian [1]. The image was successfully
>>> built, but the test failed in other reasons.
>>> And indeed, the beam_PostCommit_Python_VR_Flink is very flaky.
>>>
>>> Yifan
>>>
>>> [1] https://github.com/docker-library/python/issues/241
>>>
>>> On Mon, Oct 22, 2018 at 5:39 PM Thomas Weise <th...@apache.org> wrote:
>>>
>>>> Looks like we have more container build related errors.
>>>>
>>>> This is from beam6 -
>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/44/
>>>>
>>>> Reading package lists...
>>>> [91mW: The repository 'http://deb.debian.org/debian stretch Release'
>>>> does not have a Release file.
>>>>
>>>> W: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
>>>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages 404 Not Found
>>>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/main/binary-amd64/Packages 404 Not Found
>>>> E: Some index files failed to download. They have been ignored, or old ones used instead.
>>>>
>>>>
>>>> On Mon, Oct 22, 2018 at 2:54 PM Ankur Goenka <go...@google.com> wrote:
>>>>
>>>>> Thanks Yifan!
>>>>>
>>>>> On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yi...@google.com> wrote:
>>>>>
>>>>>> So, looks like none of us have the permissions. I filed INFRA-17167
>>>>>> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra
>>>>>> team to restart the docker on the beam15.
>>>>>>
>>>>>> Thanks.
>>>>>> Yifan
>>>>>>
>>>>>> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> I've seen the docker issue pop-up on website pre-commits as well:
>>>>>>> https://issues.apache.org/jira/browse/BEAM-5783. There were also on
>>>>>>> beam15.
>>>>>>>
>>>>>>> When I searched around the internet I found lots of instances of the
>>>>>>> same error; it seems to be some unreliability in the guts of Docker [1].
>>>>>>> Perhaps restarting the VM or docker daemon could help. Does anybody have
>>>>>>> permissions to log on and try it?
>>>>>>>
>>>>>>> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
>>>>>>>
>>>>>>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
>>>>>>>
>>>>>>>> There are two issues with
>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/
>>>>>>>> currently:
>>>>>>>>
>>>>>>>> 1) The mentioned issue with docker on beam15 - Jason, can you
>>>>>>>> possibly advise how to deal with it?
>>>>>>>>
>>>>>>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>>>>>>>> exhibited by
>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>>>>>>>
>>>>>>>> The Gradle scan is here:
>>>>>>>>
>>>>>>>>
>>>>>>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>>>>>>>
>>>>>>>> There are multiple of those in sequence on beam13
>>>>>>>>
>>>>>>>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>>>>>>>>
>>>>>>>> Any help to further investigate or fix would be appreciated!
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Thomas
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I got "Failed to restart docker.service: Interactive
>>>>>>>>> authentication required" while trying to restart the docker on
>>>>>>>>> beam15.
>>>>>>>>> Does anyone have the permission to do that? Or, we need to ask
>>>>>>>>> Apache Infra for help.
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>> Yifan
>>>>>>>>>
>>>>>>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Can we restart docker as it seems to have fixed the issue for
>>>>>>>>>> others https://github.com/moby/moby/issues/31849 ?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Ankur
>>>>>>>>>>
>>>>>>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> The docker has been installed on all Jenkins VMs. The image
>>>>>>>>>>> build process was interrupted by a grpc connection issue.
>>>>>>>>>>>
>>>>>>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>>>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Yifan
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Flink Validates Runner test cases are failing on Beam 15
>>>>>>>>>>>> because docker is not installed.
>>>>>>>>>>>> Failing tasks
>>>>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>>>>>>>> Can we install docker on all the machines as the Portable
>>>>>>>>>>>> Validates Runner tests need it.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Ankur
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Got feedback? tinyurl.com/swegner-feedback
>>>>>>>
>>>>>>
Re: Docker missing on Beam15
Posted by Thomas Weise <th...@apache.org>.
For the latter (createProcessWorker):
https://github.com/apache/beam/pull/6793
On Tue, Oct 23, 2018 at 6:47 AM Thomas Weise <th...@apache.org> wrote:
> Thanks for taking a look Yifan. Yes, it appears this was an intermittent
> issue.
>
> For beam_PostCommit_Python_VR_Flink we are left with:
>
> * beam15 docker errors
> * segmentation faults
> * "Execution failed for task ':beam-sdks-python:createProcessWorker'" -
> which should not even execute since we are using Docker
>
>
> On Mon, Oct 22, 2018 at 10:50 PM Yifan Zou <yi...@google.com> wrote:
>
>> I'm not able to reproduce that error in Beam6 (#459
>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/459/>,
>> #460 <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/460/>),
>> it probably due to some outage of Debian [1]. The image was successfully
>> built, but the test failed in other reasons.
>> And indeed, the beam_PostCommit_Python_VR_Flink is very flaky.
>>
>> Yifan
>>
>> [1] https://github.com/docker-library/python/issues/241
>>
>> On Mon, Oct 22, 2018 at 5:39 PM Thomas Weise <th...@apache.org> wrote:
>>
>>> Looks like we have more container build related errors.
>>>
>>> This is from beam6 -
>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/44/
>>>
>>> Reading package lists...
>>> [91mW: The repository 'http://deb.debian.org/debian stretch Release'
>>> does not have a Release file.
>>>
>>> W: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
>>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages 404 Not Found
>>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/main/binary-amd64/Packages 404 Not Found
>>> E: Some index files failed to download. They have been ignored, or old ones used instead.
>>>
>>>
>>> On Mon, Oct 22, 2018 at 2:54 PM Ankur Goenka <go...@google.com> wrote:
>>>
>>>> Thanks Yifan!
>>>>
>>>> On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yi...@google.com> wrote:
>>>>
>>>>> So, looks like none of us have the permissions. I filed INFRA-17167
>>>>> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra team
>>>>> to restart the docker on the beam15.
>>>>>
>>>>> Thanks.
>>>>> Yifan
>>>>>
>>>>> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org> wrote:
>>>>>
>>>>>> I've seen the docker issue pop-up on website pre-commits as well:
>>>>>> https://issues.apache.org/jira/browse/BEAM-5783. There were also on
>>>>>> beam15.
>>>>>>
>>>>>> When I searched around the internet I found lots of instances of the
>>>>>> same error; it seems to be some unreliability in the guts of Docker [1].
>>>>>> Perhaps restarting the VM or docker daemon could help. Does anybody have
>>>>>> permissions to log on and try it?
>>>>>>
>>>>>> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
>>>>>>
>>>>>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
>>>>>>
>>>>>>> There are two issues with
>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/
>>>>>>> currently:
>>>>>>>
>>>>>>> 1) The mentioned issue with docker on beam15 - Jason, can you
>>>>>>> possibly advise how to deal with it?
>>>>>>>
>>>>>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>>>>>>> exhibited by
>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>>>>>>
>>>>>>> The Gradle scan is here:
>>>>>>>
>>>>>>>
>>>>>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>>>>>>
>>>>>>> There are multiple of those in sequence on beam13
>>>>>>>
>>>>>>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>>>>>>>
>>>>>>> Any help to further investigate or fix would be appreciated!
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Thomas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I got "Failed to restart docker.service: Interactive
>>>>>>>> authentication required" while trying to restart the docker on
>>>>>>>> beam15.
>>>>>>>> Does anyone have the permission to do that? Or, we need to ask
>>>>>>>> Apache Infra for help.
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>> Yifan
>>>>>>>>
>>>>>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Can we restart docker as it seems to have fixed the issue for
>>>>>>>>> others https://github.com/moby/moby/issues/31849 ?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ankur
>>>>>>>>>
>>>>>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> The docker has been installed on all Jenkins VMs. The image build
>>>>>>>>>> process was interrupted by a grpc connection issue.
>>>>>>>>>>
>>>>>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Yifan
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Flink Validates Runner test cases are failing on Beam 15 because
>>>>>>>>>>> docker is not installed.
>>>>>>>>>>> Failing tasks
>>>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>>>>>>> Can we install docker on all the machines as the Portable
>>>>>>>>>>> Validates Runner tests need it.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Ankur
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Got feedback? tinyurl.com/swegner-feedback
>>>>>>
>>>>>
Re: Docker missing on Beam15
Posted by Thomas Weise <th...@apache.org>.
Thanks for taking a look Yifan. Yes, it appears this was an intermittent
issue.
For beam_PostCommit_Python_VR_Flink we are left with:
* beam15 docker errors
* segmentation faults
* "Execution failed for task ':beam-sdks-python:createProcessWorker'" -
which should not even execute since we are using Docker
On Mon, Oct 22, 2018 at 10:50 PM Yifan Zou <yi...@google.com> wrote:
> I'm not able to reproduce that error in Beam6 (#459
> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/459/>, #460
> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/460/>), it
> probably due to some outage of Debian [1]. The image was successfully
> built, but the test failed in other reasons.
> And indeed, the beam_PostCommit_Python_VR_Flink is very flaky.
>
> Yifan
>
> [1] https://github.com/docker-library/python/issues/241
>
> On Mon, Oct 22, 2018 at 5:39 PM Thomas Weise <th...@apache.org> wrote:
>
>> Looks like we have more container build related errors.
>>
>> This is from beam6 -
>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/44/
>>
>> Reading package lists...
>> [91mW: The repository 'http://deb.debian.org/debian stretch Release'
>> does not have a Release file.
>>
>> W: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages 404 Not Found
>> E: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/main/binary-amd64/Packages 404 Not Found
>> E: Some index files failed to download. They have been ignored, or old ones used instead.
>>
>>
>> On Mon, Oct 22, 2018 at 2:54 PM Ankur Goenka <go...@google.com> wrote:
>>
>>> Thanks Yifan!
>>>
>>> On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yi...@google.com> wrote:
>>>
>>>> So, looks like none of us have the permissions. I filed INFRA-17167
>>>> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra team
>>>> to restart the docker on the beam15.
>>>>
>>>> Thanks.
>>>> Yifan
>>>>
>>>> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org> wrote:
>>>>
>>>>> I've seen the docker issue pop-up on website pre-commits as well:
>>>>> https://issues.apache.org/jira/browse/BEAM-5783. There were also on
>>>>> beam15.
>>>>>
>>>>> When I searched around the internet I found lots of instances of the
>>>>> same error; it seems to be some unreliability in the guts of Docker [1].
>>>>> Perhaps restarting the VM or docker daemon could help. Does anybody have
>>>>> permissions to log on and try it?
>>>>>
>>>>> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
>>>>>
>>>>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
>>>>>
>>>>>> There are two issues with
>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/
>>>>>> currently:
>>>>>>
>>>>>> 1) The mentioned issue with docker on beam15 - Jason, can you
>>>>>> possibly advise how to deal with it?
>>>>>>
>>>>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>>>>>> exhibited by
>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>>>>>
>>>>>> The Gradle scan is here:
>>>>>>
>>>>>>
>>>>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>>>>>
>>>>>> There are multiple of those in sequence on beam13
>>>>>>
>>>>>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>>>>>>
>>>>>> Any help to further investigate or fix would be appreciated!
>>>>>>
>>>>>> Thanks,
>>>>>> Thomas
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I got "Failed to restart docker.service: Interactive authentication
>>>>>>> required" while trying to restart the docker on beam15.
>>>>>>> Does anyone have the permission to do that? Or, we need to ask
>>>>>>> Apache Infra for help.
>>>>>>>
>>>>>>> Thanks.
>>>>>>> Yifan
>>>>>>>
>>>>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Can we restart docker as it seems to have fixed the issue for
>>>>>>>> others https://github.com/moby/moby/issues/31849 ?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ankur
>>>>>>>>
>>>>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> The docker has been installed on all Jenkins VMs. The image build
>>>>>>>>> process was interrupted by a grpc connection issue.
>>>>>>>>>
>>>>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Yifan
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Flink Validates Runner test cases are failing on Beam 15 because
>>>>>>>>>> docker is not installed.
>>>>>>>>>> Failing tasks
>>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>>>>>> Can we install docker on all the machines as the Portable
>>>>>>>>>> Validates Runner tests need it.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Ankur
>>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>> --
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Got feedback? tinyurl.com/swegner-feedback
>>>>>
>>>>
Re: Docker missing on Beam15
Posted by Yifan Zou <yi...@google.com>.
I'm not able to reproduce that error in Beam6 (#459
<https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/459/>, #460
<https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/460/>), it
probably due to some outage of Debian [1]. The image was successfully
built, but the test failed in other reasons.
And indeed, the beam_PostCommit_Python_VR_Flink is very flaky.
Yifan
[1] https://github.com/docker-library/python/issues/241
On Mon, Oct 22, 2018 at 5:39 PM Thomas Weise <th...@apache.org> wrote:
> Looks like we have more container build related errors.
>
> This is from beam6 -
> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/44/
>
> Reading package lists...
> [91mW: The repository 'http://deb.debian.org/debian stretch Release' does
> not have a Release file.
>
> W: The repository 'http://deb.debian.org/debian stretch-updates Release' does not have a Release file.
> E: Failed to fetch http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages 404 Not Found
> E: Failed to fetch http://deb.debian.org/debian/dists/stretch-updates/main/binary-amd64/Packages 404 Not Found
> E: Some index files failed to download. They have been ignored, or old ones used instead.
>
>
> On Mon, Oct 22, 2018 at 2:54 PM Ankur Goenka <go...@google.com> wrote:
>
>> Thanks Yifan!
>>
>> On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yi...@google.com> wrote:
>>
>>> So, looks like none of us have the permissions. I filed INFRA-17167
>>> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra team
>>> to restart the docker on the beam15.
>>>
>>> Thanks.
>>> Yifan
>>>
>>> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org> wrote:
>>>
>>>> I've seen the docker issue pop-up on website pre-commits as well:
>>>> https://issues.apache.org/jira/browse/BEAM-5783. There were also on
>>>> beam15.
>>>>
>>>> When I searched around the internet I found lots of instances of the
>>>> same error; it seems to be some unreliability in the guts of Docker [1].
>>>> Perhaps restarting the VM or docker daemon could help. Does anybody have
>>>> permissions to log on and try it?
>>>>
>>>> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
>>>>
>>>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
>>>>
>>>>> There are two issues with
>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/
>>>>> currently:
>>>>>
>>>>> 1) The mentioned issue with docker on beam15 - Jason, can you possibly
>>>>> advise how to deal with it?
>>>>>
>>>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>>>>> exhibited by
>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>>>>
>>>>> The Gradle scan is here:
>>>>>
>>>>>
>>>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>>>>
>>>>> There are multiple of those in sequence on beam13
>>>>>
>>>>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>>>>>
>>>>> Any help to further investigate or fix would be appreciated!
>>>>>
>>>>> Thanks,
>>>>> Thomas
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com> wrote:
>>>>>
>>>>>> I got "Failed to restart docker.service: Interactive authentication
>>>>>> required" while trying to restart the docker on beam15.
>>>>>> Does anyone have the permission to do that? Or, we need to ask Apache
>>>>>> Infra for help.
>>>>>>
>>>>>> Thanks.
>>>>>> Yifan
>>>>>>
>>>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Can we restart docker as it seems to have fixed the issue for others
>>>>>>> https://github.com/moby/moby/issues/31849 ?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ankur
>>>>>>>
>>>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> The docker has been installed on all Jenkins VMs. The image build
>>>>>>>> process was interrupted by a grpc connection issue.
>>>>>>>>
>>>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>>>>
>>>>>>>>
>>>>>>>> - Yifan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Flink Validates Runner test cases are failing on Beam 15 because
>>>>>>>>> docker is not installed.
>>>>>>>>> Failing tasks
>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>>>>> Can we install docker on all the machines as the Portable
>>>>>>>>> Validates Runner tests need it.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ankur
>>>>>>>>>
>>>>>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>>
>>>> Got feedback? tinyurl.com/swegner-feedback
>>>>
>>>
Re: Docker missing on Beam15
Posted by Thomas Weise <th...@apache.org>.
Looks like we have more container build related errors.
This is from beam6 -
https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/44/
Reading package lists...
[91mW: The repository 'http://deb.debian.org/debian stretch Release' does
not have a Release file.
W: The repository 'http://deb.debian.org/debian stretch-updates
Release' does not have a Release file.
E: Failed to fetch
http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages
404 Not Found
E: Failed to fetch
http://deb.debian.org/debian/dists/stretch-updates/main/binary-amd64/Packages
404 Not Found
E: Some index files failed to download. They have been ignored, or old
ones used instead.
On Mon, Oct 22, 2018 at 2:54 PM Ankur Goenka <go...@google.com> wrote:
> Thanks Yifan!
>
> On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yi...@google.com> wrote:
>
>> So, looks like none of us have the permissions. I filed INFRA-17167
>> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra team to
>> restart the docker on the beam15.
>>
>> Thanks.
>> Yifan
>>
>> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org> wrote:
>>
>>> I've seen the docker issue pop-up on website pre-commits as well:
>>> https://issues.apache.org/jira/browse/BEAM-5783. There were also on
>>> beam15.
>>>
>>> When I searched around the internet I found lots of instances of the
>>> same error; it seems to be some unreliability in the guts of Docker [1].
>>> Perhaps restarting the VM or docker daemon could help. Does anybody have
>>> permissions to log on and try it?
>>>
>>> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
>>>
>>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
>>>
>>>> There are two issues with
>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/
>>>> currently:
>>>>
>>>> 1) The mentioned issue with docker on beam15 - Jason, can you possibly
>>>> advise how to deal with it?
>>>>
>>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>>>> exhibited by
>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>>>
>>>> The Gradle scan is here:
>>>>
>>>>
>>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>>>
>>>> There are multiple of those in sequence on beam13
>>>>
>>>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>>>>
>>>> Any help to further investigate or fix would be appreciated!
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>>
>>>>
>>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com> wrote:
>>>>
>>>>> I got "Failed to restart docker.service: Interactive authentication
>>>>> required" while trying to restart the docker on beam15.
>>>>> Does anyone have the permission to do that? Or, we need to ask Apache
>>>>> Infra for help.
>>>>>
>>>>> Thanks.
>>>>> Yifan
>>>>>
>>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Can we restart docker as it seems to have fixed the issue for others
>>>>>> https://github.com/moby/moby/issues/31849 ?
>>>>>>
>>>>>> Thanks,
>>>>>> Ankur
>>>>>>
>>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> The docker has been installed on all Jenkins VMs. The image build
>>>>>>> process was interrupted by a grpc connection issue.
>>>>>>>
>>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>>>
>>>>>>>
>>>>>>> - Yifan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Flink Validates Runner test cases are failing on Beam 15 because
>>>>>>>> docker is not installed.
>>>>>>>> Failing tasks
>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>>>> Can we install docker on all the machines as the Portable Validates
>>>>>>>> Runner tests need it.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ankur
>>>>>>>>
>>>>>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>> Got feedback? tinyurl.com/swegner-feedback
>>>
>>
Re: Docker missing on Beam15
Posted by Ankur Goenka <go...@google.com>.
Thanks Yifan!
On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yi...@google.com> wrote:
> So, looks like none of us have the permissions. I filed INFRA-17167
> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra team to
> restart the docker on the beam15.
>
> Thanks.
> Yifan
>
> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org> wrote:
>
>> I've seen the docker issue pop-up on website pre-commits as well:
>> https://issues.apache.org/jira/browse/BEAM-5783. There were also on
>> beam15.
>>
>> When I searched around the internet I found lots of instances of the same
>> error; it seems to be some unreliability in the guts of Docker [1]. Perhaps
>> restarting the VM or docker daemon could help. Does anybody have
>> permissions to log on and try it?
>>
>> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
>>
>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
>>
>>> There are two issues with
>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/
>>> currently:
>>>
>>> 1) The mentioned issue with docker on beam15 - Jason, can you possibly
>>> advise how to deal with it?
>>>
>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>>> exhibited by
>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>>
>>> The Gradle scan is here:
>>>
>>>
>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>>
>>> There are multiple of those in sequence on beam13
>>>
>>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>>>
>>> Any help to further investigate or fix would be appreciated!
>>>
>>> Thanks,
>>> Thomas
>>>
>>>
>>>
>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com> wrote:
>>>
>>>> I got "Failed to restart docker.service: Interactive authentication
>>>> required" while trying to restart the docker on beam15.
>>>> Does anyone have the permission to do that? Or, we need to ask Apache
>>>> Infra for help.
>>>>
>>>> Thanks.
>>>> Yifan
>>>>
>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Can we restart docker as it seems to have fixed the issue for others
>>>>> https://github.com/moby/moby/issues/31849 ?
>>>>>
>>>>> Thanks,
>>>>> Ankur
>>>>>
>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> The docker has been installed on all Jenkins VMs. The image build
>>>>>> process was interrupted by a grpc connection issue.
>>>>>>
>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>>
>>>>>>
>>>>>> - Yifan
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Flink Validates Runner test cases are failing on Beam 15 because
>>>>>>> docker is not installed.
>>>>>>> Failing tasks
>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>>> Can we install docker on all the machines as the Portable Validates
>>>>>>> Runner tests need it.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ankur
>>>>>>>
>>>>>>
>>
>> --
>>
>>
>>
>>
>> Got feedback? tinyurl.com/swegner-feedback
>>
>
Re: Docker missing on Beam15
Posted by Yifan Zou <yi...@google.com>.
So, looks like none of us have the permissions. I filed INFRA-17167
<https://issues.apache.org/jira/browse/INFRA-17167> to the Infra team to
restart the docker on the beam15.
Thanks.
Yifan
On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org> wrote:
> I've seen the docker issue pop-up on website pre-commits as well:
> https://issues.apache.org/jira/browse/BEAM-5783. There were also on
> beam15.
>
> When I searched around the internet I found lots of instances of the same
> error; it seems to be some unreliability in the guts of Docker [1]. Perhaps
> restarting the VM or docker daemon could help. Does anybody have
> permissions to log on and try it?
>
> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
>
> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
>
>> There are two issues with
>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/ currently:
>>
>> 1) The mentioned issue with docker on beam15 - Jason, can you possibly
>> advise how to deal with it?
>>
>> 2) Frequent failure due to "Segmentation fault (core dumped)", as
>> exhibited by
>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>>
>> The Gradle scan is here:
>>
>>
>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>>
>> There are multiple of those in sequence on beam13
>>
>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>>
>> Any help to further investigate or fix would be appreciated!
>>
>> Thanks,
>> Thomas
>>
>>
>>
>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com> wrote:
>>
>>> I got "Failed to restart docker.service: Interactive authentication
>>> required" while trying to restart the docker on beam15.
>>> Does anyone have the permission to do that? Or, we need to ask Apache
>>> Infra for help.
>>>
>>> Thanks.
>>> Yifan
>>>
>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Can we restart docker as it seems to have fixed the issue for others
>>>> https://github.com/moby/moby/issues/31849 ?
>>>>
>>>> Thanks,
>>>> Ankur
>>>>
>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> The docker has been installed on all Jenkins VMs. The image build
>>>>> process was interrupted by a grpc connection issue.
>>>>>
>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>>
>>>>>
>>>>> - Yifan
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Flink Validates Runner test cases are failing on Beam 15 because
>>>>>> docker is not installed.
>>>>>> Failing tasks
>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>>> Can we install docker on all the machines as the Portable Validates
>>>>>> Runner tests need it.
>>>>>>
>>>>>> Thanks,
>>>>>> Ankur
>>>>>>
>>>>>
>
> --
>
>
>
>
> Got feedback? tinyurl.com/swegner-feedback
>
Re: Docker missing on Beam15
Posted by Scott Wegner <sc...@apache.org>.
I've seen the docker issue pop-up on website pre-commits as well:
https://issues.apache.org/jira/browse/BEAM-5783. There were also on beam15.
When I searched around the internet I found lots of instances of the same
error; it seems to be some unreliability in the guts of Docker [1]. Perhaps
restarting the VM or docker daemon could help. Does anybody have
permissions to log on and try it?
[1] https://github.com/moby/moby/issues/31849#issuecomment-320236354
On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <th...@apache.org> wrote:
> There are two issues with
> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/ currently:
>
> 1) The mentioned issue with docker on beam15 - Jason, can you possibly
> advise how to deal with it?
>
> 2) Frequent failure due to "Segmentation fault (core dumped)", as
> exhibited by
> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
>
> The Gradle scan is here:
>
>
> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
>
> There are multiple of those in sequence on beam13
>
> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
>
> Any help to further investigate or fix would be appreciated!
>
> Thanks,
> Thomas
>
>
>
> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com> wrote:
>
>> I got "Failed to restart docker.service: Interactive authentication
>> required" while trying to restart the docker on beam15.
>> Does anyone have the permission to do that? Or, we need to ask Apache
>> Infra for help.
>>
>> Thanks.
>> Yifan
>>
>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com> wrote:
>>
>>> Hi,
>>>
>>> Can we restart docker as it seems to have fixed the issue for others
>>> https://github.com/moby/moby/issues/31849 ?
>>>
>>> Thanks,
>>> Ankur
>>>
>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> The docker has been installed on all Jenkins VMs. The image build
>>>> process was interrupted by a grpc connection issue.
>>>>
>>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>>
>>>>
>>>> - Yifan
>>>>
>>>>
>>>>
>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Flink Validates Runner test cases are failing on Beam 15 because
>>>>> docker is not installed.
>>>>> Failing tasks
>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>>> Can we install docker on all the machines as the Portable Validates
>>>>> Runner tests need it.
>>>>>
>>>>> Thanks,
>>>>> Ankur
>>>>>
>>>>
--
Got feedback? tinyurl.com/swegner-feedback
Re: Docker missing on Beam15
Posted by Thomas Weise <th...@apache.org>.
There are two issues with
https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/ currently:
1) The mentioned issue with docker on beam15 - Jason, can you possibly
advise how to deal with it?
2) Frequent failure due to "Segmentation fault (core dumped)", as exhibited
by
https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText
The Gradle scan is here:
https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0
There are multiple of those in sequence on beam13
Some more comments: https://issues.apache.org/jira/browse/BEAM-5467
Any help to further investigate or fix would be appreciated!
Thanks,
Thomas
On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yi...@google.com> wrote:
> I got "Failed to restart docker.service: Interactive authentication
> required" while trying to restart the docker on beam15.
> Does anyone have the permission to do that? Or, we need to ask Apache
> Infra for help.
>
> Thanks.
> Yifan
>
> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com> wrote:
>
>> Hi,
>>
>> Can we restart docker as it seems to have fixed the issue for others
>> https://github.com/moby/moby/issues/31849 ?
>>
>> Thanks,
>> Ankur
>>
>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com> wrote:
>>
>>> Hi,
>>>
>>> The docker has been installed on all Jenkins VMs. The image build
>>> process was interrupted by a grpc connection issue.
>>>
>>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>>
>>>
>>> - Yifan
>>>
>>>
>>>
>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Flink Validates Runner test cases are failing on Beam 15 because docker
>>>> is not installed.
>>>> Failing tasks
>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>>> Can we install docker on all the machines as the Portable Validates
>>>> Runner tests need it.
>>>>
>>>> Thanks,
>>>> Ankur
>>>>
>>>
Re: Docker missing on Beam15
Posted by Yifan Zou <yi...@google.com>.
I got "Failed to restart docker.service: Interactive authentication required
" while trying to restart the docker on beam15.
Does anyone have the permission to do that? Or, we need to ask Apache Infra
for help.
Thanks.
Yifan
On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <go...@google.com> wrote:
> Hi,
>
> Can we restart docker as it seems to have fixed the issue for others
> https://github.com/moby/moby/issues/31849 ?
>
> Thanks,
> Ankur
>
> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com> wrote:
>
>> Hi,
>>
>> The docker has been installed on all Jenkins VMs. The image build process
>> was interrupted by a grpc connection issue.
>>
>> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>>
>>
>> - Yifan
>>
>>
>>
>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com> wrote:
>>
>>> Hi,
>>>
>>> Flink Validates Runner test cases are failing on Beam 15 because docker
>>> is not installed.
>>> Failing tasks
>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>>> Can we install docker on all the machines as the Portable Validates
>>> Runner tests need it.
>>>
>>> Thanks,
>>> Ankur
>>>
>>
Re: Docker missing on Beam15
Posted by Ankur Goenka <go...@google.com>.
Hi,
Can we restart docker as it seems to have fixed the issue for others
https://github.com/moby/moby/issues/31849 ?
Thanks,
Ankur
On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yi...@google.com> wrote:
> Hi,
>
> The docker has been installed on all Jenkins VMs. The image build process
> was interrupted by a grpc connection issue.
>
> *11:02:12* Starting process 'command 'docker''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker Command: docker build --no-cache -t jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* Successfully started process 'command 'docker''*11:02:12* Sending build context to Docker daemon 17.65MB
> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam <de...@beam.apache.org>"*11:02:12* ---> Running in f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get update && apt-get install -y libsnappy-dev libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
>
>
> - Yifan
>
>
>
> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com> wrote:
>
>> Hi,
>>
>> Flink Validates Runner test cases are failing on Beam 15 because docker
>> is not installed.
>> Failing tasks
>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
>> Can we install docker on all the machines as the Portable Validates
>> Runner tests need it.
>>
>> Thanks,
>> Ankur
>>
>
Re: Docker missing on Beam15
Posted by Yifan Zou <yi...@google.com>.
Hi,
The docker has been installed on all Jenkins VMs. The image build process
was interrupted by a grpc connection issue.
*11:02:12* Starting process 'command 'docker''. Working directory:
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker
Command: docker build --no-cache -t
jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12*
Successfully started process 'command 'docker''*11:02:12* Sending
build context to Docker daemon 17.65MB
*11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* --->
3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam
<de...@beam.apache.org>"*11:02:12* ---> Running in
f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing
intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN apt-get
update && apt-get install -y libsnappy-dev
libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* --->
Running in 5e9b67be03f9*11:02:12* grpc: the connection is unavailable
- Yifan
On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <go...@google.com> wrote:
> Hi,
>
> Flink Validates Runner test cases are failing on Beam 15 because docker is
> not installed.
> Failing tasks
> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend
> Can we install docker on all the machines as the Portable Validates Runner
> tests need it.
>
> Thanks,
> Ankur
>