Re: [Proposal] Creating a reproducible environment for Beam Jenkins Tests

[Ismaël Mejía <ie...@gmail.com>]

BIG +1, the previous work on having docker build images [1] had a
similar goal (to have a reproducible build environment). But this is
even better because we will guarantee the exact same environment in
Jenkins as well as any further improvements. It is important to
document the setup process as part of this (for future maintenance +
local reproducibility).

Just for clarification this is independent of running the tests
locally without docker, it is more to improve the reproducibility of
the environment we have on jenkins locally for example to address some
weird Heissenbug.

I just added BEAM-5311 to track the removal of the docker build images
when this is ready (of course if there are no objections to this
proposal).

[1] https://beam.apache.org/contribute/docker-images/
On Thu, Aug 30, 2018 at 3:58 PM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> Hi,
>
> That's interesting, however, it's really important to still be able to
> easily run test locally, without any VM/Docker required. It should be
> activated by profile or so.
>
> Regards
> JB
>
> On 27/08/2018 19:53, Yifan Zou wrote:
> > Hi,
> >
> > I have a proposal for creating a reproducible environment for Jenkins
> > tests by using docker container. The thing is, the environment
> > configurations on Beam Jenkins slaves are sometimes different from
> > developer's machines. Test failures on Jenkins may not be easy to
> > reproduce locally. Also, it is not convenient for developers to add or
> > modify underlying tools installed on Jenkins VMs, since they're managed
> > by Apache Infra. This proposal is aimed to address those problems.
> >
> > https://docs.google.com/document/d/1y0YuQj_oZXC0uM5-gniG7r9-5gv2uiDhzbtgYYJW48c/edit#heading=h.bg2yi0wbhl9n
> >
> > Any comments are welcome. Thank you.
> >
> > Regards.
> > Yifan
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com

Re: [Proposal] Creating a reproducible environment for Beam Jenkins Tests

[Yifan Zou <yi...@google.com>]

Thanks all. I am struggling with the missing buildscan reports when running
jobs with containers. I believe it is a big disadvantage to use docker if
the buildscan doesn't show up. I will keep updating my progress in this
thread. In the meanwhile, any comments, suggestions and objections are
still welcome.

Regards.
Yifan

On Tue, Sep 11, 2018 at 6:08 AM Alexey Romanenko <ar...@gmail.com>
wrote:

> +1 Great feature that should help with complicated error cases.
>
> On 11 Sep 2018, at 03:39, Henning Rohde <he...@google.com> wrote:
>
> +1 Nice proposal. It will help eradicate some of the inflexibility and
> frustrations with Jenkins.
>
> On Wed, Sep 5, 2018 at 2:30 PM Yifan Zou <yi...@google.com> wrote:
>
>> Thank you all for making comments on this and I apologize for the late
>> reply.
>>
>> To clarify the concerns of testing locally, it is still able to run tests
>> without Docker. One of the purposes of this is to create an identical
>> environment as we are running in Jenkins that would be helpful to reproduce
>> strange errors. Contributors could choose starting a container and run
>> tests in there, or just run tests directly.
>>
>>
>>
>> On Wed, Sep 5, 2018 at 6:37 AM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> BIG +1, the previous work on having docker build images [1] had a
>>> similar goal (to have a reproducible build environment). But this is
>>> even better because we will guarantee the exact same environment in
>>> Jenkins as well as any further improvements. It is important to
>>> document the setup process as part of this (for future maintenance +
>>> local reproducibility).
>>>
>>> Just for clarification this is independent of running the tests
>>> locally without docker, it is more to improve the reproducibility of
>>> the environment we have on jenkins locally for example to address some
>>> weird Heissenbug.
>>>
>>> I just added BEAM-5311 to track the removal of the docker build images
>>> when this is ready (of course if there are no objections to this
>>> proposal).
>>>
>>> [1] https://beam.apache.org/contribute/docker-images/
>>> On Thu, Aug 30, 2018 at 3:58 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>> >
>>> > Hi,
>>> >
>>> > That's interesting, however, it's really important to still be able to
>>> > easily run test locally, without any VM/Docker required. It should be
>>> > activated by profile or so.
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On 27/08/2018 19:53, Yifan Zou wrote:
>>> > > Hi,
>>> > >
>>> > > I have a proposal for creating a reproducible environment for Jenkins
>>> > > tests by using docker container. The thing is, the environment
>>> > > configurations on Beam Jenkins slaves are sometimes different from
>>> > > developer's machines. Test failures on Jenkins may not be easy to
>>> > > reproduce locally. Also, it is not convenient for developers to add
>>> or
>>> > > modify underlying tools installed on Jenkins VMs, since they're
>>> managed
>>> > > by Apache Infra. This proposal is aimed to address those problems.
>>> > >
>>> > >
>>> https://docs.google.com/document/d/1y0YuQj_oZXC0uM5-gniG7r9-5gv2uiDhzbtgYYJW48c/edit#heading=h.bg2yi0wbhl9n
>>> > >
>>> > > Any comments are welcome. Thank you.
>>> > >
>>> > > Regards.
>>> > > Yifan
>>> > >
>>> >
>>> > --
>>> > Jean-Baptiste Onofré
>>> > jbonofre@apache.org
>>> > http://blog.nanthrax.net
>>> > Talend - http://www.talend.com
>>>
>>
>

Re: [Proposal] Creating a reproducible environment for Beam Jenkins Tests

[Alexey Romanenko <ar...@gmail.com>]

+1 Great feature that should help with complicated error cases.

> On 11 Sep 2018, at 03:39, Henning Rohde <he...@google.com> wrote:
> 
> +1 Nice proposal. It will help eradicate some of the inflexibility and frustrations with Jenkins.
> 
> On Wed, Sep 5, 2018 at 2:30 PM Yifan Zou <yifanzou@google.com <ma...@google.com>> wrote:
> Thank you all for making comments on this and I apologize for the late reply. 
> 
> To clarify the concerns of testing locally, it is still able to run tests without Docker. One of the purposes of this is to create an identical environment as we are running in Jenkins that would be helpful to reproduce strange errors. Contributors could choose starting a container and run tests in there, or just run tests directly. 
> 
> 
> 
> On Wed, Sep 5, 2018 at 6:37 AM Ismaël Mejía <iemejia@gmail.com <ma...@gmail.com>> wrote:
> BIG +1, the previous work on having docker build images [1] had a
> similar goal (to have a reproducible build environment). But this is
> even better because we will guarantee the exact same environment in
> Jenkins as well as any further improvements. It is important to
> document the setup process as part of this (for future maintenance +
> local reproducibility).
> 
> Just for clarification this is independent of running the tests
> locally without docker, it is more to improve the reproducibility of
> the environment we have on jenkins locally for example to address some
> weird Heissenbug.
> 
> I just added BEAM-5311 to track the removal of the docker build images
> when this is ready (of course if there are no objections to this
> proposal).
> 
> [1] https://beam.apache.org/contribute/docker-images/ <https://beam.apache.org/contribute/docker-images/>
> On Thu, Aug 30, 2018 at 3:58 PM Jean-Baptiste Onofré <jb@nanthrax.net <ma...@nanthrax.net>> wrote:
> >
> > Hi,
> >
> > That's interesting, however, it's really important to still be able to
> > easily run test locally, without any VM/Docker required. It should be
> > activated by profile or so.
> >
> > Regards
> > JB
> >
> > On 27/08/2018 19:53, Yifan Zou wrote:
> > > Hi,
> > >
> > > I have a proposal for creating a reproducible environment for Jenkins
> > > tests by using docker container. The thing is, the environment
> > > configurations on Beam Jenkins slaves are sometimes different from
> > > developer's machines. Test failures on Jenkins may not be easy to
> > > reproduce locally. Also, it is not convenient for developers to add or
> > > modify underlying tools installed on Jenkins VMs, since they're managed
> > > by Apache Infra. This proposal is aimed to address those problems.
> > >
> > > https://docs.google.com/document/d/1y0YuQj_oZXC0uM5-gniG7r9-5gv2uiDhzbtgYYJW48c/edit#heading=h.bg2yi0wbhl9n <https://docs.google.com/document/d/1y0YuQj_oZXC0uM5-gniG7r9-5gv2uiDhzbtgYYJW48c/edit#heading=h.bg2yi0wbhl9n>
> > >
> > > Any comments are welcome. Thank you.
> > >
> > > Regards.
> > > Yifan
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net <http://blog.nanthrax.net/>
> > Talend - http://www.talend.com <http://www.talend.com/>

Re: [Proposal] Creating a reproducible environment for Beam Jenkins Tests

[Henning Rohde <he...@google.com>]

+1 Nice proposal. It will help eradicate some of the inflexibility and
frustrations with Jenkins.

On Wed, Sep 5, 2018 at 2:30 PM Yifan Zou <yi...@google.com> wrote:

> Thank you all for making comments on this and I apologize for the late
> reply.
>
> To clarify the concerns of testing locally, it is still able to run tests
> without Docker. One of the purposes of this is to create an identical
> environment as we are running in Jenkins that would be helpful to reproduce
> strange errors. Contributors could choose starting a container and run
> tests in there, or just run tests directly.
>
>
>
> On Wed, Sep 5, 2018 at 6:37 AM Ismaël Mejía <ie...@gmail.com> wrote:
>
>> BIG +1, the previous work on having docker build images [1] had a
>> similar goal (to have a reproducible build environment). But this is
>> even better because we will guarantee the exact same environment in
>> Jenkins as well as any further improvements. It is important to
>> document the setup process as part of this (for future maintenance +
>> local reproducibility).
>>
>> Just for clarification this is independent of running the tests
>> locally without docker, it is more to improve the reproducibility of
>> the environment we have on jenkins locally for example to address some
>> weird Heissenbug.
>>
>> I just added BEAM-5311 to track the removal of the docker build images
>> when this is ready (of course if there are no objections to this
>> proposal).
>>
>> [1] https://beam.apache.org/contribute/docker-images/
>> On Thu, Aug 30, 2018 at 3:58 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>> >
>> > Hi,
>> >
>> > That's interesting, however, it's really important to still be able to
>> > easily run test locally, without any VM/Docker required. It should be
>> > activated by profile or so.
>> >
>> > Regards
>> > JB
>> >
>> > On 27/08/2018 19:53, Yifan Zou wrote:
>> > > Hi,
>> > >
>> > > I have a proposal for creating a reproducible environment for Jenkins
>> > > tests by using docker container. The thing is, the environment
>> > > configurations on Beam Jenkins slaves are sometimes different from
>> > > developer's machines. Test failures on Jenkins may not be easy to
>> > > reproduce locally. Also, it is not convenient for developers to add or
>> > > modify underlying tools installed on Jenkins VMs, since they're
>> managed
>> > > by Apache Infra. This proposal is aimed to address those problems.
>> > >
>> > >
>> https://docs.google.com/document/d/1y0YuQj_oZXC0uM5-gniG7r9-5gv2uiDhzbtgYYJW48c/edit#heading=h.bg2yi0wbhl9n
>> > >
>> > > Any comments are welcome. Thank you.
>> > >
>> > > Regards.
>> > > Yifan
>> > >
>> >
>> > --
>> > Jean-Baptiste Onofré
>> > jbonofre@apache.org
>> > http://blog.nanthrax.net
>> > Talend - http://www.talend.com
>>
>

Re: [Proposal] Creating a reproducible environment for Beam Jenkins Tests

[Yifan Zou <yi...@google.com>]

Thank you all for making comments on this and I apologize for the late
reply.

To clarify the concerns of testing locally, it is still able to run tests
without Docker. One of the purposes of this is to create an identical
environment as we are running in Jenkins that would be helpful to reproduce
strange errors. Contributors could choose starting a container and run
tests in there, or just run tests directly.



On Wed, Sep 5, 2018 at 6:37 AM Ismaël Mejía <ie...@gmail.com> wrote:

> BIG +1, the previous work on having docker build images [1] had a
> similar goal (to have a reproducible build environment). But this is
> even better because we will guarantee the exact same environment in
> Jenkins as well as any further improvements. It is important to
> document the setup process as part of this (for future maintenance +
> local reproducibility).
>
> Just for clarification this is independent of running the tests
> locally without docker, it is more to improve the reproducibility of
> the environment we have on jenkins locally for example to address some
> weird Heissenbug.
>
> I just added BEAM-5311 to track the removal of the docker build images
> when this is ready (of course if there are no objections to this
> proposal).
>
> [1] https://beam.apache.org/contribute/docker-images/
> On Thu, Aug 30, 2018 at 3:58 PM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
> >
> > Hi,
> >
> > That's interesting, however, it's really important to still be able to
> > easily run test locally, without any VM/Docker required. It should be
> > activated by profile or so.
> >
> > Regards
> > JB
> >
> > On 27/08/2018 19:53, Yifan Zou wrote:
> > > Hi,
> > >
> > > I have a proposal for creating a reproducible environment for Jenkins
> > > tests by using docker container. The thing is, the environment
> > > configurations on Beam Jenkins slaves are sometimes different from
> > > developer's machines. Test failures on Jenkins may not be easy to
> > > reproduce locally. Also, it is not convenient for developers to add or
> > > modify underlying tools installed on Jenkins VMs, since they're managed
> > > by Apache Infra. This proposal is aimed to address those problems.
> > >
> > >
> https://docs.google.com/document/d/1y0YuQj_oZXC0uM5-gniG7r9-5gv2uiDhzbtgYYJW48c/edit#heading=h.bg2yi0wbhl9n
> > >
> > > Any comments are welcome. Thank you.
> > >
> > > Regards.
> > > Yifan
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
>