[DISCUSS] Separating Cloud Environment(s) - Separate Dev/Test from Production 'Testing' environment(s)

[Austin Bennett <wh...@gmail.com>]

Howdy!

I'm increasingly concerned by our Cloud 'Testing' environment ( resources
associated with the 'apache-beam-testing' GCP project ).

The following might sound a little strange, given repetition of words [ and
many meanings of 'testing']:

* There is a 'production' need for ongoing testing infrastructure, tests,
pipelines, builds.
and
* There are also a 'development' and/or 'test' needs for extending the
functionality, tests, and exploring new things, before definitely adopting
and making production [ ex: getting onto master ].
and
* There is a very real possibility that someone 'testing' or 'developing'
new features/functionality, could accidentally delete/change/modify
critical resources in the solo environment -AND- We do not have super
quick/easy restore paths in the event of serious trouble - ex: if things
deleted.  [ I assume, as I don't see terraform/other code that seems
definitely easy to re-create resources ...  BUT, I could be missing and not
understanding this point ].


There could be benefits [ also costs/burdens, or pros/cons to consider ]
from separating functions/environments to minimize when/how people are
manually using resources/roles with permissions to modify the critical
infrastructure.

I have some ideas for steps to address [ and where I could step in to help
implement ], but before starting to formalize ideas and strategies [ longer
write-ups, etc ], wanted to open for discussion to ensure whether this is
thinking that makes sense, that the community believes would be valuable.
If done well, longer-term I imagine most of the to-be-solution would have
little effect on actual users of environments [ including
developers/committers/etc ] and mostly would be around better separation
and more attention paid to controls/branches/roles/permissions.

*Can people share thoughts around potentially having at least one more
environment to add additional safety around our critical code testing
infrastructure, to keep that further distinct from workflows/process/code
that is still being developed?*

Do advise,
Cheers,
Austin

Re: [DISCUSS] Separating Cloud Environment(s) - Separate Dev/Test from Production 'Testing' environment(s)

[Kenneth Knowles <ke...@apache.org>]

Seems like an inventory of our infrastructure (which at least partly exists
on the wiki) could be helpful for framing this.

On Thu, Oct 6, 2022 at 11:10 AM Austin Bennett <wh...@gmail.com>
wrote:

> Howdy!
>
> I'm increasingly concerned by our Cloud 'Testing' environment ( resources
> associated with the 'apache-beam-testing' GCP project ).
>
> The following might sound a little strange, given repetition of words [
> and many meanings of 'testing']:
>
> * There is a 'production' need for ongoing testing infrastructure, tests,
> pipelines, builds.
> and
> * There are also a 'development' and/or 'test' needs for extending the
> functionality, tests, and exploring new things, before definitely adopting
> and making production [ ex: getting onto master ].
> and
> * There is a very real possibility that someone 'testing' or 'developing'
> new features/functionality, could accidentally delete/change/modify
> critical resources in the solo environment -AND- We do not have super
> quick/easy restore paths in the event of serious trouble - ex: if things
> deleted.  [ I assume, as I don't see terraform/other code that seems
> definitely easy to re-create resources ...  BUT, I could be missing and not
> understanding this point ].
>
>
> There could be benefits [ also costs/burdens, or pros/cons to consider ]
> from separating functions/environments to minimize when/how people are
> manually using resources/roles with permissions to modify the critical
> infrastructure.
>
> I have some ideas for steps to address [ and where I could step in to help
> implement ], but before starting to formalize ideas and strategies [ longer
> write-ups, etc ], wanted to open for discussion to ensure whether this is
> thinking that makes sense, that the community believes would be valuable.
> If done well, longer-term I imagine most of the to-be-solution would have
> little effect on actual users of environments [ including
> developers/committers/etc ] and mostly would be around better separation
> and more attention paid to controls/branches/roles/permissions.
>
> *Can people share thoughts around potentially having at least one more
> environment to add additional safety around our critical code testing
> infrastructure, to keep that further distinct from workflows/process/code
> that is still being developed?*
>
> Do advise,
> Cheers,
> Austin
>
>
>
>