You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Sam Rohde <sr...@google.com> on 2020/11/25 22:52:15 UTC
Docker Development Environment
Hi All,
I got tired of my local dev environment being ruined by updates so I made a
container for Apache Beam development work. What this does is create a
Docker container from the Ubuntu Groovy image and load it up with all the
necessary libraries/utilities for Apache Beam development. Then I run an
interactive shell in the Docker container where I do my work.
This is a nice way for new contributors to easily get started. However with
the container in its current form, I don't know if this will help other
people because it is tied closely with my workflow (using VIM,
YouCompleteMe, for Python). But I think it can be a nice starting point for
improvements. For example:
- Sharing the host display with Docker to start GUI applications (like
IntelliJ) in the container
- Adding Golang development support
Here's a draft PR <https://github.com/apache/beam/pull/13430>, let me know
what you think, how it can be improved, and whether it's a good idea for us
to have a dev container like this.
Regards,
Sam
Re: Docker Development Environment
Posted by Brian Hulette <bh...@google.com>.
Thanks Alex, I'll take a look
On Mon, Dec 14, 2020 at 12:48 PM Alex Kosolapov <al...@akvelon.com>
wrote:
> Hi!
>
>
>
> Wanted to send a note that PR with changes submitted
> https://github.com/apache/beam/pull/13545 and ready for code review
>
>
>
> Thanks,
>
> Alex
>
> *From: *Alex Kosolapov <al...@akvelon.com>
> *Date: *Thursday, December 10, 2020 at 3:44 PM
> *To: *"dev@beam.apache.org" <de...@beam.apache.org>
> *Subject: *Re: Docker Development Environment
>
>
>
> Hi Brian,
>
>
>
> Yes - will update the script, created JIRA to track
> https://issues.apache.org/jira/browse/BEAM-11438
>
>
>
> I was thinking about dscl initially too - and dscl will require user to
> create docker group in this case as you noted. For some users that might be
> an extra manual step to do (group not created automatically by Docker
> install).
>
>
>
> Can approaches be combined?
>
> 1. If docker group created – use docker group’s gid
> 2. If docker group not created – assign some default, e.g. 1000 or
> some other safe way of handling “no docker group” case
>
>
>
> Thanks,
>
> Alex
>
>
>
> *From: *Brian Hulette <bh...@google.com>
> *Reply-To: *"dev@beam.apache.org" <de...@beam.apache.org>
> *Date: *Wednesday, December 9, 2020 at 12:21 PM
> *To: *dev <de...@beam.apache.org>
> *Subject: *Re: Docker Development Environment
>
>
>
> Hey Alex,
> Thanks for trying this out on macOS! Some responses inline
>
>
>
> On Tue, Dec 8, 2020 at 11:42 AM Alex Kosolapov <al...@akvelon.com>
> wrote:
>
> Hi!
>
>
>
> Thank you for creating Docker build environment - makes build environment
> setup so much easier!
>
>
>
> I ran start-build-env.sh on a macOS, and I ran into some items that wanted
> to share + propose how to improve Docker build environment for macOS
> support:
>
> 1. ./start-build-env.sh: line 75: getent: command not found and script
> build error downstream.
>
>
>
> Step 26/26 : RUN echo '. /scripts/beam_env_checks.sh' >>
> /root/.bash_aliases
>
> ---> Using cache
>
> ---> fe48b8b26e91
>
> Successfully built fe48b8b26e91
>
> Successfully tagged beam-build:latest
>
> *./start-build-env.sh: line 75: getent: command not found*
>
> Sending build context to Docker daemon 2.56kB
>
> Step 1/10 : FROM beam-build
>
> ---> fe48b8b26e91
>
> Step 2/10 : RUN rm -f /var/log/faillog /var/log/lastlog
>
> ---> Using cache
>
> ---> 11ea31f2099e
>
> Step 3/10 : RUN groupadd --non-unique -g 100 alex
>
> ---> Using cache
>
> ---> cf3f56f51d9f
>
> *Step 4/10 : RUN groupmod -g docker*
>
> * ---> Running in 5d5bc473be3b*
>
> *groupmod: invalid group ID 'docker'*
>
> *The command '/bin/bash -o pipefail -c groupmod -g docker' returned a
> non-zero code: 3*
>
>
>
> I investigated the issues and found that start-build-env.sh expected group
> "docker" created in the system groups. Creating a docker group is an
> optional post install step (
> https://docs.docker.com/engine/install/linux-postinstall/), another relevant
> reference to forum <https://forums.docker.com/t/no-more-docker-group/9123>
> gives a hint of using staff group on macOS. Solution that worked for me was
> pick another GID for DOCKER_GROUP_ID on Mac, e.g.
>
> #DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
>
>
>
> if [ "$(uname -s)" = "Linux" ]; then
>
> DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
>
> fi
>
> if [ "$(uname -s)" = "Darwin" ]; then
>
> DOCKER_GROUP_ID=1000
>
> fi
>
>
>
> I think everyone using this container should do that postinstall step so
> that docker doesn't have to be run as root. We actually point to it in an
> error message if we detect the script is run as root. I think the real
> issue here is that getent doesn't exist on MacOS, it looks like there is an
> alternative approach though:
> https://stackoverflow.com/questions/10910096/what-is-the-command-to-get-groupid-of-a-group-name-in-mac-or-linux
>
> Would you be ok updating the script to use that approach on macOS? This
> would still require users to make a docker group.
>
>
>
>
>
> 1. Step 10/10 : RUN chown -R alex: /Users/alex/.cache
>
> ---> Running in f09e2bb0e045
>
> chown: cannot access '/Users/alex/.cache': No such file or directory
>
> The command '/bin/bash -o pipefail -c chown -R alex: /Users/alex/.cache'
> returned a non-zero code: 1
>
>
>
> Fix for this was to change from HOME to DOCKER_HOME_DIR:
>
> RUN chown -R ${USER_NAME}:${GROUP_ID}
> ${DOCKER_HOME_DIR}/.cache
>
> ENV GOPATH
> ${DOCKER_HOME_DIR}/beam/sdks/go/examples/.gogradle/project_gopath
>
> Ah yes this definitely looks like a bug. It just happened to work when
> running from a machine where the HOME dir was the same as in the container.
> This one definitely makes sense.
>
>
>
> If these proposed solutions make sense I will create JIRA ticket and
> submit these improvements for Docker build support on macOS.
>
>
>
> Thank you,
>
> Alex Kosolapov
>
>
>
> *From: *Brian Hulette <bh...@google.com>
> *Reply-To: *"dev@beam.apache.org" <de...@beam.apache.org>
> *Date: *Friday, December 4, 2020 at 1:15 PM
> *To: *dev <de...@beam.apache.org>
> *Cc: *Omar Ismail <om...@google.com>
> *Subject: *Re: Docker Development Environment
>
>
>
> I think https://github.com/apache/beam/pull/13308 is about ready to
> merge. One question was whether or not to install pyenv in the container -
> I think we should try to do without it. Users of this environment will
> already be operating within a container, so they shouldn't need pyenv to
> create isolated python environments, they could just use the container's
> system python.
>
>
>
> The two issues Alex mentioned are still outstanding, but they seem to be
> issues with ./gradlew check unrelated to the container. I filed BEAM-11402
> [1] to track this separately.
>
>
>
> [1] https://issues.apache.org/jira/browse/BEAM-11402
>
>
>
> On Mon, Nov 30, 2020 at 11:43 AM Alex Amato <aj...@google.com> wrote:
>
> If any of these are suitable for at least some development. I propose we
> merge them, and update them with fixes later. Rather than trying to get
> things 100% working in the first PR.
>
>
>
> Looks like this one was opened in early Sept, and never got merged. Which
> is a pretty long time. Perhaps abandoned for the later?
>
> https://github.com/apache/beam/pull/12837
>
>
>
> This one looks like its just failing on just a few tests (Which may be
> addressed soon, but the PR was opened 19 days ago).
>
> https://github.com/apache/beam/pull/13308
>
> (Can we set a deadline for this? And just say merge it by the end of the
> week, regardless if the last two tests can be fixed or not?)
>
>
>
> (Would like to nudge this along, as it's been a pain point for many for a
> while now).
>
>
>
> Thanks for the work here Niels, Omar and Sam.
>
> Looking forward to giving this a try :)
>
>
>
>
>
> On Mon, Nov 30, 2020 at 11:32 AM Brian Hulette <bh...@google.com>
> wrote:
>
> I agree this is a good idea. I remember my first experience with Beam
> development - I ran through the steps at [1] and had `./gradlew check`
> fail. I don't think I ever got it working before moving on and just running
> more specific tasks.
>
> It would be great if we had a reliable way for new contributors to
> establish an environment that can successfully run `./gradlew check`.
>
>
>
> Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be
> close to that, so I think we should focus on getting that working and
> iterate from there. Omar concurred with that in
> https://github.com/apache/beam/pull/12837.
>
>
> [1] https://beam.apache.org/contribute/#development-setup
>
>
>
>
>
> On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com> wrote:
>
> Thank you for doing this.
>
>
>
> I have seen a few related PRs. Connecting them here in case these efforts
> could be combined:
>
> - https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail
> <om...@google.com> )
>
> - https://github.com/apache/beam/pull/13308
>
>
>
> Ahmet
>
>
>
> On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com> wrote:
>
> Hi All,
>
>
>
> I got tired of my local dev environment being ruined by updates so I made
> a container for Apache Beam development work. What this does is create a
> Docker container from the Ubuntu Groovy image and load it up with all the
> necessary libraries/utilities for Apache Beam development. Then I run an
> interactive shell in the Docker container where I do my work.
>
>
>
> This is a nice way for new contributors to easily get started. However
> with the container in its current form, I don't know if this will help
> other people because it is tied closely with my workflow (using VIM,
> YouCompleteMe, for Python). But I think it can be a nice starting point for
> improvements. For example:
>
> - Sharing the host display with Docker to start GUI applications (like
> IntelliJ) in the container
> - Adding Golang development support
>
> Here's a draft PR <https://github.com/apache/beam/pull/13430>, let me
> know what you think, how it can be improved, and whether it's a good idea
> for us to have a dev container like this.
>
>
>
> Regards,
>
> Sam
>
>
>
>
Re: Docker Development Environment
Posted by Alex Kosolapov <al...@akvelon.com>.
Hi!
Wanted to send a note that PR with changes submitted https://github.com/apache/beam/pull/13545 and ready for code review
Thanks,
Alex
From: Alex Kosolapov <al...@akvelon.com>
Date: Thursday, December 10, 2020 at 3:44 PM
To: "dev@beam.apache.org" <de...@beam.apache.org>
Subject: Re: Docker Development Environment
Hi Brian,
Yes - will update the script, created JIRA to track https://issues.apache.org/jira/browse/BEAM-11438
I was thinking about dscl initially too - and dscl will require user to create docker group in this case as you noted. For some users that might be an extra manual step to do (group not created automatically by Docker install).
Can approaches be combined?
1. If docker group created – use docker group’s gid
2. If docker group not created – assign some default, e.g. 1000 or some other safe way of handling “no docker group” case
Thanks,
Alex
From: Brian Hulette <bh...@google.com>
Reply-To: "dev@beam.apache.org" <de...@beam.apache.org>
Date: Wednesday, December 9, 2020 at 12:21 PM
To: dev <de...@beam.apache.org>
Subject: Re: Docker Development Environment
Hey Alex,
Thanks for trying this out on macOS! Some responses inline
On Tue, Dec 8, 2020 at 11:42 AM Alex Kosolapov <al...@akvelon.com>> wrote:
Hi!
Thank you for creating Docker build environment - makes build environment setup so much easier!
I ran start-build-env.sh on a macOS, and I ran into some items that wanted to share + propose how to improve Docker build environment for macOS support:
1. ./start-build-env.sh: line 75: getent: command not found and script build error downstream.
Step 26/26 : RUN echo '. /scripts/beam_env_checks.sh' >> /root/.bash_aliases
---> Using cache
---> fe48b8b26e91
Successfully built fe48b8b26e91
Successfully tagged beam-build:latest
./start-build-env.sh: line 75: getent: command not found
Sending build context to Docker daemon 2.56kB
Step 1/10 : FROM beam-build
---> fe48b8b26e91
Step 2/10 : RUN rm -f /var/log/faillog /var/log/lastlog
---> Using cache
---> 11ea31f2099e
Step 3/10 : RUN groupadd --non-unique -g 100 alex
---> Using cache
---> cf3f56f51d9f
Step 4/10 : RUN groupmod -g docker
---> Running in 5d5bc473be3b
groupmod: invalid group ID 'docker'
The command '/bin/bash -o pipefail -c groupmod -g docker' returned a non-zero code: 3
I investigated the issues and found that start-build-env.sh expected group "docker" created in the system groups. Creating a docker group is an optional post install step (https://docs.docker.com/engine/install/linux-postinstall/), another relevant reference to forum<https://forums.docker.com/t/no-more-docker-group/9123> gives a hint of using staff group on macOS. Solution that worked for me was pick another GID for DOCKER_GROUP_ID on Mac, e.g.
#DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
if [ "$(uname -s)" = "Linux" ]; then
DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
fi
if [ "$(uname -s)" = "Darwin" ]; then
DOCKER_GROUP_ID=1000
fi
I think everyone using this container should do that postinstall step so that docker doesn't have to be run as root. We actually point to it in an error message if we detect the script is run as root. I think the real issue here is that getent doesn't exist on MacOS, it looks like there is an alternative approach though: https://stackoverflow.com/questions/10910096/what-is-the-command-to-get-groupid-of-a-group-name-in-mac-or-linux
Would you be ok updating the script to use that approach on macOS? This would still require users to make a docker group.
1. Step 10/10 : RUN chown -R alex: /Users/alex/.cache
---> Running in f09e2bb0e045
chown: cannot access '/Users/alex/.cache': No such file or directory
The command '/bin/bash -o pipefail -c chown -R alex: /Users/alex/.cache' returned a non-zero code: 1
Fix for this was to change from HOME to DOCKER_HOME_DIR:
RUN chown -R ${USER_NAME}:${GROUP_ID} ${DOCKER_HOME_DIR}/.cache
ENV GOPATH ${DOCKER_HOME_DIR}/beam/sdks/go/examples/.gogradle/project_gopath
Ah yes this definitely looks like a bug. It just happened to work when running from a machine where the HOME dir was the same as in the container. This one definitely makes sense.
If these proposed solutions make sense I will create JIRA ticket and submit these improvements for Docker build support on macOS.
Thank you,
Alex Kosolapov
From: Brian Hulette <bh...@google.com>>
Reply-To: "dev@beam.apache.org<ma...@beam.apache.org>" <de...@beam.apache.org>>
Date: Friday, December 4, 2020 at 1:15 PM
To: dev <de...@beam.apache.org>>
Cc: Omar Ismail <om...@google.com>>
Subject: Re: Docker Development Environment
I think https://github.com/apache/beam/pull/13308 is about ready to merge. One question was whether or not to install pyenv in the container - I think we should try to do without it. Users of this environment will already be operating within a container, so they shouldn't need pyenv to create isolated python environments, they could just use the container's system python.
The two issues Alex mentioned are still outstanding, but they seem to be issues with ./gradlew check unrelated to the container. I filed BEAM-11402 [1] to track this separately.
[1] https://issues.apache.org/jira/browse/BEAM-11402
On Mon, Nov 30, 2020 at 11:43 AM Alex Amato <aj...@google.com>> wrote:
If any of these are suitable for at least some development. I propose we merge them, and update them with fixes later. Rather than trying to get things 100% working in the first PR.
Looks like this one was opened in early Sept, and never got merged. Which is a pretty long time. Perhaps abandoned for the later?
https://github.com/apache/beam/pull/12837
This one looks like its just failing on just a few tests (Which may be addressed soon, but the PR was opened 19 days ago).
https://github.com/apache/beam/pull/13308
(Can we set a deadline for this? And just say merge it by the end of the week, regardless if the last two tests can be fixed or not?)
(Would like to nudge this along, as it's been a pain point for many for a while now).
Thanks for the work here Niels, Omar and Sam.
Looking forward to giving this a try :)
On Mon, Nov 30, 2020 at 11:32 AM Brian Hulette <bh...@google.com>> wrote:
I agree this is a good idea. I remember my first experience with Beam development - I ran through the steps at [1] and had `./gradlew check` fail. I don't think I ever got it working before moving on and just running more specific tasks.
It would be great if we had a reliable way for new contributors to establish an environment that can successfully run `./gradlew check`.
Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be close to that, so I think we should focus on getting that working and iterate from there. Omar concurred with that in https://github.com/apache/beam/pull/12837.
[1] https://beam.apache.org/contribute/#development-setup
On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com>> wrote:
Thank you for doing this.
I have seen a few related PRs. Connecting them here in case these efforts could be combined:
- https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail<ma...@google.com> )
- https://github.com/apache/beam/pull/13308
Ahmet
On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com>> wrote:
Hi All,
I got tired of my local dev environment being ruined by updates so I made a container for Apache Beam development work. What this does is create a Docker container from the Ubuntu Groovy image and load it up with all the necessary libraries/utilities for Apache Beam development. Then I run an interactive shell in the Docker container where I do my work.
This is a nice way for new contributors to easily get started. However with the container in its current form, I don't know if this will help other people because it is tied closely with my workflow (using VIM, YouCompleteMe, for Python). But I think it can be a nice starting point for improvements. For example:
* Sharing the host display with Docker to start GUI applications (like IntelliJ) in the container
* Adding Golang development support
Here's a draft PR<https://github.com/apache/beam/pull/13430>, let me know what you think, how it can be improved, and whether it's a good idea for us to have a dev container like this.
Regards,
Sam
Re: Docker Development Environment
Posted by Alex Kosolapov <al...@akvelon.com>.
Hi Brian,
Yes - will update the script, created JIRA to track https://issues.apache.org/jira/browse/BEAM-11438
I was thinking about dscl initially too - and dscl will require user to create docker group in this case as you noted. For some users that might be an extra manual step to do (group not created automatically by Docker install).
Can approaches be combined?
1. If docker group created – use docker group’s gid
2. If docker group not created – assign some default, e.g. 1000 or some other safe way of handling “no docker group” case
Thanks,
Alex
From: Brian Hulette <bh...@google.com>
Reply-To: "dev@beam.apache.org" <de...@beam.apache.org>
Date: Wednesday, December 9, 2020 at 12:21 PM
To: dev <de...@beam.apache.org>
Subject: Re: Docker Development Environment
Hey Alex,
Thanks for trying this out on macOS! Some responses inline
On Tue, Dec 8, 2020 at 11:42 AM Alex Kosolapov <al...@akvelon.com>> wrote:
Hi!
Thank you for creating Docker build environment - makes build environment setup so much easier!
I ran start-build-env.sh on a macOS, and I ran into some items that wanted to share + propose how to improve Docker build environment for macOS support:
1. ./start-build-env.sh: line 75: getent: command not found and script build error downstream.
Step 26/26 : RUN echo '. /scripts/beam_env_checks.sh' >> /root/.bash_aliases
---> Using cache
---> fe48b8b26e91
Successfully built fe48b8b26e91
Successfully tagged beam-build:latest
./start-build-env.sh: line 75: getent: command not found
Sending build context to Docker daemon 2.56kB
Step 1/10 : FROM beam-build
---> fe48b8b26e91
Step 2/10 : RUN rm -f /var/log/faillog /var/log/lastlog
---> Using cache
---> 11ea31f2099e
Step 3/10 : RUN groupadd --non-unique -g 100 alex
---> Using cache
---> cf3f56f51d9f
Step 4/10 : RUN groupmod -g docker
---> Running in 5d5bc473be3b
groupmod: invalid group ID 'docker'
The command '/bin/bash -o pipefail -c groupmod -g docker' returned a non-zero code: 3
I investigated the issues and found that start-build-env.sh expected group "docker" created in the system groups. Creating a docker group is an optional post install step (https://docs.docker.com/engine/install/linux-postinstall/), another relevant reference to forum<https://forums.docker.com/t/no-more-docker-group/9123> gives a hint of using staff group on macOS. Solution that worked for me was pick another GID for DOCKER_GROUP_ID on Mac, e.g.
#DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
if [ "$(uname -s)" = "Linux" ]; then
DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
fi
if [ "$(uname -s)" = "Darwin" ]; then
DOCKER_GROUP_ID=1000
fi
I think everyone using this container should do that postinstall step so that docker doesn't have to be run as root. We actually point to it in an error message if we detect the script is run as root. I think the real issue here is that getent doesn't exist on MacOS, it looks like there is an alternative approach though: https://stackoverflow.com/questions/10910096/what-is-the-command-to-get-groupid-of-a-group-name-in-mac-or-linux
Would you be ok updating the script to use that approach on macOS? This would still require users to make a docker group.
1. Step 10/10 : RUN chown -R alex: /Users/alex/.cache
---> Running in f09e2bb0e045
chown: cannot access '/Users/alex/.cache': No such file or directory
The command '/bin/bash -o pipefail -c chown -R alex: /Users/alex/.cache' returned a non-zero code: 1
Fix for this was to change from HOME to DOCKER_HOME_DIR:
RUN chown -R ${USER_NAME}:${GROUP_ID} ${DOCKER_HOME_DIR}/.cache
ENV GOPATH ${DOCKER_HOME_DIR}/beam/sdks/go/examples/.gogradle/project_gopath
Ah yes this definitely looks like a bug. It just happened to work when running from a machine where the HOME dir was the same as in the container. This one definitely makes sense.
If these proposed solutions make sense I will create JIRA ticket and submit these improvements for Docker build support on macOS.
Thank you,
Alex Kosolapov
From: Brian Hulette <bh...@google.com>>
Reply-To: "dev@beam.apache.org<ma...@beam.apache.org>" <de...@beam.apache.org>>
Date: Friday, December 4, 2020 at 1:15 PM
To: dev <de...@beam.apache.org>>
Cc: Omar Ismail <om...@google.com>>
Subject: Re: Docker Development Environment
I think https://github.com/apache/beam/pull/13308 is about ready to merge. One question was whether or not to install pyenv in the container - I think we should try to do without it. Users of this environment will already be operating within a container, so they shouldn't need pyenv to create isolated python environments, they could just use the container's system python.
The two issues Alex mentioned are still outstanding, but they seem to be issues with ./gradlew check unrelated to the container. I filed BEAM-11402 [1] to track this separately.
[1] https://issues.apache.org/jira/browse/BEAM-11402
On Mon, Nov 30, 2020 at 11:43 AM Alex Amato <aj...@google.com>> wrote:
If any of these are suitable for at least some development. I propose we merge them, and update them with fixes later. Rather than trying to get things 100% working in the first PR.
Looks like this one was opened in early Sept, and never got merged. Which is a pretty long time. Perhaps abandoned for the later?
https://github.com/apache/beam/pull/12837
This one looks like its just failing on just a few tests (Which may be addressed soon, but the PR was opened 19 days ago).
https://github.com/apache/beam/pull/13308
(Can we set a deadline for this? And just say merge it by the end of the week, regardless if the last two tests can be fixed or not?)
(Would like to nudge this along, as it's been a pain point for many for a while now).
Thanks for the work here Niels, Omar and Sam.
Looking forward to giving this a try :)
On Mon, Nov 30, 2020 at 11:32 AM Brian Hulette <bh...@google.com>> wrote:
I agree this is a good idea. I remember my first experience with Beam development - I ran through the steps at [1] and had `./gradlew check` fail. I don't think I ever got it working before moving on and just running more specific tasks.
It would be great if we had a reliable way for new contributors to establish an environment that can successfully run `./gradlew check`.
Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be close to that, so I think we should focus on getting that working and iterate from there. Omar concurred with that in https://github.com/apache/beam/pull/12837.
[1] https://beam.apache.org/contribute/#development-setup
On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com>> wrote:
Thank you for doing this.
I have seen a few related PRs. Connecting them here in case these efforts could be combined:
- https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail<ma...@google.com> )
- https://github.com/apache/beam/pull/13308
Ahmet
On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com>> wrote:
Hi All,
I got tired of my local dev environment being ruined by updates so I made a container for Apache Beam development work. What this does is create a Docker container from the Ubuntu Groovy image and load it up with all the necessary libraries/utilities for Apache Beam development. Then I run an interactive shell in the Docker container where I do my work.
This is a nice way for new contributors to easily get started. However with the container in its current form, I don't know if this will help other people because it is tied closely with my workflow (using VIM, YouCompleteMe, for Python). But I think it can be a nice starting point for improvements. For example:
* Sharing the host display with Docker to start GUI applications (like IntelliJ) in the container
* Adding Golang development support
Here's a draft PR<https://github.com/apache/beam/pull/13430>, let me know what you think, how it can be improved, and whether it's a good idea for us to have a dev container like this.
Regards,
Sam
Re: Docker Development Environment
Posted by Brian Hulette <bh...@google.com>.
Hey Alex,
Thanks for trying this out on macOS! Some responses inline
On Tue, Dec 8, 2020 at 11:42 AM Alex Kosolapov <al...@akvelon.com>
wrote:
> Hi!
>
>
>
> Thank you for creating Docker build environment - makes build environment
> setup so much easier!
>
>
>
> I ran start-build-env.sh on a macOS, and I ran into some items that wanted
> to share + propose how to improve Docker build environment for macOS
> support:
>
> 1. ./start-build-env.sh: line 75: getent: command not found and script
> build error downstream.
>
>
>
> Step 26/26 : RUN echo '. /scripts/beam_env_checks.sh' >>
> /root/.bash_aliases
>
> ---> Using cache
>
> ---> fe48b8b26e91
>
> Successfully built fe48b8b26e91
>
> Successfully tagged beam-build:latest
>
> *./start-build-env.sh: line 75: getent: command not found*
>
> Sending build context to Docker daemon 2.56kB
>
> Step 1/10 : FROM beam-build
>
> ---> fe48b8b26e91
>
> Step 2/10 : RUN rm -f /var/log/faillog /var/log/lastlog
>
> ---> Using cache
>
> ---> 11ea31f2099e
>
> Step 3/10 : RUN groupadd --non-unique -g 100 alex
>
> ---> Using cache
>
> ---> cf3f56f51d9f
>
> *Step 4/10 : RUN groupmod -g docker*
>
> * ---> Running in 5d5bc473be3b*
>
> *groupmod: invalid group ID 'docker'*
>
> *The command '/bin/bash -o pipefail -c groupmod -g docker' returned a
> non-zero code: 3*
>
>
>
> I investigated the issues and found that start-build-env.sh expected group
> "docker" created in the system groups. Creating a docker group is an
> optional post install step (
> https://docs.docker.com/engine/install/linux-postinstall/), another relevant
> reference to forum <https://forums.docker.com/t/no-more-docker-group/9123>
> gives a hint of using staff group on macOS. Solution that worked for me was
> pick another GID for DOCKER_GROUP_ID on Mac, e.g.
>
> #DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
>
>
>
> if [ "$(uname -s)" = "Linux" ]; then
>
> DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
>
> fi
>
> if [ "$(uname -s)" = "Darwin" ]; then
>
> DOCKER_GROUP_ID=1000
>
> fi
>
I think everyone using this container should do that postinstall step so
that docker doesn't have to be run as root. We actually point to it in an
error message if we detect the script is run as root. I think the real
issue here is that getent doesn't exist on MacOS, it looks like there is an
alternative approach though:
https://stackoverflow.com/questions/10910096/what-is-the-command-to-get-groupid-of-a-group-name-in-mac-or-linux
Would you be ok updating the script to use that approach on macOS? This
would still require users to make a docker group.
>
>
>
>
> 1. Step 10/10 : RUN chown -R alex: /Users/alex/.cache
>
> ---> Running in f09e2bb0e045
>
> chown: cannot access '/Users/alex/.cache': No such file or directory
>
> The command '/bin/bash -o pipefail -c chown -R alex: /Users/alex/.cache'
> returned a non-zero code: 1
>
>
>
> Fix for this was to change from HOME to DOCKER_HOME_DIR:
>
> RUN chown -R ${USER_NAME}:${GROUP_ID}
> ${DOCKER_HOME_DIR}/.cache
>
> ENV GOPATH
> ${DOCKER_HOME_DIR}/beam/sdks/go/examples/.gogradle/project_gopath
>
Ah yes this definitely looks like a bug. It just happened to work when
running from a machine where the HOME dir was the same as in the container.
This one definitely makes sense.
>
>
> If these proposed solutions make sense I will create JIRA ticket and
> submit these improvements for Docker build support on macOS.
>
>
>
> Thank you,
>
> Alex Kosolapov
>
>
>
> *From: *Brian Hulette <bh...@google.com>
> *Reply-To: *"dev@beam.apache.org" <de...@beam.apache.org>
> *Date: *Friday, December 4, 2020 at 1:15 PM
> *To: *dev <de...@beam.apache.org>
> *Cc: *Omar Ismail <om...@google.com>
> *Subject: *Re: Docker Development Environment
>
>
>
> I think https://github.com/apache/beam/pull/13308 is about ready to
> merge. One question was whether or not to install pyenv in the container -
> I think we should try to do without it. Users of this environment will
> already be operating within a container, so they shouldn't need pyenv to
> create isolated python environments, they could just use the container's
> system python.
>
>
>
> The two issues Alex mentioned are still outstanding, but they seem to be
> issues with ./gradlew check unrelated to the container. I filed BEAM-11402
> [1] to track this separately.
>
>
>
> [1] https://issues.apache.org/jira/browse/BEAM-11402
>
>
>
> On Mon, Nov 30, 2020 at 11:43 AM Alex Amato <aj...@google.com> wrote:
>
> If any of these are suitable for at least some development. I propose we
> merge them, and update them with fixes later. Rather than trying to get
> things 100% working in the first PR.
>
>
>
> Looks like this one was opened in early Sept, and never got merged. Which
> is a pretty long time. Perhaps abandoned for the later?
>
> https://github.com/apache/beam/pull/12837
>
>
>
> This one looks like its just failing on just a few tests (Which may be
> addressed soon, but the PR was opened 19 days ago).
>
> https://github.com/apache/beam/pull/13308
>
> (Can we set a deadline for this? And just say merge it by the end of the
> week, regardless if the last two tests can be fixed or not?)
>
>
>
> (Would like to nudge this along, as it's been a pain point for many for a
> while now).
>
>
>
> Thanks for the work here Niels, Omar and Sam.
>
> Looking forward to giving this a try :)
>
>
>
>
>
> On Mon, Nov 30, 2020 at 11:32 AM Brian Hulette <bh...@google.com>
> wrote:
>
> I agree this is a good idea. I remember my first experience with Beam
> development - I ran through the steps at [1] and had `./gradlew check`
> fail. I don't think I ever got it working before moving on and just running
> more specific tasks.
>
> It would be great if we had a reliable way for new contributors to
> establish an environment that can successfully run `./gradlew check`.
>
>
>
> Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be
> close to that, so I think we should focus on getting that working and
> iterate from there. Omar concurred with that in
> https://github.com/apache/beam/pull/12837.
>
>
> [1] https://beam.apache.org/contribute/#development-setup
>
>
>
>
>
> On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com> wrote:
>
> Thank you for doing this.
>
>
>
> I have seen a few related PRs. Connecting them here in case these efforts
> could be combined:
>
> - https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail
> <om...@google.com> )
>
> - https://github.com/apache/beam/pull/13308
>
>
>
> Ahmet
>
>
>
> On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com> wrote:
>
> Hi All,
>
>
>
> I got tired of my local dev environment being ruined by updates so I made
> a container for Apache Beam development work. What this does is create a
> Docker container from the Ubuntu Groovy image and load it up with all the
> necessary libraries/utilities for Apache Beam development. Then I run an
> interactive shell in the Docker container where I do my work.
>
>
>
> This is a nice way for new contributors to easily get started. However
> with the container in its current form, I don't know if this will help
> other people because it is tied closely with my workflow (using VIM,
> YouCompleteMe, for Python). But I think it can be a nice starting point for
> improvements. For example:
>
> - Sharing the host display with Docker to start GUI applications (like
> IntelliJ) in the container
> - Adding Golang development support
>
> Here's a draft PR <https://github.com/apache/beam/pull/13430>, let me
> know what you think, how it can be improved, and whether it's a good idea
> for us to have a dev container like this.
>
>
>
> Regards,
>
> Sam
>
>
>
>
Re: Docker Development Environment
Posted by Alex Kosolapov <al...@akvelon.com>.
Hi!
Thank you for creating Docker build environment - makes build environment setup so much easier!
I ran start-build-env.sh on a macOS, and I ran into some items that wanted to share + propose how to improve Docker build environment for macOS support:
1. ./start-build-env.sh: line 75: getent: command not found and script build error downstream.
Step 26/26 : RUN echo '. /scripts/beam_env_checks.sh' >> /root/.bash_aliases
---> Using cache
---> fe48b8b26e91
Successfully built fe48b8b26e91
Successfully tagged beam-build:latest
./start-build-env.sh: line 75: getent: command not found
Sending build context to Docker daemon 2.56kB
Step 1/10 : FROM beam-build
---> fe48b8b26e91
Step 2/10 : RUN rm -f /var/log/faillog /var/log/lastlog
---> Using cache
---> 11ea31f2099e
Step 3/10 : RUN groupadd --non-unique -g 100 alex
---> Using cache
---> cf3f56f51d9f
Step 4/10 : RUN groupmod -g docker
---> Running in 5d5bc473be3b
groupmod: invalid group ID 'docker'
The command '/bin/bash -o pipefail -c groupmod -g docker' returned a non-zero code: 3
I investigated the issues and found that start-build-env.sh expected group "docker" created in the system groups. Creating a docker group is an optional post install step (https://docs.docker.com/engine/install/linux-postinstall/), another relevant reference to forum<https://forums.docker.com/t/no-more-docker-group/9123> gives a hint of using staff group on macOS. Solution that worked for me was pick another GID for DOCKER_GROUP_ID on Mac, e.g.
#DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
if [ "$(uname -s)" = "Linux" ]; then
DOCKER_GROUP_ID=$(getent group docker | cut -d':' -f3)
fi
if [ "$(uname -s)" = "Darwin" ]; then
DOCKER_GROUP_ID=1000
fi
1. Step 10/10 : RUN chown -R alex: /Users/alex/.cache
---> Running in f09e2bb0e045
chown: cannot access '/Users/alex/.cache': No such file or directory
The command '/bin/bash -o pipefail -c chown -R alex: /Users/alex/.cache' returned a non-zero code: 1
Fix for this was to change from HOME to DOCKER_HOME_DIR:
RUN chown -R ${USER_NAME}:${GROUP_ID} ${DOCKER_HOME_DIR}/.cache
ENV GOPATH ${DOCKER_HOME_DIR}/beam/sdks/go/examples/.gogradle/project_gopath
If these proposed solutions make sense I will create JIRA ticket and submit these improvements for Docker build support on macOS.
Thank you,
Alex Kosolapov
From: Brian Hulette <bh...@google.com>
Reply-To: "dev@beam.apache.org" <de...@beam.apache.org>
Date: Friday, December 4, 2020 at 1:15 PM
To: dev <de...@beam.apache.org>
Cc: Omar Ismail <om...@google.com>
Subject: Re: Docker Development Environment
I think https://github.com/apache/beam/pull/13308 is about ready to merge. One question was whether or not to install pyenv in the container - I think we should try to do without it. Users of this environment will already be operating within a container, so they shouldn't need pyenv to create isolated python environments, they could just use the container's system python.
The two issues Alex mentioned are still outstanding, but they seem to be issues with ./gradlew check unrelated to the container. I filed BEAM-11402 [1] to track this separately.
[1] https://issues.apache.org/jira/browse/BEAM-11402
On Mon, Nov 30, 2020 at 11:43 AM Alex Amato <aj...@google.com>> wrote:
If any of these are suitable for at least some development. I propose we merge them, and update them with fixes later. Rather than trying to get things 100% working in the first PR.
Looks like this one was opened in early Sept, and never got merged. Which is a pretty long time. Perhaps abandoned for the later?
https://github.com/apache/beam/pull/12837
This one looks like its just failing on just a few tests (Which may be addressed soon, but the PR was opened 19 days ago).
https://github.com/apache/beam/pull/13308
(Can we set a deadline for this? And just say merge it by the end of the week, regardless if the last two tests can be fixed or not?)
(Would like to nudge this along, as it's been a pain point for many for a while now).
Thanks for the work here Niels, Omar and Sam.
Looking forward to giving this a try :)
On Mon, Nov 30, 2020 at 11:32 AM Brian Hulette <bh...@google.com>> wrote:
I agree this is a good idea. I remember my first experience with Beam development - I ran through the steps at [1] and had `./gradlew check` fail. I don't think I ever got it working before moving on and just running more specific tasks.
It would be great if we had a reliable way for new contributors to establish an environment that can successfully run `./gradlew check`.
Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be close to that, so I think we should focus on getting that working and iterate from there. Omar concurred with that in https://github.com/apache/beam/pull/12837.
[1] https://beam.apache.org/contribute/#development-setup
On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com>> wrote:
Thank you for doing this.
I have seen a few related PRs. Connecting them here in case these efforts could be combined:
- https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail<ma...@google.com> )
- https://github.com/apache/beam/pull/13308
Ahmet
On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com>> wrote:
Hi All,
I got tired of my local dev environment being ruined by updates so I made a container for Apache Beam development work. What this does is create a Docker container from the Ubuntu Groovy image and load it up with all the necessary libraries/utilities for Apache Beam development. Then I run an interactive shell in the Docker container where I do my work.
This is a nice way for new contributors to easily get started. However with the container in its current form, I don't know if this will help other people because it is tied closely with my workflow (using VIM, YouCompleteMe, for Python). But I think it can be a nice starting point for improvements. For example:
* Sharing the host display with Docker to start GUI applications (like IntelliJ) in the container
* Adding Golang development support
Here's a draft PR<https://github.com/apache/beam/pull/13430>, let me know what you think, how it can be improved, and whether it's a good idea for us to have a dev container like this.
Regards,
Sam
Re: Docker Development Environment
Posted by Brian Hulette <bh...@google.com>.
I think https://github.com/apache/beam/pull/13308 is about ready to merge.
One question was whether or not to install pyenv in the container - I think
we should try to do without it. Users of this environment will already be
operating within a container, so they shouldn't need pyenv to create
isolated python environments, they could just use the container's system
python.
The two issues Alex mentioned are still outstanding, but they seem to be
issues with ./gradlew check unrelated to the container. I filed BEAM-11402
[1] to track this separately.
[1] https://issues.apache.org/jira/browse/BEAM-11402
On Mon, Nov 30, 2020 at 11:43 AM Alex Amato <aj...@google.com> wrote:
> If any of these are suitable for at least some development. I propose we
> merge them, and update them with fixes later. Rather than trying to get
> things 100% working in the first PR.
>
> Looks like this one was opened in early Sept, and never got merged. Which
> is a pretty long time. Perhaps abandoned for the later?
> https://github.com/apache/beam/pull/12837
>
> This one looks like its just failing on just a few tests (Which may be
> addressed soon, but the PR was opened 19 days ago).
> https://github.com/apache/beam/pull/13308
> (Can we set a deadline for this? And just say merge it by the end of the
> week, regardless if the last two tests can be fixed or not?)
>
> (Would like to nudge this along, as it's been a pain point for many for a
> while now).
>
> Thanks for the work here Niels, Omar and Sam.
> Looking forward to giving this a try :)
>
>
> On Mon, Nov 30, 2020 at 11:32 AM Brian Hulette <bh...@google.com>
> wrote:
>
>> I agree this is a good idea. I remember my first experience with Beam
>> development - I ran through the steps at [1] and had `./gradlew check`
>> fail. I don't think I ever got it working before moving on and just running
>> more specific tasks.
>> It would be great if we had a reliable way for new contributors to
>> establish an environment that can successfully run `./gradlew check`.
>>
>> Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be
>> close to that, so I think we should focus on getting that working and
>> iterate from there. Omar concurred with that in
>> https://github.com/apache/beam/pull/12837.
>>
>> [1] https://beam.apache.org/contribute/#development-setup
>>
>>
>> On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Thank you for doing this.
>>>
>>> I have seen a few related PRs. Connecting them here in case these
>>> efforts could be combined:
>>> - https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail
>>> <om...@google.com> )
>>> - https://github.com/apache/beam/pull/13308
>>>
>>> Ahmet
>>>
>>> On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I got tired of my local dev environment being ruined by updates so I
>>>> made a container for Apache Beam development work. What this does is create
>>>> a Docker container from the Ubuntu Groovy image and load it up with all the
>>>> necessary libraries/utilities for Apache Beam development. Then I run an
>>>> interactive shell in the Docker container where I do my work.
>>>>
>>>> This is a nice way for new contributors to easily get started. However
>>>> with the container in its current form, I don't know if this will help
>>>> other people because it is tied closely with my workflow (using VIM,
>>>> YouCompleteMe, for Python). But I think it can be a nice starting point for
>>>> improvements. For example:
>>>>
>>>> - Sharing the host display with Docker to start GUI applications
>>>> (like IntelliJ) in the container
>>>> - Adding Golang development support
>>>>
>>>> Here's a draft PR <https://github.com/apache/beam/pull/13430>, let me
>>>> know what you think, how it can be improved, and whether it's a good idea
>>>> for us to have a dev container like this.
>>>>
>>>> Regards,
>>>> Sam
>>>>
>>>>
Re: Docker Development Environment
Posted by Alex Amato <aj...@google.com>.
If any of these are suitable for at least some development. I propose we
merge them, and update them with fixes later. Rather than trying to get
things 100% working in the first PR.
Looks like this one was opened in early Sept, and never got merged. Which
is a pretty long time. Perhaps abandoned for the later?
https://github.com/apache/beam/pull/12837
This one looks like its just failing on just a few tests (Which may be
addressed soon, but the PR was opened 19 days ago).
https://github.com/apache/beam/pull/13308
(Can we set a deadline for this? And just say merge it by the end of the
week, regardless if the last two tests can be fixed or not?)
(Would like to nudge this along, as it's been a pain point for many for a
while now).
Thanks for the work here Niels, Omar and Sam.
Looking forward to giving this a try :)
On Mon, Nov 30, 2020 at 11:32 AM Brian Hulette <bh...@google.com> wrote:
> I agree this is a good idea. I remember my first experience with Beam
> development - I ran through the steps at [1] and had `./gradlew check`
> fail. I don't think I ever got it working before moving on and just running
> more specific tasks.
> It would be great if we had a reliable way for new contributors to
> establish an environment that can successfully run `./gradlew check`.
>
> Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be
> close to that, so I think we should focus on getting that working and
> iterate from there. Omar concurred with that in
> https://github.com/apache/beam/pull/12837.
>
> [1] https://beam.apache.org/contribute/#development-setup
>
>
> On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com> wrote:
>
>> Thank you for doing this.
>>
>> I have seen a few related PRs. Connecting them here in case these efforts
>> could be combined:
>> - https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail
>> <om...@google.com> )
>> - https://github.com/apache/beam/pull/13308
>>
>> Ahmet
>>
>> On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com> wrote:
>>
>>> Hi All,
>>>
>>> I got tired of my local dev environment being ruined by updates so I
>>> made a container for Apache Beam development work. What this does is create
>>> a Docker container from the Ubuntu Groovy image and load it up with all the
>>> necessary libraries/utilities for Apache Beam development. Then I run an
>>> interactive shell in the Docker container where I do my work.
>>>
>>> This is a nice way for new contributors to easily get started. However
>>> with the container in its current form, I don't know if this will help
>>> other people because it is tied closely with my workflow (using VIM,
>>> YouCompleteMe, for Python). But I think it can be a nice starting point for
>>> improvements. For example:
>>>
>>> - Sharing the host display with Docker to start GUI applications
>>> (like IntelliJ) in the container
>>> - Adding Golang development support
>>>
>>> Here's a draft PR <https://github.com/apache/beam/pull/13430>, let me
>>> know what you think, how it can be improved, and whether it's a good idea
>>> for us to have a dev container like this.
>>>
>>> Regards,
>>> Sam
>>>
>>>
Re: Docker Development Environment
Posted by Brian Hulette <bh...@google.com>.
I agree this is a good idea. I remember my first experience with Beam
development - I ran through the steps at [1] and had `./gradlew check`
fail. I don't think I ever got it working before moving on and just running
more specific tasks.
It would be great if we had a reliable way for new contributors to
establish an environment that can successfully run `./gradlew check`.
Niels Basjes' PR (https://github.com/apache/beam/pull/13308) seems to be
close to that, so I think we should focus on getting that working and
iterate from there. Omar concurred with that in
https://github.com/apache/beam/pull/12837.
[1] https://beam.apache.org/contribute/#development-setup
On Wed, Nov 25, 2020 at 3:39 PM Ahmet Altay <al...@google.com> wrote:
> Thank you for doing this.
>
> I have seen a few related PRs. Connecting them here in case these efforts
> could be combined:
> - https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail
> <om...@google.com> )
> - https://github.com/apache/beam/pull/13308
>
> Ahmet
>
> On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com> wrote:
>
>> Hi All,
>>
>> I got tired of my local dev environment being ruined by updates so I made
>> a container for Apache Beam development work. What this does is create a
>> Docker container from the Ubuntu Groovy image and load it up with all the
>> necessary libraries/utilities for Apache Beam development. Then I run an
>> interactive shell in the Docker container where I do my work.
>>
>> This is a nice way for new contributors to easily get started. However
>> with the container in its current form, I don't know if this will help
>> other people because it is tied closely with my workflow (using VIM,
>> YouCompleteMe, for Python). But I think it can be a nice starting point for
>> improvements. For example:
>>
>> - Sharing the host display with Docker to start GUI applications
>> (like IntelliJ) in the container
>> - Adding Golang development support
>>
>> Here's a draft PR <https://github.com/apache/beam/pull/13430>, let me
>> know what you think, how it can be improved, and whether it's a good idea
>> for us to have a dev container like this.
>>
>> Regards,
>> Sam
>>
>>
Re: Docker Development Environment
Posted by Ahmet Altay <al...@google.com>.
Thank you for doing this.
I have seen a few related PRs. Connecting them here in case these efforts
could be combined:
- https://github.com/apache/beam/pull/12837 (/cc +Omar Ismail
<om...@google.com> )
- https://github.com/apache/beam/pull/13308
Ahmet
On Wed, Nov 25, 2020 at 2:53 PM Sam Rohde <sr...@google.com> wrote:
> Hi All,
>
> I got tired of my local dev environment being ruined by updates so I made
> a container for Apache Beam development work. What this does is create a
> Docker container from the Ubuntu Groovy image and load it up with all the
> necessary libraries/utilities for Apache Beam development. Then I run an
> interactive shell in the Docker container where I do my work.
>
> This is a nice way for new contributors to easily get started. However
> with the container in its current form, I don't know if this will help
> other people because it is tied closely with my workflow (using VIM,
> YouCompleteMe, for Python). But I think it can be a nice starting point for
> improvements. For example:
>
> - Sharing the host display with Docker to start GUI applications (like
> IntelliJ) in the container
> - Adding Golang development support
>
> Here's a draft PR <https://github.com/apache/beam/pull/13430>, let me
> know what you think, how it can be improved, and whether it's a good idea
> for us to have a dev container like this.
>
> Regards,
> Sam
>
>